At the moment a lot of activation functions are hard coded. The idea would be to provide `nonlinearity` or `gate_nonlinearity` kwargs wherever needed.
At the moment a lot of activation functions are hard coded. The idea would be to provide
nonlinearityorgate_nonlinearitykwargs wherever needed.