Start free trial
Take Naologic for a spin today, no credit card needed and no obligations.
Start free trial

Rectified Linear Unit - What is the purpose of leaky ReLU?


To make the outputs of artificial neural networks nonlinear throughout all of their layers, Leaky ReLU is used as an activation function. A 'dead' neural network might happen during training if the normal ReLU function is used, which is why this function was created to fix the 'dying ReLU' problem.