Start free trial
Take Naologic for a spin today, no credit card needed and no obligations.
Start free trial

Rectified Linear Unit - What is the formula for ReLU?


The notion of Rectified Linear Units, or ReLU for short, is one that has just recently emerged. Maximizing the value of z is a very simple formula. It is not, in fact, linear, despite appearances to the contrary. It improves upon the Sigmoid function in efficiency while maintaining its benefits, such as learning nonlinear functions.