Rectified Linear Unit - What is the advantage of Rectified Linear Unit?

as Sigmoid functions need expensive exponential operations, ReLU (Rectified Linear Unit) offers a computational advantage over them as it just needs to choose max(0, x). When it comes to real-world applications, ReLU-based networks typically outperform Sigmoid-based networks in terms of convergence performance. (Krizhevsky & colleagues)