Vanishing/Exploding Gradients - What is vanishing and exploding gradients?

There are two possible outcomes in Neural Networks: disappearing gradients and inflating gradients. When gradients are disappearing, the derivative term shrinks to nearly nothing, and when gradients are exploding, it grows to an excess of zero.