An activation function in a neural network is a mathematical function applied to the output of a neuron. It introduces non-linearity, enabling the model to learn and represent complex data patterns. Without it, even a deep neural network would behave like a simple linear regression model.
Z equals the sum of w-sub-i times x-sub-i, plus b
Linear vs nonlinear functions.
