Activation functions: the decision point

A neuron first adds its weighted inputs, then passes the result through an activation function. This function decides how much signal moves to the next layer.

ReLU is simple: it turns negative numbers into zero and keeps positive numbers. This small bend lets a network learn nonlinear patterns.