What is Gradient Descent?
Gradient Descent is an iterative optimisation algorithm that updates model parameters in the direction of steepest loss decrease. Stochastic and mini-batch variants sample subsets of data per step for computational efficiency, introducing useful noise that aids generalisation.