Curriculum

Topic 02 · ML Fundamentals

Machine Learning

Machine Learning is the branch of AI where systems improve through data rather than explicit programming. The training loop—hypothesis, error, update—runs millions of times.

The idea

Machine learning replaces hand-crafted rules with statistical patterns extracted from data. Given enough labelled examples, a model learns to generalise. The training loop is conceptually simple: pick a hypothesis, measure the error on training data, adjust parameters in the direction that reduces error, repeat.

Supervised learning uses labelled pairs (input → output). Unsupervised learning finds structure in unlabelled data—clusters, dimensions, anomalies. Semi-supervised blends both; self-supervised invents labels from the data itself, which is how LLMs are pre-trained.

Overfitting is the central enemy: a model that memorises training examples fails on unseen ones. Validation sets, regularisation, dropout, and early stopping are all defences. Bias-variance tradeoff: simpler models underfit (high bias), complex models overfit (high variance).

L(θ) = (1/n) Σᵢ (ŷᵢ − yᵢ)²

Mean Squared Error—the most common regression loss.

What to hold onto

  • Train / Val / Test splitThe test set must never be touched until final evaluation—otherwise you have a very expensive form of overfitting.
  • Cross-validationK-fold CV gives a more reliable performance estimate when data is scarce.
  • Feature engineeringFor tabular data, hand-crafted features often outperform raw inputs. Deep models automate this for images and text.

Live: Linear & Polynomial Regression Lab

Click canvas to add sample points. Adjust polynomial degree or run real-time gradient descent optimization.

Click canvas to place data points MSE: 0.000 | Points: 0

Quick answers

What is Machine Learning?

Machine Learning is a subset of AI where algorithms learn patterns from data to make predictions or decisions without being explicitly programmed for each scenario. It encompasses supervised, unsupervised, and reinforcement learning paradigms.

What is overfitting in Machine Learning?

Overfitting occurs when a model memorises training data too closely, including its noise, and fails to generalise to unseen examples. It is detected by a gap between training and validation performance, and addressed with regularisation, dropout, or more training data.