The idea
Reinforcement Learning frames the learning problem as an agent interacting with an environment. At each step the agent observes a state, takes an action, and receives a scalar reward. The objective is to learn a
policy—a mapping from states to actions—that maximises total discounted future reward.
Markov Decision Processes (MDPs) provide the mathematical framework. An MDP is defined by states S, actions A, transition dynamics P(s'|s,a), and a reward function R(s,a). The Markov property says the future is independent of the past given the present state.
Q-Learning learns a value function Q(s,a)—the expected total reward from taking action a in state s and following the optimal policy thereafter. Deep Q-Networks (DQN) parameterise Q with a neural network, enabling RL in high-dimensional spaces like Atari games.
Q(s,a) ← Q(s,a) + α[r + γ·maxₐ'Q(s',a') − Q(s,a)]
Bellman update for Q-Learning. α is the learning rate, γ is the discount factor.