Frame Stacking: How to Teach a Machine to See Speed
Look at a still photograph of a soccer ball hovering three feet above the penalty box grass.
Is the ball rocketing toward the upper corner of the goal, or did it just bounce off the crossbar and float lazily back toward midfield?
You cannot tell. A single frozen snapshot tells you exactly where an object is, but it tells you nothing about where it is going.
Now put yourself in the gaming chair of an esports pro defending a bomb site. An enemy projectile flashes across your peripheral vision. If your eyes only captured a single static frame, you would not know whether to dodge left, duck behind cover, or hold your ground. To hit a flick shot or dodge an incoming flashbang, your brain relies on visual persistence: your visual cortex blends several micro-instants of recent motion into a vector of speed and direction.
When DeepMind first hooked a neural network up to classic video games like Pong and Breakout, their AI suffered from the exact same blindness.
The Reality
In Reinforcement Learning, an agent must know the State of the environment to choose the best action.
If you feed the neural network only the current screen image, the agent sees the paddle and the ball, but it has no idea whether the ball is traveling at fifty miles per hour toward its side of the court or bouncing away toward the opponent. A single frame is not a complete State.
In mathematics, this violates the Markov Property—the principle that the current state must contain all necessary information to decide the next move, without needing to dig through past history.
DeepMind resolved this with a surprisingly simple mechanic: Frame Stacking.
Instead of sending one single image into the convolutional neural network, the system stacks the last four consecutive frames together into a single four-layer visual sandwich (t-3, t-2, t-1, and t).
Because the neural network receives four slices of time simultaneously, its convolutional filters can instantly compare the shift of pixels across layers. The ball’s position across those four frames exposes its exact trajectory, velocity, and acceleration.
The Why
Giving an AI the sense of motion does not require giving it a complicated human memory. It only requires widening its window on the immediate present.
With four stacked frames, a neural network can calculate whether a jump has peaked, whether an opponent is accelerating out of a slide, or whether a projectile is about to hit. It transforms a series of disconnected still photos into a dynamic, living movie.
Where does this break down?
Stacking frames solves short-term velocity, but it provides zero long-term strategic memory. Stacking four frames covers a fraction of a second. It cannot tell an agent that an opponent hid behind a crate five seconds ago, or that a spell cooldown takes twelve seconds to recharge. If an environment requires genuine long-term recall across minutes, frame stacking collapses, and engineers have to bring in recurrent neural networks or transformers.
The Takeaway
A single snapshot shows you where the world stands; to know where the world is heading, you have to look at the motion between the frames.
AI specialists call it: Frame Stacking / State Representation / Markov Decision Process (MDP) Restoration
A single video frame in an Atari environment is partially observable (POMDP) because velocity and acceleration cannot be inferred from a single temporal slice. By concatenating m consecutive frames (typically m=4, often combined with frame skipping where actions repeat for k frames), the input tensor to the Convolutional Neural Network restores an effective Markovian state representation without requiring recurrent memory architectures like LSTMs.
💬 Have you ever tried to diagnose a problem at work from a single static dashboard snapshot, only to realize you could not solve it until you saw the rate of change over time?
Part 6 of 15 | #ReinforcementLearningForHumans
#ai_edu Based on Stanford and industry lectures