To a program, a photo of a number plate is just a grid of coloured dots. Somewhere in it are about ten characters, and the job is to say which ones, in order. The classical way to do that is a pipeline: each stage makes the picture a little simpler and hands it on.
The plate reader in the Y2K Lab, from my Computer Vision coursework, takes that classical route and shows every stage on screen. Here are the four stages, and how each one breaks. In a pipeline, a stage that fails quietly ruins everything after it.
Greyscale: one number per pixel
Each pixel’s red, green and blue values become a single brightness, usually weighted by how sensitive the eye is to each colour: roughly 0.3 red, 0.59 green and 0.11 blue. One number instead of three makes every later stage simpler.
What goes wrong: sometimes colour was the information. A red and a green of similar brightness become the same grey, and the edge between them disappears. The pipeline also tends to assume dark characters on a light plate. A green plate with white characters, like those on private electric cars in India, turns that upside down, so it pays to check which way the contrast runs first.
Threshold: decide what is ink
Next, every grey pixel becomes black or white: darker than the cut-off is character, lighter is background. A fixed cut-off works on a well lit test image and fails on the first photo taken at dusk.
The classic answer is Otsu’s method, from 1979. Try every possible cut-off, and keep the one that splits the pixels into two groups that are both large and far apart in brightness. No tuning, and it adapts to each image:
// grey: brightness of every pixel, whole numbers from 0 to 255
function otsuThreshold(grey) {
const hist = new Array(256).fill(0);
for (const v of grey) hist[v]++;
let sumAll = 0;
for (let t = 0; t < 256; t++) sumAll += t * hist[t];
let best = 0, bestScore = -1, nDark = 0, sumDark = 0;
for (let t = 0; t < 256; t++) {
nDark += hist[t]; // pixels at t or darker
sumDark += t * hist[t];
const nLight = grey.length - nDark;
if (nDark === 0 || nLight === 0) continue;
const gap = sumDark / nDark - (sumAll - sumDark) / nLight;
const score = nDark * nLight * gap * gap; // big groups, far apart
if (score > bestScore) { bestScore = score; best = t; }
}
return best; // at or below this is ink
}What goes wrong: Otsu picks one cut-off for the whole image. With a shadow across half the plate, no single value is right everywhere, which is why adaptive thresholding compares each pixel with its own neighbourhood instead. Get this stage wrong and characters break apart or melt together, and nothing downstream can put them back.
Segmentation: cut it into characters
The plate is now black shapes on white. One common approach is connected components: flood-fill each blob of touching black pixels and record its bounding box. Another is a projection: count the black pixels in each column, and the gaps between characters show up as columns with almost none. Screws, the border and dirt leave blobs too, so the boxes are filtered by size and shape, then sorted left to right.
What goes wrong: two characters that touch become one wide blob, and a thin stroke that broke becomes two small ones. Two-line plates, common on motorcycles, need the rows found before the columns. A tilted plate defeats a column projection, because the gaps are no longer vertical.
Template matching: compare with what you know
Each character is scaled to a fixed size and compared with a stored template of every possible character, 0 to 9 and A to Z, by counting the pixels that agree or with a correlation score. The best score wins.
What goes wrong: lookalikes. 0 and O, 1 and I, 8 and B differ by a few pixels, and a little blur erases the difference. The classical answer is to use what you know about the plate. In the usual Indian format the first two characters are letters and the last four are digits, so a character in a digit position is matched only against digits, and the question of O or 0 answers itself.
What changed, and what did not
A modern plate reader is usually two neural networks. One finds the plate in the photo, even at an angle or in poor light. The other reads the whole line at once, trained on many thousands of labelled images with a method (CTC is a common one) that never needs to be told where each character sits. There is no segmentation step to fail and no hand-picked threshold.
The classical pipeline is still worth understanding, because it is the clearest map of the problem there is. Lighting is a threshold problem, touching characters are a segmentation problem, 8 against B is a matching problem. A neural network faces the same difficulties; it just meets them inside one model, where you cannot point at the stage that failed. Knowing the stages tells you where to look when it gets a plate wrong. And where the camera, the light and the font are under your control, a pipeline like this one can still be enough.
