Draw a 7 on a 16 by 16 grid and you have made 256 numbers, one per square, from 0 for blank to 1 for full ink. A neural network that reads digits is a function from those 256 numbers to ten more, one score for each digit it might be. Learning is how it finds that function.
The digit recogniser in the Y2K Lab is this kind of network, with every matrix multiply, every backpropagation pass and the softmax written by hand in plain JavaScript. Here it is, built up from one neuron.
One neuron
A neuron multiplies each of the 256 numbers by a weight, adds them up, adds one more number called the bias, and squashes the total into the range 0 to 1 with a sigmoid curve, 1 / (1 + e^-x).
The weights are the neuron’s opinion. A positive weight on a square says ink there is evidence for; a negative one says it is evidence against. A neuron trained to spot 1s might end up with positive weights down the middle column and negative ones either side. Show it a 1 and it fires. Show it a 0, with a hole in the middle, and it stays quiet.
But one neuron can only draw one straight boundary through the space of possible images: enough for 1 against 0, nowhere near enough for ten digits in different hands.
A layer of them, then another
So use more neurons, in layers. Put, say, 32 of them in a hidden layer. Each looks at all 256 squares and learns a feature of its own, rarely as tidy as “a loop at the top” but in that spirit. An output layer of ten neurons looks at those 32 features instead of the raw squares, and each learns which features add up to its digit.
A softmax turns the ten outputs into probabilities: raise e to the power of each score and divide by the total, so all ten are positive and add up to one. Running a drawing through all of this is the forward pass: two matrix multiplications and two squashing functions.
How wrong was it?
A new network starts with small random weights, so its first guesses are noise. To improve, it needs a number for how wrong it was, usually cross-entropy: minus the log of the probability it gave the right answer. Give the right digit 0.9 and the error is about 0.1; give it 0.1 and it is about 2.3; give it 0.01 and it is about 4.6. Confident mistakes cost the most, which is the point.
Backpropagation: handing out the blame
Training nudges every weight so the error goes down. Backpropagation decides which way by passing blame backwards through the network.
Start at the outputs. With softmax and cross-entropy together, each output’s blame is simply what it said minus what it should have said. If the answer was 7 and the network gave 7 a probability of 0.3, the 7 output gets minus 0.7: push up. Every other output gets its own probability: push down, hardest on the most wrongly confident.
Then step back a layer. A hidden neuron’s blame is the output blame, weighted by its connections to each output, times the slope of its sigmoid. A neuron stuck near 0 or 1 sits on a flat part of the curve and takes little blame. Finally each weight moves against its blame, scaled by the input that flowed through it and a small learning rate. A weight from a blank square does not move, because nothing flowed through it.
That is the chain rule from school calculus, one layer at a time. Here is a complete training step for 256 inputs, 32 hidden neurons and 10 outputs:
const sigmoid = v => 1 / (1 + Math.exp(-v));
const dot = (w, v) => w.reduce((s, wi, i) => s + wi * v[i], 0);
function softmax(z) {
const top = Math.max(...z); // no overflow
const e = z.map(v => Math.exp(v - top));
const sum = e.reduce((a, b) => a + b, 0);
return e.map(v => v / sum);
}
// W1: 32 rows of 256 weights, W2: 10 rows of 32, b1 and b2: biases.
// x: 256 values from 0 to 1. label: the right digit.
function trainStep({ W1, b1, W2, b2 }, x, label, rate) {
// Forward pass
const h = W1.map((row, j) => sigmoid(dot(row, x) + b1[j]));
const p = softmax(W2.map((row, k) => dot(row, h) + b2[k]));
// Output blame: what it said minus what it should have said
const dOut = p.map((pk, k) => pk - (k === label ? 1 : 0));
// Hidden blame: back through W2, times the sigmoid's slope
const dHid = h.map((hj, j) =>
dOut.reduce((s, d, k) => s + d * W2[k][j], 0) * hj * (1 - hj));
// Move each weight against its blame
W2.forEach((row, k) => {
row.forEach((_, j) => { row[j] -= rate * dOut[k] * h[j]; });
b2[k] -= rate * dOut[k];
});
W1.forEach((row, j) => {
row.forEach((_, i) => { row[i] -= rate * dHid[j] * x[i]; });
b1[j] -= rate * dHid[j];
});
return -Math.log(p[label]); // the error
}The hidden blame is worked out before any weight changes, because it needs the old weights. Call this once per example, thousands of times, and the error trends down.
What learning looks like from outside
You never see a weight change; you see guesses change. The digit recogniser watches as you write, so it guesses from half-finished strokes. A 4 before its last stroke can honestly pass for a 1, and a network that has learned well should be unsure at that point rather than confidently wrong.
Its mistakes should make sense too: 4 and 9, 3 and 8, 1 and 7 share strokes. And a network knows only the styles it learned from, so a crossed 7 or a 1 with a long flag can throw one that never saw them. Adapting to a new hand means more passes of the step above, on examples written that way.
Old maths, new code
None of this was new when I wrote the coursework in 2000. Backpropagation had been popular since the mid-1980s, and handwritten digits were already a standard test for neural networks. What a student did not have was TensorFlow, PyTorch or Keras. Understanding a network meant writing one.
The 2026 rebuild kept that rule. The code is new and runs in today’s browsers; the maths is the coursework’s, with no machine-learning library underneath. The networks behind today’s products run the same three steps, forward pass, error and backward pass, with billions of weights where a network like this has thousands. Watching the small one learn is the cheapest way I know to make the big ones less mysterious.
