What Is Deep Learning?
Deep learning is machine learning that uses a stack of small calculators — called neurons — grouped into layers and bent with rules like ReLU, so the model can build a prediction in stages instead of one flat formula.
You already have the loop on Machine Learning: guess, measure how wrong you were, nudge the dials. This page names the building blocks practitioners use — neuron, activation, ReLU, layer, network — and shows each one with the same house example before the next word appears.
Start with something you already know
You want to guess a house price from three numbers. One sold listing looks like this:
| What we measure | Example house |
|---|---|
| Size (sqft) | 1500 |
| Bedrooms | 3 |
| Age (years) | 10 |
| Real sale price | $295,000 |
In the earlier lessons, one way to guess was: multiply each number by a dial, add them up, add one more dial called bias, and treat that sum as the price. Those dials are weights and bias. Training moves them until the guess gets closer to $295,000.
Deep learning uses that same mix, repeated in small blocks. The rest of this page names those blocks.
1. Neuron — one small calculator
A neuron is one tiny calculator. It takes several numbers in, mixes them with weights and a bias, then (usually) bends the result before sending one number out.
With our house, using made-up dials just to show the arithmetic:
That 255000 is the pre-activation — the number before any bend. If you stopped here, this neuron would be doing the same job as linear regression: one weighted sum.
Three features land on the neuron: size, bedrooms, and age.
function mix(
inputs: number[],
weights: number[],
bias: number,
): number {
let sum = bias;
for (let i = 0; i < inputs.length; i++) {
sum += inputs[i] * weights[i];
}
return sum; // pre-activation
}
mix([1500, 3, 10], [100, 20_000, -500], 50_000); // 2550002. Activation function — the bend after the mix
An activation function is a small rule you apply to the pre-activation. Its job is to bend the number.
Without a bend, stacking many mixes is still just one big mix — linear plus linear stays linear. The bend is what lets a network learn curved patterns. In practice, hidden neurons (the middle ones) almost always get an activation. The last neuron that outputs a dollar amount often has no bend, so the price can be any number.
3. ReLU — the activation you will see most often
ReLU means: if the number is negative, turn it into 0; if it is positive, leave it alone. The exact formula — the same one frameworks like PyTorch use — is ReLU(x) = max(0, x).
| Pre-activation | After ReLU | Meaning |
|---|---|---|
| 255000 | 255000 | Neuron is on — value passes through |
| 5 | 5 | Still on |
| 0 | 0 | On the hinge |
| −3 | 0 | Neuron is off |
| −5000 | 0 | Off |
So ReLU is a switch: negative evidence is ignored; positive evidence passes through.
Pre-activation is −3. ReLU turns it into 0. The neuron stays quiet.
function relu(x: number): number {
return Math.max(0, x);
}
relu(255_000); // 255000
relu(-3); // 0A full neuron in practice is mix plus ReLU:
function neuron(
inputs: number[],
weights: number[],
bias: number,
): number {
const pre = mix(inputs, weights, bias);
return relu(pre);
}
neuron([1500, 3, 10], [100, 20_000, -500], 50_000); // 255000
neuron([1500, 3, 10], [-1, -1, -1], 0); // 0 (off)4. Layer — several neurons side by side
A layer is a group of neurons that all look at the same inputs, each with its own weights and bias. Each neuron produces one number. Together they output a list of numbers (a vector).
Example: one hidden layer with two neurons, both reading [1500, 3, 10]:
Both neurons see the same house features. Only their dials differ.
type NeuronParams = { weights: number[]; bias: number };
function layer(
inputs: number[],
neurons: NeuronParams[],
): number[] {
return neurons.map((n) => neuron(inputs, n.weights, n.bias));
}
const hidden = layer(
[1500, 3, 10],
[
{ weights: [100, 20_000, -500], bias: 50_000 },
{ weights: [-2, -2, -2], bias: 0 },
],
);
// hidden → [255000, 0]5. Neural network — layers in a line
A neural network is layers hooked together: the output list of one layer becomes the input list of the next.
The middle layer is called hidden because it is not the answer you care about. You care about the final price. The middle numbers are internal values the network invents while training.
One trip from inputs to prediction is a forward pass. Weights do not change during that trip — it is only a guess.
Start with the house: square feet, bedrooms, age.
function predictPrice(
features: number[], // [sqft, beds, age]
wH: number[][], // hidden weights, shape 2×3
bH: number[], // hidden biases, length 2
wOut: number[], // output weights, length 2
bOut: number,
): number {
const hidden = [0, 1].map((j) =>
relu(mix(features, wH[j], bH[j])),
);
// linear output — no ReLU, so the price can be any number
return mix(hidden, wOut, bOut);
}6. What “deep” means
Deep means more than one nonlinear stage between input and output — usually two or more hidden layers, each with an activation like ReLU.
Deep learning is the name for machine learning that uses these deep neural networks. It is still the same three ingredients: parameters (all the weights and biases), prediction (the forward pass), and feedback (compare to the real price, then fix the dials).
What depth adds: early layers can reshape the inputs into new numbers; later layers use those numbers. Practitioners call those middle numbers learned features or a representation — just remember that middle layers invent useful internal numbers while training.
Both paths start from the same house features.
7. How the dials learn
After the forward pass, the network was wrong by some amount. Backpropagation means: send that error backward (output → hidden → …) so every weight knows which way to nudge. Then gradient descent takes a small step on each weight (the learning rate). Repeat for many epochs — full passes over the training houses.
You have a prediction and a real sale price. The gap is the error.
8. Same houses, deeper function
Use the same HOUSES table as the machine-learning lessons. The training step is still: forward → error → nudge.
const HOUSES: [number[], number][] = [
[[1200, 2, 15], 245_000],
[[1800, 3, 8], 310_000],
[[2200, 4, 3], 420_000],
[[ 900, 2, 40], 180_000],
[[1500, 3, 10], 295_000],
];
// Forward: features → hidden ReLUs → price guess
const prediction = predictPrice(features, wH, bH, wOut, bOut);
// How wrong?
const error = actualPrice - prediction;
// Then nudge every weight a little (backprop + gradient descent).
// Full training loops live in the Lessons tab when they ship.- Training
- You have sale prices. Weights move.
- Inference
- Weights are frozen. A new listing arrives; you only predict.