Terms used in ML
These words turn up in every explanation, every tutorial and every interview, and most of them name something you have already met in the Overview. Each one gets a definition, the same house example carried through, and the smallest piece of code that shows the idea.
Model
The function with adjustable numbers inside it. Features go in, a prediction comes out.
The house: price = sqft × weight is a model. Change the weight and every predicted price changes with it.
function predictPrice(sqft: number, weight: number): number {
return sqft * weight;
}Feature
One input number describing an example. You will also see it called an input or, loosely, a column.
The house: Size, bedroom count and age are three features describing a single house.
const houseFeatures = [1500, 3, 10]; // sqft, bedrooms, ageParameter
Any number the model learns during training — the dials it is allowed to turn. Weights and bias are both parameters. Every weight is a parameter; not every parameter is a weight.
The house: In price = sqft × weight there is one parameter: the weight. Add bedrooms and a bias and the parameters are [sqftWeight, bedroomWeight, bias] — everything training can change.
const parameters = {
sqftWeight: 200,
bedroomWeight: 15_000,
bias: 50_000
};Weight
A parameter that multiplies one feature — how hard that feature pushes the answer up or down. It is one kind of parameter, not a separate idea.
The house: A weight of 200 on size means every extra square foot adds about $200 to the predicted price. That weight is a parameter; so is the bias sitting next to it.
const sqftWeight = 200;
const fromSize = 1500 * sqftWeight; // 300,000Bias
A parameter added after everything is multiplied and summed — not a weight, because it is not tied to a feature. It is the answer the model gives when every feature is zero.
The house: A bias of $50,000 means houses start at $50,000 before size or bedrooms are counted at all — which is often closer to how prices really behave than starting from nothing.
const bias = 50_000;
const price = 1500 * 200 + bias; // 350,000Error and loss
How wrong a prediction is. Error usually refers to one example; loss is usually the averaged score across many, which is what training actually tries to shrink.
The house: A 1,450 sqft house is predicted at $290,000 and sells for $310,000, so the error on that house is $20,000.
const predicted = 290_000;
const actual = 310_000;
const error = actual - predicted; // 20,000Epoch
One complete pass through every training example. Training runs many of them, because a single pass moves the weights only slightly.
The house: With five houses on file, one epoch is five predictions and five corrections.
for (const [features, price] of HOUSES) {
model.train(features, price); // one epoch
}Training
Running that loop over and over: predict, compare against the real answer, adjust. The weights are moving the whole time.
The house: Five thousand epochs over the same five houses, with the weight creeping towards a sensible dollars-per-square-foot.
for (let epoch = 0; epoch < 5_000; epoch++) {
for (const [features, price] of HOUSES) {
model.train(features, price);
}
}Inference
Using the finished model on new data. It predicts and nothing else — no answer to compare against, no weights moving.
The house: A 2,000 sqft listing goes on the market this morning. Nobody knows what it will sell for, and the model returns an estimate anyway.
const estimate = model.predict([2000, 3, 5]);Gradient
The direction of steepest increase in loss. Training steps the other way — downhill.
The house: If raising the sqft weight would make error worse, the gradient on that weight points “up”; you nudge the weight down instead.
// sketch: step opposite the gradient
weight -= learningRate * gradient;Learning rate
How large each downhill step is. Too small crawls; too large overshoots or explodes.
The house: With raw square footage in the thousands, a learning rate around 1e-8 is a common starting point for these toy house models.
const learningRate = 1e-8;Batch
How many examples you look at before one weight update. Size 1 is SGD; larger batches average the gradient first.
The house: Five houses with batch size 1 means five nudges per epoch; batch size 5 means one nudge after seeing all five.
for (const [features, price] of batch) {
// accumulate gradient, then one update
}Activation (ReLU)
A bend applied after the weighted sum. ReLU is max(0, x): off below zero, pass-through above.
The house: A neuron can ignore “negative evidence” by outputting zero, then turn on when the sum crosses the hinge.
function relu(x: number): number {
return Math.max(0, x);
}Softmax
Turns a list of raw class scores into probabilities that sum to 1.
The house: Scores [2.1, 0.3, −1.0] for cheap / mid / expensive become about [0.81, 0.14, 0.05].
function softmax(logits: number[]): number[] {
const max = Math.max(...logits);
const exps = logits.map((x) => Math.exp(x - max));
const sum = exps.reduce((a, b) => a + b, 0);
return exps.map((e) => e / sum);
}Overfitting
Train performance looks great while held-out examples do poorly — memorization instead of a pattern that transfers.
The house: Loss on H1–H4 is nearly zero, but the prediction for held-out H5 is far off.
const train = HOUSES.slice(0, 4);
const heldOut = HOUSES[4];
// fit only on train, then score heldOutTrain / test split
Partition examples into ones you may train on and ones you only evaluate on. The test rows must not update weights.
The house: Train on four listings; keep the fifth as the exam.
const train = HOUSES.slice(0, 4);
const test = HOUSES.slice(4);