What generalization is
A model that only looks good on the houses it trained on has not learned the market — it has memorized the homework. Generalization means doing well on examples it did not see while the weights were moving.
This closes the supervised path from First Learning Model through Price Tier Classifier: always keep a held-out check, even when the dataset is tiny.
- Training set
- The examples you may run
train/stepon. Weights may look at these labels. - Held-out / test
- Examples you score only with
predict. If you train on them, the exam is leaked. - Overfitting
- Train loss looks great; held-out error is poor. Too much memorization (often too much capacity or too many epochs on too little data).
- Underfitting
- Train loss stays high. The model is too simple or not trained enough to capture the pattern.
Habit
Split the five houses
Smallest honest protocol on HOUSES: train on four, check the fifth. Step through the story.
Until now every lesson trained and scored on the same five rows. Loss going down felt like success — but the model had already seen every test.
What happens in this step
H1 H2 H3 H4 H5 all used for training and for “how good are we?”
Why it bites
Capacity, epochs, and five rows
A wide network ( Price Tier Classifier with many hidden units) can drive training loss to zero on four houses and still miss H5. More epochs on the same four rows make memorization easier, not harder.
- regime
Train loss stays high. Linear with one feature on a bent market — not enough shape.
sqft/1000) is not a separate “advanced” topic — it is how you give every feature a fair vote so the model can generalize.Solution in TypeScript
The code change is tiny. The habit is the lesson: never judge a model only on the rows that updated its weights.
type Vector = number[];
const HOUSES: [Vector, number][] = [
[[1200, 2, 15], 245_000], // H1 train
[[1800, 3, 8], 310_000], // H2 train
[[2200, 4, 3], 420_000], // H3 train
[[ 900, 2, 40], 180_000], // H4 train
[[1500, 3, 10], 295_000], // H5 held out
];
const train = HOUSES.slice(0, 4);
const heldOut = HOUSES[4];
// Train your model ONLY on `train`.
// Then compare:
// trainLoss = average error on H1–H4
// heldOutErr = error on H5 alone
//
// If trainLoss → 0 but heldOutErr is large, you overfit the four houses.
console.log("train size", train.length);
console.log("held out", heldOut);Next
Supervised spine is complete
You can predict a price, name the optimizer, bend with neurons, classify a tier, and refuse to trust train loss alone. Optional paths from here: online / RL loops, or deeper architectures — not required to understand the engine.
Trail: First Learning Model → Vectors → Linear Regression → Gradient Descent → Neuron → Network → Classifier → Generalization.