Machine learning · BEGINNER LESSON
Overfitting
Overfitting is when a machine learning model becomes too tailored to the training data and starts fitting noise, outliers, or dataset-specific patterns instead of general signals. A model like this can look accurate on training data but work worse on new data.
What it is and what is measured
Overfitting is a machine learning problem, meaning it concerns how well a model learns from examples and then works on new examples. The main thing being measured is performance: how accurately or reliably the model predicts or classifies data it has not already seen. A model that seems strong on its training data may still be overfitted if that performance drops on new, independent data. [0][1][2]
[1] Reference Guide on Artificial Intelligence - NCBI - NIH [2] Beyond AUC: a clinician's guide to building and trusting prediction ... [3] Machine learning for evolutionary genetics and molecular evolution
How it happens
A model can overfit when it learns noise, accidental details, or spurious correlations. Noise means random variation that is not part of the real pattern. Spurious correlations are relationships that appear in one dataset but do not hold more broadly. This risk is especially noted when the data are limited, high-dimensional, or unrepresentative, because the model may latch onto details that do not generalize. [1][2]
[2] Beyond AUC: a clinician's guide to building and trusting prediction ... [3] Machine learning for evolutionary genetics and molecular evolution
Why it matters
An overfitted model can give misleadingly good results during development and then perform poorly in real use. The sources describe this as a sharp decline when the model is applied to new patients, new populations, or data that differ from the training set. In practice, that means a model may look trustworthy inside the dataset where it was built but be less useful outside it. [0][1][2]
[1] Reference Guide on Artificial Intelligence - NCBI - NIH [2] Beyond AUC: a clinician's guide to building and trusting prediction ... [3] Machine learning for evolutionary genetics and molecular evolution
How people try to reduce it
One basic safeguard is to keep training data separate from data used for testing or later use, while making sure the training data still represent the situations the model will face. Other strategies named in the sources include regularization, feature reduction, cross-validation, bootstrap-based optimism correction, and ensemble learning. External validation, which means testing on independent and different data, is described as a strong check on generalizability. [0][1]
[1] Reference Guide on Artificial Intelligence - NCBI - NIH [2] Beyond AUC: a clinician's guide to building and trusting prediction ...
Limitations and related concerns
Overfitting is not the only reason a model can fail. The sources also mention outliers, measurement bias, selection bias, and domain shift, which is when the data in use differ from the training context. A model can also appear better than it really is if evaluation relies too much on one metric, such as discrimination, without checking calibration and clinical utility. [0][1][2]
[1] Reference Guide on Artificial Intelligence - NCBI - NIH [2] Beyond AUC: a clinician's guide to building and trusting prediction ... [3] Machine learning for evolutionary genetics and molecular evolution
SEE THE IDEA
How overfitting works
Examples the model learns from.
It may also learn noise or accidental details.
Fresh examples from use or testing.
Accuracy drops if the model relied on dataset-specific patterns.
The model is trained on examples, but overfitting appears when its learned patterns do not hold up on new data.
Overfitting links strong training-set performance to weaker performance on unseen data.
TRY IT YOURSELF
Test your understanding
Choose an answer, then check the explanation.
PUT IT TOGETHER
Match each concept to its meaning
Drag a label onto its description, or select a label and then select a description. This also works with a keyboard.
4 pairs to match
Sources and further reading
This general lesson was generated from public source material. It explains a concept; it does not interpret a private paper or provide medical advice.