Scientific concept · BEGINNER LESSON
Generalizability
Generalizability is a model’s ability to keep satisfactory performance when it is used on data or people different from those it was developed on. In these sources, it is discussed mainly for machine learning (ML) models in new hospitals, new patient groups, or new seasons and genotypes.
What it is
Generalizability is a measure of whether a result or model still works outside the original setting where it was developed. In machine learning, ML means computer methods that learn patterns from data. In healthcare, this can mean a model trained in one hospital still performs well in another hospital with different patients, practices, and data collection. The sources also distinguish generalizability from transferability: one source uses generalizability for similar data drawn from the same distribution, while transferability means handling new distributions such as unseen seasons, species, or genotypes.
[1] Toward generalizable and interpretable machine learning ... [3] Generalizability and transferability of machine learning models ...
How it works
The sources suggest that generalizability depends on what differences exist between the training setting and the new setting. In hospitals, these differences can include patient demographics, clinical practices, electronic health record systems, and how data are recorded. In crop studies, differences in seasons and genotype-by-environment interaction can change the data structure and reduce how well a model transfers. Model complexity also matters: low- and medium-complexity models, such as generalized additive models (GAMs), logistic regression (LR), and simpler approaches, often generalize better than high-complexity models like extreme gradient boosting (XGB) or multilayer perceptrons (MLPs) when data are limited.
[1] Toward generalizable and interpretable machine learning ... [3] Generalizability and transferability of machine learning models ...
Why it matters
Generalizability matters because a model that looks strong in its original dataset may fail in a new place. The healthcare source reports that locally trained models can outperform externally developed ones when enough local data are available, and that simpler models may be the better choice in low-data settings. The maize study similarly shows that no single machine learning method works best for every trait, so performance must be tested for the specific data and task being used.
[1] Toward generalizable and interpretable machine learning ... [3] Generalizability and transferability of machine learning models ...
Limitations and cautions
Generalizability is not guaranteed by using more data from somewhere else. The sources note that external models may not improve performance when local data are scarce, and that complex models can overfit, meaning they learn patterns too specific to the training data. The human-behavior source adds a broader warning: results may fail to apply when the studied population, the people who choose to participate, or the study situation differs from real life. Natural field experiments can improve realism, but they still do not solve every population difference and are not possible for every question.
[1] Toward generalizable and interpretable machine learning ... [2] Make science more reliable: study people as they go about their lives
SEE THE IDEA
Generalizability across settings
The model learns patterns from one hospital, one population, or one season.
New data may differ in patients, practices, seasons, genotypes, or recording methods.
The model is applied where it was not developed.
Researchers see whether useful performance is maintained.
A model is trained in one setting, exposed to a different setting, and then evaluated for whether performance holds up.
Generalizability is judged by how well performance survives a move from the original setting to a different one.
TRY IT YOURSELF
Test your understanding
Choose an answer, then check the explanation.
PUT IT TOGETHER
Match each concept to its meaning
Drag a label onto its description, or select a label and then select a description. This also works with a keyboard.
4 pairs to match
Sources and further reading
This general lesson was generated from public source material. It explains a concept; it does not interpret a private paper or provide medical advice.