Statistics and cognitive science · BEGINNER LESSON
Learning rate
Learning rate is a measure of how strongly new information changes a prediction or performance estimate. In the supplied sources, it appears both as an update weight in prediction models and as a measure of how quickly practice improves performance.
What it is
In these sources, learning rate is not a single universal quantity. In one framework, it is the parameter that determines how much a new prediction error changes a previous prediction. In another, it is the increase in correctness after each practice opportunity. In a motor-learning study, it is also described as the speed at which movement time drops toward a stable level. All three uses describe how quickly a system updates or improves after new experience.
[1] Understanding learning through uncertainty and bias [2] An astonishing regularity in student learning rate - PMC [3] Predicting future learning from baseline network architecture - PMC
How it works
In the prediction-error framework, a prediction error is the difference between what happened and what was expected. A larger learning rate gives more weight to the newest outcome and less weight to the older expectation; a smaller learning rate does the opposite. Some models keep the learning rate constant, while Bayesian models described in the source let it change from trial to trial according to uncertainty. Higher uncertainty about an outcome can raise the learning rate, especially when a change point, meaning the environment may have changed, is more likely.
Learning from practice data
In student practice data, learning rate was estimated as the gain in correctness per opportunity above a baseline level. The study found large differences in starting performance, but much smaller differences in learning rate across students. A typical student needed about seven practice opportunities per knowledge component to reach mastery. In the motor-learning source, learning rate was the exponential drop-off parameter for movement time; larger values meant faster convergence to steady performance.
[2] An astonishing regularity in student learning rate - PMC [3] Predicting future learning from baseline network architecture - PMC
Limitations and interpretation
These sources show that learning rate depends on how it is defined and measured. A number reported as a learning rate from one model is not directly comparable to a learning rate from another model or task. The sources also show that learning rate can be influenced by uncertainty, baseline performance, task type, and how the data are analyzed.
[1] Understanding learning through uncertainty and bias [2] An astonishing regularity in student learning rate - PMC [3] Predicting future learning from baseline network architecture - PMC
SEE THE IDEA
Two common meanings of learning rate
A reward, outcome, or practice attempt is observed.
The gap from expectation, or the improvement in correctness, is measured.
This sets how much the new information changes the estimate or performance curve.
The next prediction or movement time reflects the update.
New information is measured first; the learning rate controls how strongly it changes the next state.
The same term can describe update strength in prediction models or speed of improvement in practice data.
TRY IT YOURSELF
Test your understanding
Choose an answer, then check the explanation.
PUT IT TOGETHER
Match each concept to its meaning
Drag a label onto its description, or select a label and then select a description. This also works with a keyboard.
4 pairs to match
Sources and further reading
This general lesson was generated from public source material. It explains a concept; it does not interpret a private paper or provide medical advice.