An attrition-risk model reports 95% accuracy, a genuinely impressive-sounding number, and is deployed to flag employees at elevated risk of leaving so managers can intervene. If actual voluntary attrition in the relevant population is, say, 5% in a given year, that same 95%-accurate model can still produce a pool of flagged employees where the majority were never actually going to leave — not because the accuracy claim is false, but because of a specific, well-documented statistical effect called base rate neglect, and the underlying math of predicting a genuinely rare event.
Why a rare base rate changes what accuracy actually implies
Consider a workforce of 1,000 employees where the true annual attrition rate is 5%, meaning 50 employees will actually leave. A model with 95% accuracy, applied evenly across both leavers and stayers, will correctly identify roughly 47 or 48 of the 50 true leavers as high-risk — a strong result on its face. That same 95% accuracy rate also means the model incorrectly flags roughly 5% of the 950 employees who were never going to leave, which is around 47 or 48 people. The model's flagged pool ends up containing nearly as many false positives as true positives, despite the headline accuracy figure being genuinely, correctly reported as 95%.
This is a mathematical consequence of the base rate, not a flaw in the model
The accuracy figure and the practical usefulness of the resulting flagged list are two different things, and both the 95% accuracy claim and the roughly 50/50 split between true and false positives in the flagged pool are simultaneously, correctly true — there's no contradiction and no error in either number. The gap between an intuitively reassuring accuracy figure and a much less reassuring practical hit rate among flagged individuals is exactly what base rate neglect describes: the tendency to interpret an accuracy or reliability figure without properly accounting for how rare the underlying event actually is.
Why this matters specifically for attrition and other rare-event HR predictions
Voluntary attrition, workplace incidents, and many other outcomes HR analytics teams try to predict are, in most organizations, genuinely rare events on an annual basis — often well under 10%, sometimes under 5%. This makes HR risk-prediction models specifically vulnerable to the base rate effect, since the mathematics work out the same way regardless of the specific domain: the rarer the true underlying event, the larger the share of any reasonably accurate model's positive flags will tend to be false positives, purely as a consequence of that low base rate, not as a sign of a poorly built model.
What actually needs to be reported alongside accuracy
Precision — the share of positive flags that are actually correct, given the real base rate in the population — is the metric that directly answers the practical question managers actually care about: if this model flags someone, how likely is that flag to be right. Reporting overall accuracy alone, without precision calculated against the true base rate, allows an intuitively strong-sounding number to obscure a much less flattering practical reality about how the flagged list will actually perform in use.
What this means for building or interpreting a risk-flagging model
- Always calculate and report precision against the actual base rate of the predicted event, not just overall model accuracy
- Communicate to anyone acting on flagged results what share of flags are likely to be false positives, given the real base rate, so intervention effort is calibrated appropriately
- Be specifically cautious interpreting any accuracy figure for a model predicting a genuinely rare event without checking the base rate math first
- Consider whether the practical cost of false positives (wasted manager attention, unnecessary intervention) is acceptable given the realistic precision, not just the reported accuracy
A model can be exactly as accurate as advertised and still produce a flagged list that's mostly wrong — that isn't a contradiction, it's the ordinary, predictable mathematics of trying to catch a rare event, and it's only surprising if the base rate gets left out of the interpretation.