A pre-employment test's validity study reports a correlation of 0.25 between test scores and subsequent job performance among employees who were hired — a modest-looking number that could easily be read as evidence the test is only weakly useful. This figure likely understates the test's true predictive power considerably, for a specific, well-understood statistical reason called range restriction: the validation sample only includes people who were already selected, disproportionately excluding the lower-scoring applicants the test would have flagged as weaker candidates in the first place.
Why a narrower range mechanically shrinks a correlation
A correlation coefficient's magnitude depends partly on how much variation exists in both variables being correlated — restrict the range of one variable, and the correlation with anything else mechanically shrinks, even if the true underlying relationship across the full range is exactly the same. This is a mathematical property of correlation, not a substantive finding about the relationship's actual strength, and it applies directly to any validation sample where the range of test scores has been artificially narrowed relative to the full population the test is actually meant to be used across.
Why this specifically happens in employment test validation
A test used in hiring, by design, filters out lower-scoring applicants before they become employees — meaning any validation study conducted afterward, using only the employees who were actually hired, is working with a sample where the lowest scorers on the original test have been systematically removed from the data, precisely because the test (or a correlated selection process) did what it was designed to do. This produces a validation sample with an artificially narrow range of test scores compared to the full applicant pool the test was originally applied to, which mechanically deflates the observed correlation between test score and job performance, independent of the test's actual underlying predictive power.
Why this matters for interpreting a modest-looking validity figure
A validity coefficient of 0.25, calculated on a range-restricted sample of already-hired employees, could correspond to a considerably stronger true relationship across the full range of applicants who were actually tested, including those who scored lower and weren't hired — meaning the reported figure, taken at face value, systematically understates how useful the test actually is for distinguishing among the full range of applicants a hiring process would consider. A test dismissed as only weakly predictive based on a range-restricted validation study might, corrected for this effect, show meaningfully stronger validity than the uncorrected number suggests.
What a range restriction correction actually does
Standard statistical corrections estimate what the observed correlation would have been if calculated across the full, unrestricted range of scores, using information about how much the validation sample's variance has been reduced relative to the full applicant population's variance. These corrections are a standard, well-established part of rigorous employment test validation research specifically because uncorrected validity coefficients from range-restricted samples are a well-known and predictable source of understated results, not a subtle or rare technical footnote.
What this means for evaluating a test's reported validity
- Ask whether a reported validity coefficient was calculated on a range-restricted sample (already-hired employees) or the full range of original applicants
- Look specifically for whether a range restriction correction was applied, and treat an uncorrected figure from a restricted sample as a likely underestimate of true predictive validity
- Don't dismiss a modest-looking validity coefficient without first checking whether range restriction is a plausible explanation for the low observed figure
- Recognize that range restriction is a mathematical property of the sample, not evidence the test itself performs weakly
A test's reported validity coefficient is only interpretable in the context of the sample it was calculated on — and for any test validated after selection has already occurred, that context very often means the reported number is a meaningful underestimate of the test's real predictive value.