A testing vendor reports that its assessment shows a strong correlation with job performance, based on a study administering the test to current employees and correlating their scores with existing performance ratings measured around the same time — a concurrent validity study, genuinely useful evidence, and a meaningfully different and narrower claim than predictive validity, which requires demonstrating the test actually predicts how new candidates, not current employees, will go on to perform once hired.
What each specific validity design actually requires
A predictive validity study administers the test to actual job applicants before any hiring decision is made, then waits for a meaningful period, hires proceed independent of the test scores where possible, and performance is measured later, correlating the original test scores against this subsequent performance data — a design that directly mirrors how the test will actually be used, testing candidates before any performance information exists. A concurrent validity study instead administers the test to people already employed and already performing in the role, correlating their test scores against performance data that already exists, essentially at the same point in time.
Why concurrent validity studies are so much more common despite this limitation
A concurrent validity study can be completed relatively quickly, using an existing population of current employees and already-available performance data, while a genuine predictive validity study requires waiting through an entire hiring and performance cycle, involves genuine applicants rather than already-selected employees, and is considerably more expensive and logistically complex to conduct — practical constraints that make concurrent validity studies far more common in practice, despite predictive validity being the more directly relevant claim for an actual hiring decision.
Why current employees differ systematically from the applicant pool a test is meant to screen
Current employees have already been selected through some prior hiring process, and have already demonstrated they can perform the job adequately enough to remain employed — meaning a concurrent validity sample necessarily excludes the range of genuinely poor performers a real applicant pool would include, a form of range restriction that can distort the observed correlation in ways that don't necessarily reflect how well the test would actually distinguish among a full, unscreened pool of new applicants.
Why this distinction matters directly for hiring decisions relying on a validated test
A test's actual practical use is screening applicants before any hiring decision is made — exactly the situation a predictive validity study directly tests and a concurrent validity study does not, meaning a test validated only concurrently carries a genuine, unresolved question about whether it will actually perform as well when used for its actual intended purpose on a genuinely unscreened applicant population.
What this means for evaluating a testing vendor's validity claims
- Ask specifically whether a reported validity study was predictive or concurrent, since the two answer meaningfully different questions
- Recognize concurrent validity as useful but narrower evidence, given the range restriction and population differences involved
- Give predictive validity evidence more weight for decisions specifically about screening actual job applicants, since this directly mirrors the test's real intended use
- Be specifically cautious of vendors presenting concurrent validity results without clearly distinguishing them from the stronger predictive validity claim
The distinction between predictive and concurrent validity isn't a minor technicality — it's the difference between a test having been shown to actually do the job it's being purchased to do, and a test having been shown to correlate with performance among a group that's already been through the exact selection process the test is meant to replace or supplement.