A skills-assessment vendor demonstrates that their test scores correlate at 0.4 with actual on-the-job performance ratings — a genuinely respectable validity coefficient by the standards of employment testing research. This is real evidence, and it is not, by itself, a complete legal defense for using the test in hiring decisions. Predictive validity is one piece of a larger standard, and treating it as the whole standard is where a lot of otherwise well-validated assessments end up exposed.
Predictive validity answers whether it works, not whether it's legally defensible
Under the framework used by U.S. employment regulators, a test that produces adverse impact against a protected group (a substantially lower selection rate, commonly assessed against the four-fifths rule) has to be justified by business necessity — the test has to measure something actually required for successful performance of that specific job, and there has to be no equally effective, less discriminatory alternative reasonably available. Predictive validity evidence supports the first part of this standard. It does nothing, on its own, to establish the second part.
The less-discriminatory-alternative requirement is where validated tests still get successfully challenged
A test can have genuine, well-documented predictive validity and still fail this standard if a plaintiff can show an alternative selection method exists that predicts performance comparably well while producing less adverse impact. This is a real and recurring pattern in employment testing challenges: the employer's defense focuses entirely on demonstrating the test works, while the successful challenge focuses on showing a less discriminatory alternative was available and wasn't considered. A vendor's validity study, however rigorous, typically doesn't address this question at all, because it's not designed to compare against alternatives — it's designed to demonstrate the test's own validity in isolation.
Job-relatedness is a narrower requirement than simply correlating with success
A test item can correlate with job performance for reasons that have nothing to do with the skill it claims to measure — general cognitive ability correlates with performance in almost every job, which is exactly why cognitive ability tests are simultaneously among the most predictively valid and among the most legally scrutinized instruments in employment testing, since a generically predictive test isn't automatically a job-related one under the standard. Job-relatedness requires the test measure a skill or ability actually required to perform that specific job's actual duties, not merely something that happens to be correlated with general workplace success.
What this means for anyone evaluating or building a skills assessment
- Ask specifically whether a less discriminatory alternative was considered and compared, not just whether the test itself is validated
- Confirm the validity study was conducted for the specific job and context the test will actually be used for, not a generic version of the role
- Distinguish between a test's overall predictive validity and its specific job-relatedness — the two are related but not the same claim
- Treat a vendor's validity coefficient as a necessary but incomplete piece of the legal picture, not a complete defense on its own
None of this means predictive validity doesn't matter — it's a real, necessary piece of the standard. The mistake is treating it as the whole standard, when the regulatory framework it actually has to satisfy asks a broader question than whether the test works.