Tests used for hiring or promotions must be valid. To be valid, an assessment needs to be job-related and consistent with business necessity. It should be useful while protecting the employer in the event of any litigation.

Validity gives tests both a high utility (the degree to which they select those who are qualified) and defensibility (the degree to which the employer is justified in using the test for selection purposes). That said, validity doesn’t fit into a singular box. It is defined in a few ways, some of which are more beneficial than others.

Content vs. Criterion
Content validity shows a connection between a pre-employment assessment, an applicant’s specific knowledge, skills, or abilities, and the essential duties they’ll be required to perform on the job. When someone passes a content-valid test, an employer can be reasonably certain that the candidate has the skills necessary to perform well on the job.

Criterion validity, however, is seen as the gold standard of validation. It shows with statistical significance that the test results are predictive of one or more job performance dimensions. These dimensions take many forms, ranging from higher performance appraisal ratings to greater client satisfaction to fewer absences, among other things.

With content validity, an employer can assess whether an applicant will likely be able to perform a particular task. However, criterion validity goes beyond merely showing whether an applicant is able — it shows whether they will be successful with the task and in the role. This is the juxtaposition of “can do” versus “will do.” The benefits of that knowledge can be enormous and result in positive returns for a company.

A study conducted by the National Bureau of Economic Research shows that relying on pre-employment testing in hiring processes increased the average tenure of new hires by about 15%. Furthermore, an analysis of 17 studies on pre-employment testing showed these algorithm-based tests outperform hiring decisions made by humans at least a quarter of the time.

When you consider the importance of criterion validity in these terms, it’s easy to see why it should be a priority for all employers relying on pre-employment tests to help them make the smartest hiring decisions possible.
Establishing Criterion Validity

To establish criterion validity, you have to start by determining the performance dimensions the test is designed to measure (or possibly predict). Setting out these parameters is key before moving on to any other stage because they create the foundation for each step that follows.

Have your incumbent employees take the test as a next step. Allowing incumbent employees to see their own scores on the test will enable them to then set a defensible, job-related cutoff score for those entering into their same positions.

Once you’ve done this, supply and analyze job performance metrics for each employee. You can then correlate test scores to these job performance ratings. If you’re able to find a statistically significant relationship between the two, you’ve established criterion validity for your assessment.

As a bonus, establishing criterion validity on an assessment for one position can extend to other similar positions. Section 15B of the Uniform Guidelines on Employee Selection Procedures states that if an employer can establish that the essential job duties of the position being tested for are similar enough to a position at another organization, this criterion validity can be “transported” to the other environment — and the protection it affords can be adopted by another organization. Note that the transportability can occur either inside or outside of the primary organization in which the validity was initially established.

Establishing criterion validity — and not just content validity — whenever possible is a key element of predicting employee success and making the right hiring decisions. It’s a way for your business to go beyond just seeing whether applicants can perform job duties to let you instead see whether they can perform those duties well.

Michael Callen is the VP of products at TestGenius, an online pre-employment testing platform.