According to a report from the Society for Human Resource Management, 82% of job applicants are frustrated with the modern recruitment process. Our current recruitment system is deeply flawed, and the concerns that applicants are expressing indicate that human resource departments are dehumanizing job applicants.
In addition, employers could be setting themselves up for recruitment failure and legal risks if their recruitment process includes testing instruments that aren’t valid and reliable. The legal risk associated with test validity and reliability is not limited to pre-employment screening, but for this month’s blog, I will focus solely on psychoanalytic assessments in pre-employment testing.
Many employers have embraced assessments that ask the applicant a series of questions designed to identify the best applicant for the job. These questions might be behavior-based, or they might focus on communication styles or other personal traits or preferences. The theory is that science can select better candidates than humans can, and by using this science, recruiters are spared a significant number of hours weeding through applications and interviewing candidates. An additional benefit in using an online test could be the elimination of rater biases or examiner/test-taker relationships that could skew the outcomes. For these reasons, science and technology have become attractive alternatives to the costly interview-and-elimination process of yesteryear.
However, applicant assessments are helpful only if they are backed by solid research. They can be detrimental if they rely on weak links and assumptions. Psychoanalytic assessments should not be used for hiring decisions unless the tool has been thoroughly tested for reliability and validity and shows a high correlation of job match to job performance outcomes. Every testing tool provider should be able to present a technical manual, detailing the types and results of validity and reliability studies performed on that test. If a technical manual hasn’t been provided, ask for it.
Employers need to understand how to distinguish between testing instruments that are valid, reliable and non-discriminatory and testing instruments that could produce flawed results and even present legal liability. It is essential to review how the testing tool was scrutinized to ensure that no racial, gender, age, religious, national origin or disability biases exist.
This is a complex process. Following are just a few things to consider:
How reliable is your testing tool?
You need to know how dependable the test is. Could the test produce different results based on varied testing times and testing environments? This is a crucial consideration since online applicants will be taking the test in a wide variety of environments from quiet homes to noisy public places and at all times of the day and night.
In an effort to increase reliability, some test designers increase the number of test items. While this can be an effective way of improving reliability, it can also result in test-taker fatigue, which can create measurement errors of its own. Additionally, applicants become annoyed when asked the same questions ad nauseum. This type of experience does not provide the applicant with a good impression of your company.
Another potential reliability issue could be faulty technology that drops a test-taker’s responses part-way through the testing process. Could that experience occur, and if so, how could that impact your applicants?
How valid is your testing tool?
Technical manuals should specify the outcomes of validity studies. You need to consider the strength of each validity rating.
Predictive validity is especially important. It is evidenced by the specific outcomes hoped to be achieved. In essence, it is the test’s track record of success in the workplace. Even if a test rates high in predictive validity, it still must be validated in your workplace prior to using it in the hiring process. Does a strong correlation exist between successful incumbents and high scores on certain traits? Likewise, does an inverse correlation exist between struggling incumbents and low scores on those particular traits? If you cannot prove a strong correlation with current incumbents, how can you claim the tool can identify strong or weak candidates?
What is the applicant’s experience?
If you are concerned about your applicants’ experience – and you should be – another important step is to screen and scrutinize every question included in the testing instrument. Take the test as an applicant might. Keep in mind that your recruitment process reflects on the inner workings of your organization, and poorly worded questions can send bad messages to applicants about your organization.
Examine the justification and the specific validity and reliability of vaguely-worded, context-dependent questions since responses to such questions depend heavily on the situational severity involved. For example, a test might ask applicants how they would react if their coworker broke a rule. While this might seem like a legitimate question to ask, it could be too vague and situational to be a valid predictor of future success since it doesn’t note what rule was broken or its severity. Naturally, if a coworker ignored safety rules and jeopardized anyone’s safety, that should be approached differently than if an otherwise excellent coworker ran ten minutes late when returning for lunch, yet in both cases a rule was broken. Vague questions frustrate conscientious applicants – people you probably would most like to hire – and these questions can result in erroneous answers.
Examine the scientific soundness of multiple choice and yes/no answers that might not provide an applicant with a response that matches their genuine traits. Consider the example we just covered. What if the applicant had a good record of success at gently confronting coworkers about infractions in an effort to assess the situation and to determine the necessary steps to take next? Wouldn’t you want a calm, critical thinker like that on your team? Yet, if no choice like that is provided, you will miss out on that applicant. Poorly constructed choice options like these can fail to identify a highly adaptable person. Do you need adaptable employees, and if so, how does your testing instrument identify that trait?
Raise doubts about questions that can legitimately produce varied answers from job to job. For example, a question that asks whether applicants feel like they are heard at work can legitimately be job-dependent and not reflect on the applicant’s locus of control or communication abilities. An applicant could have a very strong internal locus of control and possess the patience, the tenacity and the relationship management skills necessary to be heard. However, that same applicant could hold a job where management makes it clear that staff input is not welcome. Let’s accept the fact that some managers have flaws and can influence an applicant’s response on a poorly worded test. This question could reflect more on the work environment than on the applicant’s traits.
Questions like these could be useful to recruiters as topics to be explored in a live interview, but in and of themselves, they cannot provide solid insights into an applicant’s personal traits. Inquire as to the scientific justification for their inclusion in the test.
Ethical considerations
Using psychoanalytic assessments in the recruitment process can be beneficial, but before you do, ensure that you understand how the science behind those tools works. Remember, you are recruiting complex human beings who deserve to be treated as individuals worthy of your time. Your applicants are important people; they are not a mass of resources to be harvested from an unwieldy electronic system.
If you use a psychoanalytic assessment to screen applicants, speak with your legal counsel about sharing the results with your applicants. We need to acknowledge that no test is 100% perfect at all times - especially since the humans taking the assessment can make mistakes of their own - so it is possible that the assessment could incorrectly evaluate the applicant. For that reason, the ethical approach is to allow applicants to challenge their test results, but before you make that decision, formulate a policy that fits your company culture in concert with your legal counsel’s advice.
Psychological tests measure human components, and humans are immensely complex, so test instruments for humans are inherently flawed. Keeping that cautionary fact in mind will help you select and administer testing devices in a responsible and ethical manner.