A growing body of evidence shows job interviews are unreliable in predicting a future employee’s work performance and job success. Unfortunately, dialogues remain at the core of pre-employment screening activities of almost all businesses, big and small. Although we cannot see the end of interviews in the recruitment and hiring process any time soon, there is a glimmer of hope for employers to improve the credibility, validity, and reliability of job interviews.
In the ongoing structured vs unstructured interview debate, experts weighing in on the matter say structured interviews are more reliable with more significant predictive power than unstructured interviews. However, most companies prefer unstructured versions because it is more convenient to execute. We are throwing our expert analysis into the mix to deliver a verdict on the structured vs unstructured interview dispute.
The Unreliability of Interviews
A New York Times opinion piece points to one inescapable fact: job interviews are impractical or useless. The outlook is that job interviews cannot reliably and accurately predict work performance because of several reasons. One expert underscored the common observation among job seekers who review answers to common interview questions days before the actual employer-job candidate dialogue. Even if the interviewer does not field questions the job seeker studied, the preparation for the activity gives the candidate an advantage.
A well-prepared job applicant has a higher chance of getting the job than someone who goes to the interview room unprepared. Morgan McKinley says a job interview is an excellent opportunity for job seekers to present themselves to employers in the best possible way. Unfortunately, the ‘best way’ might not be the job candidate’s authentic self. In short, interviewees will only showcase those attributes that give them the highest probability of getting the job.
Sadly, the job candidate is not always the same person in everyday life. The lack of clarity about the applicant’s real personality and attitude spells doom for the business, leading to poor work performance, disengagement, and mediocre quality customer service. Sadder still, it would be too late for the company to realize they made a seriously bad hire.
According to the US Department of Labor, bad hires cost businesses up to 30% of the worker’s annual compensation. For example, an employee receiving $80,000 can cost the organization $24,000 in lost productivity yearly. Unfortunately, bad hires not only impact productivity. Managers also spend 17% more time supervising underperforming employees.
A research paper presented to the University of San Francisco underscored four crucial points that make interviews highly unreliable in predicting job performance and work success.
There is an inconsistent and sometimes inaccurate interview instrument, consisting of vague and work-unrelated questions. Interviewers are also prone to making cognitive biases without realizing their impact on the interview process. Some interview questions also do not measure attributes the company considers crucial for the position. Lastly, job candidates always come prepared, presenting only the best things about them to prospective employers.
There is no questioning a job interview’s unreliability. Unfortunately, many companies still conduct interviews to complement other pre-employment activities.
Why Companies Still Conduct Job Interviews
Companies with a clear understanding of the structured vs unstructured interview differentiation are more successful in ensuring this pre-employment activity’s predictive ability, validity, and reliability. It is crucial to understand that reliability and validity issues are at the core of the structured vs unstructured interview debate. Employers, HR professionals, and recruiters must recognize and appreciate the crucial differences between the two interview types.
A job interview can be an invaluable tool for employers to get to know potential employees better if done systematically and in an organized manner. The key concepts here are ‘systematic’ and ‘organized.’ Unfortunately, the majority of the job interviews do not have organization and systematicity, increasing the risk of unreliable and invalid job candidate measures.
Job interviews allow employers to add a different dimension to a candidate’s overall job suitability. These assessment tools can also validate the results of objective pre-employment tests, such as general aptitude measures and competency skills testing. Unfortunately, the interview’s success hinges on the interviewer’s professional competence and control of unconscious biases.
This requirement emphasizes understanding and observation of structured vs unstructured interview differentiation. Interviewers can ensure more reliable and valid interviews if they can discern a structured interview from an unstructured employer-applicant dialogue.
Unstructured Interviews: Letting Unconscious Bias Rule
Unstructured interviews are a free-flowing, non-directive style of dialogue between an interviewer and an interviewee. The interviewer does not follow a set pattern or field predetermined questions, often relying on the job candidate’s responses to frame succeeding questions.
Interviewers rely on interpersonal skills and an ability to ‘connect’ with job candidates to decide whether to ask work-related questions or something out of line. Unfortunately, an interviewer might cut short the session by asking fewer questions than usual simply because she does not ‘feel’ or ‘think’ the interviewing is worth the time and effort.
As an information-gathering procedure, unstructured interviews are subjective, hence, qualitative. Unfortunately, this is where the problem lies and why this interview type fails in the structured vs unstructured interview debate.
Prone to Cognitive Biases
Everyone makes an unconscious bias now and then, where people make erroneous conclusions about something because of faulty perception. For example, most people believe that men are more successful the women or that attractive people have a greater chance of finding success than average-looking folks. There is also the idea that fair-skinned people are best for office and executive jobs, while dark-skinned and other people of color are more suitable for menial, labor-intensive work.
A Bentley University research revealed that seven out of ten people consider men are better at running businesses than women, with 63% of women also having the same beliefs. Surprisingly, less than one in four business leaders attribute male business skills and abilities to their success.
Harvard University also underscored the growing tendency of colored job applicants to ‘whiten’ their resumes. Non-white job seekers remove any information referencing their ethnicity or racial background. For example, only one in ten non-white job applicants will receive a callback for prospective employers if they do not whiten their resumes. On the other hand, resume whitening increases the chance of an interview slot by 25%.
Some hiring managers and employers still have these unconscious biases. The job candidate does not have a good chance of getting a fair interview if skin color, gender, and ethnicity become unconscious factors in the selection process.
There is also the issue of first impression bias, or the tendency to form an opinion about someone based on the first thing the person sees. Studies reveal that people only need a fraction of a second – about a tenth of a second – to form a lasting impression about someone else.
Hence, someone who adheres to an unstructured interview unconsciously injects subjectivity into the selection process. If the interviewer does not like what he sees, there is no chance the job candidate can expect the session to proceed fairly. The interviewer might show disinterest and dismiss the job applicant within the first few minutes of the dialogue.
Things are not looking good for unstructured interviews in the ongoing structured vs unstructured interview debate. Confirmation bias is a prevalent cognitive bias in unstructured interviews. Job candidates who win the interviewer’s ‘first impression test’ must also prove the interviewer’s initial perceptions. Interviewers often review application documents before asking questions. If interviewers see an interesting piece of information that reflects their beliefs, they tend to validate the information.
Confirmation bias is almost similar to anchoring bias. There is no validation of a piece of job candidate information. Instead, the interviewer focuses only on pre-existing knowledge about the candidate and disregards everything else. The interview focuses only on the ‘anchor,’ preventing the interviewer from asking questions that might uncover other attributes crucial to the job position.
Another cognitive bias prevalent in unstructured interviews is the halo effect, where interviewers focus only on the interviewer’s positive attributes, more often their attractiveness. Studies show that attractive people receive 10.5% higher compensation than average-looking job candidates. It underscores the belief that attractive people are more sociable, have impeccable social skills, intelligent, dominant, and have excellent mental health.
Focusing on a job candidate’s attractiveness (or other positive attributes) prevents the interviewer from making a more accurate assessment of the job applicant’s personality and attitude. Unfortunately, these measures are subjective – the product of the interviewer’s mental processing of information.
Unreliable Interview Questions
As mentioned, unstructured interviews do not have pre-determined questions for interviewers to ask. They might have a general guideline on the things to query, but interviewers have the discretion to phrase the questions. Hence, a question for one job candidate might have a different framing for another. Unfortunately, inconsistent interview questions lead to reliability issues.
Inconsistent interview questions are not the only things that make unstructured interviews unreliable. There is also the time element. Interviewers adhering to an unstructured process do not observe time limits, allowing the process to progress spontaneously. The danger is that some job candidates might only have a brief interview session because the interviewer formed a negative first impression. On the other hand, a job applicant who wins the interviewer’s attention can expect the dialogue to extend unnecessarily.
Studies show that unstructured interviews do not measure a job candidate’s competencies and abilities, only validate the interviewer’s beliefs and perceptions about the job requirements and whether the candidate is fit for the position or not. This is subjective data, leading to inconsistency and unreliability and increasing the company’s risk of a bad hire.
Poor Predictor of Job Performance
Predictive capability is one of the weakest points of unstructured interviews, tipping the balance in the structured vs unstructured interview argument in favor of structured interviews.
In 1998, Frank Schmidt and John Hunter conducted a meta-analysis of 85 years’ worth of research on the predictive power of various pre-employment selection methods on on-the-job performance. Of the 19 candidate selection techniques analyzed, unstructured interviews were number nine, with an r-squared value of 0.14. The results suggest that unstructured interviews can only account for 14% of employees’ actual work performance.
The figure might seem significant. However, unstructured interviews cannot compare to structured interviews (r2 = 0.26), which draws a tie for second place with general mental aptitude tests. The most powerful predictor of job performance is ‘work sample tasks,’ with an r2 value of 0.29.
In 2009, Macan supported Schmidt and Hunter’s study, strengthening the notion that unstructured interviews are poor predictors of job fit and performance.
This observation is not surprising because unstructured interviews do not adhere to standardized questions. Everything depends on how the interviewer perceives the interview should proceed. There is also the risk of straying off relevant issues if the interviewer finds the job candidate likable enough. Unfortunately, doing so also undermines the interview’s validity or credibility.
It would not be surprising to see this job candidate perform poorly at work because of the subjective basis of the hiring decision. The company would have lost thousands of dollars in productivity losses when it realized the bad hire.
Structured Interviews: Far from Perfect, But Will Do
Validity and reliability issues impair unstructured interviews in making a more accurate and objective assessment of a job candidate’s personality, attitude, and other innate qualities. Structured interviews address these issues by providing interviewers with a less-than-subjective guide to fielding questions. The HR department brainstorms with managers to devise a questionnaire the company will use to screen job applicants. All job candidates receive similar questions, and interviewers rate the responses according to a matrix or rubric.
The company tests the interview questions for internal reliability and validity before use. Most employers consider it a resource-draining process but worth it. The final interview questionnaire only contains items with high reliability and validity values.
Careful planning is necessary to ensure structured interview questions are valid and reliable. Otherwise, there is no point in using the questionnaire in screening potential employees. A well-crafted and duly-validated structured interview can guarantee the following advantages in the structured vs unstructured interview argument.
Improves Interview Reliability and Validity
Research shows that structured interviews are more reliable and valid than unstructured ones. Companies can improve the interview guide’s reliability and validity by asking situational and personality-based questions.
Situational interview questions focus on job-specific answers and goal-oriented responses, giving interviewers a more objective understanding of the candidate’s job-related cognitive abilities. Meanwhile, personality-based questions uncover a job applicant’s unique qualities, motivations, and other personality attributes. These queries are perfect for helping employers determine culture and job fit.
Studies show that structured interviews have a clearly-defined interview model, structure, and measurement. The assessment tool also focuses on measurable constructs and not on vague concepts open to misinterpretation or subjective analysis. The ideal structured interview questionnaire also has consistent definitions, measurements, and labeling of job candidate factors.
Minimizes, if not Eliminates, Unconscious Bias
Interviewers using a structured interview questionnaire cannot deviate from the tool. They must field the questions to all job candidates, regardless of preconceived notions or unconscious biases. Research indicates that structured interviews add objectivity to a subjective process. Although the company formulated the questions, giving the questionnaire a subjective element, the interview tool’s reliability and validity guarantee a more objective measurement.
Structured interviews eliminate different cognitive biases, such as first impression bias, halo effect bias, confirmation bias, stereotyping bias, anchoring bias, and other prejudicial inclinations.
Interviewers will also not have issues rating the candidate’s responses because they have a rubric to follow. Proponents of structured interviews recommend using multiple approaches to evaluating structured interviews, eliminating unconscious bias.
Companies can use a combination of horizontal and vertical evaluation to rate a job candidate’s responses immediately after the interview and compare it to other candidates after the process. Interviewers can also use Behaviorally Anchored Rating Scales to improve evaluation objectivity.
Better Predictor of Job Performance
As mentioned, structured interviews tie for second place with general mental abilities testing (r2 = 0.26) in predicting job performance, not far behind ‘work sample tasks’ with an r2 value of 0.29. It is also 12 percentage points ahead of unstructured interviews in predictive power.
It is welcome news for companies that want to avoid costly bad hires. They can save thousands of dollars annually from lost productivity and poor performance-related concerns. Businesses also guarantee better customer service, allowing them to build and strengthen their customer base. A perfect-fitting talent also contributes to organizational growth, injecting fresh insights into processes, and increasing overall productivity.
Structured interviews provide employers with a clearer picture of the candidate’s future and that of the business when used with cognitive aptitude tests, skills tests, and other objective competency assessment tools.
You might also like: The Different Types of Pre-Employment Tests You Should Know
The Verdict: Structured vs Unstructured Interview
Structured vs unstructured interview? There is no question that structured interviews have the upper hand in the structured vs unstructured interview argument. It is more reliable and lends credibility to the pre-employment screening process. Its validity also gives businesses a better picture of a job candidate’s future performance at work and organizational fit. More importantly, employers will know if the applicant is an excellent culture ad to the business.
The only downside is the preparation, time, and effort required to create, test, and validate the structured interview questionnaire. It is a time-consuming and cognitively challenging endeavor, prompting many companies to adhere to an unstructured interview process instead. However, a well-defined structured interview questionnaire gives organizations the best chances of landing the right talent, avoiding costly bad hires.
Photo credit: Pexels
