16
Each Interviewer Uses the Same Set of Questions

At this point, we’ve suggested several steps in hiring that either aren’t trained or are very different from what you’ve learned or know. (This is particularly true of this chapter.) We encourage you, once you’ve laid out your preparation for your next hire, to bring your direct reports together and walk them through what you’ve done, how this process is different, and how their roles will be, in some, cases quite different than they have perhaps grown accustomed to.

We discussed earlier that before you start this process, you’re going to develop a list of behavioral interview questions. These questions are based on the job you envision your new direct doing.

These questions form the core of each and every interviewer’s evaluation of the candidate.

This is at the heart of effective interviewing: Each candidate is being compared to one job. The job is the same, so each candidate is compared to the same job against the same basic criteria. This not only produces more effective results (more true positives and true negatives, and fewer false positives and false negatives), but it is also seen as more fair by candidates.

What is typical in interviewing today looks to those of us who have studied and measured interviews and outcomes as utter chaos. The idea that one interviewer (let alone all of them) could give three fundamentally different interviews to three different candidates is ludicrous. Easier, yes. Requires less preparation, yes. But measurable? No. Fair? Absolutely not. Repeatable? No way.

The ancient Buddhist parable of the three blind men touching an elephant comes to mind. Three blind men come upon an elephant and endeavor to determine what they have found. One touches the trunk and thinks he’s found a fat snake. One touches a leg and describes a tree. And one touches the tail and thinks he’s found a vine. And they’re all wrong. The Rigveda says, Reality is one, but wise men speak of it variously.

Do not allow your team members to come up with their own idiosyncratic set of questions. Do not allow some “experienced” direct to “go with his gut.” Don’t allow interviewers to not prepare. [Have you ever had an interviewer ask you for your résumé at the start of an interview?] If one of your team members wants to proceed this way, don’t allow that person to interview.

Very few professionals receive any interviewing training to speak of. Those who do describe a largely HR-centric briefing about how to avoid lawsuits due to discrimination. This is necessary, but hardly sufficient.

Here’s an example of most professionals’ profound lack of knowledge of effective and legal interviewing. You probably yourself believe (and have been taught) that there are “illegal” questions one must avoid when interviewing. In fact, this is not true in most locations. In the United States right now, there are only a few municipalities where asking a particular question is in itself unlawful.

The questions themselves are not illegal: Using the answers to those questions to discriminate in the hiring process is what is illegal.

We’re certainly not recommending you ask a female candidate whether she is planning on having children in the next five years. It’s a stupid and selfish question. It’s indicative of someone who would discriminate on the basis of the answer. We want you to avoid any sort of discrimination other than the natural process of selecting the best candidate, irrespective of gender or race or ethnicity or any other orientation other than ability to do the work exceptionally well. In fact, there are enough places where this kind of scurrilous discrimination is practiced that an ethical, prepared, and unbiased interviewer increases the chances of hiring those who are often discriminated against.

But don’t confuse discrimination in hiring with illegal questions. If your firm does do more in-depth, substantive interview training like we do at our Effective Hiring Manager Conferences, count yourself lucky. It’s rare.

The Importance of Asking the Same Questions

What this all means is simple: Every interviewer must ask roughly the same basic set of questions of each candidate. Again, each interviewer is comparing each candidate against the criteria for one job. We’re not comparing candidates yet—that only happens when more than one candidate gets over the high bar you’ve set for the single job you’re trying to fill. Effective interviewing is not about comparing candidates to one another—it’s about comparing each candidate to the role.

If you’ve done your preparatory work, you already have those questions. An example of an administrative assistant interview that we have used at Manager Tools is included for you in the Appendix, created using our proprietary Interview Creation Tool available to licensees of our work.

Every interviewer receives the same predetermined list of questions. The questions are one per page, in order of importance. You’ll notice that every page has the question at the top, and the vast majority of the page is blank for note taking (more on that later).

You may be thinking, Wait! Every interviewer is going to ask each candidate the same questions? Won’t the candidate get better throughout the day? Won’t every candidate sound great by the last interview? Won’t they all start to get frustrated with the same questions over and over? Won’t they say, “Well, I already answered that question?”

The answers to those questions are no, no, no, and only the weak candidates. Keep in mind that you’ve chosen the questions that get at the most important behaviors of the job. Why wouldn’t you want multiple interviewers to hear each candidate’s answers to them?

Candidates do not get better throughout their day, unless they are very well prepared and can hone an already very good answer through repetition. We have tested this repeatedly.

Candidates describe themselves as getting better throughout the day, but interviewers do not describe seeing that in candidates. What’s more, in many cases, later in the day interviewers describe candidates as becoming overconfident and not working as hard in their interviews.

The fact is, targeted, well-prepared questions make interviews much harder for less prepared candidates. This is what we want—to raise the bar. [While we are sad if we miss a good candidate every once in a while, we will tolerate that less than ideal outcome to avoid at all costs someone who isn’t right for us getting through some easier, less targeted questions.]

In general, candidates are not well prepared for interviews. You’ve probably experienced this if you’ve interviewed a few times. Candidates are poorly served by the vast majority of Internet guidance.

Some examples of current guidance that are nonsense:

  • “Preparation should be about the company.” This is ludicrous. The interview will be almost exclusively about the candidate’s background and skills. Most candidates do not prepare enough by knowing what they’ve done, how well they’ve done it, and the behaviors they can bring to the job.
  • “Interview the interviewer.” Even more ludicrous. If you’re someone who is impressed by a candidate asking you questions, you’re making decisions based on your gut. You’re vastly increasing the chances of a Hell on Earth outcome. How can we know whether a candidate has the right skills for our job without probing past performance for behaviors?
  • “Learn to be a storyteller in interviews.” Slightly less ludicrous, but still dumb. Most business conversations are not about storytelling, but about accuracy, and structure, and knowledge of facts. Too many candidates think that they can be truthfully persuasive, meaning they start by trying to persuade, and do their best to be truthful. When you think about it, what you want is for candidates to be persuasively truthful. That is, start with the truth about themselves, and then do their best to show those truths in the best possible light.
  • “Answer a question with a question.” Really? How well does that work in relationships? How well does that work with your boss where you work now? How would you feel if your colleagues always did that with you? Idiotic.

By the way, all of these nostrums are peddled by folks who either (a) don’t know any better or (b) know what it takes to succeed in interviewing (hard preparatory work and practice), but know most candidates want simple answers because they’re unwilling to do the work.

Preparation Makes for a Productive Interview

Your best candidates—those with both the right background and the preparation to demonstrate it verbally in an interview—shine in prepared behavioral interviews. They know that less prepared candidates don’t have detailed answers about their past.

Further, the best candidates don’t like unstructured interviews. They see interviewers who don’t have prepared and targeted questions as being unprepared. They report feeling like the hiring process boils down to “whomever they ‘liked’ best,” rather than who had the best background and fit.

This is not to say that cultural and personality fit are not crucially important in your hiring process. They are. It takes both cultural fit as well as behavioral job fit for your best hires. The problem with unstructured, “gut” or idiosyncratic interviews is that while they may give you a “feeling” for the candidate, you won’t learn enough about his skills and abilities to go with your estimation of his personality.

On the other hand, you will be able to gain plenty of cultural/personality fit information in a structured behavioral interview. The prepared nature of the questions means you won’t be committing the most frequently committed sin of novice interviewers: anxiously thinking about what your next question will be when you should be listening to the answer to the question you already asked. Candidates notice when you’re not totally focused on their answers.

This kind of interview also causes managers to ask, “But how will we get different perspectives on each candidate if everyone asks the same questions?” There are two easy answers to this concern.

First, all of your interviewers bring their own biases to each interview and each candidate. Interviewer perspectives on each candidate will be different because even identical answers are heard differently by different interviewers. When we have tested highly qualified candidates going through multiple similar interviews, interviewers describe different outcomes and conclusions from their interviews. Much of this is due to different communication styles, in our experience. However, those differences are much more about cultural fit rather than significant divergence on evaluating core competencies for the role.

Second, as we’ll learn in how to probe interview answers, each interviewer will ask different probing questions and learn different qualities of each candidate. This is again due to the interviewers’ own experiences, personalities, and communication habits.

You don’t have to allow your team members to conduct unprepared interviews in order to gain different perspectives on candidates. More prepared interviewers are better interviewers. The best preparation is predetermined questions designed to learn about each candidate’s previous behaviors. Previous behavioral patterns are the most accurate predictor of future behaviors, which is what you’ll be paying for when you hire someone.