Bias and Noise in Hiring Decisions

How can companies reduce not only bias, but also noise, in their hiring and other decisions?

The problem of bias in corporate decisions, such as hiring, promotion and salary decisions, is well-known. However, there is another type of error that has not been talked about as much – perhaps because it is harder to see, and harder to fit into a narrative: noise.

What is noise in corporate decisions, and how is it different from bias?

Noise is a random error in our decisions. Research has confirmed that in many tasks, experts’ decisions are highly variable. Professionals often make decisions that deviate significantly from those of their peers, from their own prior decisions, and from rules that they themselves claim to follow. This is the case even when the stakes of those judgments are high, such as when appraising real estate, valuing stocks, or sentencing criminals.

In hiring decisions, noise would be, for example, a variability in who gets hired, based on who is making the decision, what mood they’re in, or what time of the day it is when that decision is made. In other words, influences that shouldn’t play a part do play a part. What differentiates noise from bias is that the error does not always go in the same direction, as is the case with biases.

The target-analogy in the figure illustrates this difference. The shots on Target A are accurate. There is no bias and very little noise. The shots on Target B are biased, but not very noisy at all. They are systematically off in one direction – down to the right from the bullseye. Target C on the other hand shows noise, but no bias: the imprecisions in relation to the bullseye cancel each other out. Target D has both bias and noise.

Daniel Kahneman’s take on noise:

In his latest book “Noise: A Flaw in Human Judgment“, Daniel Kahneman makes a strong case that we should indeed care about noise, not only about bias. Noise is more difficult to appreciate than bias. However, it is no less real, no less costly, and no less unfair.

How can companies reduce not only bias, but also noise in their decision processes?

Trying to fix a known bias is a bit like curing a known disease. Knowing what the symptoms are, we try to work in the opposite direction. Fighting noise, on the other hand, must be preventative in nature, because we don’t know in what direction we are going to make mistakes. With that analogy, Kahneman recommends “Decision Hygiene”. Just as physical hygiene prevents all kinds of diseases, including ones we don’t fully understand yet, decision hygiene prevents all kinds of errors – noise as well as bias.

A few practical ways to apply decision hygiene, and thereby reduce noise in your hiring and other decisions:

• Whenever possible, get several independent judgments and calculate their average. Averaging judgments gets rid of noise. (Averaging judgments does not reduce bias. However, it may still be an important step in fighting bias, because it makes any bias more visible. You can see this effect in the target shot illustration above. The bias is much more striking in Target B than Target D. Anyone looking at Target B would advise the shooter to aim “up and left”. For Target D, this conclusion would be much less obvious, even though there is no less bias.)
• According to Kahneman, rank orders (= comparative judgments) contain less noise than ratings (= absolute judgments). Therefore, make judgments comparative instead of absolute. In other words: create a rank order of your options by comparing them, instead of rating each option separately on a scale.
• Break problems into subproblems that you evaluate independently. For example, in a hiring decision, create selection criteria that you evaluate separately. Then apply those in the same way to every candidate. (This values clarification exercise may help with creating criteria.) The next point is related to this one:
• Postpone your intuition. Structure the process to prevent “premature closure” driven by first impressions. In other words, don’t let your gut feelings call ALL the shots by coming to a conclusion too early. This does not mean that you shouldn’t listen to your (or your colleague’s) gut feelings, but give those feelings a place in the process. For example, in a hiring decision, make the “like-ability” of a candidate one of your official criteria. You’ll need to acknowledge that this criterion might be fraught with bias, because we tend to like people who are similar to us. However, when this is made explicit, you can decide consciously how much weight you want to give that criterion. Is it more important than the skills and experience? Does weigh in with it 10% of the total, 20%, or 80%? This kind of transparency and thoughtfulness will again not directly eliminate bias, but will make it more apparent.

As companies these days are more concerned about biases than they used to, they are hopefully also getting more attuned to the general importance of making good decisions, and to the risk of making bad decisions. That effort should naturally lead them to also tackle noise. Thankfully, many of the remedies that improve decisions will reduce both bias and noise.

by Ursina Teuscher (PhD), at Teuscher Decision Coaching, Portland OR

“When something happens, you immediately have a story and an explanation. You have that sense that you learned something and that you won’t make that mistake again.
These conclusions are usually wrong.
What you should learn is that you were surprised again. You should learn that the world is more uncertain than you think.”

Daniel Kahneman