1. Everything Seemed to Fit
We were filling a management position for the US market. The candidate’s application was impressive, the interviews went well, and her case study met our expectations. Several people involved in the process, myself included, had independently come away with a positive impression. Everything suggested that we had found the right person.
What happened after she joined was therefore all the more disappointing. Within a short time, it became clear that the working relationship was not going to succeed. The gap between our expectations and what our new colleague actually demonstrated in her day-to-day work was so substantial that the company soon decided to part ways with her.
Bad hires are nothing new. Yet this particular case stayed with us. How could the impressions we had formed during the selection process differ so dramatically from the reality that followed?
2. What Had We Actually Seen?
As we reviewed the process together, one subject increasingly came to the fore: artificial intelligence. It was something we had already encountered during the search. In several video interviews, we strongly suspected that candidates were using AI to help them answer questions. In one case, a candidate even discreetly used a laptop running an AI tool in real time during an in-person interview. I have written about that experience here before.
Of course, there is nothing inherently wrong with using AI to prepare for a job application. The difficulty arises when answers given during an interview appear to reflect someone's independent thinking but may, in fact, have been generated largely by AI. Similarly, when candidates complete a case study in advance, it has become increasingly difficult to determine how much of the analysis is genuinely their own.
We do not know whether AI played a role in this particular unsuccessful hire. But our experiences throughout the selection process had raised serious questions about how much we could still rely on our established assessment methods.
3. Back to the Eighties
My contact here in Germany described the company's response with a deliberately exaggerated phrase: "We're going back to recruiting in the eighties."
The first change effectively narrows the geographical search area. Previously, candidates from across North America were considered, with only occasional visits to the office required. Going forward, employees will be expected to work on-site three days a week. The company wants more opportunities to experience how people actually work, rather than knowing them primarily through video calls.
The second change concerns the selection process itself. In-person interviews will take place earlier, with laptops and smartphones kept out of the room. Instead of receiving a case study to complete in advance, candidates will be asked to work through an unfamiliar problem on the spot, using nothing more than pencil and paper.
There is a certain irony in this. Advances in technology are prompting a company to return to hiring methods that seem almost out of another era.
4. What These Measures Might Achieve
I can understand the company's response, although I cannot say whether these changes will actually lead to better hiring decisions. Requiring three days in the office reduces the pool of potential candidates and guarantees neither freedom from AI assistance nor protection against a bad hire. What it does offer is more frequent, direct experience of how someone works with others.
The spontaneous exercise without digital tools serves a different purpose. It is intended to reveal how a person approaches a problem, asks questions and reaches a conclusion when AI cannot provide answers in the background. Whether this produces more reliable predictions of future performance remains to be seen.
Neither decision, then, is necessarily an attempt to return to supposedly better times. Both are efforts to respond to a problem that has taken on a new dimension.
5. The Question Behind the Process
In the past, a video interview and a well-executed case study could often tell us quite a lot about how someone thinks, reasons and solves problems. Those impressions were never infallible, of course. But we could at least be more confident that the answers and approaches we were evaluating largely came from the individual.
Today, we can no longer take that for granted. AI can formulate answers, structure arguments and analyse problems in real time. As a result, an impressive performance does not necessarily tell us the same things about a person's abilities that it might have told us just a few years ago.
That does not mean we should exclude AI altogether. In many roles, knowing how to use it effectively is an important skill in its own right. But if we want to understand what someone genuinely knows and can do independently, we need to take AI out of the equation, at least for part of the assessment. Only then can we meaningfully evaluate how well that person uses these tools to extend their capabilities.
6. My Impuls
What are we actually trying to discover during a hiring process? Which of our established methods still give us reliable insight into a person's abilities, and where might we primarily be assessing the quality of their AI assistance? How can we create situations in which independent thinking becomes visible without overlooking the value of using modern tools effectively?
I do not know whether pencil and paper, combined with more face-to-face interaction, are the right answers. What I do know is that we need to reconsider both the questions we ask and the methods we use. Because when the way a result is produced changes, so does what that result can tell us about the person behind it.