The technical interview is measuring the wrong thing, again

The technical interview has been broken in a rotating set of ways for twenty years. Each fix creates the next problem.

• Brainteasers measured whether you had heard the brainteaser. Replaced by algorithms.
• Algorithm puzzles measured whether you had ground a puzzle site. Replaced, partially, by take-homes.
• Take-homes measured how much unpaid time you had, which selects against people with children and second jobs. Replaced, partially, by pairing.
• Pairing is the best of them and it measures your comfort being watched, which is correlated with experience and confidence, which is correlated with demographics.

Now there is a new one, and it is genuinely novel.

the current problem#

A candidate with a model available can pass most remote technical screens. Not "can cheat" — can pass, because the tasks we set are exactly the tasks these tools are good at.

The industry response has been mostly bad. Return-to-office for interviews. Proctoring software. Increasingly hostile monitoring. All of it makes the experience worse for honest candidates and is defeated by anyone determined.

The response has been bad because the question was framed wrong. The question is not "how do we stop candidates using AI." It is "what are we actually trying to measure, given that the job now includes using these tools."

what the job actually is#

If you hired someone today, their work would involve:

• Understanding an existing system well enough to change it safely.
• Deciding what to build, which is mostly deciding what not to build.
• Using AI tools effectively, including knowing when the output is wrong.
• Communicating a technical decision to people who will be affected by it.
• Debugging something under time pressure with incomplete information.
• Reviewing someone else's code — increasingly, a machine's — and catching the problem.

Not one of those is measured by "implement an LRU cache in forty-five minutes."

interviews that measure the real thing#

Code review. Give them a 300-line pull request with three deliberate problems: one obvious bug, one subtle design issue, one thing that is fine but looks wrong. Ask them to review it.

This is excellent. It is exactly the job, AI does not obviously help, and the conversation about the third item — where they explain why the suspicious thing is actually correct — tells you more about their judgment than any implementation task.

Debugging a real system. Give them a repository, a failing test, and thirty minutes. Let them use whatever tools they want, including models. Watch how they narrow it down. Do they read the error? Do they form a hypothesis? Do they check it? Do they notice when the model's suggestion is wrong?

Watching someone debug with AI assistance is a much better signal than watching them code without it, because it is the actual work.

Design discussion on their own past work. "Tell me about a system you built. What would you change?" The follow-up questions are where the signal is. People who genuinely understood their system can answer six levels deep. People who did not, cannot, and it becomes clear quickly.

A short, paid, scoped project. Four hours, paid at a real rate, on something close to the actual work. This is the highest-signal option and the least scalable, and it is worth it for senior roles.

what to stop doing#

Stop asking people to implement data structures from memory. They will not do this in the job, and if they need one they will look it up, correctly.

Stop the six-round loop. Every round is a coin flip with a false-negative rate. Six rounds does not make the signal six times better; it makes the process long enough that good candidates take another offer.

Stop pretending the whiteboard measures anything but whiteboard performance.

the thing nobody wants to hear#

Interviews have a low ceiling on signal. The correlation between interview performance and job performance is weak in every study anyone has run.

The highest-signal thing is working with someone. Everything else is a proxy. Which argues for: shorter processes, more willingness to take a chance, and robust ways to correct the mistake — a real probation practice, honest early feedback, and the organizational nerve to act on it.

That is a harder cultural change than redesigning the interview loop, which is why everyone redesigns the interview loop instead.