Coding Interview Platform Questions AI Tools Cannot Answer

On a coding interview platform, the questions AI tools can't answer for a candidate are the ones anchored in that candidate's own work: follow-ups on code they just submitted, trade-offs under their constraints, live debugging, and specifics about past decisions. Trivia and textbook puzzles no longer tell you much.

That shift matters because the old playbook has quietly stopped working. If your interview loop still leans on "explain the difference between a process and a thread" or a well-known array puzzle, you're mostly measuring who has a second screen. This guide walks through why that happened, which question types still separate strong engineers from fluent copy-pasters, dozens of example questions you can use tomorrow, a rubric to score them, and what your tooling has to support for any of it to work fairly.

Why do classic interview questions no longer tell you much?

Classic technical questions share one property: they have a single, well-documented correct answer. Definitions, complexity of standard algorithms, the output of a tricky snippet, the classic "reverse a linked list" or "find two numbers that sum to a target." These answers have been written up thousands of times in public, which is exactly the material AI coding assistants learned from.

So a candidate with an assistant running on a phone or a second monitor can produce a clean, correct, well-commented answer to almost any of these in seconds. Worse, the answer often looks better than what a genuinely strong engineer would write under pressure, because it's polished and complete. The question rewards retrieval, and retrieval is now free.

There's a second problem. Even before AI assistants, puzzle-style questions were a weak proxy for day-to-day work. Most engineering time goes into reading existing code, choosing between imperfect options, tracking down bugs, and explaining decisions to colleagues. A question that tests none of that was always a gamble. Now it's a gamble that's easy to game.

None of this means you should ban tools or pretend they don't exist. Your engineers will use AI assistants on the job. The goal is to ask questions where the assistant can't do the thinking for the candidate, so what you observe is the candidate's own judgment.

Scorecard · Candidate 7847 · Senior Go developer, platform team

NB

Blind review on

Name and photo hidden until shortlist

93

overall

Go language85

Concurrency82

APIs and gRPC92

Testing76

System design85

Code

94

Interview

90

Integrity

Clean

What makes a question hard for an AI assistant to answer on someone's behalf?

A question resists outsourcing when the answer depends on context only the candidate has, or when it has to be produced and revised in real time in front of you. In practice that comes down to a handful of properties.

• It's about their artifact. The question refers to code the candidate wrote minutes ago, with its specific variable names, shortcuts and bugs. Nobody else has that context.
• It has no single right answer. Trade-off questions are judged on reasoning, not on reaching a known endpoint.
• It moves. Requirements change mid-answer, so a pre-generated response goes stale immediately.
• It needs lived specifics. "Tell me about a time" questions with aggressive follow-ups reward people who were actually there.
• It's conversational and fast. Short back-and-forth turns leave no room to paste a prompt and read back an answer without it being obvious.
• It asks for estimates with stated assumptions. The number matters less than how the candidate builds it.

The six categories below each lean on one or more of these properties.

Which question types actually resist AI assistance, and what do they sound like?

Here are the question types we recommend, with concrete examples you can adapt. Mix at least three types in any interview longer than 30 minutes.

Follow-ups on the candidate's own submitted code

This is the single strongest format. Give a short take-home or timed task first, then spend the interview on what they actually wrote. If they understand their code, the conversation flows. If they don't, it falls apart within two or three questions.

• Walk me through this function line by line. Why did you pick this data structure over the obvious alternative?
• Your solution passes the tests. Which input would you expect to break it, and why didn't the tests catch that?
• You named this variable tmp. What does it actually hold at line 40?
• What's the time complexity of your solution as written, not the ideal version?
• If this function were called 10,000 times per second, what would you change first?
• Delete this helper. What happens, and how would you restructure the code without it?
• You handled the empty-list case but not a list with one element. Was that deliberate?
• Which part of this did you find hardest, and what did you try before landing here?

Trade-off reasoning

Trade-off questions don't have a correct answer, so a generic response sounds generic. Push for the candidate's choice and the conditions under which they'd change their mind.

• We need to store user sessions. Would you use a relational database, an in-memory cache or signed cookies? What would make you switch?
• A service is slow. Do you add caching or fix the query first? What would you need to know to decide?
• Monolith or separate service for a new billing feature, given a team of four? Defend your choice in two minutes.
• When would you accept duplicated code rather than introduce a shared abstraction?
• You can ship a feature in two days with a known limitation, or in two weeks without it. How do you decide, and who do you talk to?
• Strict typing or faster iteration for an internal tool used by three people?

Debugging a realistic broken snippet live

Give the candidate 30 to 80 lines of plausible production code with one or two real bugs, not syntax errors. Watch how they read, form hypotheses and test them. Ask them to think out loud.

Here's a Python function that paginates results from an API and merges them into one list. Users report that some records appear twice and the last page is sometimes missing. You have the code and a failing test. Find out why.

• Before you touch anything, what are your top two hypotheses?
• What would you log or print to confirm one of them?
• You found an off-by-one. Is that enough to explain both symptoms?
• How would you write a test that would have caught this originally?
• This passes locally but fails in CI. What's different about CI that could matter here?

Good snippets for this include race conditions in async code, timezone handling, mutable default arguments, integer overflow in aggregation, N+1 queries hidden behind an ORM call, and cache keys that ignore a parameter.

Explaining past decisions with specifics

Behavioral questions get a bad reputation because they're often asked lazily. The fix is follow-ups that demand detail only someone who was there would know.

• Tell me about a production incident you were involved in. What was the first alert, and what did you check first?
• What was the slowest query you ever fixed? What did the plan look like before and after?
• Describe a technical decision you made that turned out wrong. When did you realize it, and what did it cost?
• What's a library or tool you stopped using, and what specifically pushed you away?
• Which part of your last codebase would you rewrite first, and why hasn't anyone done it?

Keep asking "and then what?" and "how did you know?" Vague answers that stay at the level of "we improved performance" are a signal on their own.

Changing requirements mid-answer

Start a design or coding problem, let the candidate commit to an approach, then change a constraint. This tests adaptability and whether their first design was understood or borrowed.

• Design a URL shortener. (Five minutes later) Now links must expire after 24 hours. (Later) Now we need per-link click analytics by country.
• Write a function that deduplicates a list. Now the list doesn't fit in memory. Now order must be preserved.
• Build a rate limiter for one server. Now there are twenty servers behind a load balancer.
• Your API returns all orders for a user. Now some users have 2 million orders.

Estimation with stated assumptions

Estimation questions reward structured thinking. You care about the assumptions and the arithmetic, not the final figure.

• Roughly how much storage would a year of application logs take for a service handling 500 requests per second?
• How many servers would you need to send one push notification to every user of a 10-million-user app within five minutes?
• How long would it take to reindex a 200 GB table, and what would you check before starting?
• If a test suite takes 40 minutes, what's your estimate of the cost to the team per week, and where would you start cutting?

Integrity report · Submission 8964

Paste events

1 small paste, from the task text

Normal

Tab switches

5 switches, longest 63 seconds

Normal

Keystroke playback

Steady typing, edits and reruns

Normal

Code similarity

42 percent with Candidate 5826 on helper code

Check

AI-generated code likelihood

Low, style matches interview answers

Low

Signals inform your team. They never reject a candidate on their own.

How should you score these answers consistently?

Open-ended questions only work if every interviewer scores them the same way. Without a rubric, you're back to gut feel, which is where bias creeps in. Use a four-level scale per dimension and write a one-line reason for every score.

A few rules make the rubric hold up. Score each dimension right after the interview, before discussing with other interviewers. Weight dimensions by role: a senior backend hire should lean on trade-offs and debugging, while a junior hire can lean more on code ownership and communication. And never let a single brilliant or terrible moment override the whole scorecard.

What should a coding interview platform support for this style of interview?

You can run every question above with a shared doc and a video call, but it gets painful fast. A coding interview platform worth paying for should make the AI-resistant formats easy rather than heroic. Here's the checklist we'd use.

1. A real execution environment. Candidates should run code against actual test cases in a sandbox, in the language they'd use on the job. Pseudocode on a whiteboard hides too much.
2. Continuity from test to interview. The interview should open with the candidate's submitted solution already loaded, so follow-ups start immediately.
3. A shared live editor. Both sides see and edit the same code, with the ability to drop in a broken snippet for debugging rounds.
4. Question banks you control. Store your own debugging snippets, design prompts and requirement-change sequences, and rotate them so they don't leak.
5. Structured rubrics and scorecards. Each interviewer scores independently against the same dimensions, with a written reason per score.
6. Transcripts and replay. Reviewing a recording or keystroke playback beats arguing from memory in the debrief.
7. Integrity signals, not accusations. Paste events, tab switches and similarity across candidates are context for a human to review, never an automatic rejection. Our guide to detecting cheating in remote coding tests covers which signals hold up.
8. Clear candidate communication. Candidates should know what's being scored, how, and how to ask for a human review.

This is how we built DeveloperHiring. A candidate takes a browser-based coding assessment with real test cases in a sandbox across 30+ languages, either as a take-home or a timed test. Then an AI technical interview asks follow-ups on that candidate's own solution and produces a transcript scored against a rubric. Team and Scale plans add a live coding room for the human rounds.

Integrity signals (paste and tab-switch events, keystroke playback, code similarity across candidates, the likelihood that code was AI-generated, and optional webcam snapshots with consent) sit next to the scores for a person to weigh. Everything feeds a ranked top-five shortlist with scorecards, and the company always makes the final decision.

How do you keep AI-resistant interviews fair to candidates?

Making questions harder to game shouldn't make them harder to pass for honest people. A few principles keep the bar fair.

• Tell candidates the format in advance. Say that the interview will discuss their submitted code, include a debugging exercise and involve changing requirements. Surprise isn't a skill you're hiring for.
• Be explicit about tool rules. If AI assistants are allowed on the take-home, say so and adjust the follow-up questions to probe understanding. If they aren't, say that too. Ambiguity punishes the honest candidates.
• Use the same questions and rubric for everyone at a level. Improvised questions make comparisons meaningless.
• Score blind where you can. Names, photos, age and gender add nothing to an evaluation of code. DeveloperHiring scoring is blind to all four.
• Allow thinking time. Silence isn't failure. Many strong engineers go quiet for 30 seconds before a good answer.
• Accommodate when asked. Extra time, a different format or a written option for live segments should be easy to grant.
• Treat integrity signals as questions, not verdicts. A paste event might be a candidate moving their own code between files. Ask before you judge.
• Offer a human review path. Any stage scored by software should come with a clear way to request review by a person.

How do you put this into a full interview loop?

Here's a structure that fits most mid-level and senior roles and keeps total candidate time reasonable.

1. Short assessment (45 to 90 minutes). A realistic task with real test cases, take-home or timed. Keep it scoped so candidates with jobs and families can finish it.
2. Follow-up interview on their code (30 to 45 minutes). Six to eight questions from the "own code" list, plus one requirement change applied to their solution.
3. Debugging and trade-offs (45 minutes). One broken snippet, two trade-off questions tied to your stack, one estimation question.
4. Experience deep dive (30 minutes). Two past-decision questions with persistent follow-ups.
5. Debrief. Every interviewer submits scores and reasons before the meeting. Discuss disagreements, not averages.

If you're hiring in volume or across time zones, run the first two steps asynchronously so candidates choose their own time, and keep the live rounds for the shortlist. That's the structure we use to help teams hire developers without asking every applicant for a full day of interviews.

What mistakes make these questions less effective?

Even good question types fail when they're run badly. Watch for these.

• Reusing the same debugging snippet for months. Snippets leak. Keep a pool and rotate.
• Asking follow-ups that could apply to any solution. "What would you improve?" is weak. "Why is this loop nested?" is strong.
• Turning trade-off questions into quizzes. If you have a favorite answer and mark everything else down, it's trivia again.
• Letting one interviewer dominate the debrief. Independent written scores first, discussion second.
• Making the take-home too long. Long tasks select for free time, not skill, and they push candidates toward shortcuts.
• Ignoring the role. A frontend hire should debug rendering and state issues, not distributed locks.

Frequently asked questions

Should candidates be allowed to use AI assistants during a coding interview?

It depends on the stage, and you should say which rule applies before the candidate starts. Many teams allow assistants on take-home tasks because engineers use them on the job, then use follow-up questions on the submitted code to check real understanding. For live debugging and trade-off rounds, most teams ask candidates to work without assistants so they can observe reasoning directly.

Are LeetCode-style questions useless now?

Not entirely, but they've lost most of their signal when used alone. A short algorithmic task can still be a useful starting point, especially for junior roles. The value comes from the follow-up conversation about the candidate's own solution: why they chose an approach, where it breaks and how they'd change it under new constraints. Without that conversation, the answer mostly tests access to tools.

How many follow-up questions should I ask about a candidate's code?

Six to eight is usually enough in a 30 to 45 minute session. Start broad with a walkthrough, then narrow to specific lines, edge cases and complexity. Finish with one requirement change applied to their solution. If a candidate clearly understands their code after four questions, move on to a harder change rather than asking more of the same kind.

How do I write a good debugging snippet?

Start from a real bug your team has fixed, strip out anything proprietary, and keep the code between 30 and 80 lines. Include one main bug that produces a visible symptom and, for senior roles, a second subtle one. Provide a failing test or a clear bug report. Avoid syntax errors and trick questions, because they test patience rather than debugging skill.

Do AI-resistant questions disadvantage nervous candidates?

They can if run carelessly, which is why preparation matters. Share the format in advance, allow thinking time, and keep the tone collaborative rather than adversarial. Questions about a candidate's own code often feel easier than abstract puzzles, because the candidate knows the material. Offering accommodations and a written option for live segments helps too.