Infinite Depth Questions in the Technical Interview

The main goals of a technical interview are to assess what the candidate knows, how they explain what they know, and how they reason: how they handle being pushed beyond their limit of firm knowledge and have to rationally speculate. Common failure modes are: clamming up (being unwilling to reason or share their thought process), bullshitting (confidently and erroneously prognosticating), and not sanity-checking their work (ending up with answers that are obviously off by several orders of magnitude). These are bad failure modes because they’re not just bad for an interview, they are reasonably predictive of an “allergy test” with engineers. A knowledge workplace consists of a lot of problems that go beyond the known and reasonable knowledge. A new hire who comes into the workplace and who clams up, bullshits, or can’t check their work is going to be a huge drag on the team. So we are looking explicitly to filter for candidates who can reason in a clear way, are excited about going beyond the boundaries of their knowledge, and communicate their confidence levels in the correctness of their answers.

Individual “point” questions are typically used in tech interviews (“please write pseudo code to reverse a linked list on the whiteboard”), but these have a downside of coming from a fixed (though large) corpus of questions, typically which have leaked to an extent unknown to the interviewer. This allows for effective “interview prep” where prospects who brute-force-memorize reasonable answers to the corpus will get the questions “correct” and pass the interview. This will select for dedicated employees with good memory, but may unfairly punish those with reasonable breadth and depth of knowledge but who just got “unlucky” in which questions were picked.

Any test on which an applicant gets a 100% has failed to fully assess the applicant’s capability. We need a distribution where it’s basically impossible to get a perfect score to have confidence we know how to rank an applicant! So a technical interview with three memorizable answers on which the applicant gives three correct answers is discarding a lot of signal about their capabilities on the one hand and letting luck and memorization play too large a role on the other.

So the goal is instead is to do a Monte Carlo search of someone’s large knowledge space in a time bounded way, probing beyond an applicant’s “reasonable knowledge”. For a good Infinite Depth Question, the size of the Complete Answer and the random nature of the probing make “memorizing the answer” impossible. If there are areas the applicant wants to explain in more depth, they should be allowed to do so but not to wholly derail the interview, as this can allow for memorizing an answer. The goal is not to have the applicant replay a carefully memorized script!

The interviewer should have “unfairly deep” knowledge about some parts of the question that they can probe on — not to “verify the applicant has the same depth” at all, but to observe the way the applicant handles being pushed over the threshold of what they likely know or could memorize. It’s polite at the beginning of the interview to mention that some parts of the interview are explicitly going to go well beyond what is expected to be known so the applicant doesn’t panic at being presented a question to which they don’t know the answer.

One example of such a deepening probe could be around. DNS. An applicant might know that DNS sends a request to a nameserver and gets an IP address back. If we stop there, we have not probed their limit of knowledge and can’t assess how they handle this. So we proceed. How is it decided which nameserver? What kinds of name servers are there and how might they be involved in answering the request? Is the request over UDP or TCP? (An expert applicant might be expected to push back on this as a false dichotomy since other DNS resolution techniques exist, like DoT and DoH, and explain what those are and why those are interesting/useful options.) What port do DNS queries go to? What is the structuring of a DNS request packet? Response packet? What are some common DNS record types we might expect to see and what is their function? What are problems with DNS that limit its usefulness? And for forth.

Some examples of Infinite Depth Questions (there are many!):

• What happens when you turn on a computer?
• What happens when you launch a program?
• If I type in a URL into a browser and hit enter, what happens?
• What’s the difference between a VM and a container?
• How are files stored on a computer?
• If I send a SQL query to a database, how does it go about answering?
• How does WiFi work?
• How would you build a datacenter?