My interview coding test

Problem

"When I became a manager a few years ago, I started interviewing candidates. At first, I used to run interviews the way my team lead would run them - with a small coding test that applicants were asked to code in C/C++ with a few gotchas. After a while, I started noticing a problem. This exercise relied a lot on their ability and knowledge of C or C++, which was not primarily what we wanted to test."

Actions taken

"I decided to talk with my team members about this and I particularly remember one of them telling me about their perception of the issue - somebody's desire to do math is strongly correlated to being a good developer. It's not about being good at math but rather liking and being interested in math as a proxy for liking logic."

From there, and after multiple updates, I built up a new math-based coding test. I start by asking the question: "Do you know the quadratic equation?". Their answer to that says a lot. If people start saying "Argh, I hate math", it's a sign. If they don't know it, I tell them about it, as it's not about them knowing the equation or not. Then, I ask them to code a simple algorithm to solve the function I just gave them, preferably on a whiteboard (for freedom). The language doesn't matter, even pseudo-code is fine.

"What's particularly interesting about this equation is that it's easy enough to code a simple algorithm yet hard enough so that they will make mistakes (I can only think of 3-4 people who got it right the first time among 200+ interviews). The common mistakes are dividing by zero, not realizing the square root of a negative number is imaginary, and returning multiple numbers. I tell them how many mistakes they have made. 'I can see xx mistakes,' with no further details about what they are. I then ask 'Knowing that, what would you do?'. That's when it starts to become interesting. How do they react? Do they ask questions? Give up? Silently stare at their code? Test their own code? Try another method? Any method towards resolution is viewed positively but people giving up obviously fail in my mind."

Lessons

"When hiring, I'm looking a logical frame of mind and math provides a common background to test this out. In my experience, I've never seen anybody that dislikes math AND was a great algorithm programmer. There is no perfect way to evaluate if someone will be a good developer so you must rely on indicators. This test is quite a good one (among others). Moreover, engineers make mistakes all the time but what's critical is how they react to them. That's exactly why I ask 'You have xx mistakes, what would you do about it?' rather than 'What are those mistakes?' or 'How would you fix them?'. Here is what I am trying to detect in applicants through this test:

• That they don't give up
• Flexibility on how to solve a problem
• Appreciation for logic"

Key quotes:

1. "Somebody's desire to do math is strongly correlated to being a good developer."
2. "Moreover, engineers make mistakes all the time but what's critical is how they react to them."