I've read about it. Talked about it. All with a positive "it should be done" spin to it. But despite that, I've never actually done it. It being to actually have interview candidates for software engineering jobs write code. Now I'm going to do it.
What really pushed me to action was being involved in a recent round of interviews for a Scrum Master. For this, we provided the candidates with a number of scenarios prior to the interview. These might be something like a product backlog of stories, information about some teams, velocity etc. and they then had to come in and present to the interview panel what they made of it. Certain challenges were inherent in the scenarios such as story sizes larger than velocity, stories that didn't fit neatly etc.
I was part of the panel interviewing some half dozen candidates, and I found this approach very insightful. Besides the interest of just seeing who could put together a small slide stack and talk coherently about it, it gave a much firmer foundation for asking questions about. Out were the theoretical "in such and such situation how would you..?" -- in was "I noticed that in sprint 1 you didn't get in all the top priority stories, can tell us about that decision?"
So, fired up from the Scrum Master interviews, I decided that I would come up with something to use for subsequent software engineering recruitment we would do.
I started off looking at an example from a colleague who had interviewed with Thoughtworks in the past. There were a choice of three problems a candidate could choose. They were very much your run-of-the-mill "coding challenge" though. I decided to think a bit deeper about what I was trying to accomplish.
My initial thoughts were focused on setting an exercise that would hopefully steer people to:
- Demonstrate competency with key Java APIs and features such as:
- Collections
- Generics
- IO
- XML
Parsing - Annotations?
- Show us that they knew a design pattern or two
- Wrote code that had suitable comments, Javadocs
- Used a consistent code style
- Utilized unit tests
- Employed a build tool beyond their IDE
I circulated these ideas amongst my esteemed colleagues, including Jim and Rajiv to get some feedback. Rajiv offered an interesting idea which I think we might use in conjunction with a coding exercise: how about presenting people with some fragments of code and asking them to spot errors and code smells in them (e.g. not closing connections in a finally block etc.) I like this idea but was still really keen to actually see somebody's work, some actual code they'd written. As I'm sure someone else has said somewhere else, "you wouldn't hire a chef without having him cook you a meal." He might be able to talk about the culinary arts in the most poetic and comprehensive way. But if he can't cook, he ain't worth squat.
In addition to Rajiv's idea, he and Jim both helped shape my thoughts on what kind of things we wanted the exercise to focus on. Key to this was something that would have the candidate demonstrate problem solving. It had to be non-trivial, non-textbook with a variety of approaches possible. Though also we couldn't ask they work on something too demanding and time consuming...
Ultimately, here is the exercise I came up with. I'll be using it soon because I have just opened up a position for a Principal Engineer in Billerica, MA. I'm very excited to see how this works out. If you have any feedback or constructive criticism please leave comments below. Thanks.
Code Exercise: A-Maze-ingly Retro Route Puzzle
This code exercise is (very loosely) based on some of the ideas in old school text adventure games. I enjoyed these so much as a kid I even learned how to write them on my old Commodore Vic 20. Thus began my descent into programming geekery. Ah, happy days.
Moving on... in this exercise you are supplied with two files. The first is an XML document (with inline DTD) that describes an adventure game map. It will look something like this:
<map>
<room id="1" name="Hallway" north="2" />
<room id="2" name="Dining Room" south="1" west="3" east="4"/>
<room id="3" name="Kitchen" east="2">
<object name="Knife"/>
</room>
<room id="4" name="Sun Room" west="2">
<object name="Potted Plant"/>
</room>
</map>
As you can see, a room may or may not permit travel in one of the four cardinal directions and may or may not contain "objects". The second file is a plain text file where the first line indicates the ID of the room the player starts in, and each subsequent line lists the name of an object they must collect. This file will look something like this:
2
Potted Plant
Knife
The objective is to write a program that will:
- Parse the XML and create a model of the map
- Read the plain text file, noting which room to start in and which items must be collected
- Output a valid route one could take to collect all the items specified in the text file
Given the above example the following is (one of the potentially) correct outputs:
ID Room Object collected
----------------------------------
1 Hallway None
2 Dining Room None
3 Kitchen Knife
2 Dining Room None
4 Sun Room Potted Plant
There are going to be questions people have as they seek to clarify the above requirements, e.g. "Does my output have to be exactly like the example? Do I have to find the optimal route or simple a valid route?" I'm unlikely to provide any further clarification, but rather ask people to make and state their assumptions as part of their work.