Tuesday, January 19, 2010

Coding Exercises for Software Engineering Interviews


I've read about it. Talked about it. All with a positive "it should be done" spin to it. But despite that, I've never actually done it. It being to actually have interview candidates for software engineering jobs write code. Now I'm going to do it.

What really pushed me to action was being involved in a recent round of interviews for a Scrum Master. For this, we provided the candidates with a number of scenarios prior to the interview. These might be something like a product backlog of stories, information about some teams, velocity etc. and they then had to come in and present to the interview panel what they made of it. Certain challenges were inherent in the scenarios such as story sizes larger than velocity, stories that didn't fit neatly etc.

I was part of the panel interviewing some half dozen candidates, and I found this approach very insightful. Besides the interest of just seeing who could put together a small slide stack and talk coherently about it, it gave a much firmer foundation for asking questions about. Out were the theoretical "in such and such situation how would you..?" -- in was "I noticed that in sprint 1 you didn't get in all the top priority stories, can tell us about that decision?"

So, fired up from the Scrum Master interviews, I decided that I would come up with something to use for subsequent software engineering recruitment we would do.

I started off looking at an example from a colleague who had interviewed with Thoughtworks in the past. There were a choice of three problems a candidate could choose. They were very much your run-of-the-mill "coding challenge" though. I decided to think a bit deeper about what I was trying to accomplish.

My initial thoughts were focused on setting an exercise that would hopefully steer people to:

  • Demonstrate competency with key Java APIs and features such as:
    • Collections
    • Generics
    • IO
    • XML
      Parsing
    • Annotations?
  • Show us that they knew a design pattern or two
  • Wrote code that had suitable comments, Javadocs
  • Used a consistent code style
  • Utilized unit tests
  • Employed a build tool beyond their IDE

I circulated these ideas amongst my esteemed colleagues, including Jim and Rajiv to get some feedback. Rajiv offered an interesting idea which I think we might use in conjunction with a coding exercise: how about presenting people with some fragments of code and asking them to spot errors and code smells in them (e.g. not closing connections in a finally block etc.) I like this idea but was still really keen to actually see somebody's work, some actual code they'd written. As I'm sure someone else has said somewhere else, "you wouldn't hire a chef without having him cook you a meal." He might be able to talk about the culinary arts in the most poetic and comprehensive way. But if he can't cook, he ain't worth squat.

In addition to Rajiv's idea, he and Jim both helped shape my thoughts on what kind of things we wanted the exercise to focus on. Key to this was something that would have the candidate demonstrate problem solving. It had to be non-trivial, non-textbook with a variety of approaches possible. Though also we couldn't ask they work on something too demanding and time consuming...

Ultimately, here is the exercise I came up with. I'll be using it soon because I have just opened up a position for a Principal Engineer in Billerica, MA. I'm very excited to see how this works out. If you have any feedback or constructive criticism please leave comments below. Thanks.

Code Exercise: A-Maze-ingly Retro Route Puzzle

This code exercise is (very loosely) based on some of the ideas in old school text adventure games. I enjoyed these so much as a kid I even learned how to write them on my old Commodore Vic 20. Thus began my descent into programming geekery. Ah, happy days.

Moving on... in this exercise you are supplied with two files. The first is an XML document (with inline DTD) that describes an adventure game map. It will look something like this:

<map>
<room id="1" name="Hallway" north="2" />
<room id="2" name="Dining Room" south="1" west="3" east="4"/>
<room id="3" name="Kitchen" east="2">
<object name="Knife"/>
</room>
<room id="4" name="Sun Room" west="2">
<object name="Potted Plant"/>
</room>
</map>

As you can see, a room may or may not permit travel in one of the four cardinal directions and may or may not contain "objects". The second file is a plain text file where the first line indicates the ID of the room the player starts in, and each subsequent line lists the name of an object they must collect. This file will look something like this:

2
Potted Plant
Knife

The objective is to write a program that will:
  • Parse the XML and create a model of the map
  • Read the plain text file, noting which room to start in and which items must be collected
  • Output a valid route one could take to collect all the items specified in the text file


Given the above example the following is (one of the potentially) correct outputs:

ID Room Object collected
----------------------------------
1 Hallway None
2 Dining Room None
3 Kitchen Knife
2 Dining Room None
4 Sun Room Potted Plant

There are going to be questions people have as they seek to clarify the above requirements, e.g. "Does my output have to be exactly like the example? Do I have to find the optimal route or simple a valid route?" I'm unlikely to provide any further clarification, but rather ask people to make and state their assumptions as part of their work.

12 comments:

  1. I've got one:

    Given a class that you need is marked final and you just need to add stuff which choice would you take?

    1. Ask around if you could add to the class.
    2. Avoid # 1 and copy and paste the class.

    ReplyDelete
  2. Hi Jon,

    I too am an advocate for having candidates do some work to demonstrate skills by doing exercises. Here's some advice from someone who has traveled this path already...

    1.) Decide how long *you* would expect to work on a 'test' for a potential employer, decide the level of candidate you are interviewing for (5=junior, 4=dev, 3=senior, 2=lead, 1=principal), and divide your test length by the level number. The longer the test, the more generic the problem description should be.

    2.) Ask the candidate to take you through the work, similar to a design review. Ask the candidate to critique the work themselves. Generally the candidate knows where the weaknesses are (especially more skilled, experienced ones).

    3.) Don't expect finished work - and state that in your instructions to the candidates. As developers/programmers, we are a problem solving bunch and we generally get frustrated if we are asked to work out problem but are not given enough resources to complete the task (time, tools, etc...). Don't forget the candidate is interviewing you too!

    4.) Be clear about what you want demonstrated - not just to yourself (as above), but explicitly to the candidates. That way, solutions the candidate may come up with will have some overriding priorities for choosing one design over another (all else being equal).

    5.) Optionally, you can provide an 'example' solution for the candidates to review and discuss. This step holds two advantages; a. Opportunity to mentor someone; b. Opportunity to accept feedback about your own design & implementation.

    Good luck on this. It does take some time to prepare and administer. Sometimes it takes a while to get this right - I know it did for me and there are still some kinks to work out in mine (in 2nd year of use).

    If you want to compare notes further, drop me a line.

    Regards,
    Bill Berger

    ReplyDelete
  3. excellent post. definitely such interaction is going to help an organization to select a right candidate if not the perfect one.

    ReplyDelete
  4. Cool.
    Glad to see this work out finally.
    Now we are one step closer to Joel's test :)

    Another suggestion. During the interview- you can also quiz - e.g if the XML is 10G in size - what will you change in your design ?

    But mostly I agree with the excellent feedback from @Bill Berger

    ReplyDelete
  5. I for the most part agree with this approach, though it isn't clear from the post what the timeframe of the test is.

    ReplyDelete
  6. I personally like to have the Coding Exercises for the Software Engineering Interviews in a more formal way by setting up a real development desktop with those needed tools (of course the white board and the internet). I had done couple of interviews in this scenario in my previous work.

    I see your view on analyzing their code for the java comments etc., from those written by the candidates. Given the simple scenario to solve the xml, the code that will be written by the candidate would have self explanatory methods or variables unless he is running out of time or he would really focus on the logic to arrive the result in time. So, in that stand point, once the candidate is 100% done with his coding then it would be better to ask if he is given another 10 min or so what he would like to do with his completed code. This will naturally provoke or bring out the inner thinking of that candidate like whether to make java comments or to write junit etc., but essentially not TDD I guess, as he will not be given any upfront time.

    Another thought would be to solve a epic/story to arrive at a normalized tables, class diagram, sequence diagram and those technologies, that he would employ to address them like logging, audit etc.,.

    Yet another thought would be to solve couple of mathematical/logical puzzles to see and understand his logical as well as the calculative thinking.

    ReplyDelete
  7. Thanks for the feedback everyone, much appreciated. Bill, thanks in particular for those suggestions from someone who's actually done this. In particular the idea of indicating to people that I don't necessarily expect a full solution was very good.

    What may not have been crystal clear to everyone was the fact that this was a "homework" assignment following a phone screen. As for how long they have to complete the homework, I didn't specifically give a timeframe, that's almost part of the exercise to see how quick the respond to it. So far two people have completed it and they basically tool the last weekend and did it then.

    ReplyDelete
  8. Hi Jon. Wondering if you or anyone else had collected or posted some additional sample map/level files that could be exchanged by potential "solvers" of this little game?

    Cheers,

    GW

    ReplyDelete
  9. Thinking hats on! Decide upon a user interface and graphics best suited at the user end. Interface developers and Programmers work alongside hi-tech graphic designers. The participation of producers is also very important in this phase! visit this site

    ReplyDelete
  10. Through these programs, the students get to know about a lot of engineering practices of renewable energy and conventional sources of energy as well & their uses in modern society.
    Floods Pro Service Companies

    ReplyDelete
  11. Their custom software solutions are also highly compatible with the latest scientific instruments, wireless devices, medical tools and other consumer products.cheap Revit 2017

    ReplyDelete
  12. Product design and testing is estimated to continue its high growth rate and grow upon its 25% market share due to enablement of remote connectivity, which allows analysis and investigation of products in the field without being physically present on the site.buy SolidWorks 2010

    ReplyDelete