Tuesday, January 19, 2010

Coding Exercises for Software Engineering Interviews


I've read about it. Talked about it. All with a positive "it should be done" spin to it. But despite that, I've never actually done it. It being to actually have interview candidates for software engineering jobs write code. Now I'm going to do it.

What really pushed me to action was being involved in a recent round of interviews for a Scrum Master. For this, we provided the candidates with a number of scenarios prior to the interview. These might be something like a product backlog of stories, information about some teams, velocity etc. and they then had to come in and present to the interview panel what they made of it. Certain challenges were inherent in the scenarios such as story sizes larger than velocity, stories that didn't fit neatly etc.

I was part of the panel interviewing some half dozen candidates, and I found this approach very insightful. Besides the interest of just seeing who could put together a small slide stack and talk coherently about it, it gave a much firmer foundation for asking questions about. Out were the theoretical "in such and such situation how would you..?" -- in was "I noticed that in sprint 1 you didn't get in all the top priority stories, can tell us about that decision?"

So, fired up from the Scrum Master interviews, I decided that I would come up with something to use for subsequent software engineering recruitment we would do.

I started off looking at an example from a colleague who had interviewed with Thoughtworks in the past. There were a choice of three problems a candidate could choose. They were very much your run-of-the-mill "coding challenge" though. I decided to think a bit deeper about what I was trying to accomplish.

My initial thoughts were focused on setting an exercise that would hopefully steer people to:

  • Demonstrate competency with key Java APIs and features such as:
    • Collections
    • Generics
    • IO
    • XML
      Parsing
    • Annotations?
  • Show us that they knew a design pattern or two
  • Wrote code that had suitable comments, Javadocs
  • Used a consistent code style
  • Utilized unit tests
  • Employed a build tool beyond their IDE

I circulated these ideas amongst my esteemed colleagues, including Jim and Rajiv to get some feedback. Rajiv offered an interesting idea which I think we might use in conjunction with a coding exercise: how about presenting people with some fragments of code and asking them to spot errors and code smells in them (e.g. not closing connections in a finally block etc.) I like this idea but was still really keen to actually see somebody's work, some actual code they'd written. As I'm sure someone else has said somewhere else, "you wouldn't hire a chef without having him cook you a meal." He might be able to talk about the culinary arts in the most poetic and comprehensive way. But if he can't cook, he ain't worth squat.

In addition to Rajiv's idea, he and Jim both helped shape my thoughts on what kind of things we wanted the exercise to focus on. Key to this was something that would have the candidate demonstrate problem solving. It had to be non-trivial, non-textbook with a variety of approaches possible. Though also we couldn't ask they work on something too demanding and time consuming...

Ultimately, here is the exercise I came up with. I'll be using it soon because I have just opened up a position for a Principal Engineer in Billerica, MA. I'm very excited to see how this works out. If you have any feedback or constructive criticism please leave comments below. Thanks.

Code Exercise: A-Maze-ingly Retro Route Puzzle

This code exercise is (very loosely) based on some of the ideas in old school text adventure games. I enjoyed these so much as a kid I even learned how to write them on my old Commodore Vic 20. Thus began my descent into programming geekery. Ah, happy days.

Moving on... in this exercise you are supplied with two files. The first is an XML document (with inline DTD) that describes an adventure game map. It will look something like this:

<map>
<room id="1" name="Hallway" north="2" />
<room id="2" name="Dining Room" south="1" west="3" east="4"/>
<room id="3" name="Kitchen" east="2">
<object name="Knife"/>
</room>
<room id="4" name="Sun Room" west="2">
<object name="Potted Plant"/>
</room>
</map>

As you can see, a room may or may not permit travel in one of the four cardinal directions and may or may not contain "objects". The second file is a plain text file where the first line indicates the ID of the room the player starts in, and each subsequent line lists the name of an object they must collect. This file will look something like this:

2
Potted Plant
Knife

The objective is to write a program that will:
  • Parse the XML and create a model of the map
  • Read the plain text file, noting which room to start in and which items must be collected
  • Output a valid route one could take to collect all the items specified in the text file


Given the above example the following is (one of the potentially) correct outputs:

ID Room Object collected
----------------------------------
1 Hallway None
2 Dining Room None
3 Kitchen Knife
2 Dining Room None
4 Sun Room Potted Plant

There are going to be questions people have as they seek to clarify the above requirements, e.g. "Does my output have to be exactly like the example? Do I have to find the optimal route or simple a valid route?" I'm unlikely to provide any further clarification, but rather ask people to make and state their assumptions as part of their work.

35 comments:

  1. I've got one:

    Given a class that you need is marked final and you just need to add stuff which choice would you take?

    1. Ask around if you could add to the class.
    2. Avoid # 1 and copy and paste the class.

    ReplyDelete
  2. Hi Jon,

    I too am an advocate for having candidates do some work to demonstrate skills by doing exercises. Here's some advice from someone who has traveled this path already...

    1.) Decide how long *you* would expect to work on a 'test' for a potential employer, decide the level of candidate you are interviewing for (5=junior, 4=dev, 3=senior, 2=lead, 1=principal), and divide your test length by the level number. The longer the test, the more generic the problem description should be.

    2.) Ask the candidate to take you through the work, similar to a design review. Ask the candidate to critique the work themselves. Generally the candidate knows where the weaknesses are (especially more skilled, experienced ones).

    3.) Don't expect finished work - and state that in your instructions to the candidates. As developers/programmers, we are a problem solving bunch and we generally get frustrated if we are asked to work out problem but are not given enough resources to complete the task (time, tools, etc...). Don't forget the candidate is interviewing you too!

    4.) Be clear about what you want demonstrated - not just to yourself (as above), but explicitly to the candidates. That way, solutions the candidate may come up with will have some overriding priorities for choosing one design over another (all else being equal).

    5.) Optionally, you can provide an 'example' solution for the candidates to review and discuss. This step holds two advantages; a. Opportunity to mentor someone; b. Opportunity to accept feedback about your own design & implementation.

    Good luck on this. It does take some time to prepare and administer. Sometimes it takes a while to get this right - I know it did for me and there are still some kinks to work out in mine (in 2nd year of use).

    If you want to compare notes further, drop me a line.

    Regards,
    Bill Berger

    ReplyDelete
  3. excellent post. definitely such interaction is going to help an organization to select a right candidate if not the perfect one.

    ReplyDelete
  4. Cool.
    Glad to see this work out finally.
    Now we are one step closer to Joel's test :)

    Another suggestion. During the interview- you can also quiz - e.g if the XML is 10G in size - what will you change in your design ?

    But mostly I agree with the excellent feedback from @Bill Berger

    ReplyDelete
  5. I for the most part agree with this approach, though it isn't clear from the post what the timeframe of the test is.

    ReplyDelete
  6. I personally like to have the Coding Exercises for the Software Engineering Interviews in a more formal way by setting up a real development desktop with those needed tools (of course the white board and the internet). I had done couple of interviews in this scenario in my previous work.

    I see your view on analyzing their code for the java comments etc., from those written by the candidates. Given the simple scenario to solve the xml, the code that will be written by the candidate would have self explanatory methods or variables unless he is running out of time or he would really focus on the logic to arrive the result in time. So, in that stand point, once the candidate is 100% done with his coding then it would be better to ask if he is given another 10 min or so what he would like to do with his completed code. This will naturally provoke or bring out the inner thinking of that candidate like whether to make java comments or to write junit etc., but essentially not TDD I guess, as he will not be given any upfront time.

    Another thought would be to solve a epic/story to arrive at a normalized tables, class diagram, sequence diagram and those technologies, that he would employ to address them like logging, audit etc.,.

    Yet another thought would be to solve couple of mathematical/logical puzzles to see and understand his logical as well as the calculative thinking.

    ReplyDelete
  7. Thanks for the feedback everyone, much appreciated. Bill, thanks in particular for those suggestions from someone who's actually done this. In particular the idea of indicating to people that I don't necessarily expect a full solution was very good.

    What may not have been crystal clear to everyone was the fact that this was a "homework" assignment following a phone screen. As for how long they have to complete the homework, I didn't specifically give a timeframe, that's almost part of the exercise to see how quick the respond to it. So far two people have completed it and they basically tool the last weekend and did it then.

    ReplyDelete
  8. Hi Jon. Wondering if you or anyone else had collected or posted some additional sample map/level files that could be exchanged by potential "solvers" of this little game?

    Cheers,

    GW

    ReplyDelete
  9. Thinking hats on! Decide upon a user interface and graphics best suited at the user end. Interface developers and Programmers work alongside hi-tech graphic designers. The participation of producers is also very important in this phase! visit this site

    ReplyDelete
  10. Through these programs, the students get to know about a lot of engineering practices of renewable energy and conventional sources of energy as well & their uses in modern society.
    Floods Pro Service Companies

    ReplyDelete
  11. Their custom software solutions are also highly compatible with the latest scientific instruments, wireless devices, medical tools and other consumer products.cheap Revit 2017

    ReplyDelete
  12. Product design and testing is estimated to continue its high growth rate and grow upon its 25% market share due to enablement of remote connectivity, which allows analysis and investigation of products in the field without being physically present on the site.buy SolidWorks 2010

    ReplyDelete
  13. Being that it requires no movement of your joints, this means that you're getting highly effective results and all of the stress is targeting your core six pack abdominals.why are my underarms dark

    ReplyDelete
  14. Is Life a Pain in Your Neck? Perhaps one of the most common health complaints in our hectic lives today is chronic neck pain. Neck pain can disrupt our lives, limit our ability to enjoy our favorite activities and can even make a normal workday unbearably agonizing.inversion for neck pain

    ReplyDelete
  15. thanks for this usefull article, waiting for this article like this again.
    software development company in delhi

    ReplyDelete
  16. The blog has an extremely narrow focus that extends, surprisingly into wide range of concerns.
    hire a programmer

    ReplyDelete
  17. The coming of the internet age has created endless engineering job opportunities for computer science students. DB Designer

    ReplyDelete
  18. This comment has been removed by the author.

    ReplyDelete
  19. Thanks for sharing your wealthy information. This is one of the excellent posts which I have seen. I go through your all of your blog, but this blog is the best one. It is really what I wanted to see hope in future you will continue for sharing such an excellent post
    vé máy bay từ đà nẵng đi seoul

    đặt vé máy bay đi nhật bản

    vé máy bay hà nội tokyo vietnam airlines

    vé máy bay eva air đi đài loan

    đặt vé máy bay đi đài bắc

    vé máy bay hà nội cao hùng

    ReplyDelete
  20. Manage Engine Service Desk Plus is a Web-based, simple to utilize HelpDesk programming with Asset Management whose highlights incorporate agreement the executives, programming permit the board, network stock administration, buying, dynamic index and LDAP combinations and information the board functionalities. help desk softwares

    ReplyDelete
  21. A rootkit is a product program intended to furnish a gatecrasher with executive admittance to a PC without being recognized. Its motivation is quite often vindictive. linux rootkit detection

    ReplyDelete
  22. Find the best essays on is my friend's profile page. Email Extractor Software

    ReplyDelete
  23. I recently found many useful information in your website especially this blog page. Among the lots of comments on your articles. Thanks for sharing. zcode system discount

    ReplyDelete
  24. I found your this post while searching for information about blog-related research ... It's a good post .. keep posting and updating information. zcodesystem discount

    ReplyDelete
  25. This article was written by a real thinking writer. I agree many of the with the solid points made by the writer. I’ll be back. Direct Response Copywriter

    ReplyDelete
  26. Wonderful article. Fascinating to read. I love to read such an excellent article. Thanks! It has made my task more and extra easy. Keep rocking. free 3d cad software

    ReplyDelete
  27. Windows 10 has some of the most amazing features that the people who don’t use them have to lack. I got this point when I availed myself of this blog and got detailed information about how to Windows 10 download ISO 64 bit with crack full version. That is why this is an amazing blog that everyone should use for themselves in the first place. People can get to know everything about their Windows 11 download through this blog for sure 10 64 bit download

    ReplyDelete
  28. My laptop was facing a lot of screen resolution problems when it was not updated. Then I visited this blog and got to know about some amazing features of Windows 10. I also got to know how to install the Windows 10 download ISO 32 bit and use it in the best possible way. I am very impressed with this blog and will recommend it to more and more people for sure windows 10 32 bit download

    ReplyDelete
  29. Nice to be visiting your blog again, it has been months for me. Well this article that i’ve been waited for so long. I need this article to complete my assignment in the college, and it has same topic with your article. Thanks, great share. video editing software for pc

    ReplyDelete
  30. I curious more interest in some of them hope you will give more information on this topics in your next articles. FUD crypter

    ReplyDelete