Sunday, November 29, 2009

Agile adoption: Part II


In my initial post discussing the adoption of agile practices in my engineering group, I gave some background on where we were a few months back and how we started to switch from waterfall to agile.

In this post I'm going to get into some details about the practices we have tried and how they have helped us (or not). Before I get into the meat of that though, one of my colleagues, Ram, commented on that first post. He had a very interesting point which gave me pause for thought. Amongst other things Ram said, "did the points estimating really helped us??i personally could not see the benefits out of it."

Chewing this over for a few days has led me to realize there's a couple of important things anyone reading this needs to know. Firstly, what I write here are my observations, and I'm not a member of any of our scrums teams such as (for example) Ram is. For those that don't work with or otherwise know me, these postings are the observation of a manager of a team of engineers. My concerns and observations will, necessarily, be different from those actually on a scrum team doing the hard work. Secondly, I may need to make clearer why I see some things as beneficial -- perhaps they would be obvious to peers, other managers etc. but I would like to share these views also with people on actual scrum teams. I think having a more holistic view of the whys, wherefores and benefits for more than just the team would be useful and (hopefully!) interesting. I know that two of the guys on our scrum teams have their own blogs and hopefully they will post their thoughts on our agile adoption over time. You can find Ram's here, and Rajiv's here. Rajiv already has posted some preliminary thoughts.

Now on to the real substance of this post. I'm going to cover what I remember as the next four items of consequence in our agile exploration: task visibility, proper iteration kick-off meetings, iteration retrospectives and lastly Twitter. Yes really, Twitter.

Task Visibility
Truth be told, having a decently detailed understanding at any given moment of what people were working and whether it was going according to plan was historically difficult for me. In part I attributed this to not playing the role of project manager for any of our ongoing endeavors...so of course I didn't know. But finding out when I needed to know was always a tedious process. It felt like it took more time than it should and never left me with much confidence that I had the full story anyway.

Our agile adoption almost immediately started to help with this. We employed two key tools: a simple task board and Basecamp, an online project collaboration and task management tool. The task board was set up in a small room we were able to use on a daily basis for our stand up meetings and was shared amongst the entire team. It took us a while to tease out "good" tasks and longer still to realize that the RA should have tasks on there too. At this stage we weren't really story based, and so the granularity of what was on the board was fairly variable. Nonetheless, being able to see at a glance how many tasks there were for an iteration, how many remained in committed vs. in progress vs. done was very powerful.

This physical task board was augmented by Basecamp. This was mostly popular with the software engineers on the team, and we wrestled for a bit over what should "drive" the stand ups ...the physical task board or Basecamp. Our situation was complicated by the fact that we had team members in both Eastern Europe and India. This led some of us to feel that Basecamp was the way to go, but, ultimately the physical board won out. I think upon reflection this was a good thing -- the simplicity and visibility of the board was key. The software engineers continued to use Basecamp to a greater or lesser extent for most of the remainder of this release cycle.

Proper Iteration Kick-offs
Even before our first tentative steps in the agile direction, we had typically divided our product development work into two week "iterations." This didn't actually afford us that much of a benefit besides perhaps enforcing a certain regular cycle when the PM would check in with the developers and obtain some vague indication of what they might do in the next couple of weeks.

Now the kick-off meetings had much more purpose: what are we putting into the "committed" column of our task board? If we estimate that stuff in points is it <= our velocity? From my perspective this seemed to give us a much more regulated rhythm to our work. Things seemed more predictable -- the team met their commitments. They thought harder and talked more about what they were committing to. Their was greater consensus across the team even though work on different facets of the project was still silo-ed in some ways.

Having seen how this panned out I would maintain that there's really no point doing work in iterations unless you are also measuring velocity in some way. There's just no point -- if you're not measuring velocity all you're doing is arbitrarily punctuating work with some meaningless and frequent end point for no benefit. When you do iterations with velocity however, you start to gather data about the capacity of the team. From this the team can become better at predicting what they can accomplish. There's a straightforward but worthwhile sense of accomplishment knowing you can approximate what can get done in a couple of weeks. 

Retrospectives
For me retrospectives, and what they imply if done well, are the key to agile success. If (for some odd reason) I could introduce a fledgling agile team to just one new idea it would be retrospectives. I've never been a big fan of meetings, but with a retrospective you can get so much:
  • sense of team cohesion ("We're all in this together...something sucks? What would you like to do about it?")
  • shared ownership of process (so many people are used to being "told" what the process is...and now there is a chance for them to shape it)
  • an opportunity to challenge existing dogmas ("our SOPs won't let us do that")
  • a forum for people to shine with insights and ideas that often don't get an audience
  • a plan of improvement and a place to measure that improvement

What I also learned was that retrospectives need a great facilitator. It's all too easy for someone to go around the room and ask what went well and what didn't and get fairly anemic responses: "I guess what went well was we got all our committed work done...can't really think of much that didn't work out so good..." Getting people to surface the less rosy side of things is hard. There are of course those who always have views on this kind of thing (I'm kinda one of those myself) but for others it's awkward. Until people find that volunteering this kind of information is not just acceptable but welcomed some skill is required in extracting it. One way of doing this is through just lots of dialog. In a one on one situation people are often more willing to offer up something that's driving them nuts. I often would then try and find a way to toss this out for discussion without specifically identifying the source.

Twitter
I may stand alone on believing Twitter was a valuable resource that contributed to our agile adoption. Certainly I do not believe I managed to convince any of my peers of my boss to sign up (or if I did they're obviously using a pseudonym and cyber-stalking me). And this despite the copious links to informative blog posts etc. I sent them. Hmm, perhaps I did sent a few too many...

Anyway, personally I found Twitter to be a fabulous new tool for finding reading information relevant to agile. I developed a twice-a-day ritual of scanning my twitter stream and flagging interesting looking links for later reading. I started out with just a few people to follow, but through the way twitter works I was soon discovering others, from internationally known thought leaders through to ordinary people like myself writing about their experiences. With the recently added "lists" feature of twitter I have create an agile list which may be of interest to anyone wanting a starting point.


OK I think that's enough for a post. In the next installment I'll talk about some of the subsequent events including how we started to talk more about a specific agile methodology (Scrum) rather than a general notion of "agile" and some of the technical practices we started to put some focus on.

Chipotle Chicken


Ingredients

  • 4 chicken breasts
  • 1 pint heavy cream
  • 3 tbsp chipotle en adobo sauce (search google for recipe, very easy to make)
  • chopped cilantro for garnish
  • butter for frying

Directions

Cover chicken breasts with plastic wrapping and slap it around a bit with a rolling pin until it's uniformly flattened. Heat butter until melted and bubbling over medium heat in a large frying pan. Add the chicken and let it sizzle for a few minutes. Lift it up and have a look underneath; you're looking to see some nice color has been added. Once this is the case flip it over and do the same to the other side. At this stage the chicken shouldn't be cooked all the way through.

Now add the chipotle en adobo -- use more or less than suggested depending on your preference for heat. Pour in the heavy cream and kind of mix it all together. The cream and adobo sauce will blend together and should start bubbling away quite nicely. I usually turn the chicken over a couple of times during this stage. When the sauce has reduced to a thicker state, if all is well the chicken will be nicely cooked too. I usually cut the pieces in half at this point both to check done-ness and because that's a nice serving size. Taste the sauce and if you feel it's lacking add in some salt & pepper.

Transfer the chicken to serving plates. Check your sauce, it may be ready to go or you may want to reduce it further by letting it bubble away a little longer. When ready, spoon it over the chicken on each plate.

Serve with a simple salad of shredded lettuce, diced tomato seasoned with lime juice and black pepper. You may also want to add your preferred Mexi-style accompaniments, e.g. rice, refried beans etc.

Wednesday, November 25, 2009

Book Review: The No Asshole Rule

I recently read Robert I Sutton's book, The No Asshole Rule. Apparently it was borne of an article originally published in the Harvard Business Review. I cannot claim to have read that original article but the book is great; I enjoyed it immensely.

Besides the frequent smirks spawned by the amusement derived in reading a "serious" work about assholes, there's plenty of just great writing, sound advice and lots of "yeah I agree" moments.

The book is pleasantly short, with just seven exquisitely crafted chapters covering everything from the definition of an asshole in this context (not just someone who pisses you off ;-) through to eradicating them if you have the power, or dealing with them as best you can if you don't. In addition there's material on the virtues of assholes (there are some) and how to stop your own "inner jerk" getting out too often. It was nice to read that we're all assholes some of the time. The acknowledgment of this reality in the book is good, because without it I believe it would preach an unattainable level of asshole free zones. In other words it's OK to be a temporary asshole, indeed as the virtues section indicates sometimes it's necessary to jolt slackers and under-performers out of their stupor.

The section on dealing with an asshole infested workplace has some solid advice. Since most of us lack the authority (not being the CEO and all) to go on an asshole eradication spree it's probably the section most useful to most people. In particular I liked the ideas like attending meetings with assholes via telephone: if you have a meeting with an asshole in at least you can avoid having to be physically in their presence. Similarly take pleasure in answering their emails slowly, or maybe not even at all. Having a sense of control over the situation as the author suggests is indeed a very empowering feeling.

Luckily for me, there are very few assholes I have to contend with these days. Nonetheless, having finished the book there are still a few candidates whose desk I am considering leaving the book on...

Tuesday, November 24, 2009

Agile adoption: Four months in.


Actually I'm not sure if it's four months, it could be a little more, could be a little less. But that's close enough. In some respects I wish I had taken time to blog about this much more frequently, as I think it would have helped me distill my thoughts on how things were going. Possibly even a slim chance someone else may have found it interesting too ;-) But I didn't. I'm going to attribute that to a lack of habit, and maybe I can cultivate that habit more going forward. On the upside, having gone four months or so before posting my thoughts on adopting agile practices, I do feel that I have got a fair bit of experience that has some value to it. Perhaps earlier posts would have comprised vacuous meanderings around the topic of no interest to anyone.

Onwards...

Now I try and think back, I'm not 100% sure I can entirely recollect what the trigger for the turning point was. The best I can remember, there was some...shall we say, doubt or perhaps concern over whether a particular project we were running was going to be feature complete on time. Let me explain a little bit about where we were at this time:

  • we had a slew of software engineers (by our standards anyway) working away on this rather undefined "refactoring and re-architecting" phase of a major product release
  • somehow, despite some top level goals for this phase being set, there was next to know clarity on what was really going on, nor whether it was worthwhile
  • our customer certainly had no clue what was going on
  • the deadline for delivery had been fuzzy until recently
  • we had some nasty combination of huge horrible "the system shall" type requirements documents, use cases that were more use-less cases
  • nobody had a clue what the QA members of the team were doing besides "regression and exploratory testing"
  • the team was working in two week iterations, although they seemed rather purposeless upon closer inspection; the only thing that they seemed to do was ensure we more or less regularly provided a new build to QA
  • the quality of the builds was low, often emergency "patches" were needed to remedy showstopper defects that somehow made it out to QA
  • nobody was pair programming or doing TDD; the unit tests we did have were...let's just say sub-optimal *cough* not working *cough* or commented out *cough*

Anyway, back to the plot. So in a conversation about the need to critically evaluate whether this project was going to succeed, I may have blurted out something about my distaste for estimating in hours and how we should use points. I had done a little reading on this topic and sensed the potential -- estimating the work left in points and measuring velocity would give us a much better feel for progress I believed. They say you "should be careful what you ask for, as you may just get it" and yeah, I got it. So I went away and read a bit more (should we do S/M/L t-shirt size estimates? Fibonacci sequence?) before probably delivering the most disruptive message ever to my developers the next day. "Yeah this is Jon. I thought it'd be a great idea to estimate what we had left in story points. Have you heard of them? No. Don't worry. We're just gonna give it a go..."

I actually had us doing this wrong of course. Rather than estimating as a team I "pair estimated" with individual engineers for features they were responsible for developing. At this stage this was the expedient approach and looking back I still maintain it was better than do a "how many hours/days of work have you got left?" exercise. There was no QA involvement in these estimates, but I think that trying to get their involvement would have been too hard at that stage.

We settled on using 1, 2 or 3 as valid values for estimating our stories, mostly because it seemed simplest to get people to think about whether something was "easy, medium or hard" rather than agonize over a more complex set of choices. At this point I hadn't read any good material on what made for a suitable story, so many of our stories were tasks. With hindsight, although clearly not optimal this *still* was a step in the right direction.

It took the best part of a day for me to work with each of the engineers on the team and get their points based estimates for each of the stories (or tasks...) they believed they had left. Once this was done though I felt a distinct sense of progress. It was somehow easier to think in terms of there being about 100 points of work left, and how this therefore implied we needed a velocity of 20 points an iteration (not real numbers, but you get the point...)

With the "stories" and their corresponding estimates in a spreadsheet, and a daily stand up we suddenly started to be able to understand progress like never before. It was euphorically exciting to feel this degree of comprehension and transparency available to us. This feeling was a fleeting thing at first, but as iterations passed by and the velocity stabilized I think we all felt better about predicting success.

Stand up meetings were initially hard. People were late. People rambled. People didn't see the point. The chickens (myself and other managers) interrupted gratuitously to "coach" people on doing it right. We were eager but perhaps ineffective. Stand ups took ages. But, interestingly, we went from weeks slipping by with hidden issues, to impediments surfacing daily. And we'd solve them, or people would agree to work on solving them there and then. This was our second big win after understanding velocity I think.

With a toehold on our agile adoption, we were starting to build momentum. We have done much more since then, which I will write about it follow up posts to come.

Book review: Bridging the Communication Gap by Gojko Adzic

I just finished reading Gojko Adzic's book, Bridging the Communcation Gap. It's subtitled "Specification by example and agile acceptance testing." For a one sentence description, that does a pretty good job of explaining what this book covers.

I was inspired to seek out something like this after previously reading Mike Cohn's User Stories Applied (TODO: my review). It was in Mike's book that I found compelling hope that we could get away from the unwieldy requirements specifications used in my organization. These have traditionally served as the definitive record of what our software should do yet, they have never felt helpful or right to me. Indeed there is much evidence that they don't help at all, but that's another post...

In User Stories Applied, Mike answers the question I think everyone has after getting the quick overview of what a user story is: "Where do the details go then?" After all, given a story such as "As a hiker I want to view a list of nearby trail heads so I can investigate suitable hikes" one is naturally left with many questions. How far away from your current location should the software look for trail heads? Should you be able to vary that distance? What information should be in the list -- trail names? Dog / horse / mountain biking use allowed? etc.

The answer is that these details, rather than being captured in a traditional specification or use case documents can be very beneficially recorded as actual examples and acceptance tests. Understanding this was a real "aha" moment for me. Prior to this, user stories, estimating in story points, measuring velocity and using that to finally have some measure of what a team could accomplish in a fixed length iteration all felt right, but I couldn't quite figure out how we should do requirements.

Gojko's book opens by building the argument that traditional approaches to gathering and documenting requirements are flawed. I don't know that it would be hard to convince many people who've spent more than a few years in an environment where they must develop software based on the information in 100+ page requiremetns documents that this was the case. However, what Gojko does is to articulate well some reasons why this is the case. For me though, the real meat of the book is all contained in section II, which provides the concrete specifics of how to go about specifying by example and deriving acceptance tests.

The key points covered that I found valuable are:
  • Confirmation of my own belief in the necessity of customer/domain expert involvement alongside testers and developers. Business analysts should not be a conduit (or worse, filter). There's a particularly good explanation of the role of BAs mentioned in the book drawn from Martin Fowler and Dan North's presentation The Crevasse of Doom.
  • A detailed explanation of what make good examples.
  • Commentary on the benefits of a project language -- a non technical business oriented vocabulary that enables unambiguous communication for the team.
  • How to go about converting examples to acceptance tests.
  • How to run specification workshops successfully. Besides the information in the bookon this topic I am currently quite enamored with the Nine Boxes approach to interviewing people.

Since reading the book I've had a chance to get a little hands on with the technique but not in anger. I took some of the ideas and applied them to something we had already done, where I knew plenty about the system to allow be to play the role adequately of customer, tester and developer. I turned 40 pages of crap documentation (no offense intended to the original author) into about three. I'm very much looking forward to seeing us apply this approach to the next release cycles of a couple of our products.

The third section of the book discusses how to go about implementing agile acceptance testing covering items such as how to get people interested in the prospect to the synergy the technique has with user stories and the various tools available for automating tests. There is one slightly dubious (but perhaps necessary under adverse conditions) technique mentioned which is altering one's language to avoid the use of the word "test".

The fourth section is pretty short, and covers how adopting this approach to specifications and acceptance testing alters the work of various roles. This was already covered in briefer terms earlier in the book. This section expands upon that, although to some extent it feels like a bit of a rerun.

All in all the book gets a definite thumbs up from me. My only real complaint would be that the writing style is a little dry, definitely not as readable as some authors. Nonetheless, the effort to read it has been worthwhile from my point of view and I would definitely recommend the book to anyone on a team looking to do requirements better.

Basic Curry

This is my basic tomato-based curry which gives pretty good results, better than something in a jar and better than just using some generic "curry powder". It's pretty flexible and can be the basis for several variations on the theme which are noted at the end.

Ingredients
  • 4 or so chicken breasts, cut into bite sized chunks
  • Large sweet yellow onion
  • Garlic cloves -- 4 or more
  • Root ginger -- an inch or two
  • Chiles - Jalapenos and/or Serranos work well. Use your own judgment depending on how hot you like things. I would usually got with 2 x jalapenos for a middle of the road level of heat.
  • Prego tomato sauce or similar. I know that sounds weird and Italian but it's a great tomato based sauce. Just make sure you use a plain variation, not one full of oregano or mushrooms or something... 
  • Cumin seeds -- 0.5 tsp
  • Cumin/Coriander powder -- 1 tbsp. In Indian grocery stores they sell ground cumin/coriander premixed. If you have separate cumin and coriander powder then just mix them approx 50/50.
  • Garam Masala -- a spice mix. Best value is in Indian grocery stores but I have seen this in regular grocery stores too.
  • 0.5 tsp turmeric powder
  • 1.5 tsp salt
  • Lemon juice -- 1-2 tbsp
  • Cilantro -- about 8 stalks + leaves chopped


How to cook it
Chop the onion, garlic and ginger. Cut the chilies in half - I generally leave the seeds in.

Heat ~3-4 tbsp vegetable oil until hot. Drop in the cumin seeds and they should quickly darken as they sizzle in the oil. Now add the chopped onion and chilies. Fry for 5-10 minutes before adding the ginger and garlic. Cook for a few more minutes.

Add in the cumin/coriander powder, garam masala and turmeric. Stir in well and cook for a minute or so. Now add the chicken and cook until mostly sealed. Add the Prego, cilantro, lemon juice and salt. You'll probably want to add a bit of water to thin out the sauce -- I usually do this by rinsing out the Prego jar with some water and then tossing that in. Stir well and bring up to a simmer. It will probably take about 20 minutes or so for the chicken to be fully cooked.

Taste and adjust as necessary. Might need a bit more salt, lemon juice or cumin/coriander powder to bring things into balance.

You *should* serve this with white basmati rice, and if you do make sure it's not the Californian crap ;-) Get something from the Indian subcontinent. These days I quite often do it with brown rice which takes a little getting used to but is still very nice if not very authentic.

It's also nice to have some raita on the side with this which I make by combining some plain yoghurt, English (hothouse?) cucumber chunks and mint and letting sit in the fridge whilst cooking so the flavors blend. You can also stick some diced chili in that mix if you haven't got enough heat already.

This keeps and freezes pretty well in the event you can stop yourself from eating it all at once.


Variations
  • Consider adding in any of the following (all is nice) up front with the cumin seeds to create a richer flavor: half a dozen cloves, cardomom pods, a cinnamon stick, a star anise, 2 x bay leaves
  • Fenugreek seeds can be added in up front with the cumin seeds to create a very distinct flavor
  • Puree the onion/garlic/ginger and then very roughly cut up a second onion. Start by frying the whole spices then add the chilies and chunks of onion. Fry for a bit before adding the puree to create a "do piaza" (double onion) style curry
  • Cook some potatoes and add these in ~10 mins from the end
  • Substitute the Prego for diced tomatoes
  • Use fresh tomatoes cut up....they reduce down and make a nice sauce usually






Chorizo & Pepper Egg Gratin

Chorizo, Pepper & Egg Gratin

Ingredients:
  • Some Spanish (or Spanish-style...) chorizo, i.e. not the stuff commonly referred to in US grocery stores as chorizo. Whole Foods have some on their deli counter which is made in California but a fair approximation of the real thing. See http://en.wikipedia.org/wiki/Chorizo#Spanish_chorizo for more details. I'm gonna be vague on the quantity...just use your skill and judgment. I used about half of what I bought from Whole Foods to make enough for three.
  • A medium onion. Yellow, red...whatever. You can either cut slices or dice it. Recommend not making it too small as you want pieces large enough to stay moderately crisp.
  • A green bell pepper. Cut it into small slices or dice it. Again, not too small...
  • 2 tomatoes -- remove skin using the old hot water trick and then chop
  • 1 or more cloves of garlic, depending on how partial you are to it
  • some eggs
  • some grated cheese (something that melts good, cheddar, gruyere etc.)


You need to remove the "wrapping" of the chorizo, then dice fairly small.

Fry it in your fat of choice (Olive oil recommended( over high heat until it starts to brown a little. Push it to the back of the pan and drop in the onion and pepper. Fry for 5 minutes or so then drop in the chopped tomatoes and garlic. Stir everything together with a good seasoning of salt and pepper then let it sizzle as the water from tomatoes evaporates.

Transfer to an oven proof dish and then crack as many eggs as you fancy into it. It can help to make a little depression where each egg is going to go to hold it in place. Sprinkle the grated cheese over then bake in over at 350F. Original recipe says 12 - 15 minutes should do it. I haven't found this to be the case. Maybe it's an altitude thing, not sure. I've found 20 minutes to be more like it.

Good with ketchup and/or a hot pepper sauce such as Cholula, Tobasco etc.

It should end up looking like this. Being a pretentious git I have sprinkled some paprika over the top :-)