Wednesday, December 22, 2010

Mind your language!

I find it hard to pick out a favorite quote from Guy Ritchie’s cult movie “Lock, Stock and Two Smoking Barrels.” It’s just chock full of great lines.

Ignoring the obvious masterpiece from Rory, one exchange that I’ve always remembered with a smile is between Harry’s enforcer, Big Chris and his son (who is of course known as little Chris). They’re visiting someone who owes Harry money, and Big Chris’s son rifles through the indebted man’s belongings. Upon finding a serious pile of cash he exclaims: “Fuckin’ hell John, do you always walk around with this in your pocket?”

Big Chris, despite the dubious London underworld he and his son inhabit appears to have aspirations for his son to remain reasonably polite. He replies, “Hey! You use language like that again son, you’ll wish you hadn’t!” Unless you’ve seen it, it’s hard to appreciate the energy with which that warning is delivered.

Despite immersing myself in agile and Scrum and the language that comes along with that, and despite doing my best to socialize it over the last couple of years in my engineering group I think there are limits. Just as Big Chris doesn’t necessarily want his son to use the lingua franca of the underworld everywhere, I think you need to consider how far outside a development team you use agile and scrum vocabulary.

One of the three legs of Scrum is transparency. But being too transparent isn’t necessarily a good thing (think body scanners in airports, for example).

Maybe, if your entire organization has decided to operate using scrum, in everything from sales and marketing to operations, HR and accounting as well as engineering then you can happily talk about daily scrums, sprints, points and stories and velocity. But if they haven’t, then I think you have to balance carefully the education of those groups with whom you interact and the language you use.

For me, there are three key pieces of terminology that I am, with hindsight, wondering about the wisdom of sharing widely.

Yep, I don’t think we should be talking about stories. Now don’t get me wrong. That doesn’t mean I don’t think we should be using them. On the contrary, I’ve said it before and I’ll say it again, one of the most profound books for me as I was learning about agile was Mike Cohn’s "User Stories Applied."

The concept is great. As a unit of work it’s perfect. For organizing requirements it’s helped tremendously.

However, as a term to use with unsuspecting business stakeholders, customers and so forth I’m not so keen. Frankly it sounds rather ridiculous. I’ll stick with feature, thanks very much.

Few people with practical experience of dividing your development up into fixed duration timeboxes is a bad thing. But do we have to call it a sprint?

One of the things I like about Scrum is the fact that done right nobody’s having to dash like a lunatic to cram in all the last minute jobs to ship your product. Instead what we have when we do it right is a predictable, sustainable, humane way to utilize teams.

Sadly sprint conjures up anything but that vision.

These days I prefer to describe that we work in short two-week cycles, and if I need a short snappy name for them then iteration works just fine.

Points and Velocity
I’ve pulled these two terms together as they really go hand it hand. At least, if you’re estimating stories in points but not measuring your velocity you might want to reevaluate what you’re trying to achieve.

And these two are a little different for me than story and sprint. Whereas those are just unnecessarily goofy in everyday conversations with the wider business, and alternatives can serve us better, points and velocity need very careful handling, whatever you call them.

Here’s the problem – points and velocity are a tool for the team to be able to evaluate their performance, to plan and make reasonably firm commitments. Stakeholders outside the team should be concerned more with what gets done and in what order. The Product Owner and team together can use points to respond intelligently to new requests, to switch things around, and figure out what fits where and what needs to get dropped.

But start telling your stakeholders too much about average velocity, the points involved in a particular story and you inevitably risk them starting to focus on points rather than the “what should we do” and “what order should we do it in” issues. Perhaps in a really mature agile environment were a deep understanding of what points and velocity are for this wouldn’t be an issue. But early on in an agile adoption I think care is needed.

Too much data on story sizes in points, team velocities and so forth risks the following questions:
“Can’t you do more points per sprint? Why didn’t you make your commitment? Why has your average velocity dipped? Why does the tracking team have a higher velocity?”
Those kinds of discussion are not the most value one can extract from a stakeholder. I don’t want to see agile teams micromanaged by a group of well meaning people who know nothing about agile software development. But we are baiting them into doing exactly that if we report too much detailed data about points and velocity. Before you know it they’ll be looking at your definition of done suggesting we don’t need to do unit testing and who knows what else…

I say keep the dialog with stakeholders confined to steering the product, negotiating the relative priority and let the team use points and velocity data to communicate what can be done for any given release.

Put another way, what do you really want customers and stakeholders to concentrate on? The minutiae of your process? Or innovative product ideas and how they should be prioritized in the backlog?

Wednesday, December 15, 2010

Fully distributed Scrum

(Or, "How we pitched the idea of letting everyone on our team work from home and got the thumbs up")

Back at the beginning of November I wrote about my recent experience as the scrum master on a highly distributed team.

For those that didn’t read (or don’t remember) that post, a key thing to know is that the team is comprised of several people who work from home. I ended that post (partly tongue in cheek I admit) saying how it would be interesting to take those team members that were still office based and have them work from home too. Well, starting early 2011 we’re going to give it a try!

The original suggestion was first seriously discussed in a team retrospective. On the premise that it’s easier to offer an apology than ask permission I sent the appropriate managers in our group an email indicating that the team had decided we were all going to work from home next sprint. That didn’t fly. Which isn’t surprising (although it was worth a try, or at least entertaining, no?) and a more considered proposal was requested.

Myself and the team duly worked on preparing a properly coherent argument, supporting data, benefits we expected, a concrete description of how we would run this little experiment and details on how we would measure the success (or otherwise).

I know there must be many other teams trying to reconcile the demands of their managers to “do agile” and also “leverage offshore resources” so I wanted to share details of the proposal that helped convince ours that trying the rather radical proposal of sending everyone in the office home might help.

What are we trying to do?
Before we go anywhere further, it’s worth properly answering this question. Lest there be any doubt, this isn’t about folks wanting to get up late, watch daytime TV and get their laundry on during teleconferences.

This is an authentic endeavor to make our team work better. You can argue till you’re blue in the face about the benefits of team co-location, or the silliness of splitting teams into onshore and offshore components when you’re trying to simultaneously implement Scrum, but the fact remains that many, many organizations are still going to insist you can’t move the furniture and you have to structure teams this way.

Therefore, if we have to live with this, we’re going to make it work as best we can.

At the heart of much agile advice is an emphasis on communication. The trouble is, with team members scattered across different timezones and locations, communication gets clumped into localized groups. People on the edges miss out. By putting everyone on an equal footing (working from home) everything becomes more equitable and information sharing is normalized allowing for a better functioning team. At least, that’s what we believe, and what we’re trying to do is prove that out.
“Craft a philosophy and live it.” – Umair Haque,

How did we approach this?
Our presentation covered five key areas:
  • Why did we want to try this idea of everybody on the team working from home?
  • A list of benefits, beyond simply the team working ‘better’ that we anticipated
  • A description of how we wanted to run the experiment
  • How we intended to evaluate the outcomes
  • The support we needed to get this going
I’ve already talked about the why, so let’s look at the others:

Anticipated benefits
Besides the hoped for improved communication, there are some other interesting and very tangible benefits we anticipate.

As I write this blog post the time in Colorado is 2.54pm. Meanwhile, in Hyderabad it’s 3.24am the next day. We have quite the time difference between some of our team members. Before Daylight Savings Time (DST) came to an end in the US this year it was just possible (if not entirely pleasant for everyone) to have a daily scrum via phone with all team members present. Now however, since Indian Standard Time doesn’t adjust, the earliest we can hold our daily scrums is just unreasonably late for our colleague there to attend every day. Although they do stay late to attend some of the sprint planning meetings and sprint reviews they’ve been missing a lot, including retrospectives.

One very worthwhile thing we can do operating as a team that works from home is create a more humane work/life balance for people. This will pay particular dividends for our colleagues in Hyderabad, allowing them to work the majority of their days at normal local business hours. They can they stop just early enough and save the remainder for meetings in the evening. Although nobody particularly wants to have to regularly work super early or super late hours, this is a much better arrangement than having to be still stuck in an office at 9pm at night.

In addition, there are another couple of benefits we see. One is immediately realizable, the other perhaps a longer term proposition. They are reduced environmental impact through less travel and reduced office space needs.

An article in Forbes magazine in 2008 indicates there were as many as 17 million software developers in the world at that time.

More than 2 million of them are in India and the US is approaching 3 million.

Most if not all of these people spend time in cars or other transport every day commuting to and from work.

Indian emissions standards for motor vehicles lag about 5 years behind Europe, and in the US of course the majority of us love our gas-guzzling SUVs.

We may not be able to keep millions of these people off the road, but by allowing those that work in our team to do their jobs from home we can have a small but positive environmental impact. Who knows, maybe this innovative idea can spread.

As for the office space thing, if you can establish a team working at home, with perhaps continued but minimal office visits as needed then do you really need such large premises? With enough teams operating this way you could save on what you lease, on what you heat and so forth.

Running the experiment
Being a Scrum team convinced of the benefits of incremental and iterative development, we naturally intend to use the same approach to this experiment. Starting early 2011 we will try a sprint with everyone working from home.

At the end of that sprint, a large part of the retrospective will obviously explore how things worked. We’ll be asking ourselves questions like:
  • How well did that work?
  • What work well and what would we like to change?
  • Can we fix the things that didn’t work well?
If we think it was worthwhile and we can improve those aspects that didn’t go too well then we’ll repeat another sprint with any changes we need. With this approach we can pretty quickly home in on how to do this well, or equally quickly conclude that our original arrangement with two-thirds of the team staying office based was better.

Evaluating the outcomes
Many years ago I used to quite like the idea of capturing data you could analyze to objectively evaluate things. Peter Drucker is renowned for saying “What gets measured gets managed.”

I’ve since learned that if you measure the wrong things in the wrong ways then it’s more a case of “what gets measured gets gamed.” Regrettably I’m not renowned for saying that. Maybe one day.

However, we do need some way to determine how our experiment is going.

Agile thinking provides us with a few simple tools that we can employ: velocity, burn-up charts and cumulative flow spring most readily to mind. To some extent though there is the possibility that these could indeed be gamed, or unconsciously given treatment to provide the desired out come.

So besides those, we will also be looking at some other areas:

  • PO and stakeholder satisfaction. From their point of view are things better, worse or pretty much the same?
  • Team satisfaction. Do we feel that things are better, worse or pretty much the same?
  • If the experiment pans out well enough that we conduct a whole release with people working at home then we can utilize SLIM Metrics to observe productivity benefits.

Required support
Of course my team is not an island unto itself; we needed support to undertake this. First and foremost we needed the support of our management. Luckily the director of our group is open to innovative ideas and. After our presentation to him and a series of questions that the team was able to answer was willing to permit this experiment.

In more material terms we pretty much had all that we needed. People have broadband connectivity and telephones at home. They have quality spaces to work in. We do need to configure a few machines for remote access, and a few people needed new laptops. Other than that though, we’re ready to go.

In conclusion
The whole team is excited to try this. I think we’re onto something. I’m eager to see what our actual experience is like. Will things be better or just different? Next year, after we’ve run this experiment for a bit I’ll report back on how things have turned out.

Blog name change

I changed my blog name to "Coding When High."

This probably requires a little explanation lest you draw the wrong conclusions.

Peace & Happiness

Originally I named this blog "Mixed Messages" with the explanation that I was unsure I could restrict myself to any one topic. And indeed over the last couple years there were a few  posts on things other than software development  topics. However, as its turned out, the vast majority of what I write about has stemmed from my experiences managing and participating on agile and scrum teams over the last few years.

Since a clear focus has emerged for my writing here I thought a new blog name was in order. Plus I wanted to play with Tumblr, so I'll henceforth be posting my non-work related stuff over at (nothing interesting there yet...)

So how did I arrive at this name? Well, I've recently switched back from management to individual contributor on a team. Currently I'm acting as Scrum Master, but with a definite intention of writing some substantive code soon. Add in that I live at 9,200 feet above sea level, up in the Rocky Mountains west of Denver, voila, "Coding When High" sprang to mind.

So no, don't worry. I'm not doing MDD (marijuana driven development).

Thursday, December 2, 2010

The perils of rolling your own

I spent a good portion of the early years of my adult life messing up my health by smoking. Even throughout my thirties I was an off and on smoker (or “social smoker” if you will) at various times.

It’s well understood that no matter what you smoke, whether regular cigarettes, “low-tar”, cigars, hookahs or even medicinal (or illicit I suppose) marijuana: inhaling smoke will screw your lungs up pretty good. For my part, I smoked hand-rolled cigarettes, which is probably one of the worst options: no filter to take out anything at all.


In 2002 I moved from the UK to Boulder, Colorado in the US. For those that don’t know, there’s probably no place else on earth with a more militant anti-smoking stance than Boulder:
"In 1995... Boulder voters made national headlines and created a local uproar by expanding the city’s smoking ordinance to ban smoking in all indoor public place... The Boulder Dinner Theater had to get a special dispensation so an actor could light up on stage, as called for in a play."

From Insider's Guide to Boulder and Rocky Mountain National Park by By Ann Alexander Leggett, Roz Brown
As it turned out this was actually a great thing for me. It helped make quitting smoking and attempting a significant lifestyle change considerably more possible. Nonetheless, the first time I went to a gym and tried to run (for just sixty seconds!) I swear I nearly died. Humiliatingly, the gentleman on the treadmill next to me, easily older than my father, was barreling along at 6mph and hand been doing so for the last twenty minutes.

I definitely learned the hard way that rolling your own is bad.

I think this lesson translates into software development too. It’s often tempting to “roll your own” frameworks for various software development needs. At one point in time this was especially true for web applications. I’ve been to party this. Several years back we wanted an MVC web framework with AJAX support. There weren’t any at the time, or certainly none our lead engineer liked. So a couple of the guys I worked with wrote one. It was a cool project; fun and intellectually stimulating as well as educational. And we got to do things the “right” way, no more tolerating those parts of other frameworks we didn’t like. But it became the biggest albatross around our neck.

When you build your own proprietary framework there are several key problems:
  • Nobody's going to maintain it but you. This starts off as fun -- nobody else is "screwing" with your code, but quickly becomes a bind. In our situation in particular, with the AJAX/MVC web framework there was the never ending tedium of browser compatibility issues.
  • Dubious value. Is building general purpose frameworks *really* what you want to be paying your engineers to do? Is that *really* the business you’re in? Is it *really* the best use of their time?
  • It's hard! Sure, building 80% of the framework you think you want isn't too hard. But try finishing it. Try polishing it. That last 20% is tough.
  • Eventually the originators leave. You may think you've got the institutional knowledge for your framework well shared. But I'm willing to bet that if the main driver behind your home grown solution leaves it'll start to rot. A year later (if that) everyone will hate it. Not much longer and it'll be the reason why nothing can be done and everything about your product sucks (this is often melodrama but try countering it effectively...)
  • New employees have never heard of it. So they have to learn it. And, obviously you can’t hire anybody with experience in using it.
  • It lives...and lives and lives. Once you have products depending on your framework, extricating yourself from that can be difficult and costly. Just try convincing non-engineers that you spend time fixing this problem.
These are big problems and not to be underestimated. Based on my experience you should think very, very hard before you decide to commit to investing in building your own framework for anything.

The MVC/AJAX web framework we built years ago still limits us even today.

It was OK for the experienced highly-skilled programmers that built it. It was great for them to build rich one-off applications with. The trouble was, the business needed us to build dozens and dozens of cookie cutter applications…all similar but slightly different.

The trouble with that kind of work is that one doesn't tend to use experienced, highly-skilled programmers for it (or if you do, they don't stay around for long).

So then of course we had a framework that was hard for most of its users, and certainly wasn't geared to churning out cookie-cutter applications. They were frustrated and it was overly complex for the actual task at hand. Part of the consequences of that was we suffered the same defects and problems in the cookie-cutter applications over and over again. And it took longer than it needed to build them and longer than it needed to test them.

Once again, I learned the hard way that rolling your own is a really bad idea. We should have built a tool for building cookie cutter apps, not a general purpose web framework. Quickly and reliably turning out cookie cutter apps was the business we were in, not web frameworks.

P.S. if you've still got that itch to build a framework, how about contributing to an open source one instead?

Wednesday, December 1, 2010

You are not my customer

(And I am definitely not your supplier)

Recently our VP sent out one of his regular update emails and included in it a little piece about the virtues of internal customer service. He included the following in his email which appears to have come from

Internal Customer Service: Getting Your Organization to Work Together
Great customer service isn't just about serving the people outside your company.
By Scott Miller 

Providing exceptional customer service lies at the heart of the mission of many organizations. It is the central theme of books, articles, motivational seminars and business courses. Its value is undisputed in business circles. What many companies fail to focus on, however, is the primary path to exceptional customer service: internal customer service.

Internal customer service is the service we provide fellow employees and other departments within our own organizations, as well as our suppliers and anyone else with whom we work to get our jobs done. It is what we do when a colleague asks for information she needs to complete her main task for the day; it is what we say when someone from marketing asks for the addresses of good contacts; it is how we greet the vice president of sales when he walks into our office with an "I need something from you" expression on his face.

All these things can be seen as interruptions that take us away from our "real" jobs, yet they are vital to our company's success. If you see a gap between your "real" job and the needs of others in your organization, you need to rethink what your real job is. In helping others in your company, you help your company succeed. Here are some tips for creating an atmosphere of internal customer service:

1. Begin with your own perspective: Regard fellow employees and other departments as your customers. Understand that helping your colleagues do their jobs more successfully helps your organization and you. Therefore they are your customers. Treat them like VIPs.

2. View interruptions not as nuisances, but as opportunities to serve your internal customers. If you tend to view every interruption as a pothole in your road to success, reexamine those interruptions. If someone interrupts you to share gossip, that's a pothole. If someone interrupts you to ask for sales figures she needs to analyze sales team performance, that's a necessary lane change that will get your company closer to its destination. Learn to identify every real need from a colleague as a "necessary lane change," and think of every "necessary lane change" as an opportunity to move your organization closer to its goals. Take pride in helping your colleagues; enjoy your role in sharing information and providing services that help others get their jobs done. In most cases, your willingness to help others get their jobs done will lead them to readily assist you when you need it.

3. Exceed your internal customers' expectations. When someone exceeds your expectations, how do you feel? Most people feel delighted, excited, upbeat and very, very positive about that person and his or her organization. Think what you can accomplish in your organization by exceeding the expectations of fellow employees. If payroll asks for time sheets by 3 p.m., provide them by 1 p.m. so payroll can relax, knowing they have the time sheets in hand. If human resources asks for a list of important points to cover in an employee orientation, take time to think about it and provide a thorough list of what you would want to know if you were being introduced to a new job and company.

4. Say thank you. A simple, genuine "thank you" goes much farther to create an atmosphere of sharing and helping than two such small words would suggest. Even when it is a person's job to provide information or a product to you, tell them "thank you" when they have done it. Express your appreciation of their timeliness in providing it. Explain how it has made your job much easier. Show them your delight when they exceed your expectations.

I was immediately reminded of what was a very memorable counter to this way of thinking from an old colleague. This far away from the original conversation I forget the exact details, but over the years I've developed my own variant (it may be identical, I cannot say, but hopefully I've added my own nuances).

The trouble with the internal customer model is threefold. Consider:

  1. It’s not equitable. Not everybody is a customer, some people are just suppliers (accounts, engineering, HR)
  2. Those in the customer role are not paying, so they can often make unreasonable requests without consequences. As a supplier you can’t refuse their business, even if they’re a terrible customer. Real companies of course can pick and choose who they do business with. Further, you can’t charge them more when they demand unreasonable changes and extras.
  3. For the most part, people treat suppliers badly (quote from former colleague: “I love when we outsource this crap to somebody else. I can just dump all this shit on them and not have to deal with it.”) *
Is this really what you want? I say no. I say you don’t want internal customers as your model. I say you want team as your model. Put yourself in the mindset of all being on the same team. That for me is a far more pleasant, appealing and realistic model.

* OK I made that up. But it's entirely plausible is it not?

When average is good

Calling something average is one step removed from calling it mediocre. But in the mathematical sense, especially when applied to a team’s velocity it’s a good thing.

One of the certain tricky bits for any new team trying out scrum is establishing the team’s capacity. That is, without an historic record of how many story points of work a team can complete you are somewhat in the dark as to what to aim for come sprint one.

For a team that’s been working together in the past but not estimating in points it’s possible they may have a fairly accurate gut feel for things, and that can be your guide. There’s likely still the novelty of having not just to develop but also test, fix defects and truly complete (according to your definition of done) but, with experience of working together in the past a decent guess can be made.

For a newly assembled team however, or one that’s less confident in figuring out how much of the backlog to bite off initially, my advice would simply be don’t worry. Just pick more than enough and accept that you won’t make it all. From what I’ve seen the first few sprints are very much a learning experience and part of that learning is going to include figuring out your capacity.

Given just three or four sprints the team, with good coaching and guidance from a competent scrum master, will have figured out a number of recurring key items: big stories are bad, transparency between those that code and those that test is key, getting things “done” means limiting how much you work on concurrently and, ultimately, a reasonable average velocity to work with for planning future sprints.

The team I am currently working with had quite an erratic velocity initially: 16, 8, 49 (yes really!) and then 26 points. After this we have stabilized nicely with an average of 24 points a sprint.

This “we dunno, we’re just gonna try it and see” approach to working out a team’s capacity for quality work is inarguably logical. However it doesn’t necessarily sit well with those from a command and control background, especially those with management responsibilities for the team new to scrum. They like detailed planning. Predictability. Commitments. Maybe even punitive action against teams that “fail”.

This is completely the wrong approach.

Like it or not, this is when managers need to decide if they are truly supporting scrum or not. If you are, then step up. Provide the environment your team needs in this early phase of the adoption. Make them comfortable with the new regime of trying, reflecting, learning and improving. Without that support you will stifle their ability to rapidly improve, and in turn lead to a less than stellar scrum implementation. Maybe even to a failed one. This happens a lot, and people revert to traditional, comfortable but ultimately unsatisfactory approaches to building software. Don’t let this happen to your team. Create the right environment and harness the power of average.

Tuesday, November 23, 2010

What's the velocity of a turkey?

“What do you mean, an African or European turkey?”

Neither, I mean a North American Thanksgiving turkey. And no, this is not some kind of cheesy Python-esque skit. Rather, as my favorite US holiday approaches, a particular truism of turkey roasting occurred to me: your 12-16lbs bird will need about 15 minutes per pound at 325F until it reaches an internal temperature of 160-170F.

So that, I would say is the velocity of a turkey – 15 minutes per pound. And the internal temperature is your “acceptance criteria” since I’m doing dubious cookery/scrum metaphors.

Just like a scrum team’s velocity is what it is, the same is true of our turkey. You can try and "turn up the heat" to get it done quicker, but in reality the results will suck. Your end result will be all dry (for the turkey) and the product flaky (for your scrum software team).

So, to get good results, work with the reality of the situation. Plan based upon your known velocity. If you want 100 points worth of features implemented, and your team’s velocity is 25 points a sprint, reckon on it taking around 4 sprints. If your family is hoping to sit down and feast at 1pm get that 12lbs turkey in the oven with enough time for it to cook and rest afterwards. You just can’t rush it. If you have other features you want within the next 4 sprints you are going to have to make some hard choices about what you drop from your current plans. You simply can’t have it all. Similarly, if there are other things you need the oven for, plan accordingly. You can’t fit everything in there with the turkey all at once.

As much as scrum is about prioritization, it’s also about accepting the reality of what your velocity tells you can be realistically done, and working with that. That means good planning, and sometimes hard choices.

Wednesday, November 3, 2010

Highly Distributed Scrum

I'm just about to finish sprint four as the Scrum Master for a team that has a higher than average number of distributed members. And I don’t just mean that we got two offices involved, one in the US and one in Europe or India. We have three software engineers spread around the Denver Metro area, all working from their individual homes. Then we have our Product Owner in Billerica, MA. Alongside her are a couple of QA engineers. Then we have some folks over in Hyderabad, India: another software engineer and a couple more QA engineers. Finally we have me, working out of my home in the Rocky Mountains above Denver, CO. That’s six locations for just ten team members.

Add in some interested stakeholders in Berlin alongside those in the US and India and you can see what we’re up against. I’m sure we’re not alone in such a setup, but I suspect that having a number of people, including the scrum master, “decentralized” (my organization’s term for those of us that work permanently from our homes) is less common than a team made of people from two or three offices.

It’s been a fairly interesting experience, and not in the “OMG-what-a-total-disaster-I-have-a-dozen-more-horror-stories-to-take-around-with-me-warning-people-about-the-folly-of-distributed-scrum” sense.

One of the interesting things about this experience was how incredibly smoothly it went. I may just be one of the luckiest scrum masters in the world. I can’t claim superior scrum mastery as the root of this easy transition. The team was a well-established, mature team that had been working well together for a long time. The relationships were strong and team members trusted and respected one another. They had already introduced many agile practices way ahead of other groups within the organization. It seems clear to me that this helped immeasurably as we layered scrum on top of what they were already practicing. The other contributor to this, which I discovered serendipitously earlier this week on Twitter is explained in this excellent blog post by Bob McWhirter. In it, he makes the distinction between simply being a remote worker, and being a remote worker in a distributed team. In a truly distributed team there isn’t a single center of gravity where a ton of conversations occur that remote workers aren’t privy to. Due to people being spread out, a distributed team necessarily uses communication techniques that support remote workers, such as mailing lists, IRC etc.

I think the most interesting thing though is that we haven’t really seemed to need any special dispensations for what is a fairly unusual team configuration. No tweaks to scrum; no special tools, tricks, techniques etc. I’m not saying those won’t necessarily come as the team reflects more on areas for improvement during future retrospectives, just that we’ve been able to get going just by sticking to the basic principles. Our only accommodation has been for the people in Colorado to start a little early and the people in India to finish a little late (it’s a fairly brutal time zone difference).

So how are we doing it? Well, here are the basics:
  • We use Rally as our tool for agile project management, so that’s where our product and sprint backlogs are.
  • We use a wiki to organize a lot of other material. It all starts from a team home page which links to all manner of useful things: our definition of done, agreement on how we document stories, the build server, source code repository, released software etc.
  • We have a build server. This team’s current product is a rather old piece of legacy software that, presently, doesn’t lend itself to CI due to the lengthy build time. But it does still build every night.
  • We have used to do distributed, real time story estimation in points. It’s not quite got the natural feel and immediacy I would like, but it’s not half bad at all.
  • We have daily “stand-ups” held in the morning (US time) which is evening for team members in India.
  • Sprint planning, review and retrospective meetings are, similarly, timed for US AM and India PM.
  • We have a phpBB discussion forum for the team set up…we also have an email distribution list that includes all team members. Interestingly the distro list gets all the action. I’m not sure if that’s because people are less familiar with using the discussion forum medium. I think it might be because there’s a greater immediacy to email based communication.
Interestingly the team has, through retrospectives, identified all the usual early issues I’ve seen other teams come up with during scrum adoption:
  • Big stories are bad
  • Spread out completion of stories during sprint
  • Recognize and raise impediments ASAP
  • Need the right people at the sprint review to make it truly valuable
  • Be clear that we have commitment from people outside the team for stories that depend on them for success
  • We have technical debt and we need to pay it down
We have, of course, faced some challenges; the two key examples being:
  1. Meeting our commitments, i.e. getting stories “done done” by end of sprint. After a couple sprints we realized we would be much better off with smaller stories. Better to slip one small story than fail to finish two large ones.
  2. English is a second language for a number of team members. With more spontaneous verbal and (due to our distributed nature) written communication there has, understandably, been moments of confusion. This isn’t quick to fix, but with more and more practice things can surely only get better.
All things considered, I’m very pleased to be part of what’s shaping up to be a really great scrum team.

Now, I’ve argued hard against distributed scrum before. That piece was mostly borne out of frustration, seeing teams of two halves. I still stand by the views expressed there. Having your whole team co-located alongside your customer still seems, by my experience, the preferable situation. The really interesting lesson for me here though was how, with a truly distributed team, Scrum can be made to work better than I anticipated. Moreover, it seems that by adding more team members at the office based locations we might weaken, not strengthen, our existing team. Perhaps we would do better to let those who are currently office based work from home too. With the right people, it seems that perhaps a fully distributed scrum team could trump a team split between two offices. I remain skeptical that they could better a fully co-located team. That said, I wouldn’t mind the chance to find out. It’d be a fascinating experiment.

Tuesday, November 2, 2010

#Positivember and the response continuum

Over my first coffee this morning I happened across the following tweet from Corey Haines:
"Today is day 2 of a month long positivity-fest! Love your life: #positivember"
I almost feel bad for saying this, but it’s the truth: that handful of words irked me. Why? I think it’s because of my default interpretation. A month long positivity-fest sounded to me like some kind of rose-tinted denial of reality. Images of well-meaning but naïve parents that abound in a nearby town I’ll not name sprang to mind. The kind that work really hard to stop their children from expressing themselves when upset. The kind that want to insist that everything is always great. Even a cursory look at the psychological impact of that kind of suppression reveals that it probably ain’t healthy...

Then I read Corey’s blog post. It’s not (thankfully) quite that kind of hippy-dippy call to action. It’s much more an observation on how high levels of negativity are counter productive. How a concerted effort at positivity is the order of the day for advancing the notion of software craftsmanship. I’m up for that.

Still not sure I can quite buy into the notion of being completely positive about everything for the whole next month. Dropping the moaning and whining and negativity though is a good thing. It’s a bad habit I’ve had since forever. It’s easy to find fault and complain about silly little things.

A bit later, as I was walking the dog, an idea popped into my head that might help explain why the “all positive all month long” concept is hard for me. I believe gunning for all out positivity is an extreme reaction to the unpleasant other end of the scale. I think when we respond to something, that response lies somewhere along a continuum a bit like the following:
Sabotage → Meanness → Negativity → Apathy → Constructive Criticism → 100% Positivity
Reducing things to a binary negativity/positivity option over simplifies the matter. It’s the stuff to the left of the middle that’s toxic. To the right is great. I like to think I’m over there more and more these days, but still have plenty of work to do.

Not too long ago I read “The Lazy Manifesto” blog post by Leo Babauta on his website. Items number 5 – “Do less complaining and criticizing” and number 7 – “Do less judging and expecting” are, for me, a more palatable way to think about what I believe Corey is trying to achieve with #postivember.

I hope I'm not too far off track. Even if I am a bit, those two ideas resonate with me and since making a conscious effort to try and adhere to them I have to say things feel good.

Tuesday, October 26, 2010

Vive La Revolution!

In the agile revolution, are managers the over privileged aristocracy? And if so, why is nobody shouting “Off with their heads!”

I must confess there is a reason I’m partial to agile software development that might unnerve some people: I like its dark underbelly, that thread of anti-establishment flowing through it. You won't find that much on this explicitly in the Scrum guide or other key agile works. But hang out in the right places and you'll discover it's there.

Much agile thought has an egalitarian, meritocratic theme that recognizes the need for leaders in a team but simultaneously discourages hierarchies, superiority through tenure or job title and rattles the cage of old school authority. I love the shunning of traditional command and control management, the respect for people that do the work as competent individual professionals who can be trusted. And I share the distaste held by many agile practitioners for practices like detailed estimates in hours that are used to berate you later, ridiculously detailed timesheets, performance reviews and meaningless certification. These things have never sat right with me.

I’ve never been very good with rules mind you. I don’t like being told what to do or how to do it. I don’t like speed limits of 35mph when, clearly, it would be OK to go 40mph. I don’t like signs that say the park isn’t open until 6am (what if I want to go hike at 5.45am one summer morning?) I could go on, but you get the idea I’m sure… This is why so much of the thinking emanating from the agile community resonates with me.

All of which sentiment got me thinking about agile as a kind of revolution. And if it’s a revolution, who’s the aristocracy? Shouldn’t heads be rolling for the waterfall mess and soul destroying corporate drivel that preceded this?

Clearly the new nobility is management. But should their heads be on the chopping block? Take a look at this for a moment. It’s a spoof of the original agile manifesto put together by Kerry Buckley.

We have heard about new ways of developing software by
paying consultants and reading Gartner reports. Through
this we have been told to value:

Individuals and interactions over processes and tools
and we have mandatory processes and tools to control how those
individuals (we prefer the term ‘resources’) interact

Working software over comprehensive documentation
as long as that software is comprehensively documented

Customer collaboration over contract negotiation
within the boundaries of strict contracts, of course, and subject to rigorous change control

Responding to change over following a plan
provided a detailed plan is in place to respond to the change, and it is followed precisely

That is, while the items on the left sound nice
in theory, we’re an enterprise company, and there’s
no way we’re letting go of the items on the right.

It’s funny. Except it’s probably all too real in some organizations. And who makes it real? Managers. Managers drafting and implementing policy. Managers policing those policies. Managers punishing people who fail to adhere to the new policies.

But we know it’s not right. We know it’s not mean to be that way. Moreover, we’re empowered, self-organizing, self-managing teams. Do we need them any more? Can’t we vote these charlatans off the island?

Well, as someone who’s gone back and forth between individual contributor and management roles more than once I can tell you without doubt, the answer is no. Sorry about that. Yes, you really do need some organizational structure in most contemporary organizations. You do still need departmental directors and line managers and so on.

Regrettably there is an extraordinary amount of work to be done that is best suited to those outside a basic agile team. There’s financial stuff: setting budgets, planning growth, managing remuneration, capital expenditure etc. There’s medium and long term planning, setting strategy. There’s market research, marketing and PR for a group, internal and external. There’s HR, pastoral care, career development. There’s departmental coordination, networking, finding internal opportunities, connecting people together to achieve things neither could independently. There’s politics: negotiating for office space, people, budget, processes and practices that are fair and equitable. And there’s more than this too.

Could you turn every group in a large organization into an agile team, with their own backlog? Could you treat them all as peers and dispense with a great deal of hierarchy? Well maybe. But it’s a long time until something like that becomes the norm, and in the interim, even if we are in the midst of an agile revolution, beheading management is not the way to go, even if there are some that deserve it ;-)

Thursday, October 21, 2010

Seven Deadly Sins of Scrum

1. Wrath
Something that anyone on a Scrum team might succumb to with the frustrations of learning to work a new way. But this might be something the Scrum Master could be especially tempted by. As guardian of Scrum principles it can be frustrating sometimes to see people doing it "wrong." But it's important to remember that people will often learn best by doing, by earning the knowledge for themselves. Scrum is a huge cultural change and requires patience as well as guidance. Besides, the basics of Scrum are very simple and few things are truly wrong so long as you manage to keep to them. Even then, sometimes letting people step outside the framework for a bit so they can compare may help.

2. Greed
When the stakeholders or the PO start to see productive, reliable outputs from the team and take note of velocity it may be tempting for them to ask the team to do more than their historical accomplishments indicates is feasible. "Can't you just squeeze in a few extra points?" This must be guarded against, because allowing such a thing runs the risk of people cutting corners, or even gaming things  so it *appears* more points are being done per sprint by falsely inflating estimates. Trust the team to work hard at a professional standard and use velocity for what it was intended: predictably planning what will be done by release time. 

3. Sloth
To a large extent the whole-team emphasis on committing to a body of work and getting it done by end of sprint coupled with daily stand-ups leaves fewer places to hide for  slackers than a waterfall team. Nonetheless, calling out people not pulling their weight can be uncomfortable. Traditionally a persons line manager would be responsible for this, but in a self managing team there is the possibility of everyone holding each other accountable for producing. Sloth harms the whole team and cannot be ignored. Sometimes people need to be "voted off the island."

4. Pride
We should always take pride in a job well done, but this should be a shared thing. A scrum team thrives on respect for and collaboration with each other. Ego-centric prima donnas have no place. It’s a team thing. Similarly, whilst great talent is always appreciated, singling out "rock stars" and putting them on a pedestal is counterproductive.

5. Lust
Yeah, um. Usual dating people at the office advice stands methinks.

6. Envy
Yeah that other Scrum team may have their own team room/bigger whiteboard/better computers/more interesting product... Whatever. Find the interesting angle in what you’ve got, work to improve what needs improving and stop coveting thy neighbor’s 27” dual monitor set up.

7. Gluttony
Sprint planning can take a while. With inexperienced teams I've seen it start with a late breakfast and run past lunch. Too many bagels with lashings of cream cheese, Danish pastries and  pizza for lunch will catch up with you. Perhaps it's a good thing you will be sprinting afterwards!

Tuesday, October 19, 2010

Requirements are dead. Long live requirements.

One of the first “Agile” books I read that really inspired me to change how we went about creating software was Mike Cohn’s User Stories Applied. I think it’s because, perhaps more than anything else in my career in software development, I had seen problems that boiled down to “requirements problems”.

We were really suffering from some of the classic problems mentioned in Mike’s book including:

  • People not really understanding what needed to be built or why, but just focusing on making sure the software did what the requirements said because the requirements couldn’t possibly be wrong...
  • People obsessing over where commas went (when ironically the basic words in play were not always comprehensive enough for us to worry about punctuation that much)
  • People testing what they thought the requirements meant, rather than what the customer needed.
  • Requirements fatigue – huge weighty tomes of requirements typically starting “the system shall…” and yet rarely  any decent narrative providing an overview. This kind of stuff would put an insomniac to sleep.
  • Ambiguous wording, cryptic wording and just plain iffy writing: many times things were being phrased  to suit a business analyst, software engineer or tester’s view of the world. Often this rendered the document almost impenetrable to our customers who we expected to sign off on it(!)
  • Conversations and emails that involved convoluted phrases like “…but don’t you realize that requirement clearly means that in this context…”

There was so much in that book that had me nodding my head and thinking “Yep!” with every page. Of course criticizing traditional style requirements documents isn’t hard. I don’t know that I’ve ever met that many people who don’t have a story or complaint about how bad they can be. They’re an easy target and I have something more useful (I hope) to say. I’m not going to rehash what user stories are or how they may benefit you. If you’re not already familiar with them I heartily recommend that book mentioned above. For a more immediate introduction hit up Google and you’ll find plenty of material.

What I want to talk about is how to balance working with stories with environments that are looking for something a bit more traditional when it comes to requirements specifications.

The industry I work in provides services to the pharmaceutical and biotech industry. The work we do is subject to FDA guidelines and the audits (from internal quality groups, customers and potentially the FDA themselves) that go along with them. This has bred a certain set of expectations amongst auditors, not least that there will be a classically styled requirements specification from which they can trace design, implementation and verification and validation of features. Overcoming this expectation may eventually be possible. Perhaps even as soon as one, maybe two decades ;-) but for now trying to do so is tantamount to pushing water up a hill with a rake.

Given this situation, we (and I strongly suspect we are not alone here) still need to produce something that we can call a requirements specification, print out and sign and show some kind of traceability to our tests. There have been a handful of ideas over time on how we might do this. Different Scrum teams in my organization have tried different things, but I’m not sure that any of us yet have got this completely licked.

The first approach was really just business as usual. That is to say that although we had a backlog of stories, they were really just placeholders for work. We still captured the detailed aspects of what we were building in a traditional Requirements Specification. Our “Definition of Done” actually included “update RS” as an item for every story. This idea more or less worked, although it felt a little unwieldy. All the classic problems of big RS documents remained with us, we just bit things off a story at a time which definitely aided us in some ways (not least by eliminating the need to write the entire thing and sign it off before beginning to work).

We also flirted briefly with use cases, which if you do even a moments research, will allow you to find various “templates” and so forth for authoring them. The problems we had here were twofold: Firstly, a good use case is actually larger than a story. So with us trying to take story-sized pieces of work and describe them in use case format things got kind of artificial and we created more work for ourselves. If we’d carried on down that path we would have had innumerable documents, since each use case was its own Word file. Secondly, use cases seem best suited for very GUI centric functionality with obvious human “actors” interacting with the system. For web services and other non-GUI work they seemed quite hard to do well, seeming to come out quite convoluted. Sure, you can have the other system as an actor but nonetheless things came out weird. Maybe we sucked at writing use cases, but in truth I suspect they just really weren’t the right tool for capturing story requirements.

Around this time I had just finished reading Gojko Adzic’s Bridging the Communication Gap which introduced me to concepts like specification by example and acceptance testing. These ideas resonated with me and seemed like useful, sensible things we could try out. As an alternative to the use cases we were writing I proposed a story + acceptance criteria with examples template (heavily influenced by Gojko’s book) that I thought we might try. In actuality this never caught on. Perhaps it’s just as well because with hindsight, the idea of requirements scattered across hundreds of documents violated one of my own big pet peeves about our prior approach: no good narrative providing an overview of different product areas.

One reason that idea may not have got traction was perhaps due to the fact that we had started to make use of the Rally tool for agile project management. I’m actually not a big fan of Rally’s tool (more on that here if interested) but nonetheless, in our "new toy fervor" there did seem to be an exciting chance to do away with Word document based requirements altogether. In our environment this was a radical proposal which I firmly supported…if only it weren’t for those auditor types who want to see one. As a response to this the idea of exporting story details from Rally emerged. This made sense at the time too. One of the pieces of advice I’d seen from several people was, “If you need to keep a written spec, just keep the story cards” (or in our case, the story records in Rally). Rally’s product has a set of web services and various ways to interact with them, including a simple Ruby script for importing and exporting data. It does work, but it’s pretty simplistic at this time and requires quite a lot of effort to convert the CSV output into anything resembling a normal looking requirements document. It would certainly be possible to hack away on that script or create your own variation on the theme that was a bit more geared to the specific task of creating this kind of output. But, I’m no longer convinced that’s what we want to do.

Most recently, in anticipation of a brand new product starting I’ve been reading about and dabbling in behavior driven development (BDD) and acceptance test driven development (ATDD). In particular I’ve been looking at Cucumber and Cuke4Nuke (which enables one to write “test fixtures” in C#). See here for my first post after encountering Cucumber. Something this exposed me to was the “Given/When/Then” (G/W/T) form of expressing requirements. Even without automating your tests and doing full ATDD this simple approach to describing requirements seems to be pretty powerful. In many ways the G/W/T stuff seems to stand on the shoulders of use cases, but feels more lightweight, simpler and provides an easier time of things when it comes to splitting features into smaller parts.

One thing that I thought had lodged firmly in my mind after all this was the idea of tests as your specification, or “executable specifications.” The concept is simple: if your tests are readable and express what the software should do, then the running of them indicates what it actually does and you’re never playing “catch-up” with an old school requirements specification.

And yet…clearly this idea hadn’t stuck. My team was discussing last week whether we had to keep our RS for our legacy product and whether we could maybe forgo such an artifact for the new product we will be starting soon. We got onto talking about how we couldn’t just document things in Rally because the export wasn’t good enough for what we needed. I resolved to try and do something about this by investigating the export script. And I was doing so when I realized that this was completely the wrong problem to solve. I hadn’t seen it as clearly before, but I realized then: stories are transactional. They only make sense if you review them chronologically because story #42 can completely modify the feature you put in through story #7. Keeping your stories and transforming them into some kind of document as a surrogate requirements specification isn’t the right approach. Instead, we should be transforming our tests into a specification because they represent exactly what the product is meant to do in a much more succinct way than a huge transactional list of stories.

Of course to do this one needs tests that are human readable, and not just by humans that can interpret code. This brings us back to the Given/When/Then format, because it sits natural language descriptions on top of automated tests. And, lo and behold, it’s entirely possible to very easily transform your Cucumber feature descriptions into various outputs, including a PDF.

When we first embarked on transitioning to agile software development I wrote a series of four blog posts about our early experiences. I concluded with looking forward to where we would need to focus next. I said at the time I thought requirements would be a big deal for us. It’s taken longer than I anticipated, but I think now we can start to see a path forward.

We don’t want to have a big old requirements specification that we maintain. We do need to have something that helps us meet our regulatory obligations. ATDD and the G/W/T approaches can help us with this. We can generate what we need as a natural part of a work that’s far more integrated into software development than a traditional document ever could be. Requirements are dead. Long live requirements. 

Tuesday, October 5, 2010

Commitment: what can we really commit to?

I want to discuss a foundational topic that influences the management of software development: commitment. In particular I’m looking at this from the perspective of a business that develops software not for sale, but as an enabler to their operations. As such a business we commit to customers that we can do things. That we can do them by a certain date. That we can do them to a certain level of service, or quality. These commitments cascade down as implicit obligations for the operational teams that conduct the sold work and those that support them.

Except…in reality, no matter how much we want it to be so, software development teams creating products can’t really commit at this level when we have no idea what promises are being made or complexities are involved.

I know. That sounds pretty bloody useless. But it’s true. And to think otherwise is naïve optimism that is just waiting for disappointment.

What we can commit to is releasing working software regularly. In fact, when you’re working to Scrum principles, at the end of any sprint the software should be “potentially releasable.” If you can handle it that often we’re ready to go.

In many businesses dates are already determined, and features are already determined. Given that resources (that is, the people available) are also largely determined (“no you can’t hire any more people, make do with what you’ve got”) it’s rather apparent that all three variables of software development are being fixed. (We’ll assume for now that the fourth variable, quality, is considered non-negotiable by the team developing software, i.e. they as professionals will not compromise quality as a response to the inability to negotiate dates, features and resources.)

Fixing all three like this can work with a great team and a bit of luck. However, given a bit of bad luck or changes in staff that inevitably occur over time and problems will likely arise. Whether we acknowledge this and try anyway, or wait until disaster ensues and then start investigating misses the point: this is not a reliable way to develop software.

There is a “better way”. This “better way” however requires a bit of a mind-shift for those used to the more traditional approach. At its heart is prioritization. This is where things can get difficult for people who are commissioning work. It’s natural to want to say “Hey I just want it all, and by March too, we’ve sold it so it’s basically promised. Just find a way to make it work.” For a business to allow people in this position to operate this way is, in my opinion, a little bit like the proverbial Ostrich burying its head in the sand.

Indulge me by considering the following contrived little story.
If a tap* can, when fully open, fill a 5 gallon** bucket in a minute then that is just a simple fact with which you have to contend. No amount of wishing it could do more will make it so.
Further consider: you have two buildings on fire, burning away, your garage and your house and that you’d like them to be put out. The firefighting team can deploy the water anywhere. Now ordinarily you wouldn’t get to influence what firefighters did. But imagine for a minute that you can. Further imagine that the garage contains a stash of priceless artworks, but that the house is empty. Left to their own devices, the firefighters would probably think they were doing the right thing by concentrating on the house. But YOU know better. Obviously you would have them put every last drop of available water into preserving the garage. You realize there isn’t enough water to tackle both burning buildings. You realize you can’t have them save both the priceless art and the house. You realize that demanding the firefighters save both buildings ain’t gonna happen. You prioritize, and save the art.
In other words, you have recognized that You need to decide how much water to put on which fire when, and you need to accept that you can only get 5 gallons every minute to distribute amongst them.

Once you’ve realized this is the way the tap works, and there isn’t anything you can do to make it spit out water faster then you’ve learned one of the most valuable truisms about software development. You can only do so much with limited resources. To make best use of them you need to prioritize how best to deploy them. And if you want to do more, you need more resources.

* or faucet if that’s your thing
** or, again, 18.92706 liters (or litres) if you must – as you can see I’m conflicted and inconsistent with both terminology for everyday items and units of measure after living in the US for 8+ years

Friday, October 1, 2010

Delayed Gratification

(Or, "Why I believe fixed length release cycles are best")

Anyone who’s done some kind of Scrum reading or training classes will have had some exposure to the release planning techniques and ideas involved. There is plenty of information, ideas and opinions out there about that stuff, in particular how to organize your releases around themes, epics and stories, how to prioritize your backlog and how to run a release planning meeting. I, however, have been thinking a bit about things one level up from the mechanics, and here wish to discuss some of my opinions and ideas on how a business should look at managing releases. In particular I’m interested in how to balance responsiveness to changing needs and predictability on longer term commitments. I’m also interested in simplicity and optimizing the way product development works. Agile software developments offer the promise of delivering high value sooner, of responding to changing needs. But in order to deliver on this promise it is my belief that certain preexisting attitudes and tendencies that are ingrained in many of us need to be reconsidered.

In scrum, at the end of any sprint the software should be “potentially releasable.” Usually we choose not to do that though because:

  1. There isn’t enough value to justify the “cost” (deployment, distribution, training etc.) of releasing
  2. Customers (users too!) aren’t necessarily best pleased to have their software in a seemingly constant state of flux.
An easy way then to manage releases is to let business needs dictate release dates. This doesn’t require us to have any new special rules or planning processes, and it sounds reasonable: “Yeah we’re gonna need that BIG FEATURE out by CERTAIN MONTH for IMPORTANT CUSTOMER, so we should release then. And then we’ll fit in whatever else from the top of the backlog we can.” Sounds pretty good, right?

Ken Schwaber’s Scrum guide, in describing release planning meetings says nothing more than that they establish a “probable delivery date and cost that should hold if nothing changes.” He doesn’t seem to be advocating anything different.

But is this optimal? Are there potentially better alternatives? I think so. Here’s how I think it should be done and why.

In a nutshell I think we should take a leaf out of the finance and accounting book and use a year chopped up into quarters (i.e. 3 month long chunks) for planning and release purposes. In other words:
  • I wouldn’t much bother trying to look further than a year ahead
  • I would [try and] release every quarter (maybe sometimes you can skip, although I think if you have your deployment act together this is unlikely)
  • I would encourage the business to plan things around this quarterly release schedule…that is to think of things in terms of “I’d like this in the 1st quarter release” or “I can wait until the 3rd quarter release,” not “I want this in October.”
Why? Well, hopefully the idea of not looking more than a year ahead is not too controversial. All I’m saying is that things change so much in 12 months that looking further ahead is usually crystal ball gazing. That’s an idea you’ll see in many communities, whether agile, scrum, lean startup or whatever. I think it makes a lot of sense in many places, certainly in Perceptive MI.

Also fairly easy to swallow I think is the notion of thinking in 3 month chunks. Three months is around 12 weeks, or 6 two-week sprints. It’s long enough to get something of substance and value done, but a short enough time-horizon to keep everyone focused and avoid the risk of over-committing badly. It’s also probably not too long to wait for a new feature. Of course unexpected needs and critical defects occur, and interim or maintenance releases are always possible, particularly when using Scrum, since we’re potentially shippable at the end of a sprint.

But this notion of rigidly sticking to a plan of releasing every three months…I know that is a little different and needs explaining. Well, for starters it provides a nicely time-boxed predictability. There aren’t so many surprises. There isn’t the need to frequently revisit and negotiate the question of when releases will occur – everybody knows, it’s once a quarter.

This predictability is not just comforting and simple, it provides other benefits too. For starters it makes it easy to handle certain common situations. Let’s consider four:
1. The madly urgent problem/bug in production
Everyone is familiar with this situation. One moment everything is calm and ticking over nicely, the next minute chaos abounds: a bug without workaround has been found, or a really important customer has to have something and only just thought to ask.

Either way, within our proposed framework dealing with this is straightforward. There are really only two choices. The first question to ask is, “Can this wait until the next sprint?” The answer is going to depend on how truly urgent the matter is, and how long it is until the next sprint begins. With two week long sprints it’s never far away. Few problems require more immediate attention. For those that do, the team would simply abandon the current sprint and focus on the urgent issue at hand.

2. The unexpected feature
When planning for a new three month release cycle you usually have a fairly good idea of what you want to get done in a timeframe that short. Nonetheless, there are a number of very good reasons why a feature that nobody was aware of then can suddenly emerge and need attention.

This is not really a problem though when using Scrum. The Product Owner (and by extension stakeholders and other interested parties) are able to completely change the priority ordering of the product backlog every two weeks. Simply bump the new feature up to the top and it can get all the attention it needs at the next sprint. This means that, worst case, the business waits two weeks for things to start on something that was previously completely unplanned.

3. The client driven enhancement
Somewhere between the unexpected feature and pure market driving planning lies the customer driven enhancement. The distinction here is that one has the capacity to pick and choose whether to bump out current plans and place it into the release cycle that’s in progress or negotiate as needed to place it in the next.

4. The market driven plan
Above the cut and thrust of weekly operational driven needs and urgent customer requests is the longer term approach to product development. This is simply identifying desirable features based on market trends and selecting appropriate release cycles to implement them in up to a year ahead.

You see how with all of those there’s an established pattern and we work to fit things in? This reduces or even eliminates the need to frequently invest energy into discussing and negotiating with colleagues outside the Scrum team (IT, domain experts, business development, professional services etc.) over when a release will occur. The schedule is published and everyone can consult it and even, with time, comes to anticipate it. Things will happen without energy needed to push or chase it: IT can grow to anticipate doing deployments for example. Business development can talk confidently about the likely lead time for including new features to service particular customer needs.
Finally, ultimately, it encourages better planning and discipline on the business as a whole, from sales through to professional services and product development. I believe this leads to a better run business, better thinking, less burn out, and people selecting the right opportunities to pursue. I cannot see how that is a bad thing :-)

For me, this approach to managing the release cycles for a software product intuitively makes sense. The combination of simplicity, predictability and responsiveness that can be achieved is highly desirable and a worthwhile reason to move away from the more traditional approach to variable length release cycles driven by external events. What do you think?

Thursday, September 16, 2010

Scrum Master: Tools of the Trade

(or "How Rally Could Improve Their Software")

When a team is employing Scrum to project manage their software development efforts one facet for consideration is how to keep tabs on things: the backlog, the stories and associated acceptance criteria, the current state of affairs and so on.

The things one uses to do this are the “tools of the trade” for a Scrum Master, indeed for the whole team.

Much advice is given about keeping this simple. Much advice is given about the idea of visual management techniques including “information radiators” that constantly, clearly and loudly proclaim the state of affairs. These should be intuitive to read, easy to work with and not intrusive.

The quintessential approach to this is the use of a team area with a task board of some kind along with perhaps a burn down chart. We've done this at my company in our early days of agile adoption and it worked extremely well. The main benefits I saw were:
  • You can take in the state of an iteration in a glance. For instance, got most of your Post-its on the left and it's near the end of the sprint? Looks like trouble...
  • You don't really have to train anybody how to use it, it's dead simple
  • People not intimately involved in the project day-to-day (read: management and executive sponsor types) can quickly "read" it and get value from it
  • It's quick and easy to work with...adding a story, task, note, changing status...
  • It's always present, you don't have to "go" to a website, tool or what have you
Of course, it works best when you have a fully co-located team. When there are team members that can't visit the task board in person then this approach weakens considerably. This is one reason why a team may turn to a tool to aid them in creating a virtual task board. Part of me feels that this is treating the symptoms rather than the root cause...that is, you should look again at whether you really need to have a team spread out over different offices. However, I also recognize that that optimal arrangement isn't always practical, whether for financial or political or other reasons within an organization. Sometimes your talent is spread across the world and you can't bring it together. Anyway, I digress...

The other seemingly valid reason for a tool to replace the simple white board and Post-its I consider to be related to managing a group of teams or a portfolio of products. There will be very legitimate needs to roll up the status associated with individual software development teams to obtain a higher level view of the work of an engineering department. If teams are in different offices or countries, it's clearly impossible to run from one team room to another to update oneself on how all the teams and products are doing.

So clearly there are at least a couple of reasons why you may reach a point where you need a tool to help your agile teams coordinate and track their efforts.

My company reached this point. We chose Rally.

Here's a quote I happened across on the Rally Products Overview web page (it doesn't always show up, seems they cycle through various quotes) -- “The usability of Rally is one of its main selling points.”

Now I'm sorry, I don't mean to offend either the man being quoted or the people who've worked hard at Rally building this software. But it's far from a shining example of highly usable software.

It's my assertion that information radiators like a task board that a co-located team would use are basically the Gold Standard.

So, if you’re going to try and create some software that can be used in place of this then you should try damn hard to get as close as possible to that Gold Standard. I know there are other issues like concurrency, responsiveness, reporting, user management and so forth. But at the heart of a bit of software like this, it is my belief at least, should be a tool that empowers a team, not a tool that is awkward to use and gets in the way.

So what's my beef with Rally usability? Well, having thought on it a bit I believe my criticism falls into three main areas.

1. The Software is not Task-Centric

And when I say this, I'm not talking about the tasks one can break stories down into. I'm talking about the fact that different roles on a Scrum team regularly and repeatedly engage in certain tasks but that there are no options in Rally that particularly support or enable these. Instead it feels an awful lot like a small set of database tables with some views layered on top and then exposed as HTML. Various ways to slice, dice, subset and drill down my releases, iterations, stories and so forth is fine, but I'm looking for something more.
  • I want a tool that supports the Product Owner in release planning.
  • I want a tool that supports the Scrum Master in sprint planning.
  • I want a tool that supports the team on daily stand ups.
  • I want a tool that supports the team during sprint reviews.
  • I want a tool that supports the team during retrospectives.
Rally does of course contain various list views and so forth that help with this, but it seems to me that none of them are really conceived to aid any of these roles in any of these tasks. It's more a small set of Lego bricks you can arrange as you see fit to achieve your ends. And yet it would be so much more useful it the "bricks" were assembled into larger constructs that supported the various Scrum ceremonies and tasks that people on the team perform.

2. The Software Looks, um, Less Attractive than I Would Like.

I know people have worked hard on this, and I don't mean to completely disparage their efforts. I'm not even saying I could necessarily do much better (although I'd like to think I could...) but consider:
  • In general, there's a "designed by an intern programmer circa 1996" feel to things
  • The Dashboard looks like Jetspeed 1 That's not a good look for anybody.
  • In general, screen/form layouts are pretty unattractive and hard to scan: content is not well bounded, poor typography, colors, etc. Take the basic detail view of a user story, or the pop-up edit a story view…labels not emboldened, field boundaries unclear, it just looks like my 12 year old designed it after reading "HTML in 24 hours".
  • Look at the new Reports --> Epic Progress app that was just released. Was somebody really happy with how that looks? If I was Product Owner I surely would not be.
  • Why does editing a story have to spawn another browser window? Why not in-place? Rally allows a certain amount of in-place editing in list views so why not the full form? Nobody wants extra browser windows hanging around…
  • Speaking of in-place editing, there's a disconcerting "twitch" or "jump" when on the Plan --> User Stories view and you in-place edit something.
  • The tabs don’t stand out enough so it’s hard to “read” where you are, especially since so many “views” look so similar. There's not enough contrast with almost everything being in very drab black/white/blue. Use some more of the Rally Red maybe. Also I find the tabs are not very responsive, they “lag” as you move over them.
  • Why is the icon you can click and drag to reorder things different between Plan --> Backlog and Plan --> User Stories?
There's more, but that's a good representative selection of the things that strike me as looking unattractive.

3. It's Just Not Intuitive

I know agile and Scrum pretty well. The software then ought to be fairly intuitive to work my around. And yet it isn't...
  • It's Confusing how the backlog tab only shows things that *haven’t* been assigned to a sprint and/or release (can’t even remember which, but stuff disappears from here)
  • It's confusing how story hierarchies are shown in Plan --> User Stories but not Plan --> Backlog…in fact the whole parent/child thing is confusing. Why not have first class support for the common theme --> epic --> story structure?
  • Plan --> Releases gets its own tab just for defining a simple release "record"? Navigation from here to see “stuff” slated for a particular release is non-obvious. Could be much nicer.
  • “Page Tools” vs. “Actions…look different, kinda confusing…
  • Column headings etc. look's just not clear that they are column headings.
  • Plan --> Plan … really? What does that mean?
  • Track --> too many options. None seemingly optimized for daily stand-ups which is the #1 tracking activity in Scrum.
  • The "Task Board" doesn't work unless your stories have been broken down into tasks. OK, maybe that's why you call it a task board, but not everybody sees value in further decomposing stories. I don't, rendering the task board useless. Even if I did want to break stories down into tasks the view doesn't lend itself well to, say, sharing via WebEx for a stand-up with a distributed team.
  • The URLs are useless: I can't copy/paste a URL and IM it to a colleague. No matter what I'm doing the URL is always
OK roasting more or less over. Given that the product is a tool for use by agile teams and that it's developed using agile principles and practices, I'm not convinced it's an especially good "advert" for agile. I'd like to think that if Rally were being truly effective here then there would be a much nicer product as a result.

This is of course all just my own personal opinion, and I freely admit I have not worked with the product for that long. Maybe I'll grow to love it and think it the best thing since sliced bread. Presently however I do not. I think there's a lot of things that could be done to improve above and beyond adding things like the Epic Progress app (do we really need yet another view of progress?). I suspect this scathing rant bars me from applying for the Product Owner job in Boulder ;-) but if I did have that job, these are the things *I'd* be prioritizing high on my backlog. I note they are also looking to hire a Usability Analyst which has to help, although they're only looking for someone with 2+ years experience and I wonder if they don't perhaps need someone a tad more seasoned...

Monday, September 13, 2010

Scrum, Crop Rotation and the Fallow Field

Since the dawn of agriculture (well, maybe…certainly some when in the last 10,000 years) farmers have realized that growing the same crop in the same field, year in, year out depletes the nutrients in the soil. Eventually this leads to the field failing to produce as well as it used to, or even becoming unusable.

One might see a parallel here with scrum teams: although the rhythm of short release cycles, time-boxed sprint planning meetings, daily scrums, sprint reviews and retrospectives provide a stability, might they not also eventually lead to burn out and reduced productivity? I venture to say that they might.

So can agricultural techniques offer any further parallels that might inspire us with ideas for how to help keep a scrum team producing? Well, with the employment of a little imaginative (some might say dubious) metaphor, I believe they can.

First, consider the idea of leaving a field fallow: the idea here in farming terms is that one might operate a three field system, with only two fields at a time being planted. The third will be left empty, or fallow so that it might recover. Things shuffle around so that each season a different field is left fallow.

So what would it mean to leave a scrum team fallow? Simple. After a successful release, have a sprint with no formally planned activities. Sound wasteful? Consider that this would allow for the team to celebrate the release, decompress from the work and general renew their energy levels. In addition there could be some planning activities or less rigidly time-boxed investigatory work in anticipation of the next release cycle.

But there’s more. According to Wikipedia, the Romans and several civilizations in Asia and Africa used more sophisticated crop rotation methods to keep their fields productive without having to leave them fallow so often. In these schemes, by carefully selecting the right crops they were able to restore the nutrients depleted in one season with soil enriching crops subsequently.

I think the parallel for a scrum team is the idea of working on something different to the main product they are responsible for during a release cycle. This might be helping out with a different product, pursuing the development of some tool or infrastructure they think would help the team in future, training or other creative ideas.

As a child I can remember my grandmother offering up many pieces of advice, including the notion that "a change is as good as a rest." Personally, I’ll take a rest over a change any day, but in truth both work. And there’s no reason not to employ both as productive investments in the overall health of a scrum team between release cycles.