Coding When High

DICOM non-technical introduction: mind map

2011-06-07T14:48:00.002-06:00

Earlier today I went through the non-technical introduction to DICOM on the RSNA (Radiological Society of North America) website written by Steven C Horii, MD. As I was doing so I compiled a mind map using the excellent SimpleMind tool to help jog my memory on this stuff in the future.

Click the image below for the full size PNG or it's also available here as a PDF.

Understanding progress: on points, velocity and when to add new stories

2011-06-06T11:33:00.004-06:00

Building software takes time. Usually enough time that people are interested in monitoring progress and understanding when it will be done. Agile teams often use User Stories as a unit of work. Typically these are estimated in points enabling a team to record their velocity, that is, the number of points completed per sprint.

With this data, teams have a simple means to show progress. Interested parties can follow along quite easily, seeing how each sprint eats away at the features in the backlog and how many points are left to complete the product. Using the team’s average velocity will give a nice indication of how many more sprints are required to complete the features in the backlog: points remaining ÷ average velocity = sprints remaining.

There is one wrinkle to this otherwise simple scheme. As anyone who has developed software can tell you, the devil is in the detail. Something that initially looks simple can end up being more involved than originally estimated. Case in point: my team recently had a story like the following for a desktop client image processing product they are building:

 
     As an imaging assistant
     I want to be able to open JPG images
     So that I can process them

The story seemed to be straightforward and was implemented easily and quickly. All was well until we tried some particularly large JPG files. At this point things blew up with out of memory errors and the like.

So was handling large JPGs a new feature and thus a new story? Or was it obviously part and parcel of the original? In other words, the question is how do you deal with this and still report easily understandable progress?

The short answer is it doesn’t really matter. You can add new stories to the product to cover the new work that you discover. Or you can just do the work that was implied but not necessarily obvious from the original story. The formula for figuring out how many sprints remain still works either way. Taking the former approach your velocity is likely to remain fairly stable. Taking the latter approach you may see it bounce around a little bit more, or see it dip from historic levels if the team had a velocity established through more predictable types of work (e.g. maintenance on a well known product.)

The longer answer is that each approach has its own pros and cons. Depending on your situation one may be better than the other.

Adding a story, pros:

Stakeholders can see the amount of work we original imagined was involved has grown and have more chance to comment on the necessity of these items – perhaps the complexities and edge cases discovered aren’t that valuable.
It probably shouldn't matter, but velocity remains stable and nobody feels the need to cross-examine the team and ask "Why has your velocity dropped?" Done this way velocity may even serve as a crude indicator of team performance and improvements can be seen in higher velocity.

Adding a story, cons:

It probably shouldn't matter, but there's likely a class of stakeholder that will question why all this "extra" work is emerging and ask how come we failed to identify it in the first place.
If the team has the pleasure of using a computerized agile lifecycle management tool then each additional story is another thing to enter, estimate, prioritize, track and update status on etc.
The number of points to complete the originally envisaged release keeps growing making some people anxious: "We don't know how much work is left to do, how can you ever hope to predict when it will be done?"

Not adding a story, pros:

Nothing extra to track (though we probably need to clarify the acceptance criteria of the original story)
Number of points to complete original feature set for the release remains the same (unless we discover a need for genuinely new features)

Not adding a story, cons:

The potential roasting of the team: "Why is your velocity erratic/dropping?"
The team might miss an opportunity to push low value work down the backlog.

Personally I'm most strongly drawn to the idea of not adding in extra stories. I like the simplicity and minimalism of this. The more items there are in the backlog the harder it is to grok the thing as a whole and more busywork goes into managing it all. I don’t think the “pros” of adding in extra stories are powerful enough to make it the preferred approach. And although there is the potential “con” of the team getting questioned about why their velocity is erratic or dropping, I believe this can be explained quickly in simple terms. I also think it’s a lot more straightforward, even comforting, for stakeholders to see that the size of the backlog remains fairly stable unless things they too understand as new work (features) get added.

Enemies of Agility: The Dirty Dozen

2011-05-18T14:02:00.003-06:00

Developing software is hard. Agile software development represents one way to deal with the complexity and difficulty in a manageable fashion. Arguably the best way we've figured out so far. But it's not easy, and there are many impediments on the road to becoming a highly adept agile team. Below I present the "dirty dozen" impediments, or key enemies of successful agility that I've observed.

1. Management doesn’t understand agile
An individual team or two can get quite a lot of improvement by themselves. But to really scale up agility in an organization requires a deep understanding and commitment to the guiding principles from managers across a range of functional areas. Attaining that in a large corporate environment is surely hard. Even among the engineering or IT groups that might have more capacity or interest to “get” agile at a management level there are challenges. How can people many years distant from the act of actually creating software really understand what changes are occurring on individual delivery teams if they haven’t experienced them first hand themselves? How can they help shape and encourage greater adoption and performance, policies and culture if they are not actively engaged in a deep meaningful way with agile? How can they negotiate effective cross-department agreements when they don’t really understand how their teams are working?

It’s hard. Not impossible though. What it requires is a willingness at the management levels above individual delivery teams to immerse themselves in as much agile as they can. Go to conferences. Get the deep dive that your two day agile orientation course didn’t give you. Read books. Lots of books. Join mailing lists and discussion groups. Read the blog posts people put out there. Follow the thought leaders on Twitter. Participate on a team for an iteration – there lots of ways for people of different skills to do this. Pair with the PO to groom the backlog. Pair with a coder and/or tester. Do some exploratory testing, etc. etc.

Once managers have experienced being and doing agile they have a much greater chance of extending those principles up and across the organization.

2. Team doesn’t understand agile
Understanding agile requires more than sending one (or more) of the team off for a two day Certified Scrum Master course. And it’s considerably more than saying “go do agile.” I’ve talked with lots of people for whom this is all they’ve had, and it’s clear that the depth of understanding is far shallower than it needs to be to really improve.

From my experience two things are required for a team to really understand agile deeply rather than just go through the motions. Firstly, as suggested above with managers, people on agile teams need to really immerse themselves in the wider agile world. There is an immense volume of great stuff to discover, read, try and learn out there. Harnessing that will help a team understand what other people are doing and being successful with. Secondly, the team needs a coach, at least at the outset. The coach needn’t be an externally hired consultant; it could simply be an experienced scrum master or effective manager. Someone who can help guide the team away from common pitfalls and remind them of common pitfalls.

3. Team doesn’t understand end users
How can you make intelligent decisions about the software you are building and testing unless you have a deep understanding of the way it is used? Not understanding the product can lead to several problems: PO or some other key figure becomes a bottleneck; requirements become gospel and no intelligent interpretation is allowed; stupid things get done because people misunderstand needs; etc.

Depending on who and where your users are vis-à-vis the team this may or may not be easy to address. Ideally your users are accessible and team members can spend time with them regularly to understand better the world in which they work. Where this isn’t possible some effort should be put into finding ways to make sure the team understands the users as much as possible.

4. (Micro)managers who’re not on the scrum team interfering
This can be a particularly debilitating and frustrating class of dysfunction. How is a team meant to develop the ability to self-organize and self-manage if team member’s managers regularly disrupt matters?

This can manifest in a variety of ways:

Dictating how work should be done
Dictating priority thus undermining the PO
“Refining” or challenging estimates
Focusing on velocity over quality and predictability
Injecting additional work
Making promises to external parties without consulting the team
Having conversations that influence technical decisions absent the team
Confusing stakeholders by offering an uniformed, contradictory and out of date view of status

Managers may indeed have a lot of experience to offer to their teams, but micromanaging interference will sap teams’ engagement. Managers need to realize that they should go about things in a less directive fashion than they may have previously. Rather than push down decisions and work, managers need to harness the talents of a team and present the results up and out to the rest of the organization. Teams typically have (or can develop) regular and agreed upon venues and approaches for managing the how and what and when of their work. Managers who wish to influence things must utilize these to obtain team buy-in and agreement. Failing to do so invites resentment, apathy and underperformance.

5. Inappropriate or suboptimal tools
This is a problem that occurs more in larger organizations where well meaning people want to “standardize” on things. Things like last decade’s monolithic proprietary testing frameworks or agile lifecycle management tools that get slickly pitched to executives offering alluring reports to analyze all their staff but offer limited useful functionality to the actual teams using them day in and day out.

I’m not sure there are any good immediate ways to address this kind of problem. Perhaps when management obtains a deep understanding of agile principles such issues will diminish. In the interim, if the subversive option of simply going your own way, irrespective of corporate mandates is a possibility, I say go for it.

6. Poor technical and problem solving skills
Building anything but the most trivial software is hard. It requires good people. People with excellent technical and problem solving skills. This is as true of traditional approaches to software development as it is to agile of course. Good people will do better with a bad process than bad people with a good process.

Agile software development tends to “out” people who don’t make the grade more so than traditional forms of development where people could go weeks or months without having too much scrutiny applied to their work. The focus on building small, incremental, potentially shippable features in short blocks of time especially needs people with good technical and problem solving skills.

People need to be able to write good code: well designed, well factored, elegant and simple. They need to understand how to effectively use unit tests, to understand the benefits of continuous integration, pair programming, TDD, source code control, automated testing and on and on. They need to be able to approach problems with an engineering mindset, to solve problems scientifically, methodically. A team absent these skills will have problems.

7. Poor written and verbal communications
Having the technical and problem solving skills mentioned above is not enough. People need to be able to communicate effectively too, especially on agile teams.

When people are unable to explain themselves coherently there is a massive drag on the team (and potentially outside the team too). People have to clarify what people are saying are writing about. People have to dig for more information that is absent or not understandable in the original communication. People have to guess and may misunderstand. All of this is frustrating and costly.

8. Apathy
“It’s not that I’m lazy, Bob. It’s that I just don’t care.”

Some people are bored. Some people don’t understand what the purpose of their work is. Some people just want to be left alone to hack code and they perceive that agile gets in the way of that. Some people just want to be left alone to cruise Facebook. Whatever the cause, apathy will drag a team down.

Managers have a large stake in combating apathy. Assuming your company is doing something purposeful make sure people understand what it is, why it’s important and how they can help. Make sure they feel empowered to influence the work, the team, the tools. Help create intellectually interesting challenges that marry business needs with opportunities to grow. Support their learning and training needs. Shield them from soul sapping pointless drudgery. In short, make them give a shit.

For those wondering: what if we grow all these people with training, give them an opportunity to develop new skills and master new technologies and then they leave? Consider this: what if we don’t and they stay? Unmotivated, low-skilled, bored people will not a successful product make.

9. Inability to create good user stories
Admittedly one can quite happily apply agile software development principles without user stories. However, they currently sit as the most popular unit of work for agile software development.

A common “starting package” for a new agile team is comprised the set of Scrum ideas plus user stories as your unit of work, a product backlog and estimation using story points. For a team to get familiar with this is fairly easy. Scrum proscribes a set of standard meetings: sprint planning, daily scrum, sprint review and retrospective. Estimating in points requires a little more explanation and practice typically but eventually seems to work nicely for most.

The hardest part though in my experience is to understand and employ user stories well. The five most common problems seem to be:

Stories for everything: setting up new servers, writing a summary of findings, having a meeting, etc. etc. Stories are about features the software should have that make sense to users of the product. This is closely related to the misunderstanding of velocity item below.
Stories that are tasks: this is similar to the stories for everything problem, except that even when your stories are constrained just to product features it’s still possible to write them as discrete tasks rather than describing a full end-to-end feature.
Stories artificially bounded by system architecture: It’ll depend on your users, but for most users of most applications they do not know or care that your product is made up of different components. Devising stories for “the server” and “the client” may well be an indication that you’re not thinking about features from an end user point of view but from a system design perspective.
Abuse of the “as a I want so that ” formula: for many people a story is simply stating your requirements in this format. This often leads to rather convoluted looking stories. Couple this with the stories for everything or stories that are tasks problems above and you end up with things like “As the scrum master I want some administrivia completed so that I can report it to my boss.” This is pretty far adrift from where you want to be.
Poor acceptance criteria: conveying who wants what and why is a good start. But it is also essential to bound the scope of the story with good acceptance criteria. Ambiguity leads to confusion, poor estimates, frustration and potentially disappointment.

There’s no easy cure for this problem. You need someone on the team or someone coaching the team to a point where they have a good understanding of user stories. That or everyone reads Mike Cohn’s User Stories Applied…

10. Lack of automated testing
Whenever you change software there’s always the risk that you unintentionally break something. Sometimes you can make good educated guesses about what may break as a result of your changes. Other times the most innocuous seeming change creates a cascade of fail in the most unlikely of places. To make sure your changes don’t break things that were working before you need to test your software.

If you test it manually then the more your software grows the more time you need to execute your tests. One can imagine how, over time, more and more of an iteration could be taken up with making sure the newly added features haven’t inadvertently broken those that were already there. Developers are left with nothing to do and test staff are run ragged trying to keep up. Clearly a bad situation.

While an automated test suite may take more and more time to run it’s nothing like the days or more that manual testing requires.

You simply cannot hope to incrementally add features, release regularly and maintain quality without automated testing.

11. Misunderstanding the purpose of velocity
Velocity is not a measure of all the work a team has done. It’s a measure of how quickly a team can turn a backlog of product features into working software. When people add stories for training or support or infrastructure maintenance, sometimes retrospectively, ostensibly to “capture the work” or “earn credit for what we’ve done” this pollutes your velocity. It no longer provides an accurate measure of how much a team can advance a product. You might as well record that people work 40 hours a week.

By measuring velocity as just how many product features the team can implement in an iteration, and treating everything else as overhead you are in a much better position to predict how long it might take to burn through the rest of the backlog.

One of the reasons this problem can arise is a simple lack of understanding. That’s easy to correct. The more challenging problem is when teams are being evaluated by velocity: why did your velocity drop during the last iteration? Why isn’t your velocity increasing since we added Bob to your team? That thinking is missing the point. The value of velocity is improving your ability to predict when a product will have a certain set of features completed. It is not a means for measuring productivity.

12. Toxic team member
The toxic team member comes in many guises: antisocial, incompetent, arrogant, slacker, anti-agile skeptic or just general poor team fit. Irrespective of the particular flavor, one bad apple can cause a lot of damage.

I believe everyone deserves a chance to change. Some frank feedback may help. But, ultimately, if someone’s not working out they need to be…redeployed elsewhere.

Scrum and Telecommuting -- our experiment

2011-04-20T18:52:00.001-06:00

Back in December last year I wrote about a pending experiment at work. Half of our scrum team already worked from home, and we wanted to see if things worked better or worse for us if the other half all worked from home too.

After convincing our management to let us try this we targeted starting this in the first quarter of 2011. We planned to try this one sprint at a time, considering at each retrospective if it was worth continuing and what changes we might need to make to make the next sprint better.

As things transpired, we found ourselves keeping this going for the entire first quarter of 2011, and we’re continuing to do it still. As might have deduced, we though it worked our pretty good for us.

At the end of the quarter, I conducted a couple of surveys; one for the team members and one for people outside the team that we interact with. In this post I’m going to share a few of the insights from our experiment and what the surveys revealed.

How was it for the team?
Let’s start with a quick recap of the team composition. There are four people already working from home around the Denver, CO area – three software engineers plus me as scrum master. Then we have three people in Billerica, MA – our product owner and two testers. Finally we had (yes had, but have no more – they moved onto another team during the quarter) two further testers in Hyderabad, India.

There were a couple areas I polled the team on that I felt would help capture a picture of how things worked. The first was how our meetings ran now we were all working from home. Here’s a chart of the responses:

As you can see, we mostly felt that the meetings were no worse off than when half the team was office based, and in fact a few people felt they ran better. That they would run better was certainly our hypothesis going into this, based on the idea that communication would suddenly be equitable for everyone. That is to say we would now all be on a phone and there’d be no more side conversations, people failing to talk “to” the phone etc.

The one interesting point that shows up in the chart and is fairly intuitive too is that the sprint review meeting wasn’t universally considered better in a fully decentralized set up. I was actually surprised more people didn’t feel it was suboptimal, and certainly if you skip ahead to the chart where we asked people outside the team how they found things you’ll notice a few people indicated that they didn’t feel meetings were always better. I don’t know for sure, but I suspect the review more than anything might have been the meeting that triggered that feeling as the others seemed to work well.

The second major question I asked was about getting things done. Here’s the chart for that:

You can see here a lot of what you might predict:

it really wasn’t that much of a problem getting a hold of people and working with them (huzzah for IM and the telephone)
the ability to focus and get work done was better
dealing with hardware and software issues was a little trickier for a couple people

I think there are two important messages. Firstly, those who think you need to be in the office to get work done forget about how much the distractions and interruptions of cube-ville can be detrimental to work. Secondly, you need to be of the “self-serve” type and/or have a good helpdesk that can work with home based users to get through those times when you need help with hardware and software issues.

In addition to structured questions people could provide freeform comments. Here’s a sample of what people on the team had to say:

"Very good experiment, worth continuing, I can always come to office as needed."

"Meetings are more efficient: less side conversations and greater team participation."

"Work life balance was much better in this case."

The final question I asked of the team was whether they would like to carry on with this. Not surprisingly the answer was mostly yes. Those already working from home felt probably felt less excited about it all – they don’t have a choice, they’re working from home whether they like it or not.

How was it for everybody else?
Moving on to the survey of people outside out team, the one of the first questions I asked was about the respondents own inclination for working from home. Here’s the responses in chart form:

We’re not dealing with a huge number of respondents here, but still you can see that the majority of people would be interested in some kind of arrangement involving working from home, even if it wasn’t all the time.

Of course the main purpose of surveying folks we work with outside our team was find out if things worked well from their perspective. After all, it’d be no good if our team was thrilled with this but everyone else we interacted with thought it royally sucked.

Here’s a chart showing how others found things:

You can’t please all of the people of the time. As you can see, for three out of the four questions we had a few folks that felt things weren’t an improvement. But as evidenced by the tall yellow bars, for most people it was just fine. This doesn’t totally surprise me…if you look at the communication behavior in an office, people don’t have to be very far away from each other before they resort to IM or email. Put another way, although it can be useful to be able to go and talk to someone face to face, often that is not what people do even when they are in the same office.

The Problem
Yesterday I read an article from Fortune magazine about a survey conducted by technology career site Dice.com.

What the Dice survey clearly showed is that there is a lot of talent in the technology field that would happily work from home. From our small sample above this seems to be borne out for my organization too. However, Dice notes that fewer than 1% of the jobs offered on their site offer telecommuting as an option to candidates.

The article went on to point out that, given the competitive marketplace for talent, one way employers could stand out would be to consider offering people the option to telecommute. The problem I think is that so many managers are scared of this option, seeming to believe that people need to be kept in an office so they can be checked up on, managed and so forth.

Consider, for example the follow feedback I got from the second survey:

“This survey is not fair, because I don't have a baseline for how well the team performs working "centralized" versus "non-centralized". I will say, I am not happy with the level of visibility I have into how much actual work is being done. From my perspective.. You could all be at the beach all day... with one person working feverishly the day before something is due.. or you could all be working hard.. but in different directions... My gut is that I would like to see more throughput from the team regarding what is actually released.. but from my vantage it is very difficult to say if working remotely helps or hurts.”

Of course one could argue against this (I leave that as a trivially simple exercise for the enlightened reader) but the fact remains: many people still think this way. Convincing those that do that managing by outcomes rather than direct observation and the occasional ~~beating~~ pep talk is not only possible but preferable, may just be too hard.

This attitude may also help explain one question which I found the responses to puzzling. Consider the below, which shows the answers given to the question, "Should our team continue working from home?"

I was surprised by how many people answered "If they feel they must." That seems at odds with the number of people that expressed an interest in working from home themselves, but perhaps, subconsciously, people still worry about the true feasibility of it.

What I can tell you, as someone who's worked from home for over three years, and now have done so with a complete team all at home, is that it can work quite OK. Is it as good as all being in an office together? Probably not. But then many teams don't have the option of all being in the same office anyway. And there's a number of potential benefits to having a team distributed like this: attracting the right talent, providing people an appealing and innovative benefit, a good work/life balance and reduced facilities costs to name just a few.

Excuse me, I'd like to...ask you a few questions

2011-04-17T10:10:00.001-06:00

I once worked with somebody who had the following pinned up on their wall for easy reference during the many teleconferences in which they participated:

Use questions for clarity:
What?
Why?
How?
When?
Where?
Who?
Which?

Not a bad idea that, a little prompt to make sure you really understand a topic. I don’t have a reminder up on my wall but I always like to try and run through these when appropriate. (Well, and when I remember.)

I was thinking about these in the context of agile software development, and how they can help various roles and guide activities. Let’s look at each of them in turn. I’m going to approach this from the perspective of a team following scrum, but I think the general principles apply to all approaches to software development.

Where
Is everyone office based? In the same office or spread across more than one? Where is the customer? And other stakeholders? Some arrangements are better than others. The answer to the WHERE question is definitely important, but usually something few agile software development projects can influence very much. Most folks would agree that the ideal set up is collocation with an onsite customer. If you can get that…awesome. If not, recognize the shortcomings and know how to mitigate the problems they might present as best you can.

Who
The WHO question can apply to a number of things. Who is on the development team? Who is the customer? Who are the users? And so forth. From the scrum perspective it’s really important to have identified who the product owner is. The product owner is such a key role for successful scrum because of their product visioning and feature prioritization work.

Which
I struggled initially with WHICH. It has less obvious utility than the other questions. I think primarily though it can be thought of as your prompt to consider prioritization. Prioritization is absolutely key in scrum. Which projects should we fund? Which themes do we want to target for this release cycle? Which features are of highest value? Which features are risky or are we unsure how to do?

What
Everyone involved in a software development effort should be clear on WHAT they’re building. Initially WHAT is primarily the province of the product owner. They start a release cycle by consolidating input from the customer, other stakeholders, the team, their own ideas and provide a statement of what the product needs to do.

Why
The WHY is important, possibly the most important question of all to my mind for a couple of reasons.

Firstly it lets you check that a feature is really needed and worthy of a place in your product backlog. With something akin to the five why’s technique you should be able to explain the necessity for every feature in terms of one of the following business values:

Protect revenue
Increase revenue
Manage (reduce?) cost
Increase brand value
Make the product remarkable
Provide more value to your customers

Product owners should be very clear on why anything in the backlog is needed. If you as product owner find you can’t tie a feature to one of the values above then you might want to ask yourself if you’re sure it’s needed.

Secondly, once the whole team understands the WHY of a feature we can harness everybody’s input for some creative thinking. Sometimes people think they need a feature because they assume certain things can’t be changed, or they don’t realize there are other ways to approach a problem they are trying to solve. Perhaps an existing feature can be adapted to meet their needs.

How
The question of HOW is almost as important as WHAT and WHY. Those three together form a kind of virtuous trio.

HOW is a question that cannot be answered by any one single role on a scrum team. Everyone has a contribution to this question. As the saying goes, there’s more than one way to skin a cat. And with software that definitely applies. Figuring out the possibilities and weighing up the pros and cons and trade-offs involved will help lead to the best possible features implemented in the best possible ways.

Beware product owners that dream up the HOW as well as the WHAT and the WHY. Unless they’re from a development background it’s likely that not all of their HOWs are feasible or optimal.

When
Often the WHEN question is the one most businesses, customers and stakeholders obsess over. To some extent I think this is almost impossible to escape, although it is possible to put more emphasis on WHAT by stabilizing the WHEN with a fixed duration release cycle. http://www.jonarcher.com/2010/10/delayed-gratification.html

Once the team understands WHAT, WHY and HOW they can estimate the effort involved. If the team has a stable velocity this then allows the product owner to be quite predictive in answering WHEN and they can move features up and down the stack ranked backlog to accommodate the needs of the business as best as possible.

All righty then
Having reviewed all these question-words there are two key insights for me.

Certain roles on a scrum team are better positioned than others to answer certain questions (POs have a natural affinity with WHAT and WHY; developers with HOW)
Nobody on a scrum team has the exclusive preserve of any particular question: by working collaboratively and with everyone empowered to ask and answer any of these questions the talents of everyone can be harnessed to make better software.

What I got out of Mile High Agile 2011

2011-04-13T15:17:00.000-06:00

This is a slightly longer than usual, rambling somewhat whimsical post. Sorry about that. There is some good stuff toward the end though. Well I think so anyway.

Last Thursday I attended the inaugural Mile High Agile conference put on by Agile Denver. The day commenced at an unnaturally early hour with me hurtling down the mountain to Golden and then doing battle with I-70 and I-25. Perhaps due to the early travel time the traffic was lighter than the nightmare I had envisaged and well worth it compared to my regular commute (from bedroom to basement via kitchen) for what I got out of the day.

What follows are my thoughts on what I got out of the conference, not all of them directly related to agile.

Earlier in the year when I first heard about Mile High Agile I volunteered to help with a couple of items – setting up the website along with another volunteer and the corresponding registration and payment system. It’d been a while since I’d done any kind of web content creation but we elected to make things really simple by just using WordPress with a suitable theme. Although this obviously meant there were quite a number of restrictions (especially since we weren’t hosting the WordPress blog ourselves) it did mean we could get going easily and quickly. Given the chance to do it again I’d almost certainly stick with WordPress considering the simplicity of our needs, but hosting it ourselves would be well worthwhile. That would likely let us get around the problems we faced by using plug-ins which you cannot do when hosting at wordpress.com.

For the registration and payment side of things we used EventBrite. This was a service I hadn’t used before (at least not in the role of a conference organizer) and I have to say it’s a pretty impressive offering. While our needs were simple (a handful of ticket types, a few dozen discount codes, simple reporting and so on) EventBrite just seemed to work without effort. Everything we ever wanted to do (bar one thing) seemed to have already been anticipated and worked well. In addition, I had occasion to use their support services twice which are very good and incredibly quick to respond.

In the couple months leading up to the conference last week I really enjoyed working with the folks that brought this together. My contribution was pretty small compared to others, but it was fun to be part of it and work with people really intent (passionate is so overused, no?) on making this happen. It was a real joy to see folks pitching in no matter their role offering useful ideas and commentary as things unfolded. I wish I could work with more people like that more often.

At the beginning it was unclear how many sponsors we would secure or how many tickets we could hope to sell. Due to a truly outstanding effort on the part of several people we exceeded all expectations. Just check the number of sponsor logos up on the site and note that we sold nearly 500 tickets. Not bad for such an incredibly tight timeframe and the first ever Mile High Agile conference! If you build it they will come. So long as you get the word out and market it hard too.

There were a few things that struck after the day of the conference which were nothing to do with agile but interesting insights to me. I’ve been working from home for quite some time. Although I thought I had quite a rich interaction with people on my team via phone, email and IM etc. it was nice to suddenly be in the fray of lots of tech people and talking with them. I was part of the gang that stood behind the registration desk which was insanely busy for an hour or so as the best part of 500 people arrived to check in. It reminded me of a part-time job I had way back in time as a teenager, working in a home computer store on Saturdays which were always insanely busy. I loved it. Definitely makes me hanker for more local, in-person interaction.

Another thing that was weird was that driving into Denver didn’t seem too bad. I’ve always found it less than desirable in the past, but after a series of trips into the city over the last year perhaps I’ve finally got a decent sense of how its all connected up in my head and feel more at ease with it. For some reason I’m feeling the urge to visit Denver more and more.

The final odd little insight was how nice it was to be surrounded by people who were enthusiastic about various facets of agile software development. Although you might have different views and experiences from others on details, there was this common thread of “getting it” (“it” being agile) and wanting to “get it more.” Perhaps meeting people whose viewpoint reinforces many of your own beliefs is nothing but an ego stroke, but it made me think how cool it would be to work with more people like that.

Turning to the sessions I was able to attend three, all of which were in the technical track. I really enjoyed them. I wish there was more – it’d be great if next year were a multi-day conference :-) It also made me think it would have been nice to present. I’m not entirely certain what on, but I feel like I’ve learned a lot over the last couple of years and it’d be fun to share some of that beyond just writing about it on my blog. Maybe next year.

The first session I attended was Agile the Pivotal Way given by Mike Gehard of Pivotal Labs. I confess I hadn’t really realized that Pivotal Labs did anything other than make Pivotal Tracker. Turns out that really their business is building software for people, often startups, and they seem to really approach work in a way that makes a lot of sense to me.

Mike mentioned that they pair program everything. This reminded me how much I miss pair programming. I’ve had two serious periods of pairing in the past and I’m pretty sure it’s the best approach for a lot of software development work. The first time we didn’t call it pair programming because it was just my 12 year old self and my friend hacking away on a Commodore Vic 20 writing games in BASIC, PEEKing and POKEing the screen to make games that rivaled Pac Man. Almost. But it was great. The other time was much more recent. It’s how I learned Java courtesy of a colleague pairing extensively with me. It’s also how the first release of one third of our product suite got built incredibly quickly, by just me and that other guy. If I’m going to be programming in the future I hope it’s pair programming.

Mike also mentioned how their “teams” were small, and unless their clients insisted there were no independent QA or tester roles and no PMs. They (the pairs of developers) did this stuff themselves. Thinking back again, the best software I’ve ever written was when I was completely immersed with clients and talking to them directly and I had to make sure it worked as it should. No middle man (or woman) interpreting customer needs and nobody else to leave finding bugs to, and since I don’t like my things to fail I sure as heck tested the heck out of my work.

All in all, I wish I knew a lot more about Ruby and Rails because they sound like a totally cool place to work. I’m not sure if Mike’s talk was meant to be a stealth recruitment exercise, but it could certainly work as one.

The second presentation I attended was from Chris Powers of Obtiva. He was talking about test driving front end code, and by front end code he specifically meant Javascript. It’s a few years since I’ve dabbled with JS and probably don’t have any dabbling in the near future, but it was still interesting and credit is definitely deserved for live demo and coding activities. The tool Chris showed was Jasmine, and while I have nothing else to compare it to it seemed pretty neat, and I definitely liked the BDD-esque feel and the ability to express both feature and unit tests. If I were to be doing any JS in the future I think I’d be checking it out.

The third and final session I attended was given by Paul Rayner and was entitled Strategic Design using DDD. There was some though provoking stuff in this. First of all there was the Purpose Alignment Model by Niel Nickolaisen. It’s a simple idea whereby you can map out work onto a graph with four quadrants like so:

Of particular interest here is the notion that those things which fall into the top right quadrant are the ones you want to put your best effort into – you want to get the best people working on this and you want to really pay attention to the design of this stuff. By contrast, those items that fall into the lower left quadrant do not merit that kind of effort.

If I understood Paul correctly, he was advocating the use of the above to think through just how much effort you put into the design of various components of a system. That seems like a good idea. But what also struck about this from an agile and scrum perspective was how such a simple thing could be used to facilitate feature prioritization in product development. Now I’ve seen before the idea of having the stories (or themes or features or what have you) up on post its on a white board, and allowing people to move them around placing those the see as highest priority at one end and lowest priority at the other. But this would overlay a slightly more rigorous evaluation of things. If, when using the open whiteboard technique, someone places an item over on the high priority side I am not immediately sure why. If the whiteboard was divided into four quadrants like the above I would immediately see if they thought it was mission critical, a market differentiator or both. I think that could be powerful.

The second thing I got out of this session (actually a little afterwards, after Paul had blogged on the topic) was that the popular idea of “emergent design” is great if you can do it well. But how many teams really have the design and refactoring skills to pull this off? How many can spot good and bad directions to emerge to. Having a framework to first identify what components of a system merit more well-crafted design than others along with a set of good design skills to apply would indeed help I think.

The third thing Paul talked about which stuck in my mind was his "billboard test." The idea here is that if you take something you're working hard on and imagine it up on a billboard is it really the message you want conveyed? Consider his example: "Our logging framework kicks butt!" Frankly, unless your company actually sells logging frameworks realizing that you're putting this much effort into logging ought to give you pause for thought. Like say for example "Our test automation framework kicks butt." Or whatever. (Inside joke)

And that’s it. The trip home was quicker than I thought it would be too. Was very ready for the highway to have turned into parking lot but things zipped along nicely.

"If it ain't broke, don't fix it" vs. "Continuous improvement"

2011-03-17T13:30:00.000-06:00

I don’t really remember when I first heard the phrase, “If it ain’t broke, don’t fix it,” but I’ve used it plenty. Upon reflection I’ve mostly used it to avoid doing boring or unappealing work – endeavoring to cast undesirable requests as wasteful and unnecessary changes.

Thinking about it though, there is a fair bit of wisdom in the saying. Researching its origins turns up a surprisingly recent (to me anyway) provenance. Apparently one T. Bert (Thomas Bertram) Lance, Director of the Office of Management and Budget in US president Jimmy Carter’s 1977 administration is often accorded recognition for first use (although there appears to be a reasonable case for it already being in circulation in the Southern United States). He is quoted in a newsletter as follows:

Bert Lance believes he can save Uncle Sam billions if he can get the government to adopt a simple motto: “If it ain’t broke, don’t fix it.” He explains: “That’s the trouble with government: Fixing things that aren’t broken and not fixing things that are broken.”

Hard to argue with that, eh?

Having used (and possibly abused) this phrase for years, I was given pause for thought yesterday when it was tossed my way. It’s not like it’s the first time that’s occurred, but it was one of the less regular occasions where I thought , “OK, maybe what I’m trying to change isn’t broken per se, but it could be better.”

For the last few years my work has involved a lot of change. My organization has transitioned (indeed continues to transition) from a very rigid, waterfall style process of software development to an agile one. When we started out we had a pretty naïve idea of what this entailed. We were already doing work in iterations – kinda – by providing a new build to our QA/testing people every two weeks. We made a series of small changes over a relatively short period of time (I wrote about those here, here and here) and ultimately we settled upon Scrum as our specific flavor of agile.

You don’t have to spend long with Scrum to realize that one large facet of it is pushing for something a bit different to the idea of “If it ain’t broke, don’t fix it” (which henceforth I’m shortening to IIABDFI.) It’s pushing continuous improvement, or Kaizen to borrow the rather snazzy Japanese word for it. This notion is baked right in to Scrum, with every sprint having a retrospective at the end where the team looks for ways in which they might improve things.

As much as I latched onto and used the IIABDFI idea, I find Kaizen considerably more appealing. IIABDFI is mostly a crutch for me to skirt doing boring work. But Kaizen appeals to my inner perfectionist. Anyone who’s worked with me can probably tell you that I have tendency (likely somewhat annoying) to always critique the status quo. I can’t seem to help seeing things that we could change that I think would make everything just a little bit better. This characteristic hasn’t served me too badly in the past, and when I’ve worked with people of a similar bent it’s created quite a good dynamic with people all interested in continuous improvement bringing in great new ideas and accepting them from others. There can be moments of trouble if two well meaning people hold strongly differing views on what constitutes "better", but mostly things are obvious enough as improvements that a little debate is all it takes to agree upon trying them out.

I’d never really thought about these two ideas juxtaposed though, until today. Kaizen really resonates with me. But, as IIABDFI says, changing things that aren’t broken is indeed wasteful. So, how to reconcile the two?

Well, I think its all about degree. If there’s a potentially worthwhile pay off, if it’s reasonably cheap to try, maybe it is well worth changing things that aren’t especially broken. You might learn through trying your new approach that what seemed reasonable before was, if not broken, ripe for substantial improvement. And if it doesn’t pan out? That should be OK. Trying new things involves some risk. Sometimes things don’t work and you need to revert to the original way of doing things. Unless you’re drastically affecting quality or productivity or some other key facet of software development then I’d argue that trying new things should be encouraged.

If, on the other hand you’re thinking of a disruptive or expensive change, and the outcome is uncertain, and what you’re already doing is seemingly OK, perhaps that’s a red flag. Perhaps that’s when you need a slap around the head from IIABDFI to remind you not to get carried away.

What do you think?

How to completely fail at BDD

2011-02-05T16:18:00.002-07:00

Are you interested in introducing BDD to your team? Don’t try and do it like this under these circumstances. Learn from my failure.

Having experienced ambiguous requirements specifications and inadequate testing of the applications I’ve worked on during my career as a software developer, when I saw BDD I thought I saw the future. In particular the Gherkin based implementations (Cucumber, Specflow) with their Given/When/Then descriptions of scenarios for product features that could lead to executable specifications blew me away as full of potential.

I was incredibly eager to try this stuff out on a real project.

I played around a bit in my spare time with Cucumber. I learned how features were specified in plain text feature files and tests related to them were implemented in step definitions. You match the scenarios you describe in your feature files to step definitions that exercise your application code using regular expressions. I’d never really used regex that much so was initially a bit concerned that this would be a problem. It turned out though that with a small grab bag of regex tricks you could do most things you needed.

One of my first real problems I ran into was that I was looking to try BDD on an upcoming C# project. That put me in a minority group. I don’t entirely pretend to understand why, but much of the C# ecosystem seems pretty closed off, lacking in the use and acceptance of rich open source projects that I was used to in my prior life working on projects on the Java stack. (As an aside, I even notice differences when I interview C# programmers compared to Java programmers. The way they think. The things they have and haven’t read. They typically haven’t done unit testing, typically haven’t used any open source software. In general, their view of the software development world is quite shuttered. I know this is a gross generalization, but it’s noticeable. It’s a shame, because C# seems like a pretty cool language…)

I quickly found Cuke4Nuke which allows you to implement your step definitions in C# rather than Ruby (the norm for Cucumber) and Cuke4VS which would give you syntax highlighting etc. when editing feature files in Visual Studio. Here I encountered my second couple of problems: Cuke4Nuke wasn’t ready for .NET 4 and Cuke4VS even to this day doesn’t work on VS2010. Of course our project was using .NET4 and VS2010. Fixing Cuke4Nuke wasn’t too hard, I fumbled my way through that OK. Fixing Cuke4VS though was beyond me. Well, beyond the amount of effort I wanted to put in anyway. I’ve not done much with C# and nothing with Visual Studio addins like this. I sorta got it working, but nothing I could push back as a quality patch to the Cuke4VS project.

Then I discovered Specflow.

Specflow has some distinct goodness:

It’s Gherkin compatible, which means you describe your requirements in terms of features and corresponding given/when/then scenarios. While I lack the experience that would come from doing substantial real work in this, it intuitively feels “better” to me than the table based approach taken by Fit, Concordion etc.
It’s good to go with VS2010 and .NET4.
No Ruby required. With Cucumber, even though you can write your step definitions in C# courtesy of Cuke4Nuke, you still need a Ruby installation to parse your .feature files and call out over the wire to your C# steps. I’m not the first to observe that the .NET/C# community is kind of reluctant to install “weird shit” like Ruby that “doesn’t come from Microsoft”.

Sadly, at this point in time it also has some challenges:

Documentation. The main problem is the documentation, or lack thereof. Pull down the “Specflow guide” PDF and you’ll find a clear work in progress, incomplete and missing sections, editing notes etc.
Examples. Github hosts a SpecFlow-Examples project, but it needs some TLC. There are some reasonable examples in there, but quite where is not immediately obvious. All the action is inside ASP.NET-MVC/BookShop, not the features folder. Github is also potentially another barrier to entry for people. I don’t know how many C# developers are familiar with Github, but none on my team are.
VS silliness. Not Specflow’s fault, but I had hoped to use Visual C# Express for the QA people on my team to create .feature files. Sadly Microsoft has seen fit to not support Add-ins so it doesn’t work. Another barrier to adoption, at least for my team.
Stability. I never actually experienced this myself, but two of the engineers on my team said they had stability problems that they attributed directly to Specflow. It “mangled the project file.” Things “wouldn’t compile and it was complaining about Specflow stuff”. I never really got great details on this, so it’s hard to tell how much if any was directly down to Specflow. How much was me with rose-tinted glasses seeing no problem attributable to Specflow and perhaps others less interested in a new technique seeing every problem attributable to Specflow.

In general, you’re going to need a lot of patience, a genuine desire in a majority of the team to do something quite radically different to “normal” and you’ll need to draw a lot of information from the Cucumber community and then be comfortable using the mailing list for additional support. I’m actually OK with this, but it can be a real barrier to entry for a lot of people.

There were also some serious non-technical challenges; people skills and organization/political in nature.

Process and regulatory barriers.
My team makes software that is subject to 21 CFR part 11 regulations. That’s stuff that tries to help make sure any pharmaceutical drugs you take are safe. That kind of industry cascades regulations through to any software related to clinical studies to make sure the software is of high quality. Our operation can be (and frequently is) audited by our clients to make sure we comply and can (though rarely is) audited by the FDA. Having devised a “winning formula” that gets us successfully through audits people are naturally reluctant to make changes in anything that would get seen by an auditor. Having your “requirements” in .feature files alongside source code then is a serious brown trouser moment for some people.

“They could end up seeing our source code!” Yes, believe it or not, a process designed to make sure the software involved in clinical trials doesn’t actually often have people look at the source code. They prefer Word documents, signed in duplicate (or more).
“We need to show them a traditional requirements specification with numbered requirements that we can trace through to test cases.” Because that’s the only possible way to do this. Of course.
“We have to review and sign off the test scripts before they’re executed, and prove that we have done so.” I can see how this made sense 20 years ago when test scripts where a documented set of steps a manual tester followed. But what does this even mean when your “test script” is a plain text .feature file and a corresponding set of step definitions in C#?

I actually have to commend the internal quality expert I spoke to for being very receptive to new ideas and trusting my assertion that I could probably generate a traditional style requirements specification and traceability matrix that mapped features to test cases from .features files. She was less resistant than some people on the team.

Skills Deficits
The people on the team responsible for testing the product and generating the nice bundle of validation paperwork those who audit us like to see primarily have a background in testing software manually. I’d looked over their manual test scripts before (long Word documents with lots of boilerplate and then long rambling detailed steps with expected outcomes and places to mark discrepancies) and got the impression they could be better. They seemed to meander quite a lot, with a lack of clarity over exactly what precisely one was testing. I do now know how anyone could review them and spot test cases that were missing. I’d even taken a manual test script for a fairly trivial feature that ran to thirteen pages and felt pretty smug about being able to reduce it down to one short .feature file.

Some people on the team had asked me whether I thought those folks could use Specflow. Could they write good .feature files and the corresponding step definition classes? It was a fair question, and I was pretty sure some might struggle. That said, we already had a test automation tool called Test Partner that involved folks writing things in VBA, so if they could do that, could this be much harder? And I didn’t see this as a reason to indicate that Specflow and BDD was wrong. If BDD is a good idea it’s a good idea. Sure, maybe we needed to train folks or modify the composition of our team, but that was all.

But oh did I underestimate just how much they would struggle. At this point in time I wasn’t sure if Visual Studio Express would work with Specflow. So I suggested they just try. There then followed quite the flurry of emails. People who ostensibly test things for a living were unable to reliably determine whether they could use Visual Studio Express or a full featured version.

And that was just the installation.

Then we got into writing a few .feature files. The first one wasn’t…very good. This was a little disappointing as I’d circulated a number of links to articles on the web about writing “good” features but, OK, it was a first attempt. A couple of the software engineers very graciously (much more gracious than I’m being right now, but I’m currently enjoying my second Left Hand Brewing Company’s Fade to Black) pointed out some things they might do to improve them. Sadly subsequent attempts at writing .feature files were no better. It was as though the advice offered was ignored or not understood.

We didn’t even get near them implementing a step definition. I really have to wonder what they’ll be putting together in Test Partner in those VBA scripts it uses. Thank goodness we have unit tests.

Incumbent Test Framework
As indicated above, we have available (actually we’re pretty much obliged to use) a product called Test Partner. Of course it’s only for QA people, especially given the per-seat cost. It’s all kinds of horrible so far as I can tell after looking at it. It doesn’t even make it easy to place your test scripts under source code control, preferring instead to keep them inside a database (but of course…) and though you can export them (base 64 encoded, naturally) it’s hardly ideal.

In addition to the original Test Partner investment, several people have spent a not inconsiderable amount of time and money building an additional layer (or framework) on top with functionality for such things as logging and managing collections (*cough* reinventing the wheel *cough*).

Of course the political reality attached to this state of affairs is that any other pretender to the crown has got its work cut out.

Cultural suitability
My love affair with computers started back in the early 80s around age 10. A friend had an Apple IIe computer and we played Bouncing Kamungas and other mad games, we entered epically long listings that sometimes worked. The school had a Commodore PET. I learned to program simple games in BASIC. I got a Vic 20 and wrote more games, even sold some to people at school. Then later I got an Amstrad CPC 6128 complete with it's wacky 3" disk drive and after that an Amiga. I worked in an independent computer store and wrote code for their business customers at age 16. I went to college and learned more. I got a “proper” job. It’s waxed and waned ever since. There’s still nothing quite like the fun of programming I think. But the reality of a corporate job can take the shine off that – it’s not the same as sitting there with your best buddy late at night debugging a game you’ve spent all day writing. So throughout my professional career my enthusiasm has varied along with different projects, languages, technologies, jobs and bosses.

Nevertheless, several years back I took a leadership position, got interested in agile and discovered a new found passion for reading about and playing with new technologies and approaches to building software. I joined twitter, found a community of thought leaders and everyday practitioners, started reading books and blog posts on all of this, got excited, started a transition at work. Mostly successful. But not successful enough I guess. I’m still one of the very few people that does that kind of thing. And when you’re perhaps the only one on a team that reads and writes blogs, uses twitter, reads books, wants to try doing things differently it can be tough. You're seeing so much that other people aren't.

The fact is I’ve been having to sell BDD to almost everyone. Maybe one person really “gets” the potential. A few more are fairly excited about it. But it's so far outside their comfort zone.

When I first managed to get some consensus that we should try this I hadn’t done enough with Specflow myself, almost all my experimentation up to this point was with Cucumber. I wasn’t worried though, the guy that was going to try it was an excellent engineer. I was more than a little surprised then when almost every hour a new “issue” was presented on why this couldn’t possibly work. I hadn’t seen that coming.

Culturally, my current team just isn’t ready or interested in something like this. And I have to accept that I can’t necessarily change that. Maybe I should have done a lot more work with Specflow myself first and had good examples to show people. I'm not sure.

Blah, blah whine! WTF am I gonna do about it?
I don’t know yet. Not for sure. I’m thinking maybe I can contribute to the Specflow project and help out with the documentation. This isn’t entirely appealing though if we won’t even be using it. So maybe I’ll use it on the sly, even if we’re not “officially” using it for our project maybe I can keep trying it out. And use that experience to develop something like the documentation that people on my team felt needed to be there.

If you have any suggestions, let me know...

OK, self-indulgent rant over.

Shouting is not an agile value

2011-01-12T17:10:00.000-07:00

The Agile Manifesto is well known:

We are uncovering better ways of developing

software by doing it and helping others do it.

Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on

the right, we value the items on the left more.

Related, but perhaps less well known is the idea of Agile Values. Their origin lines in Extreme Programming, with there being five recognized values as of the second edition of Kent Beck’s Extreme Programming Explained.

In brief the five values are:

Communication – amongst all on the team
Simplicity – of design, emphasizing ideas such as implementing the simplest thing that will work and the acronym YAGNI (you ain’t gonna need it)
Feedback – feedback from automated tests, from the customer and from the team
Courage – to keep things simple, to implement only what is needed today, to refactor and throw away code that’s no longer needed
Respect – for other team members, for the customer

The fuller definitions can be read on Wikipedia (or of course in Kent's book).

One blindingly obvious thing you’ll notice is that shouting is not an agile value. Never was, never will be.

Well duh!

So why do I mention this? Oh well, you know. It’s not like there aren’t some managers who use that as a “technique” for trying to get what they want. It doesn’t necessarily have anything to do with agile, although regrettably I’ve seen it (and once done it myself) in response to somebody just not getting our new technical practices that came along with the agile adoption.

In my case it was due to extreme frustration with somebody who seemed almost perversely reluctant to commit code to the source control system. He was passing around changes to other colleagues by emailing them Java source code. This wasn’t the first difference of opinion about developing software I’d had with the gentleman in question. He harbored some curious ideas and was not enjoying our change over to Scrum.

I confess I rather lost the plot as we were there in the office late at night desperately trying to make sure the team made their sprint commitment. I blurted out rather loudly “What the $#@! are you doing?” I did quickly apologize and later after we’d got the job done we went to dinner and made up. I’m not proud of that moment and it’s not a “technique” I ever intend to use again.

Nonetheless, there are folks out there of a more strident nature inclined to use it far more frequently. I don’t know if their threshold for frustration is lower than mine or if they just need an anger management class or what the deal is. Suffice to say though, as an approach to trying to coach staff into doing what you want it’s not a winner. Indeed it’s a completely toxic behavior. Even if Mr or Ms. Shoutypants has great ideas people are very unlikely to ever “hear” them because they’ve written him off as an aggressive belligerent asshole.

Patience. Now there’s an Agile Value. In fact it’s a key one I believe. Stimulating, inspiring, managing and maintaining the change to agile practices takes effort. It takes leadership. And it takes patience. And so I propose that we add to communication, simplicity, feedback, courage and respect the value of patience.

Measuring the hypotenuse

2011-01-12T16:18:00.000-07:00

Consider the following right-angled triangle:

What is x? Not a trick question.

Well it’s 5. Obviously.

But how did you know that? Well maybe for this very simple example you happen to just remember that 3:4:5 are the proportions of a right-angled triangle. But given a non-trivial example with less straightforward lengths to the two known sides you would doubtlessly have employed Pythagoras’ Theorem. Certainly that’s what most people would expect to see if they presented you with such a problem.

There is of course another way. Assuming the diagram is to scale you could measure x.

But that’s not what people would expect; especially if they’d ask you to use math to solve the problem.

In previous blog posts I’ve described a programming problem I set interview candidates at my organization.

The coding exercise I set people is pretty simple. It draws its inspiration from the old text adventure games the likes of Infocom and Level 9 used to produce. If you’re not familiar with this genre the basic gist of them was that there was an unexplored territory that you could wander around with things to find and puzzle to solve. Players interacted with the game using simple text based instructions such as “go north, get axe and kill troll”.

The exercise requires the candidates to do nothing quite so complicated as that though. They simply get an XML file describing a simple collection of rooms connected in various ways and containing various objects plus a text file describing their starting location and objects they need to go find. Their task is to write a program to read the two files and then use some approach to wander through the maze and try and collect the items.

Just as you would expect something to use Pythagoras’ Theorem to find x in the problem above I would expect people to use some kind of algorithm to solve this programming exercise. I don’t care if the algorithm is as dumb as a stumbling drunk, randomly staggering from room to room until every item is found or if you’re using something like Djikstra’s shortest path, just so long as there’s some kind of intelligent solution for us to discuss.

Up until now nobody had disappointed with that expectation either. Just recently though I got a very novel submission. What the candidate had done, rather than algorithmically solve the problem was to simply issue a series of printed lines that simulated what the output might look like if they had done the exercise for real. In other words their “solution” basically was:

Console.Write("At Sculery: lamp\n At Dining Rome: plate\n At Pond: Fishing Rod\n At Library: book\n At Cave-Entrance: Pickaxe\n At pine Forest: Pine cone");

I hunted around for a bit in case there was more. Alas there was not. When I was trying to explain to the recruiter involved why there was no point proceeding further with the candidate I came up with the hypotenuse measuring analogy which he got completely.

Although I’ve never seen anyone else “solve” the programming exercise in quite this way, regrettably I have seen others who approach problems similarly, trying to find any and all obtuse approaches to avoid actually coding what’s really needed. A common variation on the theme is mangling unit tests in perverse ways to avoid actually testing what you should be.

Since this isn’t an isolated phenomenon I decided I’m coining a new term to describe it. Henceforth (for me at least) programming “solutions” that completely miss the point will be known as a “measuring the hypotenuse” approach.

Top ten tips for distributed scrum team teleconferences

2011-01-09T12:36:00.000-07:00

Distributed software teams using scrum can spend a fair bit of time on the telephone. Even if you only talk for just sprint planning, daily stand-ups, reviews and retrospectives that could easily be 6+ hours in a two week sprint.

When the team is spread around geographically you inevitably have a mix of people: some feeling tired because it’s early, some feeling tired because it’s late and always some annoying peppy ones because it’s the middle of their morning and they’ve had 7 coffees so far.

Add in cultural differences and that English (or whatever your primary language for communicating is) might not be someone’s native tongue and you are sometimes going to have challenges.

Here then, after acting as scrum master for several months on a distributed team with people in six different locations, three different time zones and two different countries, are my top ten tips to help get past those inevitable awkward silences:

1. Use a lot of questions.
Sometimes it’s not clear where to go next. If you can think of lots of different ways to frame questions, that can help. A question that wasn’t clear posed one way may be more accessible put in different terms.

2. Ask specific people for their view.
Asking, “Hey Bob, what are your thoughts on this?” will usually trigger one of two things: either Bob is going to tell you he doesn’t really understand what’s going on, or he shares the thoughts he was sitting on. Sometimes you might do this several times in a row. So after Bob you could be following up with, “And what about you Sally?”

3. Ask the quiet people.
This is a variation on #2. When conversation is flowing well sometimes the quieter members of a team can be left without a chance to contribute. When conversation isn’t flowing at all it’s often tempting to prod the more vocal members of the team. Instead of starting with the “easy” targets seek out other team members to encourage their contribution.

4. Volunteer your own opinion.
Just because the Scrum Master is meant to be a facilitator doesn’t mean he or she can’t have a useful opinion. Myself I can often hardly keep my mouth shut. I don’t think it’s a bad thing to throw out what you see, what you think should be done and solicit feedback on that.

5. Volunteer a ridiculous opinion.
This may be a bit Machiavellian for some, but I confess I use it myself: “So how about we use to solve ?” If it’s daft enough, somebody will usually be unable to resist telling you why that won’t work…which is the perfect segue into “Well what could we do instead?”

6. Use some humor.
You don’t need to be a stand up comedian and you don’t need to aim for tears streaming down people’s faces and belly laughs. But a little comedy can help lighten the mood and reenergize a meeting. Self-deprecating humor is a good way to go. Especially easy when you’ve got a ton of things like me to laugh at about yourself.

7. Take a break.
Long meetings need breaks. When you’re all sat in a conference room together nobody thinks twice about nipping out to fetch a cup of water. Everyone else can obviously see who’s gone and carry on accordingly or wait for their return if it’s crucial. But when you’re “blind” at the end of a phone I find more structure helps. I either build in a 10-15 minute break midway through a longer meeting or, if I sense things are flagging, ask if people would like to stop for a bit.

8. Switch topics.
There is, as they say, no point flogging a dead horse. If you’re making no headway as team with something perhaps there’s some other useful topic you can pursue together. Something more interesting or less ambiguous than what you were struggling with.

9. Do a mini-retrospective.
You don’t have to save retrospectives for the end of a sprint. You can retrospect on a single meeting or even midway through: “Is this working OK? Should we go about this differently?”

10. Note when people are “done.”
Now I don’t mean “done-done” in the definition of done sense. I mean worn out and you’re not going to get anywhere. Sometimes it’s just time to stop. There’s always another day.

Do you have any more tips for keeping teleconference meetings for distributed teams going? If so please share in the comments.

Mind your language!

2010-12-22T08:24:00.001-07:00

I find it hard to pick out a favorite quote from Guy Ritchie’s cult movie “Lock, Stock and Two Smoking Barrels.” It’s just chock full of great lines.

Ignoring the obvious masterpiece from Rory, one exchange that I’ve always remembered with a smile is between Harry’s enforcer, Big Chris and his son (who is of course known as little Chris). They’re visiting someone who owes Harry money, and Big Chris’s son rifles through the indebted man’s belongings. Upon finding a serious pile of cash he exclaims: “Fuckin’ hell John, do you always walk around with this in your pocket?”

Big Chris, despite the dubious London underworld he and his son inhabit appears to have aspirations for his son to remain reasonably polite. He replies, “Hey! You use language like that again son, you’ll wish you hadn’t!” Unless you’ve seen it, it’s hard to appreciate the energy with which that warning is delivered.

Despite immersing myself in agile and Scrum and the language that comes along with that, and despite doing my best to socialize it over the last couple of years in my engineering group I think there are limits. Just as Big Chris doesn’t necessarily want his son to use the lingua franca of the underworld everywhere, I think you need to consider how far outside a development team you use agile and scrum vocabulary.

One of the three legs of Scrum is transparency. But being too transparent isn’t necessarily a good thing (think body scanners in airports, for example).

Maybe, if your entire organization has decided to operate using scrum, in everything from sales and marketing to operations, HR and accounting as well as engineering then you can happily talk about daily scrums, sprints, points and stories and velocity. But if they haven’t, then I think you have to balance carefully the education of those groups with whom you interact and the language you use.

For me, there are three key pieces of terminology that I am, with hindsight, wondering about the wisdom of sharing widely.

Story
Yep, I don’t think we should be talking about stories. Now don’t get me wrong. That doesn’t mean I don’t think we should be using them. On the contrary, I’ve said it before and I’ll say it again, one of the most profound books for me as I was learning about agile was Mike Cohn’s "User Stories Applied."

The concept is great. As a unit of work it’s perfect. For organizing requirements it’s helped tremendously.

However, as a term to use with unsuspecting business stakeholders, customers and so forth I’m not so keen. Frankly it sounds rather ridiculous. I’ll stick with feature, thanks very much.

Sprint
Few people with practical experience of dividing your development up into fixed duration timeboxes is a bad thing. But do we have to call it a sprint?

One of the things I like about Scrum is the fact that done right nobody’s having to dash like a lunatic to cram in all the last minute jobs to ship your product. Instead what we have when we do it right is a predictable, sustainable, humane way to utilize teams.

Sadly sprint conjures up anything but that vision.

These days I prefer to describe that we work in short two-week cycles, and if I need a short snappy name for them then iteration works just fine.

Points and Velocity
I’ve pulled these two terms together as they really go hand it hand. At least, if you’re estimating stories in points but not measuring your velocity you might want to reevaluate what you’re trying to achieve.

And these two are a little different for me than story and sprint. Whereas those are just unnecessarily goofy in everyday conversations with the wider business, and alternatives can serve us better, points and velocity need very careful handling, whatever you call them.

Here’s the problem – points and velocity are a tool for the team to be able to evaluate their performance, to plan and make reasonably firm commitments. Stakeholders outside the team should be concerned more with what gets done and in what order. The Product Owner and team together can use points to respond intelligently to new requests, to switch things around, and figure out what fits where and what needs to get dropped.

But start telling your stakeholders too much about average velocity, the points involved in a particular story and you inevitably risk them starting to focus on points rather than the “what should we do” and “what order should we do it in” issues. Perhaps in a really mature agile environment were a deep understanding of what points and velocity are for this wouldn’t be an issue. But early on in an agile adoption I think care is needed.

Too much data on story sizes in points, team velocities and so forth risks the following questions:

“Can’t you do more points per sprint? Why didn’t you make your commitment? Why has your average velocity dipped? Why does the tracking team have a higher velocity?”

Those kinds of discussion are not the most value one can extract from a stakeholder. I don’t want to see agile teams micromanaged by a group of well meaning people who know nothing about agile software development. But we are baiting them into doing exactly that if we report too much detailed data about points and velocity. Before you know it they’ll be looking at your definition of done suggesting we don’t need to do unit testing and who knows what else…

I say keep the dialog with stakeholders confined to steering the product, negotiating the relative priority and let the team use points and velocity data to communicate what can be done for any given release.

Put another way, what do you really want customers and stakeholders to concentrate on? The minutiae of your process? Or innovative product ideas and how they should be prioritized in the backlog?

Fully distributed Scrum

2010-12-15T14:35:00.000-07:00

(Or, "How we pitched the idea of letting everyone on our team work from home and got the thumbs up")

Back at the beginning of November I wrote about my recent experience as the scrum master on a highly distributed team.

For those that didn’t read (or don’t remember) that post, a key thing to know is that the team is comprised of several people who work from home. I ended that post (partly tongue in cheek I admit) saying how it would be interesting to take those team members that were still office based and have them work from home too. Well, starting early 2011 we’re going to give it a try!

The original suggestion was first seriously discussed in a team retrospective. On the premise that it’s easier to offer an apology than ask permission I sent the appropriate managers in our group an email indicating that the team had decided we were all going to work from home next sprint. That didn’t fly. Which isn’t surprising (although it was worth a try, or at least entertaining, no?) and a more considered proposal was requested.

Myself and the team duly worked on preparing a properly coherent argument, supporting data, benefits we expected, a concrete description of how we would run this little experiment and details on how we would measure the success (or otherwise).

I know there must be many other teams trying to reconcile the demands of their managers to “do agile” and also “leverage offshore resources” so I wanted to share details of the proposal that helped convince ours that trying the rather radical proposal of sending everyone in the office home might help.

What are we trying to do?
Before we go anywhere further, it’s worth properly answering this question. Lest there be any doubt, this isn’t about folks wanting to get up late, watch daytime TV and get their laundry on during teleconferences.

This is an authentic endeavor to make our team work better. You can argue till you’re blue in the face about the benefits of team co-location, or the silliness of splitting teams into onshore and offshore components when you’re trying to simultaneously implement Scrum, but the fact remains that many, many organizations are still going to insist you can’t move the furniture and you have to structure teams this way.

Therefore, if we have to live with this, we’re going to make it work as best we can.

At the heart of much agile advice is an emphasis on communication. The trouble is, with team members scattered across different timezones and locations, communication gets clumped into localized groups. People on the edges miss out. By putting everyone on an equal footing (working from home) everything becomes more equitable and information sharing is normalized allowing for a better functioning team. At least, that’s what we believe, and what we’re trying to do is prove that out.

“Craft a philosophy and live it.” – Umair Haque, bubblegeneration.com

How did we approach this?
Our presentation covered five key areas:

Why did we want to try this idea of everybody on the team working from home?
A list of benefits, beyond simply the team working ‘better’ that we anticipated
A description of how we wanted to run the experiment
How we intended to evaluate the outcomes
The support we needed to get this going

I’ve already talked about the why, so let’s look at the others:

Anticipated benefits
Besides the hoped for improved communication, there are some other interesting and very tangible benefits we anticipate.

As I write this blog post the time in Colorado is 2.54pm. Meanwhile, in Hyderabad it’s 3.24am the next day. We have quite the time difference between some of our team members. Before Daylight Savings Time (DST) came to an end in the US this year it was just possible (if not entirely pleasant for everyone) to have a daily scrum via phone with all team members present. Now however, since Indian Standard Time doesn’t adjust, the earliest we can hold our daily scrums is just unreasonably late for our colleague there to attend every day. Although they do stay late to attend some of the sprint planning meetings and sprint reviews they’ve been missing a lot, including retrospectives.

One very worthwhile thing we can do operating as a team that works from home is create a more humane work/life balance for people. This will pay particular dividends for our colleagues in Hyderabad, allowing them to work the majority of their days at normal local business hours. They can they stop just early enough and save the remainder for meetings in the evening. Although nobody particularly wants to have to regularly work super early or super late hours, this is a much better arrangement than having to be still stuck in an office at 9pm at night.

In addition, there are another couple of benefits we see. One is immediately realizable, the other perhaps a longer term proposition. They are reduced environmental impact through less travel and reduced office space needs.

An article in Forbes magazine in 2008 indicates there were as many as 17 million software developers in the world at that time.

More than 2 million of them are in India and the US is approaching 3 million.

Most if not all of these people spend time in cars or other transport every day commuting to and from work.

Indian emissions standards for motor vehicles lag about 5 years behind Europe, and in the US of course the majority of us love our gas-guzzling SUVs.

We may not be able to keep millions of these people off the road, but by allowing those that work in our team to do their jobs from home we can have a small but positive environmental impact. Who knows, maybe this innovative idea can spread.

As for the office space thing, if you can establish a team working at home, with perhaps continued but minimal office visits as needed then do you really need such large premises? With enough teams operating this way you could save on what you lease, on what you heat and so forth.

Running the experiment
Being a Scrum team convinced of the benefits of incremental and iterative development, we naturally intend to use the same approach to this experiment. Starting early 2011 we will try a sprint with everyone working from home.

At the end of that sprint, a large part of the retrospective will obviously explore how things worked. We’ll be asking ourselves questions like:

How well did that work?
What work well and what would we like to change?
Can we fix the things that didn’t work well?

If we think it was worthwhile and we can improve those aspects that didn’t go too well then we’ll repeat another sprint with any changes we need. With this approach we can pretty quickly home in on how to do this well, or equally quickly conclude that our original arrangement with two-thirds of the team staying office based was better.

Evaluating the outcomes
Many years ago I used to quite like the idea of capturing data you could analyze to objectively evaluate things. Peter Drucker is renowned for saying “What gets measured gets managed.”

I’ve since learned that if you measure the wrong things in the wrong ways then it’s more a case of “what gets measured gets gamed.” Regrettably I’m not renowned for saying that. Maybe one day.

However, we do need some way to determine how our experiment is going.

Agile thinking provides us with a few simple tools that we can employ: velocity, burn-up charts and cumulative flow spring most readily to mind. To some extent though there is the possibility that these could indeed be gamed, or unconsciously given treatment to provide the desired out come.

So besides those, we will also be looking at some other areas:

PO and stakeholder satisfaction. From their point of view are things better, worse or pretty much the same?
Team satisfaction. Do we feel that things are better, worse or pretty much the same?
If the experiment pans out well enough that we conduct a whole release with people working at home then we can utilize SLIM Metrics to observe productivity benefits.

Required support
Of course my team is not an island unto itself; we needed support to undertake this. First and foremost we needed the support of our management. Luckily the director of our group is open to innovative ideas and. After our presentation to him and a series of questions that the team was able to answer was willing to permit this experiment.

In more material terms we pretty much had all that we needed. People have broadband connectivity and telephones at home. They have quality spaces to work in. We do need to configure a few machines for remote access, and a few people needed new laptops. Other than that though, we’re ready to go.

In conclusion
The whole team is excited to try this. I think we’re onto something. I’m eager to see what our actual experience is like. Will things be better or just different? Next year, after we’ve run this experiment for a bit I’ll report back on how things have turned out.

Blog name change

2010-12-15T12:35:00.000-07:00

I changed my blog name to "Coding When High."

This probably requires a little explanation lest you draw the wrong conclusions.

Originally I named this blog "Mixed Messages" with the explanation that I was unsure I could restrict myself to any one topic. And indeed over the last couple years there were a few posts on things other than software development topics. However, as its turned out, the vast majority of what I write about has stemmed from my experiences managing and participating on agile and scrum teams over the last few years.

Since a clear focus has emerged for my writing here I thought a new blog name was in order. Plus I wanted to play with Tumblr, so I'll henceforth be posting my non-work related stuff over at http://jonarcher.tumblr.com/ (nothing interesting there yet...)

So how did I arrive at this name? Well, I've recently switched back from management to individual contributor on a team. Currently I'm acting as Scrum Master, but with a definite intention of writing some substantive code soon. Add in that I live at 9,200 feet above sea level, up in the Rocky Mountains west of Denver, CO...et voila, "Coding When High" sprang to mind.

So no, don't worry. I'm not doing MDD (marijuana driven development).

The perils of rolling your own

2010-12-02T11:43:00.003-07:00

I spent a good portion of the early years of my adult life messing up my health by smoking. Even throughout my thirties I was an off and on smoker (or “social smoker” if you will) at various times.

It’s well understood that no matter what you smoke, whether regular cigarettes, “low-tar”, cigars, hookahs or even medicinal (or illicit I suppose) marijuana: inhaling smoke will screw your lungs up pretty good. For my part, I smoked hand-rolled cigarettes, which is probably one of the worst options: no filter to take out anything at all.

In 2002 I moved from the UK to Boulder, Colorado in the US. For those that don’t know, there’s probably no place else on earth with a more militant anti-smoking stance than Boulder:

"In 1995... Boulder voters made national headlines and created a local uproar by expanding the city’s smoking ordinance to ban smoking in all indoor public place... The Boulder Dinner Theater had to get a special dispensation so an actor could light up on stage, as called for in a play."

From Insider's Guide to Boulder and Rocky Mountain National Park by By Ann Alexander Leggett, Roz Brown

As it turned out this was actually a great thing for me. It helped make quitting smoking and attempting a significant lifestyle change considerably more possible. Nonetheless, the first time I went to a gym and tried to run (for just sixty seconds!) I swear I nearly died. Humiliatingly, the gentleman on the treadmill next to me, easily older than my father, was barreling along at 6mph and hand been doing so for the last twenty minutes.

I definitely learned the hard way that rolling your own is bad.

I think this lesson translates into software development too. It’s often tempting to “roll your own” frameworks for various software development needs. At one point in time this was especially true for web applications. I’ve been to party this. Several years back we wanted an MVC web framework with AJAX support. There weren’t any at the time, or certainly none our lead engineer liked. So a couple of the guys I worked with wrote one. It was a cool project; fun and intellectually stimulating as well as educational. And we got to do things the “right” way, no more tolerating those parts of other frameworks we didn’t like. But it became the biggest albatross around our neck.

When you build your own proprietary framework there are several key problems:

Nobody's going to maintain it but you. This starts off as fun -- nobody else is "screwing" with your code, but quickly becomes a bind. In our situation in particular, with the AJAX/MVC web framework there was the never ending tedium of browser compatibility issues.
Dubious value. Is building general purpose frameworks *really* what you want to be paying your engineers to do? Is that *really* the business you’re in? Is it *really* the best use of their time?
It's hard! Sure, building 80% of the framework you think you want isn't too hard. But try finishing it. Try polishing it. That last 20% is tough.
Eventually the originators leave. You may think you've got the institutional knowledge for your framework well shared. But I'm willing to bet that if the main driver behind your home grown solution leaves it'll start to rot. A year later (if that) everyone will hate it. Not much longer and it'll be the reason why nothing can be done and everything about your product sucks (this is often melodrama but try countering it effectively...)
New employees have never heard of it. So they have to learn it. And, obviously you can’t hire anybody with experience in using it.
It lives...and lives and lives. Once you have products depending on your framework, extricating yourself from that can be difficult and costly. Just try convincing non-engineers that you spend time fixing this problem.

These are big problems and not to be underestimated. Based on my experience you should think very, very hard before you decide to commit to investing in building your own framework for anything.

The MVC/AJAX web framework we built years ago still limits us even today.

It was OK for the experienced highly-skilled programmers that built it. It was great for them to build rich one-off applications with. The trouble was, the business needed us to build dozens and dozens of cookie cutter applications…all similar but slightly different.

The trouble with that kind of work is that one doesn't tend to use experienced, highly-skilled programmers for it (or if you do, they don't stay around for long).

So then of course we had a framework that was hard for most of its users, and certainly wasn't geared to churning out cookie-cutter applications. They were frustrated and it was overly complex for the actual task at hand. Part of the consequences of that was we suffered the same defects and problems in the cookie-cutter applications over and over again. And it took longer than it needed to build them and longer than it needed to test them.

Once again, I learned the hard way that rolling your own is a really bad idea. We should have built a tool for building cookie cutter apps, not a general purpose web framework. Quickly and reliably turning out cookie cutter apps was the business we were in, not web frameworks.

P.S. if you've still got that itch to build a framework, how about contributing to an open source one instead?

You are not my customer

2010-12-01T16:42:00.000-07:00

(And I am definitely not your supplier)

Recently our VP sent out one of his regular update emails and included in it a little piece about the virtues of internal customer service. He included the following in his email which appears to have come from Entrepreneur.com:

Internal Customer Service: Getting Your Organization to Work Together

Great customer service isn't just about serving the people outside your company.

By Scott Miller

Providing exceptional customer service lies at the heart of the mission of many organizations. It is the central theme of books, articles, motivational seminars and business courses. Its value is undisputed in business circles. What many companies fail to focus on, however, is the primary path to exceptional customer service: internal customer service.

Internal customer service is the service we provide fellow employees and other departments within our own organizations, as well as our suppliers and anyone else with whom we work to get our jobs done. It is what we do when a colleague asks for information she needs to complete her main task for the day; it is what we say when someone from marketing asks for the addresses of good contacts; it is how we greet the vice president of sales when he walks into our office with an "I need something from you" expression on his face.

All these things can be seen as interruptions that take us away from our "real" jobs, yet they are vital to our company's success. If you see a gap between your "real" job and the needs of others in your organization, you need to rethink what your real job is. In helping others in your company, you help your company succeed. Here are some tips for creating an atmosphere of internal customer service:

1. Begin with your own perspective: Regard fellow employees and other departments as your customers. Understand that helping your colleagues do their jobs more successfully helps your organization and you. Therefore they are your customers. Treat them like VIPs.

2. View interruptions not as nuisances, but as opportunities to serve your internal customers. If you tend to view every interruption as a pothole in your road to success, reexamine those interruptions. If someone interrupts you to share gossip, that's a pothole. If someone interrupts you to ask for sales figures she needs to analyze sales team performance, that's a necessary lane change that will get your company closer to its destination. Learn to identify every real need from a colleague as a "necessary lane change," and think of every "necessary lane change" as an opportunity to move your organization closer to its goals. Take pride in helping your colleagues; enjoy your role in sharing information and providing services that help others get their jobs done. In most cases, your willingness to help others get their jobs done will lead them to readily assist you when you need it.

3. Exceed your internal customers' expectations. When someone exceeds your expectations, how do you feel? Most people feel delighted, excited, upbeat and very, very positive about that person and his or her organization. Think what you can accomplish in your organization by exceeding the expectations of fellow employees. If payroll asks for time sheets by 3 p.m., provide them by 1 p.m. so payroll can relax, knowing they have the time sheets in hand. If human resources asks for a list of important points to cover in an employee orientation, take time to think about it and provide a thorough list of what you would want to know if you were being introduced to a new job and company.

4. Say thank you. A simple, genuine "thank you" goes much farther to create an atmosphere of sharing and helping than two such small words would suggest. Even when it is a person's job to provide information or a product to you, tell them "thank you" when they have done it. Express your appreciation of their timeliness in providing it. Explain how it has made your job much easier. Show them your delight when they exceed your expectations.

I was immediately reminded of what was a very memorable counter to this way of thinking from an old colleague. This far away from the original conversation I forget the exact details, but over the years I've developed my own variant (it may be identical, I cannot say, but hopefully I've added my own nuances).

The trouble with the internal customer model is threefold. Consider:

It’s not equitable. Not everybody is a customer, some people are just suppliers (accounts, engineering, HR)
Those in the customer role are not paying, so they can often make unreasonable requests without consequences. As a supplier you can’t refuse their business, even if they’re a terrible customer. Real companies of course can pick and choose who they do business with. Further, you can’t charge them more when they demand unreasonable changes and extras.
For the most part, people treat suppliers badly (quote from former colleague: “I love when we outsource this crap to somebody else. I can just dump all this shit on them and not have to deal with it.”) *

Is this really what you want? I say no. I say you don’t want internal customers as your model. I say you want team as your model. Put yourself in the mindset of all being on the same team. That for me is a far more pleasant, appealing and realistic model.

* OK I made that up. But it's entirely plausible is it not?

When average is good

2010-12-01T14:21:00.000-07:00

Calling something average is one step removed from calling it mediocre. But in the mathematical sense, especially when applied to a team’s velocity it’s a good thing.

One of the certain tricky bits for any new team trying out scrum is establishing the team’s capacity. That is, without an historic record of how many story points of work a team can complete you are somewhat in the dark as to what to aim for come sprint one.

For a team that’s been working together in the past but not estimating in points it’s possible they may have a fairly accurate gut feel for things, and that can be your guide. There’s likely still the novelty of having not just to develop but also test, fix defects and truly complete (according to your definition of done) but, with experience of working together in the past a decent guess can be made.

For a newly assembled team however, or one that’s less confident in figuring out how much of the backlog to bite off initially, my advice would simply be don’t worry. Just pick more than enough and accept that you won’t make it all. From what I’ve seen the first few sprints are very much a learning experience and part of that learning is going to include figuring out your capacity.

Given just three or four sprints the team, with good coaching and guidance from a competent scrum master, will have figured out a number of recurring key items: big stories are bad, transparency between those that code and those that test is key, getting things “done” means limiting how much you work on concurrently and, ultimately, a reasonable average velocity to work with for planning future sprints.

The team I am currently working with had quite an erratic velocity initially: 16, 8, 49 (yes really!) and then 26 points. After this we have stabilized nicely with an average of 24 points a sprint.

This “we dunno, we’re just gonna try it and see” approach to working out a team’s capacity for quality work is inarguably logical. However it doesn’t necessarily sit well with those from a command and control background, especially those with management responsibilities for the team new to scrum. They like detailed planning. Predictability. Commitments. Maybe even punitive action against teams that “fail”.

This is completely the wrong approach.

Like it or not, this is when managers need to decide if they are truly supporting scrum or not. If you are, then step up. Provide the environment your team needs in this early phase of the adoption. Make them comfortable with the new regime of trying, reflecting, learning and improving. Without that support you will stifle their ability to rapidly improve, and in turn lead to a less than stellar scrum implementation. Maybe even to a failed one. This happens a lot, and people revert to traditional, comfortable but ultimately unsatisfactory approaches to building software. Don’t let this happen to your team. Create the right environment and harness the power of average.

What's the velocity of a turkey?

2010-11-23T16:40:00.000-07:00

“What do you mean, an African or European turkey?”

Neither, I mean a North American Thanksgiving turkey. And no, this is not some kind of cheesy Python-esque skit. Rather, as my favorite US holiday approaches, a particular truism of turkey roasting occurred to me: your 12-16lbs bird will need about 15 minutes per pound at 325F until it reaches an internal temperature of 160-170F.

So that, I would say is the velocity of a turkey – 15 minutes per pound. And the internal temperature is your “acceptance criteria” since I’m doing dubious cookery/scrum metaphors.

Just like a scrum team’s velocity is what it is, the same is true of our turkey. You can try and "turn up the heat" to get it done quicker, but in reality the results will suck. Your end result will be all dry (for the turkey) and the product flaky (for your scrum software team).

So, to get good results, work with the reality of the situation. Plan based upon your known velocity. If you want 100 points worth of features implemented, and your team’s velocity is 25 points a sprint, reckon on it taking around 4 sprints. If your family is hoping to sit down and feast at 1pm get that 12lbs turkey in the oven with enough time for it to cook and rest afterwards. You just can’t rush it. If you have other features you want within the next 4 sprints you are going to have to make some hard choices about what you drop from your current plans. You simply can’t have it all. Similarly, if there are other things you need the oven for, plan accordingly. You can’t fit everything in there with the turkey all at once.

As much as scrum is about prioritization, it’s also about accepting the reality of what your velocity tells you can be realistically done, and working with that. That means good planning, and sometimes hard choices.

Highly Distributed Scrum

2010-11-03T15:12:00.003-06:00

I'm just about to finish sprint four as the Scrum Master for a team that has a higher than average number of distributed members. And I don’t just mean that we got two offices involved, one in the US and one in Europe or India. We have three software engineers spread around the Denver Metro area, all working from their individual homes. Then we have our Product Owner in Billerica, MA. Alongside her are a couple of QA engineers. Then we have some folks over in Hyderabad, India: another software engineer and a couple more QA engineers. Finally we have me, working out of my home in the Rocky Mountains above Denver, CO. That’s six locations for just ten team members.

Add in some interested stakeholders in Berlin alongside those in the US and India and you can see what we’re up against. I’m sure we’re not alone in such a setup, but I suspect that having a number of people, including the scrum master, “decentralized” (my organization’s term for those of us that work permanently from our homes) is less common than a team made of people from two or three offices.

It’s been a fairly interesting experience, and not in the “OMG-what-a-total-disaster-I-have-a-dozen-more-horror-stories-to-take-around-with-me-warning-people-about-the-folly-of-distributed-scrum” sense.

One of the interesting things about this experience was how incredibly smoothly it went. I may just be one of the luckiest scrum masters in the world. I can’t claim superior scrum mastery as the root of this easy transition. The team was a well-established, mature team that had been working well together for a long time. The relationships were strong and team members trusted and respected one another. They had already introduced many agile practices way ahead of other groups within the organization. It seems clear to me that this helped immeasurably as we layered scrum on top of what they were already practicing. The other contributor to this, which I discovered serendipitously earlier this week on Twitter is explained in this excellent blog post by Bob McWhirter. In it, he makes the distinction between simply being a remote worker, and being a remote worker in a distributed team. In a truly distributed team there isn’t a single center of gravity where a ton of conversations occur that remote workers aren’t privy to. Due to people being spread out, a distributed team necessarily uses communication techniques that support remote workers, such as mailing lists, IRC etc.

I think the most interesting thing though is that we haven’t really seemed to need any special dispensations for what is a fairly unusual team configuration. No tweaks to scrum; no special tools, tricks, techniques etc. I’m not saying those won’t necessarily come as the team reflects more on areas for improvement during future retrospectives, just that we’ve been able to get going just by sticking to the basic principles. Our only accommodation has been for the people in Colorado to start a little early and the people in India to finish a little late (it’s a fairly brutal time zone difference).

So how are we doing it? Well, here are the basics:

We use Rally as our tool for agile project management, so that’s where our product and sprint backlogs are.
We use a wiki to organize a lot of other material. It all starts from a team home page which links to all manner of useful things: our definition of done, agreement on how we document stories, the build server, source code repository, released software etc.
We have a build server. This team’s current product is a rather old piece of legacy software that, presently, doesn’t lend itself to CI due to the lengthy build time. But it does still build every night.
We have used http://planningpoker.com to do distributed, real time story estimation in points. It’s not quite got the natural feel and immediacy I would like, but it’s not half bad at all.
We have daily “stand-ups” held in the morning (US time) which is evening for team members in India.
Sprint planning, review and retrospective meetings are, similarly, timed for US AM and India PM.
We have a phpBB discussion forum for the team set up…we also have an email distribution list that includes all team members. Interestingly the distro list gets all the action. I’m not sure if that’s because people are less familiar with using the discussion forum medium. I think it might be because there’s a greater immediacy to email based communication.

Interestingly the team has, through retrospectives, identified all the usual early issues I’ve seen other teams come up with during scrum adoption:

Big stories are bad
Spread out completion of stories during sprint
Recognize and raise impediments ASAP
Need the right people at the sprint review to make it truly valuable
Be clear that we have commitment from people outside the team for stories that depend on them for success
We have technical debt and we need to pay it down

We have, of course, faced some challenges; the two key examples being:

Meeting our commitments, i.e. getting stories “done done” by end of sprint. After a couple sprints we realized we would be much better off with smaller stories. Better to slip one small story than fail to finish two large ones.
English is a second language for a number of team members. With more spontaneous verbal and (due to our distributed nature) written communication there has, understandably, been moments of confusion. This isn’t quick to fix, but with more and more practice things can surely only get better.

All things considered, I’m very pleased to be part of what’s shaping up to be a really great scrum team.

Now, I’ve argued hard against distributed scrum before. That piece was mostly borne out of frustration, seeing teams of two halves. I still stand by the views expressed there. Having your whole team co-located alongside your customer still seems, by my experience, the preferable situation. The really interesting lesson for me here though was how, with a truly distributed team, Scrum can be made to work better than I anticipated. Moreover, it seems that by adding more team members at the office based locations we might weaken, not strengthen, our existing team. Perhaps we would do better to let those who are currently office based work from home too. With the right people, it seems that perhaps a fully distributed scrum team could trump a team split between two offices. I remain skeptical that they could better a fully co-located team. That said, I wouldn’t mind the chance to find out. It’d be a fascinating experiment.

#Positivember and the response continuum

2010-11-02T16:30:00.000-06:00

Over my first coffee this morning I happened across the following tweet from Corey Haines:

"Today is day 2 of a month long positivity-fest! Love your life: #positivember http://vurl.me/WJU"

I almost feel bad for saying this, but it’s the truth: that handful of words irked me. Why? I think it’s because of my default interpretation. A month long positivity-fest sounded to me like some kind of rose-tinted denial of reality. Images of well-meaning but naïve parents that abound in a nearby town I’ll not name sprang to mind. The kind that work really hard to stop their children from expressing themselves when upset. The kind that want to insist that everything is always great. Even a cursory look at the psychological impact of that kind of suppression reveals that it probably ain’t healthy...

Then I read Corey’s blog post. It’s not (thankfully) quite that kind of hippy-dippy call to action. It’s much more an observation on how high levels of negativity are counter productive. How a concerted effort at positivity is the order of the day for advancing the notion of software craftsmanship. I’m up for that.

Still not sure I can quite buy into the notion of being completely positive about everything for the whole next month. Dropping the moaning and whining and negativity though is a good thing. It’s a bad habit I’ve had since forever. It’s easy to find fault and complain about silly little things.

A bit later, as I was walking the dog, an idea popped into my head that might help explain why the “all positive all month long” concept is hard for me. I believe gunning for all out positivity is an extreme reaction to the unpleasant other end of the scale. I think when we respond to something, that response lies somewhere along a continuum a bit like the following:

Sabotage → Meanness → Negativity → Apathy → Constructive Criticism → 100% Positivity

Reducing things to a binary negativity/positivity option over simplifies the matter. It’s the stuff to the left of the middle that’s toxic. To the right is great. I like to think I’m over there more and more these days, but still have plenty of work to do.

Not too long ago I read “The Lazy Manifesto” blog post by Leo Babauta on his zenhabits.net website. Items number 5 – “Do less complaining and criticizing” and number 7 – “Do less judging and expecting” are, for me, a more palatable way to think about what I believe Corey is trying to achieve with #postivember.

I hope I'm not too far off track. Even if I am a bit, those two ideas resonate with me and since making a conscious effort to try and adhere to them I have to say things feel good.

Vive La Revolution!

2010-10-26T15:14:00.001-06:00

In the agile revolution, are managers the over privileged aristocracy? And if so, why is nobody shouting “Off with their heads!”

I must confess there is a reason I’m partial to agile software development that might unnerve some people: I like its dark underbelly, that thread of anti-establishment flowing through it. You won't find that much on this explicitly in the Scrum guide or other key agile works. But hang out in the right places and you'll discover it's there.

Much agile thought has an egalitarian, meritocratic theme that recognizes the need for leaders in a team but simultaneously discourages hierarchies, superiority through tenure or job title and rattles the cage of old school authority. I love the shunning of traditional command and control management, the respect for people that do the work as competent individual professionals who can be trusted. And I share the distaste held by many agile practitioners for practices like detailed estimates in hours that are used to berate you later, ridiculously detailed timesheets, performance reviews and meaningless certification. These things have never sat right with me.

I’ve never been very good with rules mind you. I don’t like being told what to do or how to do it. I don’t like speed limits of 35mph when, clearly, it would be OK to go 40mph. I don’t like signs that say the park isn’t open until 6am (what if I want to go hike at 5.45am one summer morning?) I could go on, but you get the idea I’m sure… This is why so much of the thinking emanating from the agile community resonates with me.

All of which sentiment got me thinking about agile as a kind of revolution. And if it’s a revolution, who’s the aristocracy? Shouldn’t heads be rolling for the waterfall mess and soul destroying corporate drivel that preceded this?

Clearly the new nobility is management. But should their heads be on the chopping block? Take a look at this for a moment. It’s a spoof of the original agile manifesto put together by Kerry Buckley.

We have heard about new ways of developing software by

paying consultants and reading Gartner reports. Through

this we have been told to value:

Individuals and interactions over processes and tools

and we have mandatory processes and tools to control how those

individuals (we prefer the term ‘resources’) interact

Working software over comprehensive documentation

as long as that software is comprehensively documented

Customer collaboration over contract negotiation

within the boundaries of strict contracts, of course, and subject to rigorous change control

Responding to change over following a plan

provided a detailed plan is in place to respond to the change, and it is followed precisely

That is, while the items on the left sound nice

in theory, we’re an enterprise company, and there’s

no way we’re letting go of the items on the right.

(Source: http://halfarsedagilemanifesto.org/)

It’s funny. Except it’s probably all too real in some organizations. And who makes it real? Managers. Managers drafting and implementing policy. Managers policing those policies. Managers punishing people who fail to adhere to the new policies.

But we know it’s not right. We know it’s not mean to be that way. Moreover, we’re empowered, self-organizing, self-managing teams. Do we need them any more? Can’t we vote these charlatans off the island?

Well, as someone who’s gone back and forth between individual contributor and management roles more than once I can tell you without doubt, the answer is no. Sorry about that. Yes, you really do need some organizational structure in most contemporary organizations. You do still need departmental directors and line managers and so on.

Regrettably there is an extraordinary amount of work to be done that is best suited to those outside a basic agile team. There’s financial stuff: setting budgets, planning growth, managing remuneration, capital expenditure etc. There’s medium and long term planning, setting strategy. There’s market research, marketing and PR for a group, internal and external. There’s HR, pastoral care, career development. There’s departmental coordination, networking, finding internal opportunities, connecting people together to achieve things neither could independently. There’s politics: negotiating for office space, people, budget, processes and practices that are fair and equitable. And there’s more than this too.

Could you turn every group in a large organization into an agile team, with their own backlog? Could you treat them all as peers and dispense with a great deal of hierarchy? Well maybe. But it’s a long time until something like that becomes the norm, and in the interim, even if we are in the midst of an agile revolution, beheading management is not the way to go, even if there are some that deserve it ;-)

Seven Deadly Sins of Scrum

2010-10-21T08:43:00.000-06:00

1. Wrath

Something that anyone on a Scrum team might succumb to with the frustrations of learning to work a new way. But this might be something the Scrum Master could be especially tempted by. As guardian of Scrum principles it can be frustrating sometimes to see people doing it "wrong." But it's important to remember that people will often learn best by doing, by earning the knowledge for themselves. Scrum is a huge cultural change and requires patience as well as guidance. Besides, the basics of Scrum are very simple and few things are truly wrong so long as you manage to keep to them. Even then, sometimes letting people step outside the framework for a bit so they can compare may help.

2. Greed

When the stakeholders or the PO start to see productive, reliable outputs from the team and take note of velocity it may be tempting for them to ask the team to do more than their historical accomplishments indicates is feasible. "Can't you just squeeze in a few extra points?" This must be guarded against, because allowing such a thing runs the risk of people cutting corners, or even gaming things so it *appears* more points are being done per sprint by falsely inflating estimates. Trust the team to work hard at a professional standard and use velocity for what it was intended: predictably planning what will be done by release time.

3. Sloth

To a large extent the whole-team emphasis on committing to a body of work and getting it done by end of sprint coupled with daily stand-ups leaves fewer places to hide for slackers than a waterfall team. Nonetheless, calling out people not pulling their weight can be uncomfortable. Traditionally a persons line manager would be responsible for this, but in a self managing team there is the possibility of everyone holding each other accountable for producing. Sloth harms the whole team and cannot be ignored. Sometimes people need to be "voted off the island."

4. Pride

We should always take pride in a job well done, but this should be a shared thing. A scrum team thrives on respect for and collaboration with each other. Ego-centric prima donnas have no place. It’s a team thing. Similarly, whilst great talent is always appreciated, singling out "rock stars" and putting them on a pedestal is counterproductive.

5. Lust

Yeah, um. Usual dating people at the office advice stands methinks.

6. Envy

Yeah that other Scrum team may have their own team room/bigger whiteboard/better computers/more interesting product... Whatever. Find the interesting angle in what you’ve got, work to improve what needs improving and stop coveting thy neighbor’s 27” dual monitor set up.

7. Gluttony

Sprint planning can take a while. With inexperienced teams I've seen it start with a late breakfast and run past lunch. Too many bagels with lashings of cream cheese, Danish pastries and pizza for lunch will catch up with you. Perhaps it's a good thing you will be sprinting afterwards!

Requirements are dead. Long live requirements.

2010-10-19T15:32:00.000-06:00

One of the first “Agile” books I read that really inspired me to change how we went about creating software was Mike Cohn’s User Stories Applied. I think it’s because, perhaps more than anything else in my career in software development, I had seen problems that boiled down to “requirements problems”.

We were really suffering from some of the classic problems mentioned in Mike’s book including:

People not really understanding what needed to be built or why, but just focusing on making sure the software did what the requirements said because the requirements couldn’t possibly be wrong...
People obsessing over where commas went (when ironically the basic words in play were not always comprehensive enough for us to worry about punctuation that much)
People testing what they thought the requirements meant, rather than what the customer needed.
Requirements fatigue – huge weighty tomes of requirements typically starting “the system shall…” and yet rarely any decent narrative providing an overview. This kind of stuff would put an insomniac to sleep.
Ambiguous wording, cryptic wording and just plain iffy writing: many times things were being phrased to suit a business analyst, software engineer or tester’s view of the world. Often this rendered the document almost impenetrable to our customers who we expected to sign off on it(!)
Conversations and emails that involved convoluted phrases like “…but don’t you realize that requirement 2.7.3.2.17b clearly means that in this context…”

There was so much in that book that had me nodding my head and thinking “Yep!” with every page. Of course criticizing traditional style requirements documents isn’t hard. I don’t know that I’ve ever met that many people who don’t have a story or complaint about how bad they can be. They’re an easy target and I have something more useful (I hope) to say. I’m not going to rehash what user stories are or how they may benefit you. If you’re not already familiar with them I heartily recommend that book mentioned above. For a more immediate introduction hit up Google and you’ll find plenty of material.

What I want to talk about is how to balance working with stories with environments that are looking for something a bit more traditional when it comes to requirements specifications.

The industry I work in provides services to the pharmaceutical and biotech industry. The work we do is subject to FDA guidelines and the audits (from internal quality groups, customers and potentially the FDA themselves) that go along with them. This has bred a certain set of expectations amongst auditors, not least that there will be a classically styled requirements specification from which they can trace design, implementation and verification and validation of features. Overcoming this expectation may eventually be possible. Perhaps even as soon as one, maybe two decades ;-) but for now trying to do so is tantamount to pushing water up a hill with a rake.

Given this situation, we (and I strongly suspect we are not alone here) still need to produce something that we can call a requirements specification, print out and sign and show some kind of traceability to our tests. There have been a handful of ideas over time on how we might do this. Different Scrum teams in my organization have tried different things, but I’m not sure that any of us yet have got this completely licked.

The first approach was really just business as usual. That is to say that although we had a backlog of stories, they were really just placeholders for work. We still captured the detailed aspects of what we were building in a traditional Requirements Specification. Our “Definition of Done” actually included “update RS” as an item for every story. This idea more or less worked, although it felt a little unwieldy. All the classic problems of big RS documents remained with us, we just bit things off a story at a time which definitely aided us in some ways (not least by eliminating the need to write the entire thing and sign it off before beginning to work).

We also flirted briefly with use cases, which if you do even a moments research, will allow you to find various “templates” and so forth for authoring them. The problems we had here were twofold: Firstly, a good use case is actually larger than a story. So with us trying to take story-sized pieces of work and describe them in use case format things got kind of artificial and we created more work for ourselves. If we’d carried on down that path we would have had innumerable documents, since each use case was its own Word file. Secondly, use cases seem best suited for very GUI centric functionality with obvious human “actors” interacting with the system. For web services and other non-GUI work they seemed quite hard to do well, seeming to come out quite convoluted. Sure, you can have the other system as an actor but nonetheless things came out weird. Maybe we sucked at writing use cases, but in truth I suspect they just really weren’t the right tool for capturing story requirements.

Around this time I had just finished reading Gojko Adzic’s Bridging the Communication Gap which introduced me to concepts like specification by example and acceptance testing. These ideas resonated with me and seemed like useful, sensible things we could try out. As an alternative to the use cases we were writing I proposed a story + acceptance criteria with examples template (heavily influenced by Gojko’s book) that I thought we might try. In actuality this never caught on. Perhaps it’s just as well because with hindsight, the idea of requirements scattered across hundreds of documents violated one of my own big pet peeves about our prior approach: no good narrative providing an overview of different product areas.

One reason that idea may not have got traction was perhaps due to the fact that we had started to make use of the Rally tool for agile project management. I’m actually not a big fan of Rally’s tool (more on that here if interested) but nonetheless, in our "new toy fervor" there did seem to be an exciting chance to do away with Word document based requirements altogether. In our environment this was a radical proposal which I firmly supported…if only it weren’t for those auditor types who want to see one. As a response to this the idea of exporting story details from Rally emerged. This made sense at the time too. One of the pieces of advice I’d seen from several people was, “If you need to keep a written spec, just keep the story cards” (or in our case, the story records in Rally). Rally’s product has a set of web services and various ways to interact with them, including a simple Ruby script for importing and exporting data. It does work, but it’s pretty simplistic at this time and requires quite a lot of effort to convert the CSV output into anything resembling a normal looking requirements document. It would certainly be possible to hack away on that script or create your own variation on the theme that was a bit more geared to the specific task of creating this kind of output. But, I’m no longer convinced that’s what we want to do.

Most recently, in anticipation of a brand new product starting I’ve been reading about and dabbling in behavior driven development (BDD) and acceptance test driven development (ATDD). In particular I’ve been looking at Cucumber and Cuke4Nuke (which enables one to write “test fixtures” in C#). See here for my first post after encountering Cucumber. Something this exposed me to was the “Given/When/Then” (G/W/T) form of expressing requirements. Even without automating your tests and doing full ATDD this simple approach to describing requirements seems to be pretty powerful. In many ways the G/W/T stuff seems to stand on the shoulders of use cases, but feels more lightweight, simpler and provides an easier time of things when it comes to splitting features into smaller parts.

One thing that I thought had lodged firmly in my mind after all this was the idea of tests as your specification, or “executable specifications.” The concept is simple: if your tests are readable and express what the software should do, then the running of them indicates what it actually does and you’re never playing “catch-up” with an old school requirements specification.

And yet…clearly this idea hadn’t stuck. My team was discussing last week whether we had to keep our RS for our legacy product and whether we could maybe forgo such an artifact for the new product we will be starting soon. We got onto talking about how we couldn’t just document things in Rally because the export wasn’t good enough for what we needed. I resolved to try and do something about this by investigating the export script. And I was doing so when I realized that this was completely the wrong problem to solve. I hadn’t seen it as clearly before, but I realized then: stories are transactional. They only make sense if you review them chronologically because story #42 can completely modify the feature you put in through story #7. Keeping your stories and transforming them into some kind of document as a surrogate requirements specification isn’t the right approach. Instead, we should be transforming our tests into a specification because they represent exactly what the product is meant to do in a much more succinct way than a huge transactional list of stories.

Of course to do this one needs tests that are human readable, and not just by humans that can interpret code. This brings us back to the Given/When/Then format, because it sits natural language descriptions on top of automated tests. And, lo and behold, it’s entirely possible to very easily transform your Cucumber feature descriptions into various outputs, including a PDF.

When we first embarked on transitioning to agile software development I wrote a series of four blog posts about our early experiences. I concluded with looking forward to where we would need to focus next. I said at the time I thought requirements would be a big deal for us. It’s taken longer than I anticipated, but I think now we can start to see a path forward.

We don’t want to have a big old requirements specification that we maintain. We do need to have something that helps us meet our regulatory obligations. ATDD and the G/W/T approaches can help us with this. We can generate what we need as a natural part of a work that’s far more integrated into software development than a traditional document ever could be. Requirements are dead. Long live requirements.

Commitment: what can we really commit to?

2010-10-05T11:56:00.000-06:00

I want to discuss a foundational topic that influences the management of software development: commitment. In particular I’m looking at this from the perspective of a business that develops software not for sale, but as an enabler to their operations. As such a business we commit to customers that we can do things. That we can do them by a certain date. That we can do them to a certain level of service, or quality. These commitments cascade down as implicit obligations for the operational teams that conduct the sold work and those that support them.

Except…in reality, no matter how much we want it to be so, software development teams creating products can’t really commit at this level when we have no idea what promises are being made or complexities are involved.

I know. That sounds pretty bloody useless. But it’s true. And to think otherwise is naïve optimism that is just waiting for disappointment.

What we can commit to is releasing working software regularly. In fact, when you’re working to Scrum principles, at the end of any sprint the software should be “potentially releasable.” If you can handle it that often we’re ready to go.

In many businesses dates are already determined, and features are already determined. Given that resources (that is, the people available) are also largely determined (“no you can’t hire any more people, make do with what you’ve got”) it’s rather apparent that all three variables of software development are being fixed. (We’ll assume for now that the fourth variable, quality, is considered non-negotiable by the team developing software, i.e. they as professionals will not compromise quality as a response to the inability to negotiate dates, features and resources.)

Fixing all three like this can work with a great team and a bit of luck. However, given a bit of bad luck or changes in staff that inevitably occur over time and problems will likely arise. Whether we acknowledge this and try anyway, or wait until disaster ensues and then start investigating misses the point: this is not a reliable way to develop software.

There is a “better way”. This “better way” however requires a bit of a mind-shift for those used to the more traditional approach. At its heart is prioritization. This is where things can get difficult for people who are commissioning work. It’s natural to want to say “Hey I just want it all, and by March too, we’ve sold it so it’s basically promised. Just find a way to make it work.” For a business to allow people in this position to operate this way is, in my opinion, a little bit like the proverbial Ostrich burying its head in the sand.

Indulge me by considering the following contrived little story.

If a tap* can, when fully open, fill a 5 gallon** bucket in a minute then that is just a simple fact with which you have to contend. No amount of wishing it could do more will make it so.

Further consider: you have two buildings on fire, burning away, your garage and your house and that you’d like them to be put out. The firefighting team can deploy the water anywhere. Now ordinarily you wouldn’t get to influence what firefighters did. But imagine for a minute that you can. Further imagine that the garage contains a stash of priceless artworks, but that the house is empty. Left to their own devices, the firefighters would probably think they were doing the right thing by concentrating on the house. But YOU know better. Obviously you would have them put every last drop of available water into preserving the garage. You realize there isn’t enough water to tackle both burning buildings. You realize you can’t have them save both the priceless art and the house. You realize that demanding the firefighters save both buildings ain’t gonna happen. You prioritize, and save the art.

In other words, you have recognized that You need to decide how much water to put on which fire when, and you need to accept that you can only get 5 gallons every minute to distribute amongst them.

Once you’ve realized this is the way the tap works, and there isn’t anything you can do to make it spit out water faster then you’ve learned one of the most valuable truisms about software development. You can only do so much with limited resources. To make best use of them you need to prioritize how best to deploy them. And if you want to do more, you need more resources.

* or faucet if that’s your thing
** or, again, 18.92706 liters (or litres) if you must – as you can see I’m conflicted and inconsistent with both terminology for everyday items and units of measure after living in the US for 8+ years

Delayed Gratification

2010-10-01T16:32:00.001-06:00

(Or, "Why I believe fixed length release cycles are best")

Anyone who’s done some kind of Scrum reading or training classes will have had some exposure to the release planning techniques and ideas involved. There is plenty of information, ideas and opinions out there about that stuff, in particular how to organize your releases around themes, epics and stories, how to prioritize your backlog and how to run a release planning meeting. I, however, have been thinking a bit about things one level up from the mechanics, and here wish to discuss some of my opinions and ideas on how a business should look at managing releases. In particular I’m interested in how to balance responsiveness to changing needs and predictability on longer term commitments. I’m also interested in simplicity and optimizing the way product development works. Agile software developments offer the promise of delivering high value sooner, of responding to changing needs. But in order to deliver on this promise it is my belief that certain preexisting attitudes and tendencies that are ingrained in many of us need to be reconsidered.

In scrum, at the end of any sprint the software should be “potentially releasable.” Usually we choose not to do that though because:

There isn’t enough value to justify the “cost” (deployment, distribution, training etc.) of releasing
Customers (users too!) aren’t necessarily best pleased to have their software in a seemingly constant state of flux.

An easy way then to manage releases is to let business needs dictate release dates. This doesn’t require us to have any new special rules or planning processes, and it sounds reasonable: “Yeah we’re gonna need that BIG FEATURE out by CERTAIN MONTH for IMPORTANT CUSTOMER, so we should release then. And then we’ll fit in whatever else from the top of the backlog we can.” Sounds pretty good, right?

Ken Schwaber’s Scrum guide, in describing release planning meetings says nothing more than that they establish a “probable delivery date and cost that should hold if nothing changes.” He doesn’t seem to be advocating anything different.

But is this optimal? Are there potentially better alternatives? I think so. Here’s how I think it should be done and why.

In a nutshell I think we should take a leaf out of the finance and accounting book and use a year chopped up into quarters (i.e. 3 month long chunks) for planning and release purposes. In other words:

I wouldn’t much bother trying to look further than a year ahead
I would [try and] release every quarter (maybe sometimes you can skip, although I think if you have your deployment act together this is unlikely)
I would encourage the business to plan things around this quarterly release schedule…that is to think of things in terms of “I’d like this in the 1st quarter release” or “I can wait until the 3rd quarter release,” not “I want this in October.”

Why? Well, hopefully the idea of not looking more than a year ahead is not too controversial. All I’m saying is that things change so much in 12 months that looking further ahead is usually crystal ball gazing. That’s an idea you’ll see in many communities, whether agile, scrum, lean startup or whatever. I think it makes a lot of sense in many places, certainly in Perceptive MI.

Also fairly easy to swallow I think is the notion of thinking in 3 month chunks. Three months is around 12 weeks, or 6 two-week sprints. It’s long enough to get something of substance and value done, but a short enough time-horizon to keep everyone focused and avoid the risk of over-committing badly. It’s also probably not too long to wait for a new feature. Of course unexpected needs and critical defects occur, and interim or maintenance releases are always possible, particularly when using Scrum, since we’re potentially shippable at the end of a sprint.

But this notion of rigidly sticking to a plan of releasing every three months…I know that is a little different and needs explaining. Well, for starters it provides a nicely time-boxed predictability. There aren’t so many surprises. There isn’t the need to frequently revisit and negotiate the question of when releases will occur – everybody knows, it’s once a quarter.

This predictability is not just comforting and simple, it provides other benefits too. For starters it makes it easy to handle certain common situations. Let’s consider four:

1. The madly urgent problem/bug in production
Everyone is familiar with this situation. One moment everything is calm and ticking over nicely, the next minute chaos abounds: a bug without workaround has been found, or a really important customer has to have something and only just thought to ask.

Either way, within our proposed framework dealing with this is straightforward. There are really only two choices. The first question to ask is, “Can this wait until the next sprint?” The answer is going to depend on how truly urgent the matter is, and how long it is until the next sprint begins. With two week long sprints it’s never far away. Few problems require more immediate attention. For those that do, the team would simply abandon the current sprint and focus on the urgent issue at hand.

2. The unexpected feature
When planning for a new three month release cycle you usually have a fairly good idea of what you want to get done in a timeframe that short. Nonetheless, there are a number of very good reasons why a feature that nobody was aware of then can suddenly emerge and need attention.

This is not really a problem though when using Scrum. The Product Owner (and by extension stakeholders and other interested parties) are able to completely change the priority ordering of the product backlog every two weeks. Simply bump the new feature up to the top and it can get all the attention it needs at the next sprint. This means that, worst case, the business waits two weeks for things to start on something that was previously completely unplanned.

3. The client driven enhancement
Somewhere between the unexpected feature and pure market driving planning lies the customer driven enhancement. The distinction here is that one has the capacity to pick and choose whether to bump out current plans and place it into the release cycle that’s in progress or negotiate as needed to place it in the next.

4. The market driven plan
Above the cut and thrust of weekly operational driven needs and urgent customer requests is the longer term approach to product development. This is simply identifying desirable features based on market trends and selecting appropriate release cycles to implement them in up to a year ahead.

You see how with all of those there’s an established pattern and we work to fit things in? This reduces or even eliminates the need to frequently invest energy into discussing and negotiating with colleagues outside the Scrum team (IT, domain experts, business development, professional services etc.) over when a release will occur. The schedule is published and everyone can consult it and even, with time, comes to anticipate it. Things will happen without energy needed to push or chase it: IT can grow to anticipate doing deployments for example. Business development can talk confidently about the likely lead time for including new features to service particular customer needs.

Finally, ultimately, it encourages better planning and discipline on the business as a whole, from sales through to professional services and product development. I believe this leads to a better run business, better thinking, less burn out, and people selecting the right opportunities to pursue. I cannot see how that is a bad thing :-)

For me, this approach to managing the release cycles for a software product intuitively makes sense. The combination of simplicity, predictability and responsiveness that can be achieved is highly desirable and a worthwhile reason to move away from the more traditional approach to variable length release cycles driven by external events. What do you think?