Every 5,000 miles or so I take my car in to the shop for an oil change. It’s part of the routine, preventative maintenance I do to keep my car running in tip top shape. Routine maintenance gives me the peace of mind that my car won’t leave me stranded somewhere, extends the life of my car so I’ll get the most out of it, and makes sure my car is always functioning effectively which is extremely important with gas as expensive as it is. Routine maintenance just makes sense – cars are expensive and mine plays an important role in my day-to-day life. So it is only logical that I would want to take good care of my car.

Routine maintenance is good for other things too. I go to the dentist twice a year for a routine cleaning and inspection to make sure my chompers are in good form. I remove the cruft from the gutters on my house once a year so they don’t clog and cause bigger problems. I take my dog to the vet for blood work and a general checkup every year for her birthday to make sure she stays happy and healthy. I don’t want to lose my teeth. I don’t want the gutters falling off my house. I don’t want my dog getting worms or licking my face with bad breath. Regular checkups and maintenance help me avoid these things.

Routine maintenance is important for software teams too. Part of the total cost of ownership for any software development team is the cost of the routine maintenance that keeps that team healthy, happy, and working at peak efficiency. The last thing I want is for my team to break down in the middle of an iteration. Even a team failing to live up to their full potential is something that can be avoided with simple routine maintenance. Burnouts, flare-ups, missed commitments, firefighting – these are all signs of wear and tear that must be repaired to bring a team back to center. Wouldn’t it be easier just to avoid all that though? Preventative team maintenance is the single most cost effective way for ensuring teams have a full and happy life, operate effectively, and won’t break down at critical times such as in the middle of a project.

Teams don’t have oil to change every 5,000 miles, but there are still activities that should be a part of every team’s regular routine maintenance plan. Here are some of the things I’ve found useful on my teams.

Retrospectives and reflection – This is the oil change, the physical, the teeth cleaning of the software development world. Every member of the team gathers to think about and discuss how their practices are actually working for the team and project.

What do I do? At its most basic, a retrospective is a ritualistic, round table discussion where everyone on the team has a chance to share their thoughts and feelings on how the team is doing. Agile Retrospectives” by Esther Darby and Dianna Larson is a great reference for getting started with team retrospectives.

How often? Retrospectives should be done once every iteration, generally at the end. If iterations are long it might be worth considering doing them more often. If using a continuous release schedule, pick a date every 2 weeks to 3 months to reflect as a team even if you use continuous improvement practices.

Exploration – Always doing the same thing day in and day out creates ruts in our minds and spirits. Most people who are great at developing software will get bored if left to do the same things for too long.

What do I do? Toss things up by letting folks try new things. This might include running an experiment with a new technology or methodology, learning or taking full responsibility for a new-to-you team role, or taking on non-development tasks like writing documentation or coaching. Carefully consider risks when exploring and mitigate appropriately to ensure you still satisfy your threshold of success.

How often? This depends on the team. One tool I’ve found useful for understanding how prone to exploration teammates might be is the VIEW Problem Solving Style assessment. Retrospectives coupled with solid software risk management will also help determine the right amount of exploration.

Upgrade tools – Sticking with old and busted when there’s a new hotness available not only makes us feel unprofessional but can also hurt us.

What do I do? Buy the best you can within the technical constraints of your project.

How often? Upgrading doesn’t have to happen with every new release. There is a diminishing return by upgrading too often. Major upgrades that only happen every few years for tools that are your team’s lifeline (like, say, Visual Studio) are a great candidate for upgrades. For less critical applications like Office you might skip releases. And if the team says a tool is critical then don’t give them a hard time – figure out how to meet their needs!

Quality downtime – Vacation and holidays, time away from computers, projects, and even each other helps keep the team fresh and ready for work.

What do I do? Encourage folks to leave the office every once in a while. Part of this is culture – make it OK to go on vacation and *Gasp* not check your email! The big trick is that this needs to be quality time off, not just time away from the office.

How often? At least 4 weeks a year, more is better. That’s enough for the standard American bank holidays plus another 10 days. Time off should be taken in good size blocks, spread out throughout the year (keep in mind that American holidays are sparse throughout the spring/summer), and preferably not interfere with day-to-day team operations. With enough forward notice, anyone should be able to take a vacation whenever they want.

Status meetings (with visible progress) – Check the pulse and blood pressure of the team.

What do I do? Get together to review progress, risks, tasks, sanity, problems, and make sure there are no big surprises. The key here is to get a real sense of the important things the team wants or needs to track. Big visual charts are great for this – use data when possible to show real status. A round table will catch the outliers that can be challenging to measure.

How often? Generally a one hour weekly status meeting is enough, even if you’re already doing daily stand-ups.

Parties, downtime together – Take time to celebrate your successes! A team that parties together is a team that builds great software together. Do things with your teammates other than writing software.

What do I do? The sky’s the limit – happy hours, LAN parties, hikes, concerts, dinners, lunch. You don’t have to have the whole team for every event, but folks should feel comfortable hanging out together. Designate a social committee to come up with fun things to do or make it a team role that moves around.

How often? At least once every iteration or once a month. Just go have fun and you’ll be OK.

What else?

Look, you’re going to pay to keep teams working one way or another. Either you’re team will break down and you can pay to rebuild it (which may not always be possible) or you can set aside the time and resources to do the routine maintenance. You don’t always get to choose when a catastrophic breakdown will happen, but you do have the choice of when to schedule your next oil change. Retrospectives are a no brainer – just do it.

When building relationships we look for commonalities between ourselves and others. When we have things in common it makes us feel good, like we’re not alone in the world. And the things we have in common become the basis for lasting relationships. Homogony in a group brings comfort, you know what everyone else is thinking, you can predict how the group will react to new ideas, and you become confident, even emboldened because you are surrounded by people who are similar to you. Let’s face it, it feels great when you share a new idea to nodding heads of agreement and a strong show of support from your peer group.

Over time you’ll find your niche, a great group of people who think like you and see value in the same things you do. The more time you spend together the closer you’ll become and the more awesome your community will get. You’ll feel supported and you’ll give support to your peers. Together you’ll build an exclusive community of trust, mutual respect, and consensus; one that feeds on itself with demonstrations of perpetual support free of criticism, negativity, and ridicule.

With a large enough group, it’s possible to warp reality around yourself so that you are only exposed to the ideas and feelings produced by your group, the place where you feel safe, comforted, and welcome. With so much support it is easy to dismiss ideas created outside the group as “radical” or “stupid,” and miss out on huge opportunities in the process.

This mentality is everywhere in the software world.

The software craftsmanship movement largely molded by Pete McBreen in his book pitted the enlightened craftsmen against the ignorant engineers. I strongly identify as a software engineer and was duly offended by this book. It wasn’t until about mid-way through the book, when McBreen suggested that the Personal Software Process was one of the greatest processes a craftsman could use that I realized we agreed more than we disagreed. A line in the sand had been drawn and two groups formed but there was more overlap between these groups than anyone was willing to admit. As an engineer I don’t want to “sweep the shop floor” as someone’s apprentice but I recognize the importance of mentoring. Software Craftsmanship highly values the contributions of the individual but recognizes that teams must somehow be organized so projects can succeed. Software engineering (at least as I’ve always thought of it) and software craftsmanship are nearly identical ideas and McBreen’s book reads as a jaded rant produced after an unfortunate project. Two communities with similar passion and ideals, divided by a label!

Agile and architecture are two other groups that (only until very recently) seem to ignore each other out of spite or perhaps ignorance. The main principle of the agile movement is a focus on delivering meaningful software that helps solve customers’ problems: “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.” The Software Engineering Institute has spent the better part of two decades researching software architecture and how it affects a team’s ability to deliver valuable software to customers. Software architecture is a communication and organization mechanism. The primary purpose is in helping teams deliver software effectively. It is a means to achieving the very same principles outlined in the Agile Manifesto. There’s nothing wrong with documenting a design, writing down quality attribute scenarios in addition to user stories, or making a plan before your start laying down production-quality code. It is true that the wrong kinds of up front design will do more harm than good for all but the most trivial systems, but that doesn’t make all architecture practices as bad as agile zealots will have you believe.

Is Kanban agile? Who cares?!? If the process works for your team then use it! The more important (and interesting) question we should be asking is – how do I know if my process is working?

As someone who regularly experiments and actively seeks innovation, I’ll be the first to admit that I get things wrong often, but I pride myself on not allowing my likes and dislikes, my preconceived perception of my reality, to get in the way. While I don’t enjoy leaving my comfort zone, I recognize that it is important both for personal growth and the growth of the communities in which I participate.

And when it comes to building software, I recognize that though I may have a preferred approach or process, and set of tools and languages I like, there is no one-size-fits-all solution and the most important thing is to use the best tools for the job at hand. While I may have strong beliefs they are weakly held, my one conviction is to always do what I think is best for the project and the people involved with it.

I used to think hacking was bad. It was something you did when you didn’t have a plan, when you didn’t know what you were doing; it’s what amateurs do, noodling around the code without clear direction or intent. Or it was something you did, quick and dirty, just to get something out the door. Hacking is the best, surefire way to waste time and create mountains of unmaintainable code you’ll have to rewrite anyway. I’ve always struggled with this since I feel so drawn to hacking culture – DIY, playing with things just to see how they work, ignoring the instruction manual, reveling in the aesthetic of a job well done, making what you can within limited constraints – and many of the brightest minds in software development speak quite passionately about the merits of hacking.

Thinking about this for a while, I’ve come to the conclusion that there’s nothing wrong with hacking. Quite the contrary, hacking could be one of the most positive things a team can do. In fact, many of the agile processes and techniques that have worked for me in the past seem to be designed to cultivate teams of hackers.

It seems I had fallen for some of the myths about hackers.

Myth: Hackers don’t have a plan

Hackers do have a plan but they acknowledge that the plan will change and then plan with this in mind. This means creating plans only to the level of detail to which you are currently comfortable making plans. For example we all know that making a detailed plan at the start of a project, down to the code level, for all but the tiniest programs, is silly since the plan is mere conjecture of a positive outcome. Rather than writing a fantasy on paper, hackers lay plans based on the current knowledge. These plans then expose areas for further exploration (hacking!), the resultant knowledge of which can be turned into more detailed plans.

In XP these detailed plans are created just in time during the iteration planning game, sometimes even later than that. That’s not to say that there wasn’t a plan before the iteration began. Not so. A great degree of planning, communication, and coordination was required to be able to plan so adeptly, so quickly.

Myth: Hackers don’t know what they’re doing

In the literal sense, a lot of times hackers don’t know what they are doing. That is why they are hacking. The purpose is to figure out how something works or the best approach for something. And that’s the key. At a meta-level, hackers know exactly what they are doing – they are searching for information, for knowledge, for a means to achieve an end.

Action research is a relatively new-to-software-engineering research method that seemed to take agile researchers by storm at XP2010. The basic premise is that a researcher works closely with a consenting organization to observe practices in software engineering. Rather than acting as a passive observer, as is traditionally done, the researcher injects practices directly into the team and observes before and after reactions. For example, in recent work by Tor Erlend Fægri, having observed issues with team collaboration he suggested several techniques to help solve the problem. Some suggestions worked, some didn’t. Fægri was able to take the time to understand what happened and why because he was dedicated to understanding almost as a participant, watching the events unfold. This take on research makes me think of Jane Goodall’s work with chimpanzees in which she directly interacted with wild primates in order to better observe their behavior.

When I first heard about action research it sounded a lot to me like hacking – trying things out to understand how something works.

Myth: Hackers don’t create value

Yes, the code that comes out of a hard core hacking session might be a big pile of spaghetti, but consider the purpose of this work in context. Another perspective is that hacking may incur technical debt which sooner or later will need to be paid off. But it’s important to keep in mind that not all technical debt is bad. When you are hacking, you are either in learning mode or in finish mode. Learning mode is never wasted effort. Finish mode, where you are just trying to get something finished so it can be used, can leave scars on your code, but the immediate value provided to customers can be enormous. When you have a desperate and immediate need, anything that works today is better than something perfect that works next week.

The same goes for architecture. Of course, many of the architecture researchers use words and phrases like “experiments” or “trade-off analysis” to describe this same act of quickly creating something for the sake of knowledge. That’s really just hacking too.

Sometimes the artifacts you produce are throwaway. Sometimes the working solution is good enough and you’ll never have to worry about it again. Sometimes you’ll have to pay off the technical debt. But hacking for the right reasons is never wasted effort.

Agile Development Helps Cultivate Good Hackers

Agile software development, be it controlled through time boxes or flow, is all about ensuring that work doesn’t teeter out of control a day at a time. The focus on providing customers with value creates an environment that encourages action and creation. The agile manifesto gently nudges hacking behaviors. Frequent releases of working software, welcoming changing requirements, focus on simplicity and continuous attention to technical excellence, helping the customer understand what they need and then doing your best to make it for them. Even the idea of continuous process improvement is about hacking your team to make yourselves better. Hacking is the very lifeblood of agile software development.

Hacking isn’t a dirty word. Hacking could be the most important thing you do all day.

[This post is a recap on the second talk I gave at XP2010. This was the big one, the experience report talk, one of 15 experience reports published at XP2010. You can download the slides (pdf) or the full paper (pdf) from this website or from XP2010.org.]

Process improvement is important for nearly all teams but it can sometimes be difficult for a team to know what is working, what isn’t working, and what techniques or methods to try when attempting to improve. Performing a scientific experiment is one way help overcome these problems but as academic research has shown us, while experimentation can yield interesting results, running an experiment is time consuming, expensive, and requires some serious thinking and control to pull off. From a practitioner’s standpoint this means that experimentation is a non-starter.

Of course, that’s only if you run experiments like an academic.

Back Story

Just over a year ago, my MSE studio team at Carnegie Mellon had a problem. We had decided we would use Extreme Programming for the construction phase of our project but some team members had doubts concerning pair programming. We had decided that we would use some kind of peer review, having already seen the many benefits of inspection when reviewing other artifacts. The dispute arose over whether pair programming would give similar enough results. Also, not all team members had experience with pair programming but everyone on the team knew and enjoyed solo programming.

The number one concern was whether pair programming would allow us to meet our very strict deadline. We had just over three months to complete the construction phase of the project. According to our threshold of success this meant implementing all “must have” requirements with a minimum level of quality. Did we really have time to waste by having two people working on the same code at the same time? Wouldn’t working independently and inspecting code on an as needed basis allow us to get more work done faster?

At the time it just so happened that I was taking a reading class with Mary Shaw and in that class we discussed some research findings that might help settle this debate. Research from Laurie Williams, Ward Cunningham, Barry Boehm, and many others showed that pair programming requires more effort (although never double the effort) but is faster than programming alone (pdf). Also pair programming creates code of about the same quality as coding alone with inspection (pdf). Of course, the research may not apply to us since Square Root is closer to a professional team working on a large project with a real client, not undergrads working on short term toy projects.

After an iteration where some teammates used pair programming and others refused, we decided to try an experiment to see which practice actually worked better. The original idea was that we might be able to validate some of the research but decided instead that it was more important just to resolve our own internal conflicts and figure out which processes worked better.

Conducting a Lightweight Experiment

With the scientific method as our guide we planned and executed a lightweight experiment which pitted programming alone against pair programming. The results were amazing (and you can find the raw data in our project archive). In conducting the experiment we used a set of novel techniques which I think can be useful in conducting other lightweight experiments. There’s more background in the experience report so I’m only putting the meaty stuff in this post.

Focus narrowly on a single question – The essential key to keeping an experiment light is to only tackle one thing at a time. In this case we focused on comparing and contrasting a single technique, pair programming, rather than multiple techniques or an entire process (such XP vs. TSP).

Divide work, not teams – If I were comparing pair programming to programming alone in an academic setting, I would put together two teams of about the same experience and have them each build their own version of the same software, one team using pair programming, the other programming alone. In a business setting this is a complete waste and few companies can afford to have two teams duplicating effort. By dividing work instead of teams you may lose some control over variables in the experiment but in most cases isolating more variables doesn’t add any further clarity to helping answer the narrowly focused question. To divide work successfully you need to have some way of estimating work units for division. We used use case points as shown in the figure depicting our modified planning game.

Continue making releases – Since we still needed to make a comparison, rather than dividing into teams and duplicating effort we divided the features that were released each iteration. In this way we built about half the features released during an iteration using each technique. Working on about half the features using pair programming meant that at least some features were being built by individuals. At the time this was a risk reduction decision to make sure that if pair programming completely failed we’d still have something to ship at the end of the iteration. Explicitly managing risks is the only way to know if the lightweight experiment may cause problems for making releases. Also, we had a strictly defined cut-off for stopping the experiment if it ever stopped us from shipping to our client.

Use the data you have – In almost all cases we were able to get the data we needed to evaluate our hypothesis from our current process. When we couldn’t, we only had to make minor modifications to our data collection practices, for example adding a check box to our SharePoint server for indicating whether a task was paired or individual.

One of the more interesting things we did was to create a “tally sheet” for collecting pair programming issue detection statistics in real time, as the issues were discovered. Given the near instantaneous code-inspect-fix cycle when programming in pairs, this was the only way to collect similar data for comparing pair programming to inspection.

Statistical significance is overrated – The whole point of running a lightweight experiment is to collect just enough data to help you make a better decision or validate your gut feeling. This technique is not meant for uncovering universal truths or proving something to the rest of the world. In exchange for keeping the experiment light, the results will only apply to your team. Over the course of an iteration or two, 4-6 weeks, you’ll only get enough data to start to see trends. In our case the results were not statistically significant using individual T-tests but that didn’t matter. The most important thing is that we had data that could be used for comparison, data that everyone felt good about and that helped us gain clarity into what we did and how well it worked.

Retrospectives get immediate value – The whole reason the experiment is light is to reduce cost and decrease the lag time to providing value to the team. Just to give you a little perspective, it took us 6 weeks to run the experiment and had enough data and casual observations to make a decision during the retrospective when the analyzed data was shared. That event occurred in early August of 2009. This experience report required almost nine full months of gestation from the paper proposal to the talk I gave at the conference. The gestation period on “universal truth” research can be even longer. We, as practitioners, don’t have to wait for those universal truths to be born to get value from research. By running your own quick and dirty, lightweight experiments, you can get results in a timely fashion that you know will apply to your team because your team was the subject of the experiment. It’s all about closing the gaps between research and practice and taking the information you need now instead of waiting for academic research to catch up.

Overall Conclusions

For the Square Root team it turned out that pair programming was faster, cheaper, and produced code that had more predictable albeit slightly worse quality. The more important lesson is that we discovered a technique, lightweight experimentation, for learning other interesting things about our team and about software engineering in general.

My paper and this blog post were all about trying to describe the technique, using our experiment as an example. I think it would be awesome if teams around the world conducted lightweight experiments on a variety of topics. If enough folks share what they learn, we might start to see trends emerge across teams that could lead to universal truths, validate research, or at least discover some great rules of thumb.

What else might make for a great experiment? Anything you’ve got a question about on your team!

• What is the clearer way to write requirements, user stories or use cases?
• Which estimation technique is more accurate of X and Y?
• Can we skip unit testing if we use inspection (looking at quality, knowledge sharing)?
• Is UML a better design notation than the one we made up as a team?
• What else…?

If you do a lightweight experiment, let me know! Share what you learn as a blog post or whitepaper. Let others know what you’ve learned! Even if the specific results only apply to your team and the way you’ve executed your project, your experiences help form a baseline, a sort of shared understanding for how software development works, how some of these practices work. And there’s so much about software engineering that we have yet to learn.

Acknowledgements

This paper was my first experience report and it was an awesome journey. Naturally a lot of folks helped me along the way and I would like to take a moment to make sure they know that I appreciate their influences and support. The Square Root team: Marco Len, Yi-Ru Liao, Abin Shahab, and especially my fellow experiment co-champion Sneader Sequeira for having the guts to go along with this idea in the first place. Some of the faculty at Carnegie Mellon: Dave Root and John Robert (my studio mentors) for bringing up the idea of writing a paper, and Jonathan Aldrich for helping review my proposal. Artem Marchenko was my XP2010 paper shepherd after the proposal was accepted, and the quality of each draft only improved because of his inputs. A group of my fellow employees at Net Health Systems sat through an early draft of the presentation I gave and shared valuable feedback for improving it. And finally I thank, Marie, my wife, who was with me from start to finish and read more drafts and sat through more practice talks than anyone else. She’s probably as much an expert on this subject by now as I.

A Final Aside

I wrote the initial draft of this paper as my final reflection paper for my Master of Software Engineering degree (pdf). That draft has a very different tone, approach, conclusion, and direction than what I eventually published for XP2010. This is half due to there not being a hard page limit but also I had a lot more time to think about what was really important when writing for XP2010. There’s some interesting information, mostly in the lessons learned, that might prove interesting to those who are interested. You should check out my Square Root teammates’ reflection papers as well since they are all interesting and well written.

As I’m writing this I am making final preparations to leave for Trondheim, Norway to present an experience report at the 11th International Conference on Agile Software Development and Extreme Programming, or XP2010. After my experience report was accepted, the conference organizers opened up a second round for lightning talk proposals, and in a moment of whimsy I decided to propose another talk. That was accepted as well. So I’ll be giving two talks at XP210.

If you’re at the conference I would very much like to meet you! This is my first time presenting at a conference and only my second conference attended (my first was OOPSLA this past October) so I’m a little nervous and unsure what to expect. Above all I hope that I’m able to share some information that is useful, interesting, and inspirational. I think I have some interesting insights and perspectives that can help make the way we build software just a little bit better and I want to hear your opinions, your thoughts and experiences too. If you can’t make it to my talks, come find me, I’d be glad to discuss any ideas with you. I plan to have a copy of the paper and slide decks available on this website along with a brief synopsis of the talks shortly after the conference. And of course I’ll be looking for people to eat meals with and hang out while I’m here on Wednesday and Thursday. I’m here to present, to learn, and to discuss cool ideas with other practitioners.

Here’s a summary of the two talks I’m giving along with some background information. The background isn’t necessary but just interesting information and context related to the talks.

Put it to the Test: Using Lightweight Experiments to Drive Process Improvement

This experience report tells the story of how my team ran a lightweight experiment to figure out whether we should use pair programming or program alone and review our code with Fagan inspection. With only a few hours of work and only a few weeks time we discovered that pair programming was instrumental to our eventual success.

In this talk I will discuss what we learned about setting up and running the experiment so you can run lightweight experiments of your own on whatever topics your team finds most interesting or pressing. Experimentation doesn’t have to be this overbearing, lofty, academic thing that it has become. My hope is that teams around the world will use this technique to discover just a little more about how software engineering works and that they’ll share what they learn in white papers, blog posts, and future experience reports. We can help close the gap between research and industry with just a sprinkling of scientific thinking.

Background Information

• Paper and talk abstract
• Pair programming experiment work plan as executed by the team (archived)
• Square Root’s team reflection blog
• A brief analysis of why pair programming seemed controversial to the team while we were planning.
• Complete Square Root project archive (You’ll find data from the experiments along with tons of context information on the project. This is a very raw archive, but there’s tons of great data if you’re willing to root around.)

Lessons from a Software Engineering Dojo: The MSE at Carnegie Mellon University

At OOPSLA in Orlando, Florida this past October I heard proclamations like this more than a few times: “The only way to teach software engineering is through experience. What we really need is a software engineering program that uses a capstone project, a non-trivial, long term project that lets you practice what you learn. No university programs currently have such a project in their curriculum.” I agree. In fact, I agree so much that I attended the only university in the world in which a long-term, realistic (in both scope and complexity), team-based capstone project is an essential part of the software engineering curriculum.

I have two goals with this lightning talk. First, I aim to spread the word about the existence of the Master of Software Engineering program at Carnegie Mellon University. Carnegie Mellon is on the forefront of software engineering education research and the Master of Software Engineering program has been teaching professional software engineers to become true masters in the field for over 20 years. Second, since the studio component (the capstone project) of the MSE degree is so similar to an industrial setting, there’s a lot industry teams can use for training and educating software engineers as you work. There are tons of lessons that can be taken from the MSE in the form of both research and battle tested experiences. This second goal will be the greater emphasis of the talk.

Background Information

• Lightning talk abstract
• The 9 crafts of software engineering proposed by Alistair Cockburn
• Master of Software Engineering homepage
• Wikipedia entry about the MSE program
• MSE studio projects archive

“Better is the enemy of good enough” is a phrase often held up as the reason for not making changes on a team. If everything seems “good enough,” the effort to make something better is regarded as waste. A lot of times, “good enough” is defied in terms of “providing value to the customer,” often stated as the “shipping working software” metric. So if you are shipping working software and receiving generally positive feedback from your customers, then what you’re doing is good enough and there is no need to do things differently.

But if good enough is really all you need, why is it so dissatisfying?

I’ve mentioned before that it’s a good idea to have a project threshold of success, a set of minimum goals that must be completed for a project to be considered successful. Failure to meet all goals in the threshold of success means you’ve failed the project. So while you will succeed if you meet your threshold goals, only meeting the goals means you’ve done the absolute minimum amount of work necessary to be successful. Satisfying the threshold of success for a project is like getting a C in school. It’s certainly good enough, but it isn’t exactly awesome.

While “good enough” is perfectly acceptable (You didn’t fail!), it always feels nice to achieve more. It feels good when I’m able to gently exceed my client’s expectations. It also feels good to do things right and not merely get things done. This is why merely shipping software, the minimum requirement for succeeding on a project, is not enough for me.

Equally important to me is how the work is done, not just that the work gets done. Is overtime or heroic effort necessary? Are we reflecting on our work and attempting to improve? Does my team work together well, take risks, and innovate in many different areas of the project? Do we try to use data to understand what is happening in the project? Are teammates given the support they need to grow as professionals? Am I having fun, looking forward to work every day, and happy with my contributions on the team? Is the software something I am proud of and actually useful to people? Is the product well supported by documentation? Is my code beautiful and maintainable? And most importantly, did I either learn something new or achieve something great while working on this project?

This is actually a lesson in understanding what “good enough” really means and why tools such as Threshold of Success are so important. Only when everyone agrees at a conceptual level that what the team is doing is “good enough” can everyone on the team move forward and be happy. Sometimes this will mean getting working software out the door no matter the cost. For most teams “good enough” will be a mix between a working software product, a happy and healthy team, and laying foundation for future work.

Making improvements in how my team operates or to the software itself just feels good. It’s fun. So while “better is the enemy of good enough,” avoiding change is paramount to avoiding the very things that makes me happy. The trick is making sure that I’m not making changes just for the sake of change. Sound engineering and a good understanding of the project’s threshold of success can help to avoid this fate. Once you’ve met your threshold, earn some extra credit by improving your code base or making process improvements. Or, better still, choose a threshold of success which requires you to do just a little more than the bare minimum. Because no one should settle for good enough when awesome is within reach.

This post is a recap of a talk I gave this weekend at the Carnegie Mellon University Master of Software Engineering 20th Anniversary Mini-Conference. I’ve made the paper this talk is based on (pdf) as well as the slides I used during the talk (pdf) available. I’ve also linked to as many of the primary sources I used in research as I could so please, check out those papers if this is something that interests you.

About a year ago I discussed the idea of using affordances to help figure out how to make software processes work more smoothly for a team. Back then the idea spawned from a moment of crisis and self-reflection on my studio team, but having thought about it for a while and noticing the phenomena occurring on other teams I decided to revisit this idea and see if there is a way to proactively use affordances to avert problems rather than merely explaining problems as they occur. As it turns out there is a precedent for affordance-driven design in object engineering. With a few basic assumptions I think affordance-driven design can be extended to software processes as well.

To better explain how to proactively use affordance to pick or tailor a software process it helps to know a little about the Theory of Affordances (pdf). If you’ve read The Design of Everyday Things by Donald Norman then you probably already have a pretty good understanding of how to use affordances in the context of object design and usability. From an ecological psychology perspective, affordances are a way of helping explain or predict how an animal will behave in the context of their environment. In the context of humans, our environment is influenced by our experience, culture, and background as well as our current goals within that environment.

A simple example is the play button on a DVD player. A triangle to the right means play. This is obvious to us because we know what a DVD player does (we have experience with similar devices), we turn to the device when we want to watch movies (we’re looking for the play button), and culturally, our notion of time is left to right (so a triangle pointing right means play while a triangle facing left means reverse). But what if you come from a culture where time is generally represented as passing from down to up vice left to right? In this case, a triangle facing up might be a better symbol on a play button than a triangle pointing right. The value of the affordance changed based on a cultural bias and the user’s background.

This is all well and good, but what does it have to do with software process? For the Theory of Affordances to apply to software processes (or any process) we have to assume that process is a part of the environment. This is an interesting proposition since process really only exists in our minds. Process is something that we make up, and like our understanding of an object your knowledge and experience with a process will influence your perception of that process as an environmental influence. As long as you believe in, understand, and follow a process, the steps of that process exist as much as any other object in the real world. Logically this seems to make sense. Even the US patent office will grant patents for a process just as it will a physical invention.

Back to the core problem of identifying affordances, something I did not have an answer for in my original post a year ago. As best as I can tell, the only way to identify affordances is to reverse engineer a process focusing on the affordances using a technique known as Affordance Driven Design. Affordance Driven Design has three basic steps (pdf).

1. Identify a user’s needs in terms of functions.
2. Identify the desired functional affordances necessary to achieve the previously identified user’s functions.
3. Choose affordances to design into artifacts which are mostly likely to help achieve those desired functional affordances.

As an example, consider the task of blending a drink using a blender (pdf). Typical functions a user might want to perform are preparing the blender, blending, and cleaning up afterward. Functional affordances might include the “countertop-ability” (the ease with which a blender can be moved to a countertop), the “clean-ability” (the ease with which a user can clean a blender), and “transportability” (the ease with which a user can move a blender around). A person might choose any number of blenders to perform the desired functions but each blender will fulfill the functional affordances in different ways. A hand blender is extremely portable while a gas-powered “whacker” (powered by a 26cc engine, complete with motorcycle throttle – it makes the smoothest margarita you’ll eve have) isn’t really intended for indoor use.

To software engineers, functional affordances should look very familiar – they’re essentially quality attributes. Thinking about affordances from this perspective gives us a huge advantage since, as software engineers, we are already extremely familiar with quality attributes and quality attributes scenarios. Affordances, therefore either promote or inhibit desired quality attributes in your process.

Thinking about software processes, the functions will all be related to software: writing, designing, testing, and releasing are just a few possibilities. Some process quality attributes might include:

• Plan-ability (How far ahead does a process help you to plan?)
• Predictability (How well can you see into the future?)
• Changeability (When the course of a project needs to shift, how well does the process support chaging plans or direction?)
• Quality (The degree to which your process promotes “quality”)
• Cost (The amount of resources you’re willing to spend on process to achieve specific functions)
• Harmony (How well the team gets along)
• Reliability (How consistently the process helps you perform)
• Performance (Could be speed of development or quantity of code – define what you mean in the quality attribute scenario.)

As an example, say changeability is a desired quality on your team. A specific changeability scenario might go something like this. In order to meet business needs in an aggressive market, the team needs to be able to shift focus and answer competitors’ challenges within five business days. What are the things that might get in the way of this kind of rapid change? Heavy documentation could be one thing, as it nudges teams into keeping a single course. Long iterations also make it difficult to shift focus since more effort is required to make longer term plans. Going light and having short iterations, on the other hand promotes a team’s ability to change. But like every design decision there are trade-offs. Going light in terms of documentation might make it harder to achieve certain kinds of quality, for example.

The main idea here is that it’s relatively easy to identify process affordances by thinking like a designer and applying the skills we’ve already acquired as software engineers. I propose that evaluating process affordances as I’ve discussed here is a great way to pick a process and also to tailor processes. When tailoring simply identify affordances that are helping the team (be sure to keep those), and identify the affordances that are nudging the team in the wrong direction (replace those with affordances that help you do the right things). And above all, remember that if things are going wrong, it isn’t always your fault. The process is a part of the environment and if your process is giving you the wrong cues for your project or team, then it’s the wrong process for you. So change it!

I’ve had to help more than a few teams get their version control systems sorted out over the past few years, and so I thought it would be easier if I just wrote down the philosophies I use for initializing a repository and getting the whole system set up. If you’re looking for some specific advice on how to set up and use a specific version control system, the Pragmatic Starter Kit Series for CVS, Subversion, or Git is a great place to start.

What should go into a source code repository?

The short answer: the repository should contain everything necessary to perform a clean build of your system. In most cases, this includes the code, third-party binaries necessary for building, tests, and documentation.

It’s ok to assume that everyone has their build environment “properly configured” for building. To make sure, make a list of everything that must be setup in the environment to build the software and put it on the team wiki. These things don’t need to be stored in the repository but you should at least write down what the standard build environment is supposed to look like. Depending on what the required software is, it might also be a good idea to keep a copy of it, just in case something happens to vendor in the future. The last thing you want is for a vendor to stop supporting the version of something you need, forcing you to upgrade because your hard drive crashed and you had to setup a new environment.

Include at least the following in your standard build environment list:

• Compiler versions
• Team sanctioned IDEs
• Required frameworks, toolkits, and build tools
• IDE extensions that the team has decided are so critical/awesome to the project they have to be used. Critical/awesome IDE extensions might enable a required tool-kit (such as GWT in Eclipse) or configure the IDE is specific ways (such as coding styles or static analysis settings)

Putting code and tests in the repository is fairly obvious, but third-party binaries (e.g. libraries) might not be. Put these in version control so that it’s easy to check out a project from source control and build without monkeying around with anything. I’ve found it best to create an “ext_lib” folder for storing all the external libraries. This way there is no confusion over what versions to use, and all the build paths can be set so that anyone can build just by checking out the code.

Here’s a real life example. Let’s say you’re writing a web application using the Google Web Toolkit and you rely on a caching library. The caching library should go into your ext_lib folder and you should tuck a zip of the GWT version you use away in a safe place just in case you need it later. Say your team is also using JUnit. Put the version you use in the ext_lib folder. This way everyone can build and use whatever GUI they want to run tests, be it the JUnit GUI or an Eclipse Plug-in.

Another real life example. Let’s say you use the excellent Sharp AutoUpdated component. Should you version the binary or the source? That was a trick question since it depends. The best answer is to only keep the binary of the library, but this isn’t always possible. One of the awesome things about open source software is that you have access to the source if you need it. So, let’s say you find a bug in the AutoUpdater and for some reason the maintainers aren’t responding quickly enough for your immediate needs. You can’t live with this bug so you have no choice but to fix it yourself. Congratulations, you just took ownership over your own fork of the AutoUpdater component. You now are responsible for maintaining the code – either in your version control library or in a public fork, and merging with the original code base may be more difficult in the future.

What doesn’t go into the source code repository?

Remember the DRY Principle for writing code (Don’t Repeat Yourself)? Well, that applies to your version control system too. Anything that can be derived shouldn’t be held under version control. Since your source code is already in the repository, storing the built binary is a violation of the DRY Principle. The penalty? Confusion, mistakes, and avoidable headaches. Third party libraries in the ext_lib folder don’t violate DRY since you can’t build them – you don’t own the source.

Also, do your fellow developers a favor and keep your personal stuff out of the repository. If you’re testing, nobody else wants to see your test reports. When you run the application, keep your logging messages to yourself. Also keep anything related to how you set up your personal environment in your personal environment. The last thing I want is to open up my IDE and see the last tabs you had open because you committed your personal user settings.

The easiest way to keep these sorts of undesirables out of the repository is by setting up an ignore list. Share it among the team.

How often do I commit?

Generally you should commit your changes anytime you think you’ve finished something useful that doesn’t introduce problems into the system. On the average, you should be committing changes at least once a day.

There’s two parts to this commit rule. “Finished something useful” might mean many things. This is by design. When you’ve finished a logical chunk of code that does something, feel free to commit it. “Doesn’t introduce problems” is a common courtesy to your fellow developers. Make sure, at a minimum, the system builds and passes any automated tests you have. And always update before you commit. Depending on your team size and how important the code is, you might establish a checklist for committing. Google has theirs automated. Every change the system has to build, pass tests, and pass a peer review before it can be committed.

Remember this mantra: Commit early, commit often.

But if everyone is committing all the time, isn’t that going to cause problems?

When you’re working with people and coordinating effort, problems will inevitably arise. Just remember, if you’re going to fail, fail early. It’s better to cause a conflict today through miscommunication while there’s plenty of time to fix it than the day before it’s time to deploy. Why? The conflicts will be smaller since you’re incrementally growing your code base. Also, since you made the changes recently they are fresh in your head and easier to work with. Code more than a week old might as well have been written by someone else.

Taking a risk management approach makes mitigating this easy. The risk: “Developers use a shared repository and commit changes frequently; might cause code conflicts that break the build.” The source of this risk is communication; therefore anything which helps facilitate communication can reduce the likelihood of this risk becoming a problem. Daily stand-up meetings are perfect for getting the word out about what everyone is working on. Automatically generated email updates from the version control system keep folks abreast throughout the day as changes are made. Continuous integration acts as a smoke test for uncovering integration problems while they’re small. Good merge tools can help reduce the impact of the consequence.

Once everyone gets used to the update-then-commit cycle, most of these problems go away. In my experience, big problems with code in the repository are usually a symptom of larger problems such as poor communication or failing processes.

What are some of your version control philosophies? What helps you keep things organized so you can get things done?

Everyone has a chore they hate doing. For me it’s cleaning the dishes. I’m a busy guy so I usually don’t get around to cooking and eating dinner until fairly late. Rather than cleaning anything, I stack the dishes in the sink and maybe soak a pan if something burned to the bottom. If the dishwasher has space, I’ll load it and set it going but most nights I leave a big pile of dirty dishes sitting around. After two or three nights of this, all the pots and pans are dirty and there’s no room for cooking thanks to the piles of dirty dishes. It’s actually kind of disgusting.

My wife takes a slightly different tact. After the meal is finished she immediately cleans all dishes, pots, pans, and utensils used while she was cooking. She has the forethought to run the dishwasher beforehand so there is plenty of room to load dirty dishes after the meal. She even wipes down the counter and stove so everything is ready for the next meal we cook. She’s quite amazing actually and a good cook to boot (especially when she’s following a recipe).

Professional chefs take matters a step further still. They clean as they cook.

I procrastinate doing something I hate and the result is a monumental, exhausting chore which takes an hour or more to finish. My wife spends 10, 15 minutes tops a night “tidying up” and though she hates doing dishes just as much as I, she makes it seem effortless. Professional chefs make miracles in the kitchen minutes at a time.

So it is with software.

Integrating software, even with a small team can be a chore.  Which would you prefer?  Approach A: write a lot of code, get everything working individually, and then do a big bang integration at the end; or Approach B: write a little code and integrate a little. While putting off integrating might satisfy your immediate needs, much like skipping dishes and moving straight to dessert, Approach A is going to cost more than Approach B in the end. Why?

• Integration problems aren’t uncovered until you integrate (profound, I know) so the longer you wait to integrate, the longer it takes to find out if there is a problem. Of course, no one ever plans for problems…
• Conflicts have further reaching consequences the longer you wait to fix them. Modern version control systems usually do a pretty good job merging changes but even magic has its limits.
• The full power of refactoring can’t be realized because the turnaround time on changes is too long.  The side effect is that you don’t refactor which means the code becomes more brittle over time.
• More code changes means more time to bring it all together and a higher likelihood of introducing defects through integration.  Unless you’ve planned knowing that integration will take time, chances are good you’re going to ship late.

Even better than big bang and nightly builds: continuous integration, cleaning as you code. Automated build servers have made huge advancements over the past few years. I highly recommend Hudson. It’s super easy to install and get started and has plug-ins for practically everything. There are even Hudson plug-ins for C# and just about every version control system you could want (and even some you don’t).

Doing chores sucks, but don’t make it worse than it has to be by putting things off. Continuous integration is a no brainer. If I had a dishwashing machine that constantly washed dishes as I finished using them I would dance naked in the streets, celebrating the marvels of modern technology. (You should be thankful such a machine doesn’t exist.)

You really don’t have an excuse for not using continuous integration.

Most software processes are light in two areas: quality assurance and process improvement. Most processes prescribe specific techniques for ensuring the production of quality code. XP for example advocates unit testing with TDD, continuous integration with smoke tests, pair programming, and acceptance tests written by the customer (using something like FIT so you can run automated regressions). Process improvement in XP is accomplished in a ’round the campfire, Kumbaya singing, get in touch with your feelings brainstorming session. Assumptions abound and there is no systematic way of ensuring that either testing or process improvement is handled adequately.  As we know, assumptions are never good enough.

The biggest assumption in XP (and indeed most software processes) is about bug tracking. Common sense dictates that you will create some kind of bug database. Hopefully it will at least be some kind of third party bug tracker such as Bugzilla or Bug Genie. Excel will work in a pinch but quickly becomes unsuitable for teams larger than one developer.  But how does the bug tracking actually work? What bugs get reported? Will you record issues from inspections in your bug tracker or only “true” bugs? What is the process for fixing a bug? What is the process for closing a bug? Who has access to the bug tracker? What information is required in your bug database and what information is optional? How do you determine defect priorities? Or the severity of bugs?

The majority of software processes provide answers for almost none of these questions. You are largely on your own to make up whatever you think makes the most sense for your development environment based on the best practices for your software process and your understanding of “good” software quality assurance practices.

No matter what quality process you follow, you will need a defect control philosophy. Once again, in absence of guidance I turn to the Team Software Process, one of the few processes to define what it means to track defects. In the Team Software Process, defects are treated as blight, a horrific mistake injected through the ineptitude of a developer. To remove these blights, the TSP relies on a series of filters in the form of code reviews, code inspections, unit tests, function tests, and so on. XP has a similar, though somewhat less rigorous set of filters in the form of pair programming, TDD, continuous integration, and acceptance tests. Each filter is meant to remove more and more defects, until finally “all” defects are removed from the system by the time the software has passed through all the filters. Generally each filter is intended to remove different types of defects, though it is conceivable to capture escaped defects from a previous filter in a later filter.

Just as water passing through layers of sand and rock will remove debris, so too will code passing through layers of unit tests and inspections sift out injected defects.

Defect Data

With these ideas in mind, bug tracking has three basic goals.

1. Record defects so they can be analyzed and fixed.
2. Identify the means by which defects are injected.
3. Identify the means by which defects are removed.

We write down bugs so we can go back and fix them later. This is obvious. But bug data can be used to measure process improvement also.

Since many parts of a software process are dedicated to filtering out the defects we’ve injected, understanding how defects are injected is essential to preventing similar defects from being seen in the future. The idea is that we want to learn from our mistakes. To achieve this, record the type of defect and the reason it was injected. The TSP gives us a good starting point for each of these, shown in the tables below. You should feel free to modify the types and reasons so they make sense for you, your team, and your project.

The defect type characterizes what kind of defect is injected and captures the essence of what is needed to fix the defect.

The defect’s reason characterizes why the defect was inject.

Assigning the reason can be a little tricky. I have found it is best if the person who injected the bug assigns the reason. On small teams (or if you’re following the PSP), this is generally pretty easy. It’s not about rubbing their nose in the problem – well, actually it is. Ok, it’s not about embarrassing or punishing the person but creating an opportunity for learning from our mistakes. If you can understand the reason why a defect was injected, it’s possible to prevent the defect from occurring again. For example, if there seems to be a rash of education related defects in a particular module, perhaps some training is in order.

Ideally we’d also like to know when the defect was injected, as in at what phase of development. It is possible to realize this information with additional analysis but I’ve found the return to be rather diminutive. Basically, you’ll learn what we’ve known all along – that the longer a defect is in the system, the harder it is to get out and that the most expensive defects are injected during the earlier phases of development (e.g. design defects are costly). Rather than track when things were injected I think it makes more sense to track when defects are detected. The point is to gain an understanding of how well the quality process filters out defects. To accomplish this, simply write down what you were doing when you found the bug. If you’re using XP, the list might include designing, writing new code (in a pair), writing new code (alone), refactoring, unit testing, integration, and acceptance testing. With this information you should be able to determine how effective each activity filters defects and over time whether the quality process is having issues. For example, I would expect interface defects to be detected during integration. If they are being detected earlier, say during unit testing, or later, say during acceptance testing, then my continuous integration and smoke test suite might not be as robust as it should be.

Better Bug Tracking

The strategies I’ve outlined here are a little more sophisticated than your average bug tracker, but add a lot of punch for very little effort. Tracking defect type, reason injected, and phase detected allow you to get a better handle not only on how defects are being injected into the software, but also how they are being detected. Both these chunks of information are necessary for understanding how defects are making their way into the system and how your process is helping you ferret them out of your system.