Extreme Programming’s system metaphor, as traditionally presented, has a fatal flaw. It has a nudge which encourages teams to describe the system at too high a level, as one large monolithic thing. The result is nearly always the same: a generic metaphor that doesn’t really describe the software and provides next to no guidance for implementation. This has led many in the Agile community, including Kent Beck, XP’s creator, to abandon the system metaphor entirely. I have three big worries with this.

1. While Beck may have changed his mind about many practices from XP, both versions of Extreme Programming, available in two editions of Extreme Programming Explained, are readily available and actively promoted by the publisher.  Treating both editions as equals curbs the adoption and spread of better practices.  The metaphor lives on in spite of being abandoned by its creator.
2. Agile teams, while improving, have been generally slow to adopt now common knowledge on software architecture.  Deemphasizing the system metaphor has created a black hole in the software lifecycle around architecture and design.  Those teams that are software architecture-focused have a hard time figuring out how to remain agile while incorporating architecture best practices.
3. Metaphors aren’t going anywhere.  They are a natural reaction to describing complex things – in literature, in life, and in software.  Ignoring the system metaphor makes it more difficult to use effectively.

So rather than fighting the metaphor as a tool for describing software architectures, we should embrace it.  It’s time to refresh the system metaphor by combining current best practices from software architecture with what we know works for metaphors in literature.

The System Metaphor, Refreshed

The ultimate goal of the system metaphor is to create a common vision and vocabulary for discussing and reasoning about a system – with all stakeholders.  This stands for the refreshed system metaphor too.  Communication is king in the abstract world of software architecture, and so communication is the ultimate metric for determining whether a system metaphor is good or bad.  Metaphors that enhance communication are better than ones that confuse or mislead the team.

To this end, there are five guidelines for evaluating metaphors to determine their fitness as aids for describing software architecture, and one corollary for sharing and using metaphors.

A good metaphor:

1. Represents a single view.
2. Deals with only one type of structure.
3. Gives clear guidance concerning design decisions.
4. Sheds light on system properties.
5. Draws on a shared experience.

[This post is a recap on the second talk I gave at XP2010. This was the big one, the experience report talk, one of 15 experience reports published at XP2010. You can download the slides (pdf) or the full paper (pdf) from this website or from XP2010.org.]

Process improvement is important for nearly all teams but it can sometimes be difficult for a team to know what is working, what isn’t working, and what techniques or methods to try when attempting to improve. Performing a scientific experiment is one way help overcome these problems but as academic research has shown us, while experimentation can yield interesting results, running an experiment is time consuming, expensive, and requires some serious thinking and control to pull off. From a practitioner’s standpoint this means that experimentation is a non-starter.

Of course, that’s only if you run experiments like an academic.

Back Story

Just over a year ago, my MSE studio team at Carnegie Mellon had a problem. We had decided we would use Extreme Programming for the construction phase of our project but some team members had doubts concerning pair programming. We had decided that we would use some kind of peer review, having already seen the many benefits of inspection when reviewing other artifacts. The dispute arose over whether pair programming would give similar enough results. Also, not all team members had experience with pair programming but everyone on the team knew and enjoyed solo programming.

The number one concern was whether pair programming would allow us to meet our very strict deadline. We had just over three months to complete the construction phase of the project. According to our threshold of success this meant implementing all “must have” requirements with a minimum level of quality. Did we really have time to waste by having two people working on the same code at the same time? Wouldn’t working independently and inspecting code on an as needed basis allow us to get more work done faster?

At the time it just so happened that I was taking a reading class with Mary Shaw and in that class we discussed some research findings that might help settle this debate. Research from Laurie Williams, Ward Cunningham, Barry Boehm, and many others showed that pair programming requires more effort (although never double the effort) but is faster than programming alone (pdf). Also pair programming creates code of about the same quality as coding alone with inspection (pdf). Of course, the research may not apply to us since Square Root is closer to a professional team working on a large project with a real client, not undergrads working on short term toy projects.

After an iteration where some teammates used pair programming and others refused, we decided to try an experiment to see which practice actually worked better. The original idea was that we might be able to validate some of the research but decided instead that it was more important just to resolve our own internal conflicts and figure out which processes worked better.

Conducting a Lightweight Experiment

With the scientific method as our guide we planned and executed a lightweight experiment which pitted programming alone against pair programming. The results were amazing (and you can find the raw data in our project archive). In conducting the experiment we used a set of novel techniques which I think can be useful in conducting other lightweight experiments. There’s more background in the experience report so I’m only putting the meaty stuff in this post.

Focus narrowly on a single question – The essential key to keeping an experiment light is to only tackle one thing at a time. In this case we focused on comparing and contrasting a single technique, pair programming, rather than multiple techniques or an entire process (such XP vs. TSP).

Divide work, not teams – If I were comparing pair programming to programming alone in an academic setting, I would put together two teams of about the same experience and have them each build their own version of the same software, one team using pair programming, the other programming alone. In a business setting this is a complete waste and few companies can afford to have two teams duplicating effort. By dividing work instead of teams you may lose some control over variables in the experiment but in most cases isolating more variables doesn’t add any further clarity to helping answer the narrowly focused question. To divide work successfully you need to have some way of estimating work units for division. We used use case points as shown in the figure depicting our modified planning game.

Continue making releases – Since we still needed to make a comparison, rather than dividing into teams and duplicating effort we divided the features that were released each iteration. In this way we built about half the features released during an iteration using each technique. Working on about half the features using pair programming meant that at least some features were being built by individuals. At the time this was a risk reduction decision to make sure that if pair programming completely failed we’d still have something to ship at the end of the iteration. Explicitly managing risks is the only way to know if the lightweight experiment may cause problems for making releases. Also, we had a strictly defined cut-off for stopping the experiment if it ever stopped us from shipping to our client.

Use the data you have – In almost all cases we were able to get the data we needed to evaluate our hypothesis from our current process. When we couldn’t, we only had to make minor modifications to our data collection practices, for example adding a check box to our SharePoint server for indicating whether a task was paired or individual.

One of the more interesting things we did was to create a “tally sheet” for collecting pair programming issue detection statistics in real time, as the issues were discovered. Given the near instantaneous code-inspect-fix cycle when programming in pairs, this was the only way to collect similar data for comparing pair programming to inspection.

Statistical significance is overrated – The whole point of running a lightweight experiment is to collect just enough data to help you make a better decision or validate your gut feeling. This technique is not meant for uncovering universal truths or proving something to the rest of the world. In exchange for keeping the experiment light, the results will only apply to your team. Over the course of an iteration or two, 4-6 weeks, you’ll only get enough data to start to see trends. In our case the results were not statistically significant using individual T-tests but that didn’t matter. The most important thing is that we had data that could be used for comparison, data that everyone felt good about and that helped us gain clarity into what we did and how well it worked.

Retrospectives get immediate value – The whole reason the experiment is light is to reduce cost and decrease the lag time to providing value to the team. Just to give you a little perspective, it took us 6 weeks to run the experiment and had enough data and casual observations to make a decision during the retrospective when the analyzed data was shared. That event occurred in early August of 2009. This experience report required almost nine full months of gestation from the paper proposal to the talk I gave at the conference. The gestation period on “universal truth” research can be even longer. We, as practitioners, don’t have to wait for those universal truths to be born to get value from research. By running your own quick and dirty, lightweight experiments, you can get results in a timely fashion that you know will apply to your team because your team was the subject of the experiment. It’s all about closing the gaps between research and practice and taking the information you need now instead of waiting for academic research to catch up.

Overall Conclusions

For the Square Root team it turned out that pair programming was faster, cheaper, and produced code that had more predictable albeit slightly worse quality. The more important lesson is that we discovered a technique, lightweight experimentation, for learning other interesting things about our team and about software engineering in general.

My paper and this blog post were all about trying to describe the technique, using our experiment as an example. I think it would be awesome if teams around the world conducted lightweight experiments on a variety of topics. If enough folks share what they learn, we might start to see trends emerge across teams that could lead to universal truths, validate research, or at least discover some great rules of thumb.

What else might make for a great experiment? Anything you’ve got a question about on your team!

• What is the clearer way to write requirements, user stories or use cases?
• Which estimation technique is more accurate of X and Y?
• Can we skip unit testing if we use inspection (looking at quality, knowledge sharing)?
• Is UML a better design notation than the one we made up as a team?
• What else…?

If you do a lightweight experiment, let me know! Share what you learn as a blog post or whitepaper. Let others know what you’ve learned! Even if the specific results only apply to your team and the way you’ve executed your project, your experiences help form a baseline, a sort of shared understanding for how software development works, how some of these practices work. And there’s so much about software engineering that we have yet to learn.

Acknowledgements

This paper was my first experience report and it was an awesome journey. Naturally a lot of folks helped me along the way and I would like to take a moment to make sure they know that I appreciate their influences and support. The Square Root team: Marco Len, Yi-Ru Liao, Abin Shahab, and especially my fellow experiment co-champion Sneader Sequeira for having the guts to go along with this idea in the first place. Some of the faculty at Carnegie Mellon: Dave Root and John Robert (my studio mentors) for bringing up the idea of writing a paper, and Jonathan Aldrich for helping review my proposal. Artem Marchenko was my XP2010 paper shepherd after the proposal was accepted, and the quality of each draft only improved because of his inputs. A group of my fellow employees at Net Health Systems sat through an early draft of the presentation I gave and shared valuable feedback for improving it. And finally I thank, Marie, my wife, who was with me from start to finish and read more drafts and sat through more practice talks than anyone else. She’s probably as much an expert on this subject by now as I.

A Final Aside

I wrote the initial draft of this paper as my final reflection paper for my Master of Software Engineering degree (pdf). That draft has a very different tone, approach, conclusion, and direction than what I eventually published for XP2010. This is half due to there not being a hard page limit but also I had a lot more time to think about what was really important when writing for XP2010. There’s some interesting information, mostly in the lessons learned, that might prove interesting to those who are interested. You should check out my Square Root teammates’ reflection papers as well since they are all interesting and well written.

[Here's a recap of the first talk I gave at XP2010. Since it was a Lightning Talk, rather than posting slides I've summarized my talk and added references directly in this post. I welcome comments and discussion.]

Craftsmanship is an interesting model for thinking about how to teach someone to become a great software engineer. Industry hasn’t always done the best job taking advantage of this metaphor for enabling training and instruction. Sure, there’s agile coaches and conferences like XP2010 where peers can collaborate, but rarely does an organization, a business, deliberately encourage and enable engineering growth for the software engineers they hire.

As we learn how to build software we go through the three stages of craftsmanship.  For most of us, we are apprentices in university, taking courses and learning the basics of computer science and software development by imitating our professors and the books we read.  We are journeymen the first few years on the job as we start our careers, applying the lessons we learned in school in practical setting and trading tips with fellow journeymen.  Eventually some of us pass some kind of test under the tutelage of a master and are ourselves declared as such.  The frustrating part is that so few people find masters to help when attempting to cross the threshold from journeyman to master.  How do you know when you’ve made it?  Where are these great masters, these mentors for helping to learn how to be a great software engineer?

Wouldn’t it be great if there were a place, a dojo if you will, that we could go to practice with other journeymen under the guidance of masters, interacting with apprentices just starting out on their craftsman journey?  As it turns out there is such a place.

The Master of Software Engineering program at Carnegie Mellon has been teaching professional software engineers how to build software better for just over 20 years now. The faculty and staff at the MSE have honed some practices that can be directly applied in industry. Normally I wouldn’t advocate transitioning academic education practices to an industrial environment but the MSE is a near perfect hybrid of industry and academia. The studio project, the capstone project which forms about 50% of the curriculum is a long duration (16 months), real project with business clients who expect software that will provide real business value. Commitment varies and during the summer semester, student teams are working on the studio project as a full time job, dedicating over 40 hours a week to the project. In addition, unlike most academic programs, all students are experienced engineers with at least 2 years of industry experience.

A dojo is a place for training, a place where a variety of students with different backgrounds come together to practice and become better at their craft. So while the MSE makes for an excellent dojo, it’s not easy for everyone to move to Pittsburgh for 16 months of intense study.  So, how can the success of the MSE be applied within industry? I think that there are six key practices where the MSE excels that industry should take note for training and professional development.  These are practices which can be applied in nearly any business setting with effective results.

Education – In school we can take classes. On the job we can read books, start discussion groups, read blogs, and go to conferences. Education becomes a catalyst for growth.

Mentoring – Mentors are guides who encourage growth by asking probing questions and pushing those being mentored out of their comfort zone. Mentors are there to dust you off when you fail and never directly solving problems for those being mentored (a favorite technique of the MSE mentors is to answers questions with questions).  In the MSE program, every student meets with a mentor once a week for 30 minutes to discuss how the project is going and thoughts on software engineering. This is a significant commitment for industry and so holding mentor meeting perhaps over lunch maybe once a month is sufficient. The point is to help novices and journeymen to find masters for guidance.

Proposals – Proposals help teams focus on the think-act-reflect cycle for approaching software from an engineering perspective. In a proposal, teams think through methods, processes, and techniques that will be used and this written (it may be brief or as detailed as necessary) proposal becomes the basis for evaluation and reflection for the team. In essence the proposal acts as a plan for determining an approach to software engineering practices.  The whole point is to get student engineers to start thinking in terms of the simple to see but takes a lifetime to master, cyclic think-act-reflect approach to problem solving.  See this article from my studio team’s reflection blog on understanding when decisions are made and the complete archive of proposals from my team and others are available from the MSE studio archive for some concrete examples of how proposals work.

Presentation & Critique – Communication and collaboration is a powerful tool for learning. During a presentation and critique, a team presents a proposal and that proposal is the critiqued by both mentors and peers. The comments and questions are then taken into consideration when revising or changing proposals. This is a powerful tool that doesn’t cost much and fosters knowledge sharing across an organization.

Peer Collaboration – This is so obvious I shouldn’t have to say it but simply talking to peers is one of the most often overlooked sources of information and learning. Many professional environments inadvertently create physical barriers which further prevent collaboration. Team lunches are nice for getting to know each other, but genuine collaboration must involve asking hard questions and then collaborating with a diverse group of individuals to help answer those questions.  Presentation & Critique is one way to facilitate this.  Setting up the environment to encourage collaboration is another.

Reflection – I have come to believe that this is the single most important practice in software engineering. If only more professionals would take the time to reflect on what they do and use that reflection to drive improvements then many of the most difficult problems we face as an industry would begin to resolve themselves. Effective reflection is ongoing, mentally intense, and difficult to do well. It involves both hard data and soft feelings.

All of this combines to create a place filled with passionate software engineers of all different levels of mastery, each learning from one another, and taking the field of software engineering to an entirely new and better plane of existence.  If you are ever in the Pittsburgh area, please stop by the Cave (the place where the studio teams do their work) for a tour!  You can contact information and further details about the program on the MSE Website.

References

• D. Garlan , D.P. Gluch , J.E. Tomayko, Agents of Change: Educating Software Engineering Leaders, Computer, v.30 n.11, November 1997, p.59-65.
• D. Root, M. Rosso-Llopart, G. Taran, Proposal Based Studio Projects: How to Avoid Producing “Cookie Cutter” Software Engineers, CSEE&T 2008, pp. 145-151.
• D. Root, M. Rosso-Llopart, G. Taran, Key Software Engineering Concepts for Project Success: The Use of “Boot Camp” to Establish Successful Software Projects, CSEE&T 2007, pp. 203-210
• J. Tomayko. Teaching Software Development in a Studio Environment, Association for Computing Machinery, ACM 0-89791-377-9/91/0002-03000, September, 1991.
• The 9 crafts of software engineering proposed by Alistair Cockburn
• Schon, Donald, The Reflective Practitioner: How Professionals Think In Action
• MSE Studio Archives – a record of over 70 past studio projects since the late 1990s

This post is a recap of a talk I gave this weekend at the Carnegie Mellon University Master of Software Engineering 20th Anniversary Mini-Conference. I’ve made the paper this talk is based on (pdf) as well as the slides I used during the talk (pdf) available. I’ve also linked to as many of the primary sources I used in research as I could so please, check out those papers if this is something that interests you.

About a year ago I discussed the idea of using affordances to help figure out how to make software processes work more smoothly for a team. Back then the idea spawned from a moment of crisis and self-reflection on my studio team, but having thought about it for a while and noticing the phenomena occurring on other teams I decided to revisit this idea and see if there is a way to proactively use affordances to avert problems rather than merely explaining problems as they occur. As it turns out there is a precedent for affordance-driven design in object engineering. With a few basic assumptions I think affordance-driven design can be extended to software processes as well.

To better explain how to proactively use affordance to pick or tailor a software process it helps to know a little about the Theory of Affordances (pdf). If you’ve read The Design of Everyday Things by Donald Norman then you probably already have a pretty good understanding of how to use affordances in the context of object design and usability. From an ecological psychology perspective, affordances are a way of helping explain or predict how an animal will behave in the context of their environment. In the context of humans, our environment is influenced by our experience, culture, and background as well as our current goals within that environment.

A simple example is the play button on a DVD player. A triangle to the right means play. This is obvious to us because we know what a DVD player does (we have experience with similar devices), we turn to the device when we want to watch movies (we’re looking for the play button), and culturally, our notion of time is left to right (so a triangle pointing right means play while a triangle facing left means reverse). But what if you come from a culture where time is generally represented as passing from down to up vice left to right? In this case, a triangle facing up might be a better symbol on a play button than a triangle pointing right. The value of the affordance changed based on a cultural bias and the user’s background.

This is all well and good, but what does it have to do with software process? For the Theory of Affordances to apply to software processes (or any process) we have to assume that process is a part of the environment. This is an interesting proposition since process really only exists in our minds. Process is something that we make up, and like our understanding of an object your knowledge and experience with a process will influence your perception of that process as an environmental influence. As long as you believe in, understand, and follow a process, the steps of that process exist as much as any other object in the real world. Logically this seems to make sense. Even the US patent office will grant patents for a process just as it will a physical invention.

Back to the core problem of identifying affordances, something I did not have an answer for in my original post a year ago. As best as I can tell, the only way to identify affordances is to reverse engineer a process focusing on the affordances using a technique known as Affordance Driven Design. Affordance Driven Design has three basic steps (pdf).

1. Identify a user’s needs in terms of functions.
2. Identify the desired functional affordances necessary to achieve the previously identified user’s functions.
3. Choose affordances to design into artifacts which are mostly likely to help achieve those desired functional affordances.

As an example, consider the task of blending a drink using a blender (pdf). Typical functions a user might want to perform are preparing the blender, blending, and cleaning up afterward. Functional affordances might include the “countertop-ability” (the ease with which a blender can be moved to a countertop), the “clean-ability” (the ease with which a user can clean a blender), and “transportability” (the ease with which a user can move a blender around). A person might choose any number of blenders to perform the desired functions but each blender will fulfill the functional affordances in different ways. A hand blender is extremely portable while a gas-powered “whacker” (powered by a 26cc engine, complete with motorcycle throttle – it makes the smoothest margarita you’ll eve have) isn’t really intended for indoor use.

To software engineers, functional affordances should look very familiar – they’re essentially quality attributes. Thinking about affordances from this perspective gives us a huge advantage since, as software engineers, we are already extremely familiar with quality attributes and quality attributes scenarios. Affordances, therefore either promote or inhibit desired quality attributes in your process.

Thinking about software processes, the functions will all be related to software: writing, designing, testing, and releasing are just a few possibilities. Some process quality attributes might include:

• Plan-ability (How far ahead does a process help you to plan?)
• Predictability (How well can you see into the future?)
• Changeability (When the course of a project needs to shift, how well does the process support chaging plans or direction?)
• Quality (The degree to which your process promotes “quality”)
• Cost (The amount of resources you’re willing to spend on process to achieve specific functions)
• Harmony (How well the team gets along)
• Reliability (How consistently the process helps you perform)
• Performance (Could be speed of development or quantity of code – define what you mean in the quality attribute scenario.)

As an example, say changeability is a desired quality on your team. A specific changeability scenario might go something like this. In order to meet business needs in an aggressive market, the team needs to be able to shift focus and answer competitors’ challenges within five business days. What are the things that might get in the way of this kind of rapid change? Heavy documentation could be one thing, as it nudges teams into keeping a single course. Long iterations also make it difficult to shift focus since more effort is required to make longer term plans. Going light and having short iterations, on the other hand promotes a team’s ability to change. But like every design decision there are trade-offs. Going light in terms of documentation might make it harder to achieve certain kinds of quality, for example.

The main idea here is that it’s relatively easy to identify process affordances by thinking like a designer and applying the skills we’ve already acquired as software engineers. I propose that evaluating process affordances as I’ve discussed here is a great way to pick a process and also to tailor processes. When tailoring simply identify affordances that are helping the team (be sure to keep those), and identify the affordances that are nudging the team in the wrong direction (replace those with affordances that help you do the right things). And above all, remember that if things are going wrong, it isn’t always your fault. The process is a part of the environment and if your process is giving you the wrong cues for your project or team, then it’s the wrong process for you. So change it!