Extreme Programming’s system metaphor, as traditionally presented, has a fatal flaw. It has a nudge which encourages teams to describe the system at too high a level, as one large monolithic thing. The result is nearly always the same: a generic metaphor that doesn’t really describe the software and provides next to no guidance for implementation. This has led many in the Agile community, including Kent Beck, XP’s creator, to abandon the system metaphor entirely. I have three big worries with this.

1. While Beck may have changed his mind about many practices from XP, both versions of Extreme Programming, available in two editions of Extreme Programming Explained, are readily available and actively promoted by the publisher.  Treating both editions as equals curbs the adoption and spread of better practices.  The metaphor lives on in spite of being abandoned by its creator.
2. Agile teams, while improving, have been generally slow to adopt now common knowledge on software architecture.  Deemphasizing the system metaphor has created a black hole in the software lifecycle around architecture and design.  Those teams that are software architecture-focused have a hard time figuring out how to remain agile while incorporating architecture best practices.
3. Metaphors aren’t going anywhere.  They are a natural reaction to describing complex things – in literature, in life, and in software.  Ignoring the system metaphor makes it more difficult to use effectively.

So rather than fighting the metaphor as a tool for describing software architectures, we should embrace it.  It’s time to refresh the system metaphor by combining current best practices from software architecture with what we know works for metaphors in literature.

The System Metaphor, Refreshed

The ultimate goal of the system metaphor is to create a common vision and vocabulary for discussing and reasoning about a system – with all stakeholders.  This stands for the refreshed system metaphor too.  Communication is king in the abstract world of software architecture, and so communication is the ultimate metric for determining whether a system metaphor is good or bad.  Metaphors that enhance communication are better than ones that confuse or mislead the team.

To this end, there are five guidelines for evaluating metaphors to determine their fitness as aids for describing software architecture, and one corollary for sharing and using metaphors.

A good metaphor:

1. Represents a single view.
2. Deals with only one type of structure.
3. Gives clear guidance concerning design decisions.
4. Sheds light on system properties.
5. Draws on a shared experience.

The system metaphor is one of the 12 key practices from XP and it is probably also the most controversial. Researchers at the Software Engineering Institute have observed that, of all the practices from Extreme Programming, the system metaphor is used the least. Other research done on graduate students at Carnegie Mellon concluded that natural language metaphors are useless for either communication or architecture design.

These are damning findings for the system metaphor, and yet on nearly every technical team with which I’ve built software, natural language metaphors were a daily part of how we communicated with each other. Maybe it has to do with the inherent abstractness of software, something that can’t be touched or tasted, only described. Maybe my teams have simply lacked the technical vocabulary needed to communicate in more precise words.

In spite of this nearly all of these projects were successful. Certainly there must be some merit in using natural language metaphors to describe a software system?

This got us thinking. Metaphors frequently come up in our everyday lives quite naturally and they are used effectively in books and movies. Is there something special about software that makes metaphors not work? Why would folks have such a hard time applying the system metaphor in practice when it’s obvious that metaphors in general are so effective in other places throughout our lives?

The simple answer is that we’ve been doing it wrong this whole time.

There’s nothing wrong with the system metaphor. It’s our application of the technique that has been wrong.

What’s missing from the prevailing literature on the System Metaphor is clear guidance that shows what a good metaphor should look like. Without this guidance, and without the background in software architecture to backfill this knowledge, teams are left alone to cobble together our own metaphors based on the limited information available on the Internet; a web of information that is largely an echo chamber of the same few sentences from the XP book. It should be no wonder that the system metaphor is unsuccessful in many cases.

What does a good metaphor look like? How do you know if you’ve created an effective metaphor? When should you create or update a metaphor? Why is one metaphor better or worse than another metaphor?

In looking for answers to these questions we combined what we knew about literary metaphors and software architecture practices to come up with a clear set of guidelines for the System Metaphor. These guidelines completed the System Metaphor definition and provided us with the guidance we needed to effectively design, analyze, and communicate the architecture of the software system we were building. We were no longer fighting our natural tendencies to create metaphors but instead were embracing them. And we created a more solid architecture as a result.

Interested in hearing more? If you’re going to Agile2011, why don’t you come to hear my talk? During this session I will explore our six guidelines for making better metaphors and walk through examples from our project. In only 30 minutes you’ll learn how to make metaphors that matter to your project, your team, and the system you are building in a way that improves team communication and encourages architectural thinking. This is a session you won’t want to miss!

[Here's a recap of the first talk I gave at XP2010. Since it was a Lightning Talk, rather than posting slides I've summarized my talk and added references directly in this post. I welcome comments and discussion.]

Craftsmanship is an interesting model for thinking about how to teach someone to become a great software engineer. Industry hasn’t always done the best job taking advantage of this metaphor for enabling training and instruction. Sure, there’s agile coaches and conferences like XP2010 where peers can collaborate, but rarely does an organization, a business, deliberately encourage and enable engineering growth for the software engineers they hire.

As we learn how to build software we go through the three stages of craftsmanship.  For most of us, we are apprentices in university, taking courses and learning the basics of computer science and software development by imitating our professors and the books we read.  We are journeymen the first few years on the job as we start our careers, applying the lessons we learned in school in practical setting and trading tips with fellow journeymen.  Eventually some of us pass some kind of test under the tutelage of a master and are ourselves declared as such.  The frustrating part is that so few people find masters to help when attempting to cross the threshold from journeyman to master.  How do you know when you’ve made it?  Where are these great masters, these mentors for helping to learn how to be a great software engineer?

Wouldn’t it be great if there were a place, a dojo if you will, that we could go to practice with other journeymen under the guidance of masters, interacting with apprentices just starting out on their craftsman journey?  As it turns out there is such a place.

The Master of Software Engineering program at Carnegie Mellon has been teaching professional software engineers how to build software better for just over 20 years now. The faculty and staff at the MSE have honed some practices that can be directly applied in industry. Normally I wouldn’t advocate transitioning academic education practices to an industrial environment but the MSE is a near perfect hybrid of industry and academia. The studio project, the capstone project which forms about 50% of the curriculum is a long duration (16 months), real project with business clients who expect software that will provide real business value. Commitment varies and during the summer semester, student teams are working on the studio project as a full time job, dedicating over 40 hours a week to the project. In addition, unlike most academic programs, all students are experienced engineers with at least 2 years of industry experience.

A dojo is a place for training, a place where a variety of students with different backgrounds come together to practice and become better at their craft. So while the MSE makes for an excellent dojo, it’s not easy for everyone to move to Pittsburgh for 16 months of intense study.  So, how can the success of the MSE be applied within industry? I think that there are six key practices where the MSE excels that industry should take note for training and professional development.  These are practices which can be applied in nearly any business setting with effective results.

Education – In school we can take classes. On the job we can read books, start discussion groups, read blogs, and go to conferences. Education becomes a catalyst for growth.

Mentoring – Mentors are guides who encourage growth by asking probing questions and pushing those being mentored out of their comfort zone. Mentors are there to dust you off when you fail and never directly solving problems for those being mentored (a favorite technique of the MSE mentors is to answers questions with questions).  In the MSE program, every student meets with a mentor once a week for 30 minutes to discuss how the project is going and thoughts on software engineering. This is a significant commitment for industry and so holding mentor meeting perhaps over lunch maybe once a month is sufficient. The point is to help novices and journeymen to find masters for guidance.

Proposals – Proposals help teams focus on the think-act-reflect cycle for approaching software from an engineering perspective. In a proposal, teams think through methods, processes, and techniques that will be used and this written (it may be brief or as detailed as necessary) proposal becomes the basis for evaluation and reflection for the team. In essence the proposal acts as a plan for determining an approach to software engineering practices.  The whole point is to get student engineers to start thinking in terms of the simple to see but takes a lifetime to master, cyclic think-act-reflect approach to problem solving.  See this article from my studio team’s reflection blog on understanding when decisions are made and the complete archive of proposals from my team and others are available from the MSE studio archive for some concrete examples of how proposals work.

Presentation & Critique – Communication and collaboration is a powerful tool for learning. During a presentation and critique, a team presents a proposal and that proposal is the critiqued by both mentors and peers. The comments and questions are then taken into consideration when revising or changing proposals. This is a powerful tool that doesn’t cost much and fosters knowledge sharing across an organization.

Peer Collaboration – This is so obvious I shouldn’t have to say it but simply talking to peers is one of the most often overlooked sources of information and learning. Many professional environments inadvertently create physical barriers which further prevent collaboration. Team lunches are nice for getting to know each other, but genuine collaboration must involve asking hard questions and then collaborating with a diverse group of individuals to help answer those questions.  Presentation & Critique is one way to facilitate this.  Setting up the environment to encourage collaboration is another.

Reflection – I have come to believe that this is the single most important practice in software engineering. If only more professionals would take the time to reflect on what they do and use that reflection to drive improvements then many of the most difficult problems we face as an industry would begin to resolve themselves. Effective reflection is ongoing, mentally intense, and difficult to do well. It involves both hard data and soft feelings.

All of this combines to create a place filled with passionate software engineers of all different levels of mastery, each learning from one another, and taking the field of software engineering to an entirely new and better plane of existence.  If you are ever in the Pittsburgh area, please stop by the Cave (the place where the studio teams do their work) for a tour!  You can contact information and further details about the program on the MSE Website.

References

• D. Garlan , D.P. Gluch , J.E. Tomayko, Agents of Change: Educating Software Engineering Leaders, Computer, v.30 n.11, November 1997, p.59-65.
• D. Root, M. Rosso-Llopart, G. Taran, Proposal Based Studio Projects: How to Avoid Producing “Cookie Cutter” Software Engineers, CSEE&T 2008, pp. 145-151.
• D. Root, M. Rosso-Llopart, G. Taran, Key Software Engineering Concepts for Project Success: The Use of “Boot Camp” to Establish Successful Software Projects, CSEE&T 2007, pp. 203-210
• J. Tomayko. Teaching Software Development in a Studio Environment, Association for Computing Machinery, ACM 0-89791-377-9/91/0002-03000, September, 1991.
• The 9 crafts of software engineering proposed by Alistair Cockburn
• Schon, Donald, The Reflective Practitioner: How Professionals Think In Action
• MSE Studio Archives – a record of over 70 past studio projects since the late 1990s

This post is a recap of a talk I gave this weekend at the Carnegie Mellon University Master of Software Engineering 20th Anniversary Mini-Conference. I’ve made the paper this talk is based on (pdf) as well as the slides I used during the talk (pdf) available. I’ve also linked to as many of the primary sources I used in research as I could so please, check out those papers if this is something that interests you.

About a year ago I discussed the idea of using affordances to help figure out how to make software processes work more smoothly for a team. Back then the idea spawned from a moment of crisis and self-reflection on my studio team, but having thought about it for a while and noticing the phenomena occurring on other teams I decided to revisit this idea and see if there is a way to proactively use affordances to avert problems rather than merely explaining problems as they occur. As it turns out there is a precedent for affordance-driven design in object engineering. With a few basic assumptions I think affordance-driven design can be extended to software processes as well.

To better explain how to proactively use affordance to pick or tailor a software process it helps to know a little about the Theory of Affordances (pdf). If you’ve read The Design of Everyday Things by Donald Norman then you probably already have a pretty good understanding of how to use affordances in the context of object design and usability. From an ecological psychology perspective, affordances are a way of helping explain or predict how an animal will behave in the context of their environment. In the context of humans, our environment is influenced by our experience, culture, and background as well as our current goals within that environment.

A simple example is the play button on a DVD player. A triangle to the right means play. This is obvious to us because we know what a DVD player does (we have experience with similar devices), we turn to the device when we want to watch movies (we’re looking for the play button), and culturally, our notion of time is left to right (so a triangle pointing right means play while a triangle facing left means reverse). But what if you come from a culture where time is generally represented as passing from down to up vice left to right? In this case, a triangle facing up might be a better symbol on a play button than a triangle pointing right. The value of the affordance changed based on a cultural bias and the user’s background.

This is all well and good, but what does it have to do with software process? For the Theory of Affordances to apply to software processes (or any process) we have to assume that process is a part of the environment. This is an interesting proposition since process really only exists in our minds. Process is something that we make up, and like our understanding of an object your knowledge and experience with a process will influence your perception of that process as an environmental influence. As long as you believe in, understand, and follow a process, the steps of that process exist as much as any other object in the real world. Logically this seems to make sense. Even the US patent office will grant patents for a process just as it will a physical invention.

Back to the core problem of identifying affordances, something I did not have an answer for in my original post a year ago. As best as I can tell, the only way to identify affordances is to reverse engineer a process focusing on the affordances using a technique known as Affordance Driven Design. Affordance Driven Design has three basic steps (pdf).

1. Identify a user’s needs in terms of functions.
2. Identify the desired functional affordances necessary to achieve the previously identified user’s functions.
3. Choose affordances to design into artifacts which are mostly likely to help achieve those desired functional affordances.

As an example, consider the task of blending a drink using a blender (pdf). Typical functions a user might want to perform are preparing the blender, blending, and cleaning up afterward. Functional affordances might include the “countertop-ability” (the ease with which a blender can be moved to a countertop), the “clean-ability” (the ease with which a user can clean a blender), and “transportability” (the ease with which a user can move a blender around). A person might choose any number of blenders to perform the desired functions but each blender will fulfill the functional affordances in different ways. A hand blender is extremely portable while a gas-powered “whacker” (powered by a 26cc engine, complete with motorcycle throttle – it makes the smoothest margarita you’ll eve have) isn’t really intended for indoor use.

To software engineers, functional affordances should look very familiar – they’re essentially quality attributes. Thinking about affordances from this perspective gives us a huge advantage since, as software engineers, we are already extremely familiar with quality attributes and quality attributes scenarios. Affordances, therefore either promote or inhibit desired quality attributes in your process.

Thinking about software processes, the functions will all be related to software: writing, designing, testing, and releasing are just a few possibilities. Some process quality attributes might include:

• Plan-ability (How far ahead does a process help you to plan?)
• Predictability (How well can you see into the future?)
• Changeability (When the course of a project needs to shift, how well does the process support chaging plans or direction?)
• Quality (The degree to which your process promotes “quality”)
• Cost (The amount of resources you’re willing to spend on process to achieve specific functions)
• Harmony (How well the team gets along)
• Reliability (How consistently the process helps you perform)
• Performance (Could be speed of development or quantity of code – define what you mean in the quality attribute scenario.)

As an example, say changeability is a desired quality on your team. A specific changeability scenario might go something like this. In order to meet business needs in an aggressive market, the team needs to be able to shift focus and answer competitors’ challenges within five business days. What are the things that might get in the way of this kind of rapid change? Heavy documentation could be one thing, as it nudges teams into keeping a single course. Long iterations also make it difficult to shift focus since more effort is required to make longer term plans. Going light and having short iterations, on the other hand promotes a team’s ability to change. But like every design decision there are trade-offs. Going light in terms of documentation might make it harder to achieve certain kinds of quality, for example.

The main idea here is that it’s relatively easy to identify process affordances by thinking like a designer and applying the skills we’ve already acquired as software engineers. I propose that evaluating process affordances as I’ve discussed here is a great way to pick a process and also to tailor processes. When tailoring simply identify affordances that are helping the team (be sure to keep those), and identify the affordances that are nudging the team in the wrong direction (replace those with affordances that help you do the right things). And above all, remember that if things are going wrong, it isn’t always your fault. The process is a part of the environment and if your process is giving you the wrong cues for your project or team, then it’s the wrong process for you. So change it!

Over Spring break I was fortunate to receive an invitation to the exclusive CERT (Computer Emergency Response Team) 2009 Technical Symposium.  The symposium was held in celebration of the twentieth anniversary of CERT but the main theme of the symposium was an examination of the internet, how the choices made just over 20 years ago have resulted in a huge security mess and what we can do about it today to avert the utter destruction of our modern way of life.  I wish I was exaggerating but the internet is rapidly becoming the exclusive means for sharing and managing the world’s data.  The world’s economy is following suit which heightens the need for security even more.  Speakers at the symposium included Vinton Cerf (sometimes known as “The Father of the Internet”), Scott Charney (VP for Microsoft’s Trustworthy Computing Division), Lawrence Roberts (invented packet switching), Steve Crocker (invented “Request for Comments”), Paul Mockapetris (invented DNS), and many other brilliant people who have been instrumental in creating and defining the internet.  Each speaker brought slightly different experiences to bear but everyone had the same message: if we don’t do something to secure the internet soon, something very, very bad will happen.

First, a little history

Security was never considered when the internet was originally conceived.  I guess that’s not entirely accurate, security was considered but quickly dismissed in favor of “the smallest set of tools that would give us the largest set of possibilities for the future,” according to Mockapetris.  Considering the original operating environment, the closed ARPANET/DARPANET, it makes sense that security was traded in favor of other quality attributes such as performance, reliability, and modifiability.  In the early 1980s, hosts numbered in the hundreds and there was an established and respected community of trust among the fraternity of hosts.

Once the internet became a public entity things changed.  Slowly, the community of trust that kept everything together began to unravel.  It started with news group flame wars.  Shortly after viruses started showing up.  There was even some direct hacking, albeit primarily by misled white hat hackers trying to help.  Then on November 2, 1988 the Morris Worm was released by a researcher from MIT and within 72 hours most of DARPANET was taken offline by the resulting accidental denial-of-service attack.  The community of trust that kept the internet running was permanently shattered.

CERT was created to deal with the problems that resulted from the world’s first denial-of-service attack.  While CERT’s mission is noble and its function critical to dealing with all the threats faced on the internet today, CERT is not the panacea required to save the world from the ever increasing internet threats coming from all angles in the form of botnet armies, distributed denial-of-service-attacks, and targeted attacks by sophisticated hackers motivated purely by profit.  The main problem is that CERT, by nature, is a reactive organization put in place to respond to emergencies as they happen.

As Vinton Cerf pointed out at the symposium, “the internet is always under attack.  It’s not enough to respond to each of the different incidents as they happen.  Reactivity is not enough.”  To save the world, we need to proactively address the security holes that were originally designed into the internet.

Short Term Solutions

Fixing the internet is an architectural problem.   It’s impossible to focus on a single layer of the protocol stack to make the internet more secure.  Each layer has different kinds of vulnerabilities and each layer can be exploited using different methods to achieve the same ends.  Unfortunately, replacing the entire protocol stack is difficult, thanks in a large part to how successful the internet has been.  But there are several short term strategies we can begin working toward today that work within the constraints of the current system.

One of the easiest and most significant things we can do is take humans out of the security loop.  It’s been proven time and again that people don’t follow instructions for securing systems.  Both John Gilligan and Vinton Cerf stated that improperly configured software accounted for over 80% of attacks on the internet.  Fix this problem and we’ve essentially removed 80% of cyber security threats.  Fixing the way we identify ourselves online (i.e. passwords) is another easy way to take humans out of the security loop.  A recurring theme from most speakers but most prominently discussed by Scott Charney was redefining identity online.  There are solutions that can work now to solve each of these problems.

It turns out that fixing the improper configuration problem is easy but it’s going to be a difficult sell.  The basic idea is that a computer or device has to start with a good configuration and then control mechanisms have to be put in place to prevent the configuration from getting hosed.  Some vendors such as Apple, Tivo, and Blackberry are already doing a great job at keeping their device configurations under control.  The trade-off for security is going to be freedom (modifiability if you prefer quality attributes).  In exchange for a usable, safe, and enjoyable internet experience that even your grandmother can handle you have to give up the ability to tweak anything you want in the system.  If iPod, iPhone, Tivo, and Blackberry sales are any indicator, people are willing to exchange freedom for something that just works.

Is it really fair to compare devices such as phones and PDAs with full blown computers?  Absolutely.  The key phrase is “Internet Enabled Device.”  An iPhone or Tivo can be hijacked and recruited into a botnet army just as easily as a standard desktop PC.  If even only 1% of the more than 4 billion mobile phones in use throughout the world were compromised, that would make 40 million bots ready to unleash mountains of spam and unending DDoS attacks.  The only reason this doesn’t happen is because mobile devices are effectively locked down to prevent such compromises from occurring.

With regards to passwords, I hate to say it but I think the Department of Defense is actually on the right track.  Every DoD employee and contractor is issued what is known as a Common Access Card (CAC), an ID card attached to a digital identification certificate issued by a third party authority that has verified that the person to whom the certificate was issued is indeed that person.  Both Vinton Cerf and Scott Charney were very clear that online identity can only be solved by combining something you know, usually a password, with something you have, a physical thing.  This something you have could be a digital identification certificate issued by an authority.

While it would be awesome to issue CACs to every citizen in the United States, perhaps attached to their driver’s licenses as Charney suggested, a staged success approach is possible.  PayPal has taken the initiative with a hardware device called Security Key.  Security Key is small enough to fit on a keychain and it generates a secret token (in this case a string of numbers) every thirty seconds.  When you purchase through PayPal you enter your username, password, and the token shown on the device.  Even if you accidentally gave this information to a phishing website, the token will be worthless after 30 seconds protecting your online wallet from fraud.

Though Vinton Cerf didn’t know the first thing about what I was talking about when I asked him about it, I see a lot of potential with OpenId.  OpenId is a protocol that provides something akin to a universal password that you can use to access thousands of websites and, with a little more work, could solve the identification problem across the internet.  First, OpenId providers would need to take on the role of verifying user’s real life identities and connecting the physical identity with the digital identity.  Second, OpenId providers have to offer some kind of physical verification when logging in, the something you have.  This could be in the form of a CAC, something biometric like a fingerprint reader, or even something as simple as PayPal’s Security Key.  In my opinion, OpenId is poised to become the third-party identification authority the world needs.

The Next Step

Another big theme at the symposium was trying to come up with a plan of action for moving forward, for fixing these problems.  Most speakers agreed that it would be unfortunate if it took a Pearl Harbor-like event to be the catalyst that motivated people to action.  Given the sophistication of attacks and the amount of value on the internet, the magnitude of damages could bring the United States, and in turn the world thanks to the global economy, to its knees.  The most frustrating part is that we know about the problems and I think there are a lot of smart people who know how to technically solve the problems.

By the end of the second day of the symposium I was left with more questions than answers and I am completely convinced that there will be a cyber security failure of a magnitude we have not yet seen in the near future.  I’m not suggesting that everyone start stocking canned goods in the cellar, but now is the time to proactively start working for change.  For as much flack as Microsoft has taken over the years, they seem to be on the forefront of effecting change both in terms of technology and public policy.  It’s difficult to know where to start, but Microsoft’s Trustworthy Computing resources are both comprehensive and practical.  The knowledge to fix the problems seems to be out there, the trick is motivating the world to change.