“Postmodern Project Management,” an article released in Defense AT&L in May of 2008, makes a convincing argument for why modern project management philosophy fails to help projects reach success [1]. As software architecture is largely a tool to facilitate communication and provide guidance for a project, it is not difficult to see parallels between current and emerging trends in software architecture with the idea of modernism vs. postmodernism project management practices. Tell me if these sentiments don’t ring true for the upfront vs. emergent design approaches.

Modernism is indeed an effective approach for a rational, static world where surprises are rare, measurements are precise, humans are tools, and our understanding of the system dynamics is very nearly complete. If the PM’s world was linear and predictable, then Modernism would work just fine. But the reality is, reality is messier than that. Things change unexpectedly, surprises surprise us, people are people, and the system dynamics are both unstable and nonlinear. In this sort of environment, Modernism breaks down.

Modernist project management refers to taking a rational, calculated, purely scientific (some might say cold) approach to managing people and projects. By extension, modernist software architecture approaches designing a software system in a similarly rational, calculated, and scientific way. The object of modernist software architecture is to create a process by which great software architecture designs can be consistently and predictably repeated by any team working on any project. The best example of modernist software architecture comes out of the excellent work and research from Software Engineering Institute.

Postmodernist software architecture on the other hand aims to inject some humanity into how we approach the architectural design and ultimately implementation of a software system. In other words, postmodern software architects acknowledge that no matter how much we plan or how good our models are, we can’t predict everything as the modernist software architect attempts to do [1].

For us normal folks, [Postmodernism] can be understood as a worldview that is skeptical of Modernism’s certainties. Postmodernism doesn’t necessarily deny Modern certainties—it just questions, examines, and deconstructs them, investigating the underlying assumptions, particularly when those assumptions are flawed, hidden, ignored, or otherwise not made explicit.

Historically most software architects (and developers in general) tend to approach the design of a complex system in a rational, linear way:  first you gather requirements, then you create a model (which precisely defines the solution), then you verify your model before finally starting implementation. In my experience this approach works extremely well – until things change (and change they always do). Models are imprecise by their very nature and it’s easy to forget important details or get details wrong. Changes happen throughout the life of a project – requests from customers, risks becoming problems, even changes you would have never expected.

When changes happen, the modernist approach to software architecture crumbles. All that documentation, all those models, all that analysis – these carefully laid plans become not only useless but also a burden and a resource drain. Over time systems built by relying too heavily on models constructed through a rational process become rigid and the quality decreases. This is not to say that models are bad. When dealing with something as abstract as software architecture models are essential. It’s how the models are created and how we use them that make postmodernist software architecture work better.

Change is an opportunity to deliver even more value to customers and users. Postmodernists react with confidence and excitement (and a healthy dose of skepticism as any architect modernist, postmodernist or otherwise should before evaluating trade-offs) where modernists react with pessimism, frustration, and disappointment. But lightweight representations are only part of the story. By recognizing that people are not interchangeable and that all humans are inherently flawed, postmodern architects naturally tend to design systems that are more flexible, testable, and maintainable. By not only allowing change but embracing and encouraging it, architectural drivers can evolve and converge more rapidly, actually shortening the “period of uncertainty.”

Faith in the power of individuals and accounting for their humanity is what makes postmodern software architecture so powerful. No amount of up front system analysis will ever change human nature.  And as architects we can never forget this when designing systems.

[1] Maj. Dan Ward, USAF, Maj. Chris Quaid, USAF, and Capt. Gabe Mounce, USAF, “Postmodern Project Management”. Defense AT&L: May-June 2008.  [Being a defense journal it's no wonder that I missed it (thanks to the Rogue Project Leader for pointing it out!]

Extreme Programming’s system metaphor, as traditionally presented, has a fatal flaw. It has a nudge which encourages teams to describe the system at too high a level, as one large monolithic thing. The result is nearly always the same: a generic metaphor that doesn’t really describe the software and provides next to no guidance for implementation. This has led many in the Agile community, including Kent Beck, XP’s creator, to abandon the system metaphor entirely. I have three big worries with this.

1. While Beck may have changed his mind about many practices from XP, both versions of Extreme Programming, available in two editions of Extreme Programming Explained, are readily available and actively promoted by the publisher.  Treating both editions as equals curbs the adoption and spread of better practices.  The metaphor lives on in spite of being abandoned by its creator.
2. Agile teams, while improving, have been generally slow to adopt now common knowledge on software architecture.  Deemphasizing the system metaphor has created a black hole in the software lifecycle around architecture and design.  Those teams that are software architecture-focused have a hard time figuring out how to remain agile while incorporating architecture best practices.
3. Metaphors aren’t going anywhere.  They are a natural reaction to describing complex things – in literature, in life, and in software.  Ignoring the system metaphor makes it more difficult to use effectively.

So rather than fighting the metaphor as a tool for describing software architectures, we should embrace it.  It’s time to refresh the system metaphor by combining current best practices from software architecture with what we know works for metaphors in literature.

The System Metaphor, Refreshed

The ultimate goal of the system metaphor is to create a common vision and vocabulary for discussing and reasoning about a system – with all stakeholders.  This stands for the refreshed system metaphor too.  Communication is king in the abstract world of software architecture, and so communication is the ultimate metric for determining whether a system metaphor is good or bad.  Metaphors that enhance communication are better than ones that confuse or mislead the team.

To this end, there are five guidelines for evaluating metaphors to determine their fitness as aids for describing software architecture, and one corollary for sharing and using metaphors.

A good metaphor:

1. Represents a single view.
2. Deals with only one type of structure.
3. Gives clear guidance concerning design decisions.
4. Sheds light on system properties.
5. Draws on a shared experience.

The system metaphor is one of the 12 key practices from XP and it is probably also the most controversial. Researchers at the Software Engineering Institute have observed that, of all the practices from Extreme Programming, the system metaphor is used the least. Other research done on graduate students at Carnegie Mellon concluded that natural language metaphors are useless for either communication or architecture design.

These are damning findings for the system metaphor, and yet on nearly every technical team with which I’ve built software, natural language metaphors were a daily part of how we communicated with each other. Maybe it has to do with the inherent abstractness of software, something that can’t be touched or tasted, only described. Maybe my teams have simply lacked the technical vocabulary needed to communicate in more precise words.

In spite of this nearly all of these projects were successful. Certainly there must be some merit in using natural language metaphors to describe a software system?

This got us thinking. Metaphors frequently come up in our everyday lives quite naturally and they are used effectively in books and movies. Is there something special about software that makes metaphors not work? Why would folks have such a hard time applying the system metaphor in practice when it’s obvious that metaphors in general are so effective in other places throughout our lives?

The simple answer is that we’ve been doing it wrong this whole time.

There’s nothing wrong with the system metaphor. It’s our application of the technique that has been wrong.

What’s missing from the prevailing literature on the System Metaphor is clear guidance that shows what a good metaphor should look like. Without this guidance, and without the background in software architecture to backfill this knowledge, teams are left alone to cobble together our own metaphors based on the limited information available on the Internet; a web of information that is largely an echo chamber of the same few sentences from the XP book. It should be no wonder that the system metaphor is unsuccessful in many cases.

What does a good metaphor look like? How do you know if you’ve created an effective metaphor? When should you create or update a metaphor? Why is one metaphor better or worse than another metaphor?

In looking for answers to these questions we combined what we knew about literary metaphors and software architecture practices to come up with a clear set of guidelines for the System Metaphor. These guidelines completed the System Metaphor definition and provided us with the guidance we needed to effectively design, analyze, and communicate the architecture of the software system we were building. We were no longer fighting our natural tendencies to create metaphors but instead were embracing them. And we created a more solid architecture as a result.

Interested in hearing more? If you’re going to Agile2011, why don’t you come to hear my talk? During this session I will explore our six guidelines for making better metaphors and walk through examples from our project. In only 30 minutes you’ll learn how to make metaphors that matter to your project, your team, and the system you are building in a way that improves team communication and encourages architectural thinking. This is a session you won’t want to miss!

In Choosing a Software Design Strategy I proposed that your understanding of the design space will determine what design strategies are appropriate for your project. That is, how much you know about the problem domain and solution space will determine how much design work is appropriate for your project. Rather than fussing over dogma, the power is clearly in your hands. If you feel uncomfortable with what you are capable of designing, then you need to further explore the design space! Otherwise, you are free to design in a way that is appropriate to your understanding of the system and a way that makes sense to your team be it Big Design Up Front or a more minimalistic design approach.

Personally I’m a visual person so having a graphical representation which shows how to think about what I know about the design space is extremely helpful to me. In this graphical representation, there is a real boundary which will eventually prevent me from exploring the infinitely large design space. This boundary is formed by things like time, budget, and schedule as well as thought stuff such as your team’s training or overall knowledge and the existence of research or even existing technologies. For example, once I was involved with a project that required large amounts of data to be transferred from Australia to the United States in real-time with an aggressive deadline, on the order of tens of milliseconds. Crunching the numbers, we quickly realized that this was completely impossible with the existing telecommunications infrastructure. In fact, the only technology we could find which might promise this kind of performance was entangled quantum particles which are only today becoming a reality…sort of. The solution boundary in this case severely limited our ability to find a viable design. Sometimes you can move these real boundaries, for example by increasing the budget and schedule or hiring a consultant, but any system you design will come from your understanding of the area in the highlighted space shown below.

Another interesting thing about this representation is that it’s actually parametric. The X and Y axes will tell you what you know, but your understanding of the problem you aim to solve and the solution you are using to solve it will change over time. It’s also possible that you might not start at (0, 0) in every case. And the way you learn about the design is going to be different every time, rarely (if ever) taking the direct, efficient route to enlightenment. As time goes on, as you learn more about the problem and solution, you’ll create a parametric curve, the shape of which will give you a pretty good idea of where your knowledge gaps might be. And thinking about design in this way exposes some interesting things about what you know, and what you might need to learn in order to make a satisficing design.

So if you have a design curve that starts off almost horizontal, it means that you know a lot about the problem but don’t know very much about solutions. Teams with lots of specific domain knowledge might face a curve like this. Most of the projects I did with the US Navy started with a curve that looked like this. The problems the Navy was trying to solve were well understood and the teams were initially made up of veteran subject matter experts. One of my jobs then was to help understand how to solve these well understood problems using more recent advances in hardware and software.

A curve which starts off steep presents the exact opposite case. Here the team has plenty of knowledge about possible solutions but knows very little about the problem. This is the case with most software development teams. We’re experts in building software and in most cases, when someone comes to us with a problem they want a software solution. The team’s main focus will be on understanding requirements and then applying their software engineering knowledge in creating a solution. A great example of this scenario is web development shops who specialize in creating web applications for clients. For these teams, every solution will involve some kind of 3-tiered system, probably using one of a handful of languages the team is an expert in. Now it’s just a matter of understanding the problem so they can iron out the specifics of the solution.

Most of the time your team is going to have a pretty good mix of knowledge, and as you work to complete the project you’ll embark on a circuitous journey through the design space, eventually coming to a point where you’ll feel comfortable moving forward with design and full scale development.

Your team will have some base knowledge at the start of the project. From there you’ll start to explore requirements which might lead you to thinking of certain solutions. Of course any solution will involve trade-offs which will expose new problems to consider so it’s back to the client to talk about the positives and negatives of one solution over another (quality attribute scenarios work great in this situation). As you explore these new problems, you’ll gain new knowledge which will then help you learn even more about the solution. Throughout this journey you might create prototypes, architecture descriptions, sketches, or code stubs. Each of these things will serve as a communication aide for your team and clients and give you tangible evidence of your understanding of the design space.

The ultimate goal of this line of thinking is to figure out a better way to understand how to effectively explore the design space so you can more effectively reach a point when you have enough knowledge to be able to move forward with implementation. At some point, you’ll feel comfortable with what you know and create an appropriately detailed design. Or maybe you’ll just jump right into implementation. Exploration might take a few minutes or several weeks, all depending on what you already know about the problem and potential solutions, not to mention the size of the design space that needs exploring (more space implies more complexity and bigger systems). You might leave large swaths of the design space unexplored when you create your design or move onto implementation or you might explore everything fully. Regardless of your knowledge, it is important to realize that the amount of detail you use in your design and when you do that design work – the design strategy you choose – is an independent matter but your knowledge will limit your design capabilities.

The awesome part about thinking about design in this way is that even though you’ve moved on to a different phase of your project’s lifecycle, your knowledge and understanding of the design space will not cease. You’ll continue to learn more about the problem and solution with each step of implementation until the project ends. And by the end you will have the most complete understanding of the problem and solution yet. Of course, this will also be the time when this understanding is least helpful to you since the project is over! The essential key then is to apply methods which help you to quickly and efficiently understand when enough is enough so you can create useful plans, organize your team, and begin implementation with confidence that you will succeed.