[Here's a recap of the first talk I gave at XP2010. Since it was a Lightning Talk, rather than posting slides I've summarized my talk and added references directly in this post. I welcome comments and discussion.]

Craftsmanship is an interesting model for thinking about how to teach someone to become a great software engineer. Industry hasn’t always done the best job taking advantage of this metaphor for enabling training and instruction. Sure, there’s agile coaches and conferences like XP2010 where peers can collaborate, but rarely does an organization, a business, deliberately encourage and enable engineering growth for the software engineers they hire.

As we learn how to build software we go through the three stages of craftsmanship.  For most of us, we are apprentices in university, taking courses and learning the basics of computer science and software development by imitating our professors and the books we read.  We are journeymen the first few years on the job as we start our careers, applying the lessons we learned in school in practical setting and trading tips with fellow journeymen.  Eventually some of us pass some kind of test under the tutelage of a master and are ourselves declared as such.  The frustrating part is that so few people find masters to help when attempting to cross the threshold from journeyman to master.  How do you know when you’ve made it?  Where are these great masters, these mentors for helping to learn how to be a great software engineer?

Wouldn’t it be great if there were a place, a dojo if you will, that we could go to practice with other journeymen under the guidance of masters, interacting with apprentices just starting out on their craftsman journey?  As it turns out there is such a place.

The Master of Software Engineering program at Carnegie Mellon has been teaching professional software engineers how to build software better for just over 20 years now. The faculty and staff at the MSE have honed some practices that can be directly applied in industry. Normally I wouldn’t advocate transitioning academic education practices to an industrial environment but the MSE is a near perfect hybrid of industry and academia. The studio project, the capstone project which forms about 50% of the curriculum is a long duration (16 months), real project with business clients who expect software that will provide real business value. Commitment varies and during the summer semester, student teams are working on the studio project as a full time job, dedicating over 40 hours a week to the project. In addition, unlike most academic programs, all students are experienced engineers with at least 2 years of industry experience.

A dojo is a place for training, a place where a variety of students with different backgrounds come together to practice and become better at their craft. So while the MSE makes for an excellent dojo, it’s not easy for everyone to move to Pittsburgh for 16 months of intense study.  So, how can the success of the MSE be applied within industry? I think that there are six key practices where the MSE excels that industry should take note for training and professional development.  These are practices which can be applied in nearly any business setting with effective results.

Education – In school we can take classes. On the job we can read books, start discussion groups, read blogs, and go to conferences. Education becomes a catalyst for growth.

Mentoring – Mentors are guides who encourage growth by asking probing questions and pushing those being mentored out of their comfort zone. Mentors are there to dust you off when you fail and never directly solving problems for those being mentored (a favorite technique of the MSE mentors is to answers questions with questions).  In the MSE program, every student meets with a mentor once a week for 30 minutes to discuss how the project is going and thoughts on software engineering. This is a significant commitment for industry and so holding mentor meeting perhaps over lunch maybe once a month is sufficient. The point is to help novices and journeymen to find masters for guidance.

Proposals – Proposals help teams focus on the think-act-reflect cycle for approaching software from an engineering perspective. In a proposal, teams think through methods, processes, and techniques that will be used and this written (it may be brief or as detailed as necessary) proposal becomes the basis for evaluation and reflection for the team. In essence the proposal acts as a plan for determining an approach to software engineering practices.  The whole point is to get student engineers to start thinking in terms of the simple to see but takes a lifetime to master, cyclic think-act-reflect approach to problem solving.  See this article from my studio team’s reflection blog on understanding when decisions are made and the complete archive of proposals from my team and others are available from the MSE studio archive for some concrete examples of how proposals work.

Presentation & Critique – Communication and collaboration is a powerful tool for learning. During a presentation and critique, a team presents a proposal and that proposal is the critiqued by both mentors and peers. The comments and questions are then taken into consideration when revising or changing proposals. This is a powerful tool that doesn’t cost much and fosters knowledge sharing across an organization.

Peer Collaboration – This is so obvious I shouldn’t have to say it but simply talking to peers is one of the most often overlooked sources of information and learning. Many professional environments inadvertently create physical barriers which further prevent collaboration. Team lunches are nice for getting to know each other, but genuine collaboration must involve asking hard questions and then collaborating with a diverse group of individuals to help answer those questions.  Presentation & Critique is one way to facilitate this.  Setting up the environment to encourage collaboration is another.

Reflection – I have come to believe that this is the single most important practice in software engineering. If only more professionals would take the time to reflect on what they do and use that reflection to drive improvements then many of the most difficult problems we face as an industry would begin to resolve themselves. Effective reflection is ongoing, mentally intense, and difficult to do well. It involves both hard data and soft feelings.

All of this combines to create a place filled with passionate software engineers of all different levels of mastery, each learning from one another, and taking the field of software engineering to an entirely new and better plane of existence.  If you are ever in the Pittsburgh area, please stop by the Cave (the place where the studio teams do their work) for a tour!  You can contact information and further details about the program on the MSE Website.

References

• D. Garlan , D.P. Gluch , J.E. Tomayko, Agents of Change: Educating Software Engineering Leaders, Computer, v.30 n.11, November 1997, p.59-65.
• D. Root, M. Rosso-Llopart, G. Taran, Proposal Based Studio Projects: How to Avoid Producing “Cookie Cutter” Software Engineers, CSEE&T 2008, pp. 145-151.
• D. Root, M. Rosso-Llopart, G. Taran, Key Software Engineering Concepts for Project Success: The Use of “Boot Camp” to Establish Successful Software Projects, CSEE&T 2007, pp. 203-210
• J. Tomayko. Teaching Software Development in a Studio Environment, Association for Computing Machinery, ACM 0-89791-377-9/91/0002-03000, September, 1991.
• The 9 crafts of software engineering proposed by Alistair Cockburn
• Schon, Donald, The Reflective Practitioner: How Professionals Think In Action
• MSE Studio Archives – a record of over 70 past studio projects since the late 1990s

Late in the afternoon on December 25, during one of the loudest, howling winter storms I’ve ever experienced we lost power. Normally this wouldn’t be a big deal except I was in a vacation house with 20 other people, basically my wife’s entire extended family.

After the power went out, the heat did not fire up. The tree went dark. So did the TV, DVD player, and Wii.

Using a pair of LED headlamps my wife and I received for Christmas the 20 of us took turns rolling and stuffing homemade ravioli dough for dinner. Christmas ravioli making is extremely serious business and I had finally been promoted to “unmonitored ravioli stuffer” this year. Luckily we had manual pasta rollers to flatten the dough. There’s always talk of “upgrading” from the hand cranked system but this year, tradition trumped technology.

Once ravioli are stuffed, they have to dry for a few hours before cooking. In the years past this was the time to play with new games or watch a new movie. Of course, without electricity most of our new toys were rendered useless.

Since the heat was off and the vacation house was gigantic, the fireplace in the living room was our only option for warmth. Lighting the room was the handful of candles we had, originally intended for dining ambiance, and the low glow from the fire.

To pass the time, we sang Christmas carols in the near dark. It was all sort of surreal, a setting I am certain we would not have created on our own had we not lost electricity, had we not been so completely pushed out of our comfort zone. Even more fantastic was that after over 20 years of putting it on my wish list, someone actually gave me a pair of night vision goggles for Christmas. Surprisingly, this “toy” is the real deal. I was able to wander around a completely pitch dark house with no problems. The night vision, along with the head lamps (which single handedly saved dinner), were by far the best gifts of the day. The electricity turned on around 1:00am that morning and by the next morning, everything went back to the way it was.

Sitting in the dark on Christmas night got me thinking. This was an experience that I never would have chosen, a situation I would never have intentionally embraced, but it turned out to be pretty fun. How many opportunities have I missed because my default attitude was to stay inside my comfort zone? While happy accidents are great, can’t I do more to create opportunities rather than relying on happenstance? Given the time of year it only seems appropriate to ask these sorts of questions.

I’m a technologist, a scientist, and an engineer at heart. I love playing with gadgets, tinkering with software, and working on interesting and challenging problems. Sitting in the dark with no power, my normal pursuits removed, it was easy to remember that people are important too. Sure, when building software everyone always talks about how people are important, but when building software we call people “users” essentially reducing their humanity. After all, there are only two industries where the customers are called “users” – software is one of them.  Is it really appropriate to use the same term to describe software clients and drug addicts?

My lesson from the night: the technology and gadgets and programming and processes and everything else are awesome, but they are meaningless if they fail to create a genuine relationship with people.  I do believe that relationships matter and that people are important. I’d like to do a better job of thinking about people first this year. In the rush of excitement surrounding every new technological achievement, it’s sometimes easy to forget that helping people is why I build software.  It’s too bad it took a harsh winter snow storm to remind me of this.

It’s easy to convince a group of people to follow you when you’ve got a great idea that you’re passionate about. Passion is like a highly communicable virus, easily spreading from one host to another. Something funny happens, though once enough people are on board with an idea: new ideas become less infectious over time as if the group has built up antibodies against risk. It’s always unfortunate to see this happen since most organizations are initially brought together by an idea that was so risky and so contagious that everyone wanted to be a part of it. Eventually, if an organization is not careful, it becomes a place where Big Ideas go to die, a sort of idea graveyard.

Kathy Sierra explains it best:

Any time a new idea is brought to the table, especially a Big Idea, the group acts like white blood cells, attacking the Big Idea as if it were a foreign invader, reducing the idea to a benign and much less exciting version of itself. The end result is something that no one can really get all that excited about. It may get the job done but it certainly doesn’t inspire anyone.

It’s really difficult to fight off this group tendency. It’s much, much easier to say no to something than yes, especially when there’s an established status quo. Once a group (company, club, team, non-profit, whatever) feels safe, group risk aversion magnifies problems into insurmountable barriers. Kathy Sierra (boy, do I miss her blog!) talked about this too, avoiding risks leads to the safest route, but the safest route, ultimately will only yield incremental improvements:

In most cases, the barriers preventing new Big Ideas from being achieved are no bigger than the barriers overcome by the group when it formed. The main difference is that there’re more people to get over the wall now than when it was just you and your Big Idea which brought the group together.

As a group it’s important to fight risk aversion which is often reinforced by groupthink. It’s a difficult job, but if growth and innovation are goals, taking risks is an essential part of success. Figuring out how to work with Big Ideas rather than fight against them (and in turn deal with risk as an organization) makes it easier to jump the wall as a group when and if the time comes. Delegating is a great way to build practice taking risks If you’re able to trust someone else to do something that you could reasonably do yourself you’re on your way to letting others run with their Big Ideas.

As an individual with a Big Idea, it’s important to quickly figure out whether your idea is something that has legs or whether it’s something better left to the back burner. Seth Godin’s The Dip has some simple advice but ultimately it’s up to you to determine. If you yourself are not prepared to pull each and every member of the group over the wall that is blocking you from realizing your ultimate vision then it may not be time to unveil your idea to the group.

Both the group and the innovator have a part in killing a Big Idea or making it fly. Recognizing that you have to work together is the first step toward achieving greater success.

Building a great software product is only half the battle. The relationship the user builds with your software has the single greatest influence on how awesome that software is. Sorry. It’s not the language the software is written in. It’s not the algorithms. It’s not all the processes you used behind the scenes to create it. If the relationship sucks, your software sucks. Period.

To prove this I’m going to use a tangentially-related personal experience that has absolutely nothing to do with software. In this case, it’s ok, because realistically, this bit of reflection actually has nothing to do with software – remember it’s about relationships. Let’s talk about two concerts I’ve seen this summer.

Coldplay: “You’re Awesome!”

I’ve heard Coldplay’s radio hits but I was never really a huge fan of the band. Had it not been for a friend’s invitation I would never have gone to this concert and I would have missed out on one of the best live shows I’ve seen. Acceptable radio hits became handcrafted masterpieces, more amazing live than on FM. The most interesting part of the show, and the thing that left the greatest impression on me, was the band’s absolute humble devotion toward the fans in attendance. At one point the band walked out to mini-stages set up in the seated area under the pavilion and way out in the lawn, giving all fans an equal chance to be a part of the show, not just the diehards that purchased overpriced tickets.

Coldplay’s attitude created a completely positive vibe that made me want to enjoy myself.  After they had exhausted themselves for close to two hours, doing everything possible to create a positive experience for the audience, Coldplay thanked us, not for coming to see them, but for allowing them to play for us, “You’re awesome, thank you for letting us play for you tonight!”

Dave Matthews Band – “We’re Awesome!”

I didn’t realize that Dave Matthews Band live shows are really just one long meandering jam session.  Every one of the band members is an amazing musician, but 14+ minute songs start to wear after a while.  Dave Matthews rarely spoke (when he did it was incomprehensible) and most of the band wore sunglasses throughout the entire show. Overall, it was a very mediocre concert that was better suited to an afternoon picnic setting than an evening at a large outdoor amphitheatre.  From a technical perspective it was amazing but overall it failed to entertain me.

The most interesting thing about this concert was the crowd. In spite of the fact that the band seemed to be playing solely for their own amusement, the audience was full of absolute fanatics, complete with their own dress code – plaid shorts and tie dyed shirts (though rarely together) with sandals – and a shared emotional connection to every song the band played. Dave Matthews Band is here to play music – If you like it, whatever, if you don’t, whatever.  “We had a fun time tonight and I hope you did too!”

It’s all About the Relationship

It turns out that the relationship the bands built with the audience during the show was directly related to how much I enjoyed it. It should be pretty obvious how this extends to software.  The relationship your software builds with your users will determine how much they enjoy using it. Is your software helping users be amazing or does your software expect users to change their behaviors to use it? Are you building something for yourself that others might happen to like or are you targeting a specific audience?

Software that is technically the best will always fall short to similar software that builds a relationship. Kathy Seirra expressed a similar sentiment with her Dating Rules for Software. But there’s a small catch to Kathy’s rules as exhibited by the Dave Matthews Band. Apple tends to act this way too. Sometimes if you do your own thing and you actually are great, you can afford not to focus on the relationship. It’s extremely difficult but sometimes it works. Sacrificing the relationship creates a razor thin margin for error but when you get it right, you’ll breed fanatics that will love you (almost) unconditionally and critics who will absolutely hate you – there is no mediocrity.

Sacrifice the relationship and you could be the best, but only until someone almost as good comes along who thinks about others before themselves.  I think Microsoft has learned that lesson the hard way.  No matter what you do, relationships rule the day.

The setting is a small library filled with various books on software engineering. Five people are sitting around a single, small table, the room overcrowded with the group. The Square Root team is reviewing the mid-semester presentation I threw together last night before we were to show it to our mentors in a few hours. I was showing the team the last slide. Up to that point everyone had pretty much agreed with the content in the presentation.

“Does anyone have any other risks they’d like to bring up?” I asked, confident that, as the team leader, I had a firm grasp on how the team was doing and where our current problems lay.

I waited a few seconds.

“OK, if nobody’s got anything I’ll go ahead and email this out…”

“Communication,” the quietest member of the team chimed in. “We have problems with our team communication.”

I was in shock. Surely this must just be an isolated problem. “Do you mean team communication or do you mean you don’t understand some aspect of the project?”

“I agree. Team communication seems to be problematic.” Now there were two dissenters.

A few seconds later there were three. Three fifths of the team felt we had team communication problems that were directly impacting the outcome of the project. I sat there shocked for a moment. I had just been blindsided by a team communication problem, therefore, obviously, we have a team communication problem.

Looking back at this incident I have decided that leadership, my leadership, was to blame. From the beginning, I had led the way I liked to be led: hands-off with enough space to make my own decisions and get things done the way I wanted to do them, the polar opposite of micromanagement. Of course, for me this worked out really well. This was the way I liked to work and the way I was used to working. Unfortunately, for a new team, a team lacking in trust, whose members didn’t know one another and had other commitments and priorities (school work) my ideal working environment and preferred leadership style was disastrous.

To prevent utter team destruction I changed perspectives. From what I could tell, team members didn’t know what they were supposed to be doing. I had assumed that this team would operate similarly to my last team where everyone would come together like a well-oiled machine to get work done. Shortly after the incident in the library I remembered that my last team took almost a year to become that awesome. Of course there were going to be problems in my new team.

Since my team seemed not to know how to take initiative in completing tasks and getting work done I thought I’d set an even better example for them. By doing so, maybe they’d get the hint and follow suit. The next day I kicked things into gear. I took on more work. I picked up all the slack I could and then some. I took on tasking outside of work specifically assigned to my role. I set high standards for myself and my team and we were going to meet those standards if I had anything to do with it.

Leadership disaster number two was prevented thanks to a timely assignment in my Managing Software Developers course. Leadership that Get’s Results by Daniel Goleman defines six distinct leadership styles and gives a little advice on when each style is appropriate. Up to that point I had no knowledge of such styles and had been flying on instincts. The following table is taken from Goleman’s paper but I encourage you to read the full paper to gain a better understanding of these ideas in a more complete context. There’s also tons of information on the web, a search away.

As it turns out, I started with the right idea by using an authoritative style of leadership even though I didn’t know the name for it. I failed with this style because the team wasn’t ready for it yet. We didn’t have a clear vision or clear goals. We weren’t yet working well together. There was little buy-in to my leadership or the few goals the team did have. We were a full-fledged dysfunctional team. Switching to a pacesetting style was just about the worst possible thing I could have done. If I had gone on for too long I am fully confident that I would have destroyed the team and we wouldn’t have gotten anything done during our first semester. With my new found knowledge of the different leadership styles I now had new options. I immediately put three styles to use: affiliative, coaching, and coercive.

The team was clearly broken. My hope was that affiliative leadership might help build trust and better bonds among teammates. Being a programmer I tend to be a little more logical than emotional so trying to tune in more to how people felt was tough. I found that affiliative leadership goes hand in hand with vulnerability and trust and that individually thanking someone for their hard work, even if it’s relatively small, and meaning it is one of the most important things you can do as a leader.

In addition to team harmony, it was obvious that some team members were struggling with the tasks they had been given. Rather than taking over those tasks as I had been doing, I decided to try taking on more of a coaching role. In some cases I would help team members directly, other times I encouraged other team members to work together on tasks. The end result was a team better able to work together to accomplish tasks.

In spite of all this, I had the feeling that if we didn’t do something immediately, the entire semester would be a wash. To prevent this from happening I used coercive leadership in an attempt to get us back on track quickly even though the overall impact could be negative in the long term. In a sense, I was willing to be a bad guy so the team had a chance of meeting its goals. The gamble paid off and the way the team operated turned around almost instantly.

About two months after I had been blindsided by unseen communication problems, my team seemed to be working together much better. Problems were being flushed out into the open more quickly and everyone on the team seemed to enjoy working with one another on the project. I am not going to take full credit for the change but I will take credit for being the catalyst that put the change into motion. All because I changed how I led the team.

There is a small downside to the changes, but it’s only really a downside because I enjoy doing technical work. The leadership role on my team has evolved. By the end of the semester I found myself directly responsible for almost no work but rather, I was the go to guy for all problems the team was facing. I had my hands in everything but I wasn’t able to really sink my teeth into anything. If this is what management is like and I ever do become a project manager I will need to find ways to remain technically involved in something or I think I’d eventually go insane.

The biggest lesson I’m taking away from this experiences is that sometimes the team’s needs and my needs aren’t going to line up. Recognizing when this is occurring and finding a balance between these conflicting needs is critical to team success.