Five years ago I thought that if I did a better job of programming, other people would like me and want to follow my example. It turns out that applying XP is not as simplistic as this. XP takes place in a complex social context, so simply applying techniques doesn't give you control of the organization or even your project.
Applying XP and seeing dramatic results takes a while, more like years than weeks. You should see big improvement in the first weeks and months, but those improvements only set the stage for the big leaps forward that happen further down the road. There is so much waste in software development. The waste is rooted more in what we believe and feel than in what we do. Becoming aware of and addressing those beliefs and feelings takes time and experience.
The word "adoption" as applied to a style of software development has all the wrong implications. Taking on a style of software development does not cover or eliminate your pre-existing problems. The problems you have are still your problems. With XP you will have a new context in which to go about solving them. XP is not what addresses them. You are going to address them, in your own way, in your own time, in the context of XP or whatever process you use. While you may have a vision of how you are going to develop in a few years, by the time you get there you'll be developing in your own style. Your own style may be significantly influenced by a vision like XP's, but it'll still be your style.
Starting with XP is more like getting into a pool than adopting a child. There are many ways to get into a pool: you can dip in a toe; you can sit on the edge and dangle your feet; you can walk down the steps; you can perform a smooth, powerful racing dive; or you can do a cannonball, making a lot of noise and getting everyone around you wet. There is not one right way to get in the water.
Once teams start applying XP, there is always the danger of reverting to the old way of doing things. Programmers who know better still change code without writing a failing test first. Managers who know better, who have experienced the benefits of clear and honest communication, still demand more of teams than anyone believes is possible. Organizations that see dramatic improvement elect to revert to old methods and waste in times of stress. Reverting to past behaviors regardless of their effectiveness is common.
Organizational reversion is hard to address because it is out of the team's control. The XP team does a great job, deploys more functionality than originally requested with minimal defects right on time with no extra stress or hours. Another team pulls all-nighters, slashes scope viciously at the last moment, and deploys amid a blizzard of defects. When the organization downsizes, the XP team is fired because the other team showed more "commitment". XP teams work differently, standing out in a way that can have negative social and political effects. Teams need to emphasize their commitment to the organizational goals and show how their style of work supports these goals.
Finding an executive sponsor to champion XP within the organization smooths your interaction with the company while you are transitioning to a new style of work. Expect to be accountable to the person who is standing up for you. If you don't have support from higher up in the organization, your own satisifaction with your team's rapport and productivity may have to suffice.
The way to begin organizational change is still to start with yourself. It is the one change you have control over. First develop your skills, then put them into service. Leading by example is a powerful form of leadership. Dave Thomas and Andy Hunt, the "Pragmatic Programmers", are excellent examples of this strategy.
I was talking recently with a technical lead who told me that he totally supported programmers writing automated tests. "Great," I said, "you've tried JUnit then?"
"Oh, no, I've never written any tests. I just think it's a great idea."
Expecting others to do what you are not willing to try yourself is disrespectful and ineffective. Asking them to take risks you aren't willing to take undermines your relationships and destroys the cohesiveness of the team. This misalignment of authority and responsibility creates distrust. You also lose the opportunity for learning, feedback, and self-improvement.
The strategy of learning skills and putting them into service works at many scales:
You learn test-first programming, then share it with your team.
Your team learns to estimate and develop story-by-story, then invites internal customers to pick stories.
Your organization learns to deploy solid software predictably, then invites external customers to be part of planning.
In each case, the gesture is the same: change myself, then offer the fruits of that change to others. Both steps create value. When I change, it's because I've found a way to improve. When I offer new skills to my customers, I pass along these benefits.
"Continuous" improvement is a bit of a misnomer. It means continuous awareness, responsiveness to feedback, and openness to improvement. When you know how to improve, then you improve. You make a change; observe the effects; then digest the change, turning it into a solid habit. Eventually you hit a plateau from which you can absorb more feedback and identify the next opportunity (Figure 23).
Sometimes you'll try a new practice only to discover that your performance actually falls (Figure 24).
When this happens, it doesn't mean that improvement is impossible or that the practice you tried is inherently bad. It may mean you don't have enough experience with the practice. It may mean you weren't ready for the new practice, because some hidden prerequisites weren't in place. Change back and address the underlying issues. Later, the practice in question may appear at the top of your improvement list again, and this time you'll be ready.
The course of improvement is not smooth or predictable. It is sensitive to both the context in which it begins and the course of improvement itself. A sequence of practices that worked magic for one team might be a disaster for another.
I don't want to imply that the course of improvement is always slow and tortuous. I have seen teams turn themselves around in weeks. Conditions that facilitate sudden turnarounds are:
Aligned values. The team and organization are willing to accept and work with the XP values.
Pain. The team has been through a recent loss like layoffs or a failed deployment. Clear memory of recent pain makes people more willing to try dramatic changes.
While the course of such improvement looks dramatic and discontinuous, it's really just the normal wavy curve compressed because the team is so receptive to change (Figure 25).
Forcing these conditions for rapid change is not ethical. However, if you find yourself in this situation, you have an opportunity for seemingly miraculous change. It's not magic, just the regular stuff happening very quickly.