The Dorset House Quarterly Interviews

James A. Highsmith III
Author of Adaptive Software Development: A Collaborative Approach to Managing Complex Systems

ISBN: 0-932633-40-4  
©2000  392 pages   softcover  
$50.95 (includes $6.00 for U.S. shipping by UPS)

DHQ: How is Adaptive Software Development (ASD) relevant to today's software development environment?

JH: I've recently been asking two questions at the beginning of my speaking engagements. First, "How many of you think the Internet, e-business, and e-commerce will significantly change your company's business over the next five years?" It's a somewhat silly question, and nearly everyone raises their hand. The second question engenders a more confused response: "How many of you think the Internet, e-business, and e-commerce will significantly change your practice of software project management?" When I ask this one, no one makes a move.

Why do we continue to think that the Internet changes everything except "me"? I think the closing lines of the introduction to the book say it best:

. . . somehow, while admonishing our businesses to adopt technology more rapidly, we, as the purveyors of technology, have failed to anticipate the impact that the speed and turbulence we have created has had on our own practice of management. Adaptive Software Development is ultimately about rethinking how to manage in the turbulent times we have brought upon ourselves.

DHQ: The front cover shows a mountain climber. Tell us about climbing.

JH: I used a number of climbing analogies in the book. Climbing has been a part of my life for the last 15 years, and I think there are useful analogies between climbing and project management. Let me mention a couple: adaptation to the environment, and the sharp end of the rope.

In climbing, you always have a planned route up the mountain. However, climbers understand that the mountain is in control and the only reasonable approach is to adapt to what the mountain presents. Weather may be the most obvious obstacle. If a storm drops four feet of new snow on an unstable base, you go home. Continuing in the face of inevitable disaster isn't macho—it's stupid. And yet, for some reason, we often convince ourselves that we are in complete control of our software projects, and we make stupid decisions like climbing in high-avalanche conditions. Maybe you adapt to the snowfall—waiting a day or two to let the snow consolidate before continuing. We often need to adapt similarly to project obstacles instead of blundering on, following our original plan. Traditional project management is an extension of a command-control style of managing. It assumes we can predict the future and control our progress toward it. Adaptive project managers are more flexible; they are like climbers who adapt to the landscape rather than try to change it.

Climbers have a term called the "sharp end of the rope." A lead climber places protective anchors as he or she advances. However, since protection takes time to establish and good anchor points are not always available, the lead climber always risks falling twice the distance he or she is above the last anchor.

Furthermore, poor rock or ice conditions may even make anchor points tentative, raising the prospect of a more serious fall. Being on the sharp end is a constant decision-making position. Go another five feet to a better anchor point, but risk a fifteen-foot fall? Being too tentative or too aggressive can be equally dangerous. With some projects, the leader's heightened sense of awareness may be losta detriment to both speed and quality. Extreme projects—those characterized by high change, high speed, and uncertainty—provide the same adrenaline rush that the sharp end of a rope does. Managing a traditional project can be analogous to "top-roping," in which the leader is safely anchored and the chance of a serious fall is nil.

DHQ: The science of complex adaptive systems (CAS) is featured in your book. How can software development managers use CAS theories to improve their effectiveness?

JH: Traditional management practices have been around for several hundred years, and during the early twentieth century, "scientific" management came to the fore. Justification for the rise of scientific management was based in part on Darwinian evolution (survival of the fittest) and Newtonian determinism (we can predict falling objects, therefore we can predict the future of a business). However, just as quantum mechanics and Einstein's theory of relativity have impacted theoretical physics, speed and uncertainty are impacting traditional business management practices. The science of complex adaptive systems provides a powerful new metaphor, a new sense of how organizations function, and I think this can help project managers. The era of the networked organization has just gotten started, and we don't have good models for managing this type of organization yet. One of the things that drew me to using complexity science to better understand organizational dynamics was that it provided a model for both delivering concrete results and producing healthy project teams. We are five years into a 20- to 25-year transition to an e-business economy. Speed to market will be important (and critical, in selected markets), but ultimately, over this transition period, an organization's ability to adapt will be its strategic strength. Understanding complex adaptive systems concepts will help organizations build the skills necessary to adapt.

DHQ: You assert that chaos can actually help a manager. How can software development managers use ASD theories to manage chaos and embrace change?

JH: A large part of it lies in understanding a fundamentally different view of how the world operates. If we think the world is linear and predictable, then we will use a certain set of management practices. However, once we acknowledge uncertainty, another set of practices is needed. One of the tenets of traditional software engineering management seems to be, "If a little process is good, more process is better." This view of a large, monolithic "methodological" approach to software management has actually scared many organizations away from implementing appropriate practices. They are afraid of becoming bureaucratic and drowning in documents. ASD provides a very different model, one based on CAS's "edge of chaos" theory. The ASD approach would rather use too little process rather than too much. It advocates being just organized enough to keep things from becoming chaotic. It is in this "zone" that creativity and innovation flourish.

DHQ: You were featured on the Extreme Projects track and panel at Software Development 2000. What is Extreme Programming (XP)?

JH: Extreme Programming is a development approach that has been around for three or four years now, but recently it's become "hot" following Kent Beck's Extreme Programming Explained (Addison-Wesley, 2000). XP concentrates on a minimal set of practices (pair programming, collective ownership, and refactoring) but emphasizes simplicity, collaboration, and communications—not documentation and process-itis. However, XP is very rigorous with the practices it does advocate. Test cases are an integral part of the development process.

There seems to be an increasing interest in approaches like XP and ASD that focus more on collaboration than on process (Lean Development, SCRUM, and Crystal Light Methods are other examples). Also, an entire genre of software tools is emerging to implement these ideas.

DHQ: How do XP practitioners use ASD for larger-scale projects?

JH: Unlike proponents of the Capability Maturity Model, XP practitioners are circumspect about the domain in which their practices are relevant. They concentrate on small, collocated project teams of fewer than ten. There are efforts under way in the XP community to address the scaling issue. In my book, I discuss three levels of collaboration—interpersonal, cultural, and structural. XP basically addresses collaboration at the interpersonal level (and without the interpersonal level foundation, the other two won't succeed). By looking at structural collaboration issues (workstate life cycles, knowledge management, and collaboration tools) and cultural collaboration issues (management style), ASD may offer one path for scaling XP.

DHQ: What kind of projects do you pursue at Information Architects?

JH: I like working with companies over time to help them adapt to the dictates of the new e-business economy. I've always believed in Jerry Weinberg's Secrets of Consulting principle that you are fooling yourself if you think it's possible to change an organization by more than 10 percent (Dorset House Publishing, 1985). Rather than impose some grand plan, I shoot for small changes that provide immediate results. Hopefully, the small changes will accumulate over time into larger change. Much of my work usually boils down to improving project management and collaboration skills.

DHQ: Thanks, Jim!