Building muscles for change: Complexity and collaboration

Suppose you are a person of average health. You try to take the stairs; you're a little out of breath when you get to the top. You go to the gym for a while; the results are discouraging compared to the effort. The muscles in your body are optimized for your current lifestyle, which doesn't require a lot of physical activity.

Now suppose you decide to run the Boston Marathon. Do you have the potential to do it? Almost certainly -- but you know it'll take lots of training. You'll need to build the discipline, the muscles, the stamina, the techniques. And then you can do it.

Like our human bodies, our organizations have the potential to escape sabre-toothed tigers with split-second reflexes, or to bring down a woolly mammoth to feed the tribe. But do we have the current ability? Identifying the boundaries of an organization's current abilities and proposing a path for increased organizational capacity are core activities for business architecture.

Sometimes, though, we're surprised by the gap between our potential and actual ability. Maybe the problem is inherently different from what it seemed to be? Others have proposed that there are significantly different kinds of problems an organization might try to work on: [1]

• Simple problems that are routinely solved using known procedures (e.g., helping a customer at the help desk);
• Complicated problems that can be understood and overcome through analysis, expertise, and best practices (e.g., designing a house for a family);
• Complex problems that cannot be fully analyzed up front and require a highly adaptable, iterative, and experimental approach (e.g., starting a new business in a new market); or,
• Chaotic problems that are so fast-changing and open-ended that they are not subject to any repeatable approach (e.g., bringing about lasting world peace).

Further, others have proposed that some problems are wicked, meaning they are difficult or impossible to work on because of "incomplete, contradictory, and changing requirements that are often difficult to recognize" [Wikipedia]. This term was coined for large-scale social problems (poverty, sustainability, etc.), but some writers suggest (and I agree) that wicked problems can appear in large organizations or even large IT projects, with characteristics such as: [2]

• People aren't yet committed to working on the same thing; they don't agree what the right problem to solve is
• People aren't yet in enough agreement on right or wrong approaches to decide between alternative solutions
• People can't yet agree when enough has been done to address the problem
• The problem and solution are very interrelated with other similarly intractable problems
• Attempts to make progress are being judged as a total success or failure to solve the whole problem, rather than as a necessary experiment

(To avoid diluting the term "wicked problem" and its relevance for public policy, I want to emphasize that within organizations, while it is useful to identify apparently "wicked" internal problems, the purpose of doing so is to focus effort on transitioning the problem into one that is tractable. Unlike societies, organizations have a lot of control over the problems they choose and how they scope them.)

Applying the above concepts to my experience in higher education, I think large initiatives typically require a progression from wicked, to complex, to complicated, to simple, like this:

• Wicked - The problem is poorly defined and there is lack of consensus about how (or even whether) to approach the problem space; so we bring stakeholders together until we get to:
• Complex - The problem is scoped and people are committed to help solve it, but it is unique in our experience; so we try out different approaches until we get to:
• Complicated - We've identified (or created) expertise that makes the problem (or parts of it) directly tractable; so we work on it until we can implement:
• Simple - We have a solution that can be maintained by trained people in a well-documented, repeatable way.

In effect, every major change for an organization starts as a wicked problem -- for that organization, relative to its abilities -- until enough stakeholders can commit to work on something definite together; then it is a complex problem until people working together have tried enough approaches to to make it tractable; then it becomes a complicated problem that sheer resources and expertise can overcome in a predictable timeframe and budget. (If you agree with that, then an important corollary might be that time and budget are not predictable until people are collaborating effectively and the problem is transitioning from complex to complicated.)

Here's where collaboration comes in. In those early stages while the problem is wicked or complex, making progress requires people to build shared understanding and commitment -- often many people. Collaboration doesn't always work out, but it is still the best tool we have as human beings (it's how we reform a health care system or put a person on the moon). The basic reasons for this are two sides of the same coin:

• No single person has the capacity to maintain full understanding of the problem, design a balanced solution, implement it, and drive its adoption. Addressing complexity starts from admitting that no lone genius is going to solve the problem, and no individual heroic leader is going to transport us to the future.
• Whether the problem can be considered solved is inherently a matter of agreement and perception. There isn't an obvious right answer; there just might be an outcome that most people are satisfied with. Solving this kind of problem relies on bringing people along -- some as direct collaborators, and many others as willing participants in the solution.

Simply: without collaboration, an organization is likely to fail at big changes because it has neither involved enough people to arrive at a sound solution, nor convinced enough people for the proposed solution to be adopted.

Crucially, the degree of collaboration required to address the problem at hand may not yet exist in your organization. It is a muscle, and if it hasn't been used recently, it has atrophied. So the proposed problem comes with a meta-problem: the organization isn't ready yet to collaborate on the "real" problem.

This is pretty well understood in professions such as change management or management consulting. These professions bring practices such as organizational readiness assessment, maturity models, and specific competencies to help organizations build the muscles they need to start the marathon they really wanted to run.

As a business architect it is crucial to identify the need for this muscle-building and help management address it (because business architecture is management support). In my experience, initiatives that try to skip their "strength training" (or remain oblivious to it) become very costly. In the worst case I've experienced, it looked like this:

The organization hasn't fully agreed on the problem to be addressed (scope and commitment) but starts the project anyway. Existing capacity for collaboration is insufficient for key partners to contribute effectively, and the project isn't resourced (doesn't have the time and skills) to build up organizational capacity for collaboration. Key stakeholders start to disengage (because their input doesn't appear to matter), and so the project team starts to disengage (because it's too difficult to obtain relevant input). The project team turns inward to focus on its deadlines -- from here on, anyone not inside the team "bunker" is regarded as an obstacle, not a partner.

More complexity continues to surface. As the project continues to add detail without enough context, it diverges from the organization's needs. Senior managers no longer have enough insight or influence to guide the project from the outside. After several years of design and implementation, the project comes to a grinding halt as it becomes obvious to everyone, including the project team, that there is no way to validate the proposed solution, so it can't be adopted. The project has to be largely restarted with a new way of working together, revisiting all the major design decisions, taking it 50% or more over time and budget.

Obviously all your major projects can't be paused just because some business architect says the organization needs to build more muscles! Rather, the practical challenge is to build up enough organizational capacity for collaboration in the early days of a project (consider it like a project deliverable), so the capacity is there when it's needed.

What's involved in building up collaboration? There are many sources more qualified than me on this; I would encourage any business architect to look to other professions on this topic, and defer heavily to management on what collaboration practices have already worked or should be tried. A few key points I feel confident about stating:

• Collaboration means working together toward a shared goal; it is not just communicating better or being present in meetings.
• Collaboration relies on trust, which is built through working together on something, which results in growing to respect each other's strengths and constraints.
• Collaboration relies on motivation, which is heavily influenced by leadership, including role modeling collaborative behavior, rewarding collaboration, making it part of organizational culture long-term, and reinforcing shared goals.
• Collaboration can start small with tactical changes (such as managing each meeting toward a shared result and following up on next steps) -- but managers need to periodically assess whether all the small steps toward collaboration are adding up to the level of collaboration needed for the next phase of work.
• Collaboration can be accelerated by good facilitation, which provides external structure and support (like your personal trainer for the marathon, or the Scrum Master for your Agile team).
• Collaboration does not scale linearly. The effort it takes to get two people to collaborate closely is probably not 1/100th the effort it takes to get 100 people to collaborate at that same level. Because of this, other management tools such as hierarchies and delegation are also important in large efforts.

If you're an IT manager and this all sounds very theoretical to you, think about the last time you formed a new team to do a project or provide a service. How much time and effort did it take for that team to "form, storm, norm, and perform", compared to the actual design and implementation? If you're a non-IT manager, similarly, the last time you saw a major change happen, how much lead-up did it take to get the organization on board? I think you'll find that the cost of the necessary collaboration is quite significant, and it would be worthwhile to plan for it.

Those are my thoughts. In your work, what have you learned about complex and wicked problems and the degree of collaboration needed to address them? For each major initiative you're involved in, is the problem still wicked, or complex or complicated? I would love to hear from you in the comments or in email. Thanks for reading!

Endnotes:

1. See, for example: Snowden & Boone, A Leader's Framework for Decision Making (Harvard Business Review 2007); Nason, It's Not Complicated: The Art and Science of Complexity in Business (2017) as quoted in Kinni, The Critical Difference Between Complex and Complicated (MIT Sloan Management Review 2017).
2. See, for example: Wicked problem (Wikipedia); What's a Wicked Problem (Stony Brook University); Wicked Problems (wickedproblems.com); based on concepts originated in Rittel & Webber, Dilemmas in a General Theory of Planning (Policy Sciences 1973).