Leading With an Issue Portfolio

This post is part of a series for a current Itana community discussion of "organized anarchies." I'd like to share a way of thinking about decision-making in large universities that I've gleaned from watching my mentors over the years, and offer a general method.

How do decisions come about?

In a prior post, I quoted Kenneth P. Ruscio describing decision-making in a university as a process in which "solutions chase problems and problems chase solutions. Decisions are opportunities. They come about when various problems attach themselves to various solutions in complicated, unpredictable and sometimes mysterious ways.” [2]

From my experience I think there are patterns in how change happens in a university that has characteristics of an "organized anarchy". Rather than a linear path of decision-making, change is more like a social process of people collecting around ideas. To me it looks like this:

The first thing to recognize is that in many workplaces, what most people think of as a management decision is the choice to take a shared approach to some issue. But not taking a shared approach is also a choice, and it is the one most often selected in an organized anarchy. So what I often see happening in a university is really a progression through different modes [3]:

• Parallel mode: Is it sufficient for people to handle the issue in diverse ways? If so, the issue stays in this mode until some additional urgency arises, or it's no longer relevant.
• Example: To move functionality to the cloud, should teams at our university use Amazon, Google, or Microsoft? What about virtual machines, containers, or functions? Initially, this is up to individual teams to experiment with.
• Associative mode: Is there interest in (and benefit from) exchanging findings and practices around the issue? This often results in converging on fewer (but still several) ways of addressing the issue. Many issues stay here until no longer relevant.
• Example: As teams start using cloud infrastructure in production, they exchange what they've learned. A few options become clearly better for most purposes than others. Maybe some case studies or recommendations get written down.
• Cooperative mode: Is there high urgency for (or value from) a shared approach? If so, people will try to collaborate on one. After some effort, a shared approach may be agreed on -- or not. If not, at least there is broader awareness of the issue and possible approaches.
• Example: A shared service is defined for teams to easily obtain certain kinds of pre-configured cloud infrastructure from certain vendors. The service takes care of shared needs such as identify management integration, default security setup, etc. Teams are not required to use the service, but it becomes the most common approach.

What can leaders do?

For new leaders this kind of decision flow can be very surprising and frustrating. It doesn't map closely to leadership or change management practices that are all about driving in a straight line from a vision to a result. Leaders often end up with burnout that looks like this:

I've seen people I respect greatly and who are great managers leave higher education in frustration with this non-linear decision-making environment. The non-linear pattern resists strong advocacy (especially early on) and does not reward highly visible, structured leadership.

But there is a role for leaders in this non-linear pattern. In each mode, there is important work to help critical issues get attention and advance to the next mode as quickly as possible. Doing this reduces waste, opportunity costs, and risk to the institution. Leaders can:

Note that even if what you achieve as a leader is to more quickly make clear that there will not be a shared approach to an issue, that is a benefit to the institution. It reduces the time others spend on the issue. It completes a decision-making experiment (or "test balloon"), so to speak, so that other experiments can be tried.

Leading with an issue portfolio

For me as an individual leader, there are issues I have a stake in -- whether I'm explicitly accountable for their resolution (rare) or they're related to my general role (more common). New related issues arise all the time, while others become irrelevant (see above).

Think of these issues as an issue portfolio. Some issues have a high risk/reward for the organization, others less. Some issues are getting too little attention, some too much. As a leader my goal can be to shepherd key issues through the organization. Let's say each dot on this quadrant is an issue within my scope:

This approach is probabilistic -- there's no guarantee that the issue I think is most important will reach a shared approach, no matter how hard I work at it. But I can help create the circumstances in which an issue will go from one mode to the next. Tools for this might include personal influence and networking, meetings, and various kinds of groups from less formal (e.g., brown bags) to more formal (e.g., committees).

How is this better for the institution? Even if every individual just clearly prioritizes the top issues they are advocating for, that helps advance the non-linear process. Leaders can also use traditional approaches to build consensus on a vision around an issue, gather allies, and build a coalition of support.

When even a few key leaders in an institution do this, it quickly clarifies which issues are likely to reach a shared approach, and focuses the effort of collaboration and change on those. Likewise, when teams communicate their key issues, useful collaborations can be found more quickly. And so on.

Starting points for leadership in organized anarchy

Based on the above approach, here are some suggested starting points:

1. 
Define the scope of your “portfolio” of issues. What kinds of stuff do you most need to care about? Don’t just react to everything others bring to the table.
2. Know the current issues in your leadership portfolio. What mode is each issue in, who are current and potential players, and what are the experiments and findings so far?
3. Prioritize which issues to expend your limited energy on (based on what you can glean about importance and feasibility).
4. Apply energy to your top issues as appropriate to the mode each issue is in; help each issue move along when it is ready.
5. Prepare to invest more effort into your top issues in later modes (plan and gather support out ahead of each issue).
6. Look for like-minded leaders and take opportunities to build a coalition.
7. Admit when an issue has become intractable despite your best efforts, and shift your energy elsewhere for now.

What are your thoughts? Do you already do something like this for yourself or your team? What's working for you?

Endnotes

1. Matthew House, A Career in Organized Anarchy: Building Interpersonal Relationships in Higher Education, ACM SIGUCCS Annual Conference (2018).
2. Kenneth P. Ruscio, Leadership in Organized Anarchy, Public Administration Review (2016) (emphasis added).
3. I'm borrowing the terms parallel, associate, and cooperative from writing on child development -- see Wikipedia, Parten's stages of play.
4. See Wikipedia, Hype cycle.

Management Approaches Within Organized Anarchy

This post is part of a series for a current Itana community discussion of "organized anarchies" -- the idea that the business architecture of some higher education institutions is characterized by "many autonomous actors operating with bounded rationality in an environment with ambiguous goals, an unclear link, between cause and effect, and fluid participation with the activities and subgroups of the organization." [1]

How are universities organized?

If you've worked in higher education for a while, you've heard an "origin story" along the lines of: Lo these many years ago in the Middle Ages, the first universities were founded to enable individual faculty to independently create and preserve knowledge. Since then, universities have been governed by faculty to protect their academic freedom, and this history drives the management culture of modern universities -- even as they have grown to include hundreds or thousands of non-faculty employees in all the functions needed to run a mid size corporation. [2]

While I agree that the universities I've worked in demonstrate characteristics of an organized anarchy, my own take on this is that it's less because of individual autonomy (as in the classic origin story), and more because of a multiplicity of unit-level management approaches. Many units within a university are far from anarchic (try asking your registrar or campus police chief what kind of team they lead.) But at a large university, the sheer diversity of management approaches in play, and their interactions, results in a complex system that is very challenging to steer. [3]

As a business architect, thinking of it this way makes the university tractable to some analysis and planned change (though often at such great effort that the change still isn't worth it). If I believed every individual in the institution were autonomous, I'd throw up my hands at any change involving more people than I can fit in a conference room. But if we can understand and connect up diverse management approaches, then change leaders have a fair chance at larger initiatives (though still at great effort).

So what's really going on inside a university?

Thinking about the many different university units I've worked with, I can see a diversity of management approaches. You may see others, and have different names for them, but here's a range:

There are good reasons for this diversity. It's natural for each unit to shift toward a management approach suited to its missions and strategies. Managers in a unit may consciously choose an approach to execute on a strategy [4], or the approach may be evolving and not "self aware" yet.

So even when a university is an organized anarchy with a very Autonomous management approach overall, it contains a multitude of "enclaves" with their own management approaches and cultures. In a large university a map of a few units and their management approaches might look like this:

These different management approaches are constantly in friction when work crosses units, as in this very common example:

Admissions applications to the university have increased dramatically, and the Financial Aid office is desparate to improve technology for processing financial aid offers. The unit is Process Driven, with a flat organization and specialized roles responsible for each part of an intense process that has hard deadlines throughout the academic year.

When Financial Aid goes to Central IT for help, it encounters a Podular unit, where IT service teams work semi-autonomously within their own service strategies, with just a few lightweight shared processes (for IT service management) and shared functions (such as a Project Management Office). [5] Few of the managers in either unit are self-aware of their unit's management approach or the differences between units.

Over years, each time Financial Aid and Central IT try to work together on major improvements, they drift apart again in mutual frustration. Financial Aid is staffed to focus on its processes, and can't free up resources to do business analysis for IT changes. When Financial Aid does state requirements, it needs results by specific points in the year when changes can safely occur. The Podular teams in Central IT have little practice with hitting hard deadlines, and they don't collaborate often enough with each other to be able to pull together the full package of IT that Financial Aid actually needs. 

Multiplied over many units, initiatives, and years, the net effect for the university is organized anarchy. Senior leaders struggle to understand why obviously beneficial changes aren't executed. Participants grow frustrated with how hard it is to obtain a decision or follow through, and develop defense mechanisms to be less accountable. Key opportunities are missed and the backlog of unresolved problems multiplies.

How can we do better?

I think change leaders (and the architects supporting them) can improve the success of cross-functional initiatives by recognizing the multiplicity of management approaches and applying that recognition as suggested below.

Identify the management approaches in play. You may be the first person to ask this question for the initiative, and the participants may not be self-aware of their management approaches yet. Ask questions about how people expect to see goals set, accountability established, decisions made, and outcomes assessed. Note differences that could be pitfalls for the initiative.

Plan for the extra effort and skills needed to bridge different management approaches. At what points in the initiative will differences be the greatest obstacle? It might be in discovery, design, implementation, or operations -- or in other factors such as enforcing a time or budget constraint. The approaches can be bridged, but it takes time and it takes a team with the skills, experience, and position to do so. If that will be a problem for the initiative, you've identified a major risk to escalate.

Help units and individuals solidify their own management approach. Crossing management approaches is even more difficult when the people involved aren't following a consistent approach within their own units. If you can't help management of the unit clarify their intent, at least help individuals clarify how they are going to work. How will they represent their unit? Contribute to decisions in the initiative? To what degree can they realistically commit to the initiative? How do they plan to participate in the work of the initiative?

Manage expectations with sponsors and key stakeholders. These often come from a management approach that is different from that of the participants or the initiative, and get frustrated as a result. For example, a CFO from a Hierarchical unit sponsoring a cross-functional initiative that works Collaboratively with participants from a variety of management approaches is going to start with unrealistic expectations.

Be clear about the management approach of the initiative. Each initiative needs a defined approach to manage its own work, and it may well be different from that of the participating units. Here are some examples of efforts "layered" onto the sample university above:

The approach your initiative takes is what is most under your control, so pick a viable approach and communicate it clearly, early and often. If necessary, train people in the approach. Time spent "onboarding" participants to how they are going to work together is never wasted.

With that effort applied, the example started above might continue something like this:

A new change leader in Central IT hears about the history of working with Financial Aid and decides to look more closely. She spends time in Financial Aid to understand its management approach and needs, and also takes a skeptical look at how well Central IT is applying its management approach. She finds allies and willing participants, and convenes a sponsor group that is ready to understand the cross-functional challenge and exercise influence accordingly.

A cross-functional initiative is formed with a clear management approach, and the participants understand how they will need to behave differently from their normal ways of working. Though the problem space is still very challenging, the initiative now has a chance of addressing it.

And yes, that is a real-life example and the initiative is still producing results after several years.

Is it worth it?

Is it worth it to do the above analysis? Only sometimes. The kind of change leadership and business architecture work needed to bridge management approaches is effortful and time-consuming. It might not take place for any of several reasons, including:

• The benefits of the initiative simply aren't worth the effort
• The urgency isn't there yet; the stakeholders are fine with letting the initiative "drift" rather than work on being more aligned
• The key decision-makers involved aren't ready to discuss, understand, or recognize the consequences of the multiplicity of management approaches

You might also be asking yourself: Is it worth it for universities to operate this way? In my opinion that's a personal leap of faith, similar to the "glass half full" metaphor. Organized anarchy is an extremely wasteful way for a university to operate -- or a quite reasonable way to operate -- depending on whether you tend to see:

• Duplication of effort -- vs. -- Adaptability to meet local needs
• Costly unexpected course changes -- vs. -- Agility to respond in the moment
• Ineffective use of resources -- vs. -- Setting aside resources for experimentation
• Lack of transparency -- vs. -- Protection from interference
• Lost opportunities -- vs. -- Freedom to pursue diverse goals

And so on. Personally I think a better version of the question is, is the university getting the most it can out of its management approaches? That applies to the overall model, each different approach selected for a different purpose, and initiatives that cross the university. And that's something we can all work on.

So how do you see it?

Endnotes

1. Matthew House, A Career in Organized Anarchy: Building Interpersonal Relationships in Higher Education, ACM SIGUCCS Annual Conference (2018).
2. There are problems with this version of history. For a debunking, see George Keller, Academic Strategy: The Management Revolution in American Education (1983).
3. That is, parts of the system are easily understood, but their multiplicity and interactions result in unpredictable behaviors at the level of the system. See Wikipedia, Complex system.
4. As an example framework for this, see Jay Galbraith, The Star Model.
5. I'm borrowing the Podular label from Dave Gray, The Connected Company (2014). For a good intro to the topic see this blog post by Dave Gray: The Future is Podular (2015).

The University as Organized Anarchy

If you work in higher education (or another large nonprofit), do you encounter situations in which your institution's decision-making seems unpredictable or intractable? You may be working in an "organized anarchy," an organizational model described by observers of higher education as far back as the 1970s.

For our March 8th and 22nd Itana calls the higher education architecture community is discussing a paper on this topic by Matthew House, Enterprise IT Architect at Washington University in St. Louis, titled A Career in Organized Anarchy: Building Interpersonal Relationships in Higher Education (also available for download here) [1]. Matt's paper and presentation wonderfully connect our community with scholarly thinking on this topic.

For me, Matt's research has triggered a lot of thinking about business architecture and change leadership in higher education, including questions like:

• Are there systemic reasons why change leadership (supported by business architecture) faces special challenges in higher education?
• What are the particular responsibilities of a leader or architect in an organization with very limited intentionality?
• What tools does an architect or leader have for working in this context?

I'd like to offer a few blog posts to inspire further discussion and I hope everyone will join in with their own thoughts -- especially on the Itana mailing list, which you can join here. I strongly recommend you start with Matt's paper, which is very concise and packed with great concepts.

So what is an "organized anarchy"?

In 2016, Kenneth P. Ruscio, then President of Washington and Lee University, described his job like this:

“College and university presidents preside, if that word can be used, over ‘organized anarchies,’ … Goals are shifting. The boundaries of the organization are constantly being redrawn. The referees are sometimes making up the rules as the game progresses … solutions chase problems and problems chase solutions. Decisions are opportunities. They come about when various problems attach themselves to various solutions in complicated, unpredictable and sometimes mysterious ways.” [2]

Now that is a pretty remarkable statement for the leader of a venerable institution. Anarchy? Making up the rules?! Decisions are mysterious?!! Strong words, but the situation may sound familiar to you, and it is recognized by scholarly observers of higher education.

The term "organized anarchy" was coined in 1974 in a book by Michael Cohen and James March [3] to describe institutions of higher education with "many autonomous actors operating with bounded rationality in an environment with ambiguous goals, an unclear link, between cause and effect, and fluid participation with the activities and subgroups of the organization." [1] Cohen and March identified five properties of decision-making in this kind of organization, which Matt summarized for us as:

• Most issues, most of the time, have low salience for most people;
• The total system has high inertia;
• Any decision can become a garbage can for almost any problem;
• The processes of choice are easily subject to overload; and
• The organization has a weak information base. [1]

Put another way, I've sometimes described it to peers (before reading about this research) as the "near-zero accountability" organization. A place where people often just walk away from what they don't feel strongly about, can often set aside larger organizational goals (unless they become absolutely critical), and often aren't challenged to inform the choices they make on behalf of the organization.

If that sounds familiar, read on.

Who cares? And what is our responsibility?

Seeing our context systematically described this way is first of all a kind of mental relief to those of us who have worked in higher education for many years, trying to bring about planned change and finding it surprisingly slow going. Over the last 20 years I've regularly had conversations with newer employees (from other industries) in which I reassure them that no, they're not crazy, the environment really is quite unusual. It's reassuring to also affirm that for myself.

More importantly, many of you reading this, and many colleagues I've worked with over the years, are leaders trying to help higher education institutions overcome challenges and change for the better. We feel responsible for some part of the best contributions universities can make to society.

How is that responsibility changed by knowing you work in an organized anarchy -- a place where decisions are often non-deterministic and the institution as a whole has little specific intentionality about advancing its mission and the public interest? This is a professional existential question for leaders when they "hit the wall" of what can be done in a large university. Confronted with organizational anarchy, which path will you choose:

1. Do you regard the organizational model as unacceptable (whether practically for purposes of doing work, or ethically), and walk away from the particular situation?
2. Will you work within the means the system offers, finding the path of least resistance to do what can be done relatively easily?
3. Can you effect structural change within the system that enables it (and you) to work differently?

I've taken all the above paths in different situations. Regarding the first path, I do think there are situations in which a university's organizational model is simply counter to the public interest in higher education, and professionals have to make hard choices. I'm not going to focus on that today (but would love to hear your thoughts!).

Let's suppose we're on the second or third path ...

What tools do we have as leaders?

Knowing more about the institution's organizational model and decision-making properties, we can be more clear-eyed and effective in our work and choice of tools as architects and leaders.

(A) Building interpersonal relationships. In his paper, Matt describes building interpersonal relationships to become more effective in an organized anarchy. This is an essential tool, Matt laid it our really well, and I don't have more to add. I see this as a crucial "working within the system" path, as well as layering some informal structure onto the organization.

Matt also cites several other tactics attributed to Cohen and March [3], including: influencing the system by focusing to spend time and persist on fewer issues; creating visible groups and recognizing their efforts to advance an issue; overloading the system with issues; and analyzing the system to find small actions with big effects. [1]

In subsequent blog posts, I'd like to dig into some of these areas to offer additional tools that I've seen leaders use successfully:

(B) Identifying and connecting management approaches. Though the institution may work as an organized anarchy, not all its units do. (Try asking your registrar or campus police chief whether they run an organized anarchy.) Leaders can support change that crosses units by identifying the different management approaches in play and helping them connect up.

• More on Management Approaches Within Organized Anarchy

(C) Managing an issue portfolio. Leaders can identify a "portfolio" of issues they feel responsible for, and shepherd them through several stages to increase their likelihood of being worked on and resolved. This tool complements using relationships.

• More on Leading With an Issue Portfolio

(D) Forming enclaves. Leaders have the ability (and sometimes an obligation) to form "enclaves" within the larger organization. In these enclaves, key participants are shielded from the surrounding anarchy and supported in doing focused work. This is an example of the structural change path.

• More on this topic soon

I hope this will inspire others in our community to share their ideas and tools, because we all have a lot to learn and can use all the help we can get. I look forward to hearing everyone's ideas!

Endnotes

1. Matthew House, A Career in Organized Anarchy: Building Interpersonal Relationships in Higher Education, ACM SIGUCCS Annual Conference (2018).
2. Kenneth P. Ruscio, Leadership in Organized Anarchy, Public Administration Review (2016) (emphasis added).
3. Michael D. Cohen and James G. March, Leadership and Ambiguity: the American College President (1974), as cited in [1] above.

False dilemmas for architects

I was introduced to the idea of the false dilemma (aka false polarity, false dichotomy, or fool's choice) as a leadership concept by Alisa Hata some years ago. In political debate, a false dilemma is a common rhetorical device (e.g., "you're either with us or against us"). But false dilemmas also show up regularly when leaders (including architects), are asked to participate in decisions.

Once someone points this out to you, you may start seeing them everywhere. Here's a little collection of false dilemmas from my work as an architect. How would you respond to these?

Breadth or depth

• Q: "We don't have unlimited time to do analysis. Should we go for the most breadth or go deep in one area?"
• A: "Neither. All problem-solving moves naturally between breadth and depth. We'll need breadth for context, and we'll need to validate it with enough detail, and as we go we'll see we need depth in some but not all areas."

High level or detailed

• Q: "Senior leadership doesn't have time for details. We should keep the presentation very high level and not go into any detail."
• A: "It can be helpful to do both. Make the high level points quickly, but also pick one issue for more detailed explanation to help people understand what is behind each issue."

Top-down or bottom-up

• Q: "Should our management make this decision top-down, or should it be left to each team to choose bottom-up?"
• A: "Neither. We should help management gather input from employees and consider it in setting goals. Within those goals, teams should have latitude to meet local needs in the most sensible way."

Central or decentralized

• Q: "If groups start doing this themselves, it will become a decentralized activity! Wouldn't it be more efficient to do this centrally?"
• A: "Over time and across many activities, some of both. New activities are often best explored by a few groups with the biggest stake. There may turn out to be long-term benefit from centralization, and if there is, some groups will still have specialized enough needs that are most cost-effective to meet locally."

Standard or exceptional

• Q: "We have a standard process for this. If we start making exceptions to it, won't we lose the value of our standard?"
• A: "Probably not. Leaving aside very precise manufacturing processes, most business processes involve variation and judgment based on circumstances. Design and management of the process should account for this, within reason."

Waterfall or Agile

• Q: "For this project, should we iterate in an Agile way, or do traditional sequential planning, requirements gathering, design, and implementation?"
• A: "Some of both. Some up front planning and early discovery work will validate the scope, provide necessary context, identify priorities, and ensure a more successful project. But once enough good information is in place, the team should start designing and implementing iteratively."

Planned or unplanned

• Q: "It would take too long to plan this in enough detail to guide our work, and anyway the plan will change. Can we just identify one or two good next steps?"
• A: "Some of both. In order for the organization to stay on track over time, it should have a long-term vision, choose a high-level strategy, and maintain an evolving roadmap. But it is equally important to start working on short-term wins and learning from them. And certainly not everyone has to be involved in planning to the same degree."

Buy or build

• Q: "We'll have to decide whether to buy or build the solution. That will result in a totally different project and team."
• A: "To some degree. Unless the problem/solution is trivial, both approaches will require very similar planning, analysis, design, testing, integrations, change management, and even operations activities. Some of the detailed work and essential technical skills will certainly be different."

Secure or unsecured

• Q: "Should we design the solution to this security standard, or leave it unsecured?"
• A: "Neither. Just designing to a standard isn't enough to consider something secure. There is always a risk, so security is always a risk calculation. Work with the stakeholders to define the appropriate amount of effort to mitigate different kinds of risks."

Compliant or non-compliant

• Q: "We have to do it this way, even though it's costly, or the enterprise won't be legally compliant."
• A: "Maybe -- let's find out more. Like security, compliance is risk management; we'll never avoid all compliance risk, so we should prioritize. And do the users really need to rely on this particular solution for compliance? Also, the regulator may not have expected this outcome. What was really intended and what other approaches would be acceptable?"

Completed or failed

• Q: "It doesn't look like we can complete this work as scoped -- it turns out it won't be worth the effort. So the project should be considered failed."
• A: "Well, not exactly. Learning is also a return on investment in a project, if we apply what we learned. Based on what we've learned, we can better define successful future work. Continuing to do something with no further value would have been a failure."

In each case the catch is that taking into account more possibilities will probably take more effort and time. It may change the intended outcome of the meeting you're in or even the scope of a project. The stakeholders may not respond well initially. Deciding when to push back on a false dilemma is one of the judgments involved in being an architect or leader.

Building muscles for change: Complexity and collaboration

Suppose you are a person of average health. You try to take the stairs; you're a little out of breath when you get to the top. You go to the gym for a while; the results are discouraging compared to the effort. The muscles in your body are optimized for your current lifestyle, which doesn't require a lot of physical activity.

Now suppose you decide to run the Boston Marathon. Do you have the potential to do it? Almost certainly -- but you know it'll take lots of training. You'll need to build the discipline, the muscles, the stamina, the techniques. And then you can do it.

Like our human bodies, our organizations have the potential to escape sabre-toothed tigers with split-second reflexes, or to bring down a woolly mammoth to feed the tribe. But do we have the current ability? Identifying the boundaries of an organization's current abilities and proposing a path for increased organizational capacity are core activities for business architecture.

Sometimes, though, we're surprised by the gap between our potential and actual ability. Maybe the problem is inherently different from what it seemed to be? Others have proposed that there are significantly different kinds of problems an organization might try to work on: [1]

• Simple problems that are routinely solved using known procedures (e.g., helping a customer at the help desk);
• Complicated problems that can be understood and overcome through analysis, expertise, and best practices (e.g., designing a house for a family);
• Complex problems that cannot be fully analyzed up front and require a highly adaptable, iterative, and experimental approach (e.g., starting a new business in a new market); or,
• Chaotic problems that are so fast-changing and open-ended that they are not subject to any repeatable approach (e.g., bringing about lasting world peace).

Further, others have proposed that some problems are wicked, meaning they are difficult or impossible to work on because of "incomplete, contradictory, and changing requirements that are often difficult to recognize" [Wikipedia]. This term was coined for large-scale social problems (poverty, sustainability, etc.), but some writers suggest (and I agree) that wicked problems can appear in large organizations or even large IT projects, with characteristics such as: [2]

• People aren't yet committed to working on the same thing; they don't agree what the right problem to solve is
• People aren't yet in enough agreement on right or wrong approaches to decide between alternative solutions
• People can't yet agree when enough has been done to address the problem
• The problem and solution are very interrelated with other similarly intractable problems
• Attempts to make progress are being judged as a total success or failure to solve the whole problem, rather than as a necessary experiment

(To avoid diluting the term "wicked problem" and its relevance for public policy, I want to emphasize that within organizations, while it is useful to identify apparently "wicked" internal problems, the purpose of doing so is to focus effort on transitioning the problem into one that is tractable. Unlike societies, organizations have a lot of control over the problems they choose and how they scope them.)

Applying the above concepts to my experience in higher education, I think large initiatives typically require a progression from wicked, to complex, to complicated, to simple, like this:

• Wicked - The problem is poorly defined and there is lack of consensus about how (or even whether) to approach the problem space; so we bring stakeholders together until we get to:
• Complex - The problem is scoped and people are committed to help solve it, but it is unique in our experience; so we try out different approaches until we get to:
• Complicated - We've identified (or created) expertise that makes the problem (or parts of it) directly tractable; so we work on it until we can implement:
• Simple - We have a solution that can be maintained by trained people in a well-documented, repeatable way.

In effect, every major change for an organization starts as a wicked problem -- for that organization, relative to its abilities -- until enough stakeholders can commit to work on something definite together; then it is a complex problem until people working together have tried enough approaches to to make it tractable; then it becomes a complicated problem that sheer resources and expertise can overcome in a predictable timeframe and budget. (If you agree with that, then an important corollary might be that time and budget are not predictable until people are collaborating effectively and the problem is transitioning from complex to complicated.)

Here's where collaboration comes in. In those early stages while the problem is wicked or complex, making progress requires people to build shared understanding and commitment -- often many people. Collaboration doesn't always work out, but it is still the best tool we have as human beings (it's how we reform a health care system or put a person on the moon). The basic reasons for this are two sides of the same coin:

• No single person has the capacity to maintain full understanding of the problem, design a balanced solution, implement it, and drive its adoption. Addressing complexity starts from admitting that no lone genius is going to solve the problem, and no individual heroic leader is going to transport us to the future.
• Whether the problem can be considered solved is inherently a matter of agreement and perception. There isn't an obvious right answer; there just might be an outcome that most people are satisfied with. Solving this kind of problem relies on bringing people along -- some as direct collaborators, and many others as willing participants in the solution.

Simply: without collaboration, an organization is likely to fail at big changes because it has neither involved enough people to arrive at a sound solution, nor convinced enough people for the proposed solution to be adopted.

Crucially, the degree of collaboration required to address the problem at hand may not yet exist in your organization. It is a muscle, and if it hasn't been used recently, it has atrophied. So the proposed problem comes with a meta-problem: the organization isn't ready yet to collaborate on the "real" problem.

This is pretty well understood in professions such as change management or management consulting. These professions bring practices such as organizational readiness assessment, maturity models, and specific competencies to help organizations build the muscles they need to start the marathon they really wanted to run.

As a business architect it is crucial to identify the need for this muscle-building and help management address it (because business architecture is management support). In my experience, initiatives that try to skip their "strength training" (or remain oblivious to it) become very costly. In the worst case I've experienced, it looked like this:

The organization hasn't fully agreed on the problem to be addressed (scope and commitment) but starts the project anyway. Existing capacity for collaboration is insufficient for key partners to contribute effectively, and the project isn't resourced (doesn't have the time and skills) to build up organizational capacity for collaboration. Key stakeholders start to disengage (because their input doesn't appear to matter), and so the project team starts to disengage (because it's too difficult to obtain relevant input). The project team turns inward to focus on its deadlines -- from here on, anyone not inside the team "bunker" is regarded as an obstacle, not a partner.

More complexity continues to surface. As the project continues to add detail without enough context, it diverges from the organization's needs. Senior managers no longer have enough insight or influence to guide the project from the outside. After several years of design and implementation, the project comes to a grinding halt as it becomes obvious to everyone, including the project team, that there is no way to validate the proposed solution, so it can't be adopted. The project has to be largely restarted with a new way of working together, revisiting all the major design decisions, taking it 50% or more over time and budget.

Obviously all your major projects can't be paused just because some business architect says the organization needs to build more muscles! Rather, the practical challenge is to build up enough organizational capacity for collaboration in the early days of a project (consider it like a project deliverable), so the capacity is there when it's needed.

What's involved in building up collaboration? There are many sources more qualified than me on this; I would encourage any business architect to look to other professions on this topic, and defer heavily to management on what collaboration practices have already worked or should be tried. A few key points I feel confident about stating:

• Collaboration means working together toward a shared goal; it is not just communicating better or being present in meetings.
• Collaboration relies on trust, which is built through working together on something, which results in growing to respect each other's strengths and constraints.
• Collaboration relies on motivation, which is heavily influenced by leadership, including role modeling collaborative behavior, rewarding collaboration, making it part of organizational culture long-term, and reinforcing shared goals.
• Collaboration can start small with tactical changes (such as managing each meeting toward a shared result and following up on next steps) -- but managers need to periodically assess whether all the small steps toward collaboration are adding up to the level of collaboration needed for the next phase of work.
• Collaboration can be accelerated by good facilitation, which provides external structure and support (like your personal trainer for the marathon, or the Scrum Master for your Agile team).
• Collaboration does not scale linearly. The effort it takes to get two people to collaborate closely is probably not 1/100th the effort it takes to get 100 people to collaborate at that same level. Because of this, other management tools such as hierarchies and delegation are also important in large efforts.

If you're an IT manager and this all sounds very theoretical to you, think about the last time you formed a new team to do a project or provide a service. How much time and effort did it take for that team to "form, storm, norm, and perform", compared to the actual design and implementation? If you're a non-IT manager, similarly, the last time you saw a major change happen, how much lead-up did it take to get the organization on board? I think you'll find that the cost of the necessary collaboration is quite significant, and it would be worthwhile to plan for it.

Those are my thoughts. In your work, what have you learned about complex and wicked problems and the degree of collaboration needed to address them? For each major initiative you're involved in, is the problem still wicked, or complex or complicated? I would love to hear from you in the comments or in email. Thanks for reading!

Endnotes:

1. See, for example: Snowden & Boone, A Leader's Framework for Decision Making (Harvard Business Review 2007); Nason, It's Not Complicated: The Art and Science of Complexity in Business (2017) as quoted in Kinni, The Critical Difference Between Complex and Complicated (MIT Sloan Management Review 2017).
2. See, for example: Wicked problem (Wikipedia); What's a Wicked Problem (Stony Brook University); Wicked Problems (wickedproblems.com); based on concepts originated in Rittel & Webber, Dilemmas in a General Theory of Planning (Policy Sciences 1973).

What is Architectural Thinking?

In his 2015 book Chess and the Art of Enterprise Architecture, Gerben Wierda makes the case that to be effective, enterprise architecture can't rely on handing off reference architectures and principles to others to read and implement. Rather, EA should promote collaboration and design skills that are widely held, using "our capability of cooperation to create the virtual enterprise architect that can handle ... complexity and unpredictability."

What would it mean for lots of people in an organization to do "architectural thinking"? Many concepts contribute to an architecturally sound design, but what makes thinking architectural? In our team we've been considering what to put in an introductory workshop that encourages architectural thinking by everyone.

As an opener for 2019, here's my personal take on what architectural thinking looks like, in five elements. There's even a mnemonic! (It's my first one, so be kind to it.)

Architectural thinking is:

• Sustainable
• Holistic
• Accountable
• Realistic
• Participatory

You can change the terms to suit your organization of course, but here's how I apply them:

Sustainable

"Sustainable (adj): Able to be maintained at a certain rate or level." [1]

Take responsibility for thinking about the long term. Strive for solutions that are sustainable for your organization, such as:

• A system that is cost effective to implement, operate, expand, and eventually retire
• An organization structure that can scale to accommodate likely growth
• A data model that is designed to be extensible

Many detailed principles and guidelines have been written in pursuit of sustainability, but I suggest that the important thing is for each person participating in a project to take ownership of sustainability on behalf of the organization. Pretend you are the CFO or CIO. How would you think about long term value, cost, and risk? Ask yourself whether what you are proposing will be, for example:

Cost-effective ... Easily adoptable

Reusable ... Extensible ... Adaptable

Secure ... Compliant

Thinking sustainably typically leads to decisions about tradeoffs. Building in flexibility and scalability has a cost; will it be worth it? The leading vendor's product is more expensive, do we really need it? These tradeoffs are difficult; digging into them and discussing them openly leads to better design decisions.

Sustainability may seem like a low bar for architectural thinking -- why not strive for amazing, innovative, groundbreaking solutions? Those can certainly be the starting point for open-ended design thinking and may turn out to be the most sustainable because of the very high new value they bring -- or not. Just leave time to consider their total cost of ownership like any other alternative.

Holistic

"Holistic (adj): Characterized by the belief that the parts of something are intimately interconnected and explicable only by reference to the whole." [1]

It is very common for an idea that would greatly improve one part of a system or process to have negative impacts on other parts. Take responsibility for seeing the big picture and the relationship of all the parts. Strive for solutions that increase connectedness across the organization and internally, such as:

• A system that integrates well in the enterprise and is also modular internally
• A business process design that considers the full value stream across organizational units
• A data model designed for reporting across subject matter domains

Thinking holistically usually adds information, complexity, and therefore effort. It requires balancing breadth and depth; it requires balancing the cost of further analysis with the benefit of knowing more about context that is likely to impact the solution. Ask yourself whether enough has been done to ensure that the proposed path will be, for example:

Aligned ... Integrated 

Well-understood ... Coherent ... Intentional

Complete ... Balanced

Holistic thinking typically challenges both the proposed problem and solution. The initial stated problem or goal often turns out not to be the highest priority or the right scope, once more context is considered. Likewise, the initial stated solution often turns out not to be the best available one, once all its impacts on the context are considered.

Working holistically also has a strong people dimension. To get to the right scope, problem, and solution, groups need shared language, concepts, and analytical tools. Architectural methods of all kinds exist to provide this foundation for people to work together on "systems thinking", dealing with complex problems and solutions.

Crucially, thinking holistically requires being willing to learn fearlessly and as a generalist. In any reasonably complex undertaking, no single specialist can provide the big picture that is needed -- but it can be achieved by a team willing to learn just enough from each other to see the relationships of the most important parts.

Accountable

"Accountable (adj): Required or expected to justify actions or decisions; responsible; able to be explained or understood." [1]

Take responsibility for enabling outcomes. Strive for solutions that are accountable to the goals of the organization, such as:

• A systems roadmap that best supports the long-term strategy of the organization
• A business process that is highly responsive to the needs of the customer
• A plan that is based on the best available data and has agreed-on measures of success

We've all seen some version of the "solution in search of a problem." A major challenge for any team is to agree on and stay focused on a set of goals all the way from ideation through the many twists and turns of design and implementation. At each stage, ask yourself whether what is being proposed is still, for example:

Value-driven ... Customer-centric ... Strategic

Measurable ... Testable ... Auditable ... Traceable

Transparent ... Documented

It can be surprisingly difficult to identify goals that are widely agreed on, clearly relevant, and also specific enough to drive design decisions. Lack of stated goals is never an excuse, however. It just means becoming responsible for helping others define or refine goals. This includes working with stakeholders, escalating issues to sponsors, or even identifying stakeholders and sponsors to establish accountability where it is missing.

Realistic

"Realistic (adj): Having or showing a sensible and practical idea of what can be achieved or expected." [1]

Take responsibility for charting a course that your organization can be successful in. Strive for approaches that are realistic for the organization as it works today or can feasibly work in the future, such as:

• A system that users can adopt and that IT can maintain with available expertise
• An operating model change that the organization can successfully bridge to
• A data management plan that builds in continuous improvement of data quality

Organizations progress over time and so do the solutions and changes they can adopt. At the same time, unpredicted changes occur. Think in terms of progressive maturity and iterative work that responds to changing conditions. Compare what has been done elsewhere, and consider past results in the same organization. Ask yourself whether what is being proposed is, for example:

Iterative ... Repeatable

Feasible today -- or achievable as a stretch
Benchmarked ... Researched

Bring realism to the role of architecture as well. In designing for the long term and the big picture, uncertainty about the future is an inherent part of the job, and there are diminishing returns to usefully reducing uncertainty.

Participatory

"Participatory (adj): Allowing people to take part in or become involved in an activity." [2]

Take responsibility for enabling groups to collaborate, gain new shared understanding, and  ultimately make better decisions. Strive to work in ways that help individuals and teams align and buy in to a shared approach, such as:

• A system developed with the active participation of business stakeholders and end users
• A business process change identified and prioritized by the participants in the process
• A data model conceptually validated by its future users 

Involving more people can seem daunting and undoubtedly adds near-term effort. Experience shows that the right level of participation ultimately saves time by generating better ideas, building a coalition that can overcome roadblocks, identifying conflicts that need to be resolved, and increasing adoption. At each stage, ask yourself whether your team's approach is, for example:

Accessible ... Understandable

Collaborative ... Interactive ... Diverse

Visible ... Communicated

This aspect of architectural thinking also brings us back to our starting point: as you practice architectural thinking, try to enable as many people around you as possible to incorporate architectural perspectives in their thinking as well.

Conclusion

In compiling notes for this post I was tickled that the mnemonic "SHARP" spells the last name of my favorite teacher and architect of recent years, Alec Sharp, author of Workflow Modeling. So this first post of 2019 is dedicated to Alec -- Happy New Year, sir!

Endnotes

1. Oxford English Living Dictionaries
2. Cambridge Dictionary