Waterfall, Agile, and Organizational Focus

A colleague asked me the following question, which I thought was great and could be helpful to others to discuss.

Question: "Many of our IT projects use a project management style that is more Waterfall than not. This creates all the problems associated with a Waterfall process, such as very long delivery times and poor fit for requirements.

Some attempts have been made at introducing an Agile methodology. The most common reason these didn't work out seems to be that cross-functional teams compete with the regular/operational work duties of team members. People claim they aren’t able to attend meetings.

When multiple members are regularly not coming to meetings, the results are predictable: frustration, failure to meet deadlines, poorer deliverables. The end result seems to be a return to a Waterfall approach that silos stakeholders, PMs, and developers.

Is there a best practice way of dealing with what must be a common occurrence?"

Response: In my career the "best practice" response I've seen to this has been something like:

• Commit the organization to growing a Lean/Agile approach over time
• Hire a Lean/Agile coach to provide thorough training and coach some teams in Agile practices
• Use a continuous improvement cycle to help the initial teams improve while spreading the approach to more teams

I work in an IT organization where that approach was started over five years ago. I would say success is very mixed. On the plus side, Agile has stuck around, and the teams using it seem happy with the control and flexibility it gives them. Ongoing challenges in my organization are similar to what my colleague describes in his:

• Difficulty creating and meeting estimates for long-term work
• Difficulty keeping up with business priorities
• Difficulty getting business participation in the Product Owner role

What would be a standard "best practice" followup? Something along the lines of: do more training and coaching. Help teams improve their practices. Start smaller, demonstrate success, then grow the new practices.

Those are reasonable tactics, but I think there might be organizational root causes to consider as well. I think it has to do with the organization's basic ability to truly commit to Lean.

A basic tactic of Lean is to identify and eliminate instances of waste. A basic form of waste is work in progress. One way this this translates into IT is trying to do too many things at once compared to the resources available. (Another way to say this is that Lean is about respecting individuals, which includes enabling them to do their best work.)

Inability to focus and retain resources on top priorities is a problem that spans Waterfall and Agile approaches. Lack or loss of resources is a classic "top ten" reason for project failure in traditional project management. Another example: most organizations can name "bottleneck" resources who are clearly overloaded and (unwillingly) put outcomes at risk. And in Agile, because of the emphasis on continuous teamwork, it becomes especially apparent which individuals have too many competing duties.

From what I've seen and read, success with Lean and Agile requires a leap of faith. The organization's leaders, customers, and teams must truly believe that in the long run, focusing resources is better because:

• Completing a few high priority outcomes sooner (and moving on to the next ones) is more valuable than making slow progress on numerous competing priorities (and falling short in most)
• People are more productive when they do less switching between jobs, so total productivity increases
• Quality increases when people have enough focus to improve how they work

The value proposition is straightforward, but the reality for organizations is quite difficult. That's where it's important to be honest about the organization's ability to commit. Can the organization truly:

• Make and stick to a long-term commitment that affects everyone -- are there relevant examples of doing this?
• Ensure that decision-makers prioritize some goals and defer others -- when their success so far has been based on appearing to satisfy or appease everyone?
• Bring along everyone who will be impacted -- not just implementation teams but also operations and the customer?
• Push through a difficult learning process, a lot of contentious tradeoffs, and an initial apparent reduction in productivity to get to the benefits of a new way of working?

If we're being honest, I suspect the correct answer in many organizations is: no, not yet. In their book The Toyota Way to Service Excellence, Liker and Ross reflect that in most organizations Lean transformation will fizzle out or turn into a set of formulaic, low-value tools. The basic premises of focus and respect for people don't fit how those organizations define success in their culture.

What this suggests to me is looking for opportunities to shift an organization's culture over time and strengthen "muscles" over time.

One opportunity, as old-fashioned as it may seem, is to get better at Waterfall -- because many of the same muscles will help strengthen Agile teams later. For example:

• Improve resource tracking to provide better information about how many and which projects can realistically be done
• Improve project prioritization to ensure the most important projects get the most attention
• Proactively identify and resolve resource contentions
• Improve estimation and tracking of projects to provide a "feedback loop" for resource management and prioritization
• Improve communication with stakeholders to demonstrate where progress is being made and why other goals are being deferred

For IT organizations, the cloud and DevOps offer another kind of opportunity at the moment, because they shift how resources are used and how teams work. If teams for new services can be encapsulated and grown with limited dependency on existing operations, and those services can maintain focus, then gradually the culture of an organization can change as other services fall away. Of course, that's a long game with serious management commitment and risk.

What other opportunities have people seen work to overcome barriers to organizational focus in either Agile or Waterfall? I'd love to see your comments.

Maker - A Missing CliftonStrength?

I've benefited from using Clifton's StrengthsFinder method for myself and teams, and I recommend it to people who are thinking about their jobs and careers. I think the method contributes to opening up important internal and team conversations.

Nevertheless, every time I've taken the assessment and looked at my personal strengths, I've felt something missing. So I'd like to propose an additional CliftonStrengths theme that I think will resonate with others to, from what I've seen at work and in my hobbies.

Maker 

You have a natural drive to make things. What you want to make may vary -- for example, it could be a physical object, a piece of writing or software, a team, an organization, or a process. You find great satisfaction in making, which can include evaluating ideas for new things, planning how to make something, designing it, collecting inputs for it, the process and craft of making it, finishing it, sharing it with others, and observing how it is used. You probably experience making as a "flow state." After making something you feel a sense of achievement from materializing an idea, but also soon want to go on to make the next thing. The thing that was made is only partly an end in itself; the desire to make is a flywheel that keeps you going from one project to the next.

To others, the opportunity that you see to make something may not appear as valuable as it seems to you, because of how you value the act of making and the intrinsic value of the thing to be made. To your team, making something may seem merely necessary or incidental to achieving an outcome. Others might see something as "good enough" and ready to use or set aside while you still see many opportunities to improve it. 

You may find it helpful to partner with others who are focused on engaging people in the work, or as a source of ideas, or to help apply and realize the value of what was made. Using your Maker strength, you can help others execute on their ideas; they can experience a sense of participation in the result. As a leader, you can share the excitement of making something bigger together. If you work with or manage Makers, help ensure they have something to make and guide their energy toward the most valuable outcome for the team.

In thinking about this theme, I wondered why it is not already included in StrengthsFinder. I suspect it is because Donald Clifton was interested in "soft skills" that were -- and often still are -- under-recognized or undervalued in the workplace. From this perspective, making is associated more with the "hard skills" involved in a specific discipline or craft.

However, I'm pretty confident that some people are more driven Makers than others, and that the drive to make has important implications for how someone looks for opportunities, participates in a team, and gains satisfaction from their work. For example, when I reflect on and journal my own work, along with appreciating how I've helped others, I list specific things I've made as personal milestones.

With our contemporary emphasis on teams, soft skills, value, and so forth, sometimes the joy of making something, and the discovery and unexpected value that can result, can take a back seat. I've even felt guilty about proposing to make something because it seems "tactical" and not "strategic" to do so. Of course I would agree that the team and the outcomes are ultimately more important; making good use of the Makers in your team is one way to support that.

Regarding existing themes in StrengthsFinder, I see some overlap with Activator, but with the emphasis on a specific thing to be made over generally "making things happen." Related strengths could be Achiever (motivated by working on something defined), Focus (following through on the vision for something), and Intellection (thinking about something for its own sake).

Is Maker a theme you identify with too? Does this definition seem like it closes a gap in StrengthsFinder for you? I'd be curious to hear. Thanks for reading!

Reflections on Growing a Business Analysis Community of Practice

Since 2014 I've helped grow our organization's Business Analysis Community of Practice. Today this active CoP brings together practitioners and practices from all over our university. The CoP:

• Creates a place for new practitioners to feel welcomed and meet others doing similar work
• Enables practitioners to build each other's knowledge of best practices and business domains throughout the university 
• Helps practitioners converge on shared methods, making it easier for teams to get started and share informations
• Helps practitioners understand business analysis career paths and retain great people in the organization

If you don't already do something like this in your organization, I really encourage you to try it! Here are my reflections and lessons learned.

The early days: peer networking

The CoP was founded around 2012 by Colleen Kelly, a business analyst in central IT at the time. It started as an invitation to meet once a month in the cafeteria for afternoon coffee. Colleen used her network to reach out to peers, word got around, and by the time I joined in 2014 there was a core group of five to ten "regulars." We met to chat, network, and hear about each other's projects.

Just a few simple tools helped the group get started: a mailing list that anyone could join, and an internal web page with basic meeting information. The CoP also benefited from the work of our fantastic Organizational Development team in central IT, which promoted the concept of CoPs and regularly brings together the volunteer CoP leaders to exchange their experiences. (You can find all the CoPs that this great work has resulted in on this site.)

In retrospect I think some key aspects of this phase were:

• Concept of the CoP -- some organizational encouragement for practitioners to get together
• Someone to take the lead -- it only takes one volunteer to get the ball rolling
• Open community -- anyone could come for a friendly chat regardless of their level of experience
• Low threshold of participation -- no expectation for anyone to prepare content

Evolution to topical sessions

As a new business analyst at the university I had a strong interest in hearing about and sharing business analysis best practices. I found other CoP members were interested in this too, though we also found that we all had slightly different roles, approaching business analysis from different directions.

To dig into this we started to alternate a few of the networking sessions with topical meetings. A few members shared projects they were working on, and we had a few sessions on general business analysis concepts. We found that this led to a new level of interest and new participants. Practitioners like to hear about each other's work and learn about different parts of the organization. But it was also very important to keep the informal networking going.

Hiatus, restart, and broader reach & leadership

During these years the CoP also had a hiatus. Colleen and I were co-leading and we both had job changes. There were no CoP meetings for a year or more. I mention this to point out that a  CoP can go through some fits and starts because it may rely on just a few people to keep momentum.

But it can also come back! In late 2017 I had a new role and time to start the CoP again. I joined forces with Taryn Rex, another business analyst, and we reached out to everyone we knew across the university who might be interested, encouraging them to also reach out to others around them.

At this juncture we changed the name of the CoP from "Business Analyst" to "Business Analysis" Community of Practice. We knew that people at the university were doing business analysis in many different roles -- a few with a Business Analyst job title, but many more as project managers, service managers, program analysts, data analysts, financial analysts, and so on.

As new members joined the list we asked them to complete a short survey on their interests and level of experience. This greatly helped us understand the community. We also started to offer online participation (through Zoom) since the university has three campuses. The community grew to over 150 mailing list recipients across the university, with 15-25 members attending each session based on their interests.

The restart focused on introducing business analysis practices in a more structured way. For the initial session I gave a one-hour introduction to business analysis to level-set expectations. Monthly sessions going forward would be focused on a survey of methods, a particular method, or a prepared presentation of project work. (You can view the calendar of topics for 2017-2018.)

When Taryn was unable to continue as co-leader, I decided to form a larger group to co-lead. From the first few sessions there were some clearly very interested participants, and I was delighted to find peers to form a wonderfully engaged steering group (listed on this page in the CoP web site). The steering group meets monthly to guide the direction of the CoP, develop agendas, and identify speakers.

Key reflections here are:

• If you're leading, understand what motivates you -- what's going to sustain you in taking this extra responsibility?
• A community is best led by a community -- find others who are also motivated to keep it going
• Many people in your organization could be doing business analysis, from many different roles
• Use the network of everyone you know -- interested people are everywhere

Transition to shared practices

The interests of new members showed a top recurring theme of learning about the most common business analysis methods in our enterprise. As a steering group we thought the CoP could build on this by sharing our top methods and starting to converge on common practices. We tried the following experiment.

For two two-hour meetings, we ran workshops in which small teams of 3-4 worked on a selected business analysis method. Each team chose from a list of common general methods. Each team drafted a short document summarizing:

• Description: What is this method?
• Step by Step: How to go about it in 5-7 high level steps
• Benefits: What value to expect from the method
• When to Use
• References: Probably the most important part -- links to external resources as well as examples from people's work within our university

We found a volunteer editor for each method and turned the drafts into web pages. The result is the following resource in the CoP's web site:

• Business Analysis Methods

The content is a mix of viewpoints since it was drafted in groups. But I think that is the power of taking this step: it is truly a peer-to-peer resource. Within the community, this work is a source of pride, an excellent basis for future meetings, and a foundation for continuous improvement. And as a practitioner, when I meet with teams I can now say, "these are common practices identified by business analysis practitioners across the university, and we should consider them in this project."

(On a side note, by this point organizing the CoP and developing business analysis in the organization had become  part of my job description as a business architect on the Enterprise Architecture team. While this might not be essential for every CoP, it has been helpful to know that putting 5-10 hours a month into planning and running meetings and doing outreach is supported by my management.)

Some key things I learned from this have been:

• Ask people to contribute -- more than you expect will be glad to
• There is no "right" version of a business analysis method -- the best fit for the organization is probably to start from what people have already found works, and improve on that 

What the future might hold

Where are we headed with our CoP? The steering group has lots of ideas. Some potential areas on our list include:

• Linking up with related CoPs -- for example our organization also has CoPs for project management, testing, and Agile practitioners
• Our new Slack workspace -- how to encourage more ongoing peer-to-peer conversation
• Bringing back more networking time for people to just get to know each other
• Creating ongoing "new to business analysis at the UW" sessions
• A track focused on helping hiring managers define business analysis roles, recruit, interview, and retain
• Sessions more focused on particular professional subdomains within business analysis, such as operational business analysis, process improvement, facilitation, data analysis, financial analysis, etc.

In an enterprise as big and diverse as ours, the potential topics are endless!

What has your experience been with CoPs or similar efforts? Hope to hear from you in the comments.

Leading With an Issue Portfolio

This post is part of a series for a current Itana community discussion of "organized anarchies." I'd like to share a way of thinking about decision-making in large universities that I've gleaned from watching my mentors over the years, and offer a general method.

How do decisions come about?

In a prior post, I quoted Kenneth P. Ruscio describing decision-making in a university as a process in which "solutions chase problems and problems chase solutions. Decisions are opportunities. They come about when various problems attach themselves to various solutions in complicated, unpredictable and sometimes mysterious ways.” [2]

From my experience I think there are patterns in how change happens in a university that has characteristics of an "organized anarchy". Rather than a linear path of decision-making, change is more like a social process of people collecting around ideas. To me it looks like this:

The first thing to recognize is that in many workplaces, what most people think of as a management decision is the choice to take a shared approach to some issue. But not taking a shared approach is also a choice, and it is the one most often selected in an organized anarchy. So what I often see happening in a university is really a progression through different modes [3]:

• Parallel mode: Is it sufficient for people to handle the issue in diverse ways? If so, the issue stays in this mode until some additional urgency arises, or it's no longer relevant.
• Example: To move functionality to the cloud, should teams at our university use Amazon, Google, or Microsoft? What about virtual machines, containers, or functions? Initially, this is up to individual teams to experiment with.
• Associative mode: Is there interest in (and benefit from) exchanging findings and practices around the issue? This often results in converging on fewer (but still several) ways of addressing the issue. Many issues stay here until no longer relevant.
• Example: As teams start using cloud infrastructure in production, they exchange what they've learned. A few options become clearly better for most purposes than others. Maybe some case studies or recommendations get written down.
• Cooperative mode: Is there high urgency for (or value from) a shared approach? If so, people will try to collaborate on one. After some effort, a shared approach may be agreed on -- or not. If not, at least there is broader awareness of the issue and possible approaches.
• Example: A shared service is defined for teams to easily obtain certain kinds of pre-configured cloud infrastructure from certain vendors. The service takes care of shared needs such as identify management integration, default security setup, etc. Teams are not required to use the service, but it becomes the most common approach.

What can leaders do?

For new leaders this kind of decision flow can be very surprising and frustrating. It doesn't map closely to leadership or change management practices that are all about driving in a straight line from a vision to a result. Leaders often end up with burnout that looks like this:

I've seen people I respect greatly and who are great managers leave higher education in frustration with this non-linear decision-making environment. The non-linear pattern resists strong advocacy (especially early on) and does not reward highly visible, structured leadership.

But there is a role for leaders in this non-linear pattern. In each mode, there is important work to help critical issues get attention and advance to the next mode as quickly as possible. Doing this reduces waste, opportunity costs, and risk to the institution. Leaders can:

Note that even if what you achieve as a leader is to more quickly make clear that there will not be a shared approach to an issue, that is a benefit to the institution. It reduces the time others spend on the issue. It completes a decision-making experiment (or "test balloon"), so to speak, so that other experiments can be tried.

Leading with an issue portfolio

For me as an individual leader, there are issues I have a stake in -- whether I'm explicitly accountable for their resolution (rare) or they're related to my general role (more common). New related issues arise all the time, while others become irrelevant (see above).

Think of these issues as an issue portfolio. Some issues have a high risk/reward for the organization, others less. Some issues are getting too little attention, some too much. As a leader my goal can be to shepherd key issues through the organization. Let's say each dot on this quadrant is an issue within my scope:

This approach is probabilistic -- there's no guarantee that the issue I think is most important will reach a shared approach, no matter how hard I work at it. But I can help create the circumstances in which an issue will go from one mode to the next. Tools for this might include personal influence and networking, meetings, and various kinds of groups from less formal (e.g., brown bags) to more formal (e.g., committees).

How is this better for the institution? Even if every individual just clearly prioritizes the top issues they are advocating for, that helps advance the non-linear process. Leaders can also use traditional approaches to build consensus on a vision around an issue, gather allies, and build a coalition of support.

When even a few key leaders in an institution do this, it quickly clarifies which issues are likely to reach a shared approach, and focuses the effort of collaboration and change on those. Likewise, when teams communicate their key issues, useful collaborations can be found more quickly. And so on.

Starting points for leadership in organized anarchy

Based on the above approach, here are some suggested starting points:

1. 
Define the scope of your “portfolio” of issues. What kinds of stuff do you most need to care about? Don’t just react to everything others bring to the table.
2. Know the current issues in your leadership portfolio. What mode is each issue in, who are current and potential players, and what are the experiments and findings so far?
3. Prioritize which issues to expend your limited energy on (based on what you can glean about importance and feasibility).
4. Apply energy to your top issues as appropriate to the mode each issue is in; help each issue move along when it is ready.
5. Prepare to invest more effort into your top issues in later modes (plan and gather support out ahead of each issue).
6. Look for like-minded leaders and take opportunities to build a coalition.
7. Admit when an issue has become intractable despite your best efforts, and shift your energy elsewhere for now.

What are your thoughts? Do you already do something like this for yourself or your team? What's working for you?

Endnotes

1. Matthew House, A Career in Organized Anarchy: Building Interpersonal Relationships in Higher Education, ACM SIGUCCS Annual Conference (2018).
2. Kenneth P. Ruscio, Leadership in Organized Anarchy, Public Administration Review (2016) (emphasis added).
3. I'm borrowing the terms parallel, associate, and cooperative from writing on child development -- see Wikipedia, Parten's stages of play.
4. See Wikipedia, Hype cycle.

Management Approaches Within Organized Anarchy

This post is part of a series for a current Itana community discussion of "organized anarchies" -- the idea that the business architecture of some higher education institutions is characterized by "many autonomous actors operating with bounded rationality in an environment with ambiguous goals, an unclear link, between cause and effect, and fluid participation with the activities and subgroups of the organization." [1]

How are universities organized?

If you've worked in higher education for a while, you've heard an "origin story" along the lines of: Lo these many years ago in the Middle Ages, the first universities were founded to enable individual faculty to independently create and preserve knowledge. Since then, universities have been governed by faculty to protect their academic freedom, and this history drives the management culture of modern universities -- even as they have grown to include hundreds or thousands of non-faculty employees in all the functions needed to run a mid size corporation. [2]

While I agree that the universities I've worked in demonstrate characteristics of an organized anarchy, my own take on this is that it's less because of individual autonomy (as in the classic origin story), and more because of a multiplicity of unit-level management approaches. Many units within a university are far from anarchic (try asking your registrar or campus police chief what kind of team they lead.) But at a large university, the sheer diversity of management approaches in play, and their interactions, results in a complex system that is very challenging to steer. [3]

As a business architect, thinking of it this way makes the university tractable to some analysis and planned change (though often at such great effort that the change still isn't worth it). If I believed every individual in the institution were autonomous, I'd throw up my hands at any change involving more people than I can fit in a conference room. But if we can understand and connect up diverse management approaches, then change leaders have a fair chance at larger initiatives (though still at great effort).

So what's really going on inside a university?

Thinking about the many different university units I've worked with, I can see a diversity of management approaches. You may see others, and have different names for them, but here's a range:

There are good reasons for this diversity. It's natural for each unit to shift toward a management approach suited to its missions and strategies. Managers in a unit may consciously choose an approach to execute on a strategy [4], or the approach may be evolving and not "self aware" yet.

So even when a university is an organized anarchy with a very Autonomous management approach overall, it contains a multitude of "enclaves" with their own management approaches and cultures. In a large university a map of a few units and their management approaches might look like this:

These different management approaches are constantly in friction when work crosses units, as in this very common example:

Admissions applications to the university have increased dramatically, and the Financial Aid office is desparate to improve technology for processing financial aid offers. The unit is Process Driven, with a flat organization and specialized roles responsible for each part of an intense process that has hard deadlines throughout the academic year.

When Financial Aid goes to Central IT for help, it encounters a Podular unit, where IT service teams work semi-autonomously within their own service strategies, with just a few lightweight shared processes (for IT service management) and shared functions (such as a Project Management Office). [5] Few of the managers in either unit are self-aware of their unit's management approach or the differences between units.

Over years, each time Financial Aid and Central IT try to work together on major improvements, they drift apart again in mutual frustration. Financial Aid is staffed to focus on its processes, and can't free up resources to do business analysis for IT changes. When Financial Aid does state requirements, it needs results by specific points in the year when changes can safely occur. The Podular teams in Central IT have little practice with hitting hard deadlines, and they don't collaborate often enough with each other to be able to pull together the full package of IT that Financial Aid actually needs. 

Multiplied over many units, initiatives, and years, the net effect for the university is organized anarchy. Senior leaders struggle to understand why obviously beneficial changes aren't executed. Participants grow frustrated with how hard it is to obtain a decision or follow through, and develop defense mechanisms to be less accountable. Key opportunities are missed and the backlog of unresolved problems multiplies.

How can we do better?

I think change leaders (and the architects supporting them) can improve the success of cross-functional initiatives by recognizing the multiplicity of management approaches and applying that recognition as suggested below.

Identify the management approaches in play. You may be the first person to ask this question for the initiative, and the participants may not be self-aware of their management approaches yet. Ask questions about how people expect to see goals set, accountability established, decisions made, and outcomes assessed. Note differences that could be pitfalls for the initiative.

Plan for the extra effort and skills needed to bridge different management approaches. At what points in the initiative will differences be the greatest obstacle? It might be in discovery, design, implementation, or operations -- or in other factors such as enforcing a time or budget constraint. The approaches can be bridged, but it takes time and it takes a team with the skills, experience, and position to do so. If that will be a problem for the initiative, you've identified a major risk to escalate.

Help units and individuals solidify their own management approach. Crossing management approaches is even more difficult when the people involved aren't following a consistent approach within their own units. If you can't help management of the unit clarify their intent, at least help individuals clarify how they are going to work. How will they represent their unit? Contribute to decisions in the initiative? To what degree can they realistically commit to the initiative? How do they plan to participate in the work of the initiative?

Manage expectations with sponsors and key stakeholders. These often come from a management approach that is different from that of the participants or the initiative, and get frustrated as a result. For example, a CFO from a Hierarchical unit sponsoring a cross-functional initiative that works Collaboratively with participants from a variety of management approaches is going to start with unrealistic expectations.

Be clear about the management approach of the initiative. Each initiative needs a defined approach to manage its own work, and it may well be different from that of the participating units. Here are some examples of efforts "layered" onto the sample university above:

The approach your initiative takes is what is most under your control, so pick a viable approach and communicate it clearly, early and often. If necessary, train people in the approach. Time spent "onboarding" participants to how they are going to work together is never wasted.

With that effort applied, the example started above might continue something like this:

A new change leader in Central IT hears about the history of working with Financial Aid and decides to look more closely. She spends time in Financial Aid to understand its management approach and needs, and also takes a skeptical look at how well Central IT is applying its management approach. She finds allies and willing participants, and convenes a sponsor group that is ready to understand the cross-functional challenge and exercise influence accordingly.

A cross-functional initiative is formed with a clear management approach, and the participants understand how they will need to behave differently from their normal ways of working. Though the problem space is still very challenging, the initiative now has a chance of addressing it.

And yes, that is a real-life example and the initiative is still producing results after several years.

Is it worth it?

Is it worth it to do the above analysis? Only sometimes. The kind of change leadership and business architecture work needed to bridge management approaches is effortful and time-consuming. It might not take place for any of several reasons, including:

• The benefits of the initiative simply aren't worth the effort
• The urgency isn't there yet; the stakeholders are fine with letting the initiative "drift" rather than work on being more aligned
• The key decision-makers involved aren't ready to discuss, understand, or recognize the consequences of the multiplicity of management approaches

You might also be asking yourself: Is it worth it for universities to operate this way? In my opinion that's a personal leap of faith, similar to the "glass half full" metaphor. Organized anarchy is an extremely wasteful way for a university to operate -- or a quite reasonable way to operate -- depending on whether you tend to see:

• Duplication of effort -- vs. -- Adaptability to meet local needs
• Costly unexpected course changes -- vs. -- Agility to respond in the moment
• Ineffective use of resources -- vs. -- Setting aside resources for experimentation
• Lack of transparency -- vs. -- Protection from interference
• Lost opportunities -- vs. -- Freedom to pursue diverse goals

And so on. Personally I think a better version of the question is, is the university getting the most it can out of its management approaches? That applies to the overall model, each different approach selected for a different purpose, and initiatives that cross the university. And that's something we can all work on.

So how do you see it?

Endnotes

1. Matthew House, A Career in Organized Anarchy: Building Interpersonal Relationships in Higher Education, ACM SIGUCCS Annual Conference (2018).
2. There are problems with this version of history. For a debunking, see George Keller, Academic Strategy: The Management Revolution in American Education (1983).
3. That is, parts of the system are easily understood, but their multiplicity and interactions result in unpredictable behaviors at the level of the system. See Wikipedia, Complex system.
4. As an example framework for this, see Jay Galbraith, The Star Model.
5. I'm borrowing the Podular label from Dave Gray, The Connected Company (2014). For a good intro to the topic see this blog post by Dave Gray: The Future is Podular (2015).

The University as Organized Anarchy

If you work in higher education (or another large nonprofit), do you encounter situations in which your institution's decision-making seems unpredictable or intractable? You may be working in an "organized anarchy," an organizational model described by observers of higher education as far back as the 1970s.

For our March 8th and 22nd Itana calls the higher education architecture community is discussing a paper on this topic by Matthew House, Enterprise IT Architect at Washington University in St. Louis, titled A Career in Organized Anarchy: Building Interpersonal Relationships in Higher Education (also available for download here) [1]. Matt's paper and presentation wonderfully connect our community with scholarly thinking on this topic.

For me, Matt's research has triggered a lot of thinking about business architecture and change leadership in higher education, including questions like:

• Are there systemic reasons why change leadership (supported by business architecture) faces special challenges in higher education?
• What are the particular responsibilities of a leader or architect in an organization with very limited intentionality?
• What tools does an architect or leader have for working in this context?

I'd like to offer a few blog posts to inspire further discussion and I hope everyone will join in with their own thoughts -- especially on the Itana mailing list, which you can join here. I strongly recommend you start with Matt's paper, which is very concise and packed with great concepts.

So what is an "organized anarchy"?

In 2016, Kenneth P. Ruscio, then President of Washington and Lee University, described his job like this:

“College and university presidents preside, if that word can be used, over ‘organized anarchies,’ … Goals are shifting. The boundaries of the organization are constantly being redrawn. The referees are sometimes making up the rules as the game progresses … solutions chase problems and problems chase solutions. Decisions are opportunities. They come about when various problems attach themselves to various solutions in complicated, unpredictable and sometimes mysterious ways.” [2]

Now that is a pretty remarkable statement for the leader of a venerable institution. Anarchy? Making up the rules?! Decisions are mysterious?!! Strong words, but the situation may sound familiar to you, and it is recognized by scholarly observers of higher education.

The term "organized anarchy" was coined in 1974 in a book by Michael Cohen and James March [3] to describe institutions of higher education with "many autonomous actors operating with bounded rationality in an environment with ambiguous goals, an unclear link, between cause and effect, and fluid participation with the activities and subgroups of the organization." [1] Cohen and March identified five properties of decision-making in this kind of organization, which Matt summarized for us as:

• Most issues, most of the time, have low salience for most people;
• The total system has high inertia;
• Any decision can become a garbage can for almost any problem;
• The processes of choice are easily subject to overload; and
• The organization has a weak information base. [1]

Put another way, I've sometimes described it to peers (before reading about this research) as the "near-zero accountability" organization. A place where people often just walk away from what they don't feel strongly about, can often set aside larger organizational goals (unless they become absolutely critical), and often aren't challenged to inform the choices they make on behalf of the organization.

If that sounds familiar, read on.

Who cares? And what is our responsibility?

Seeing our context systematically described this way is first of all a kind of mental relief to those of us who have worked in higher education for many years, trying to bring about planned change and finding it surprisingly slow going. Over the last 20 years I've regularly had conversations with newer employees (from other industries) in which I reassure them that no, they're not crazy, the environment really is quite unusual. It's reassuring to also affirm that for myself.

More importantly, many of you reading this, and many colleagues I've worked with over the years, are leaders trying to help higher education institutions overcome challenges and change for the better. We feel responsible for some part of the best contributions universities can make to society.

How is that responsibility changed by knowing you work in an organized anarchy -- a place where decisions are often non-deterministic and the institution as a whole has little specific intentionality about advancing its mission and the public interest? This is a professional existential question for leaders when they "hit the wall" of what can be done in a large university. Confronted with organizational anarchy, which path will you choose:

1. Do you regard the organizational model as unacceptable (whether practically for purposes of doing work, or ethically), and walk away from the particular situation?
2. Will you work within the means the system offers, finding the path of least resistance to do what can be done relatively easily?
3. Can you effect structural change within the system that enables it (and you) to work differently?

I've taken all the above paths in different situations. Regarding the first path, I do think there are situations in which a university's organizational model is simply counter to the public interest in higher education, and professionals have to make hard choices. I'm not going to focus on that today (but would love to hear your thoughts!).

Let's suppose we're on the second or third path ...

What tools do we have as leaders?

Knowing more about the institution's organizational model and decision-making properties, we can be more clear-eyed and effective in our work and choice of tools as architects and leaders.

(A) Building interpersonal relationships. In his paper, Matt describes building interpersonal relationships to become more effective in an organized anarchy. This is an essential tool, Matt laid it our really well, and I don't have more to add. I see this as a crucial "working within the system" path, as well as layering some informal structure onto the organization.

Matt also cites several other tactics attributed to Cohen and March [3], including: influencing the system by focusing to spend time and persist on fewer issues; creating visible groups and recognizing their efforts to advance an issue; overloading the system with issues; and analyzing the system to find small actions with big effects. [1]

In subsequent blog posts, I'd like to dig into some of these areas to offer additional tools that I've seen leaders use successfully:

(B) Identifying and connecting management approaches. Though the institution may work as an organized anarchy, not all its units do. (Try asking your registrar or campus police chief whether they run an organized anarchy.) Leaders can support change that crosses units by identifying the different management approaches in play and helping them connect up.

• More on Management Approaches Within Organized Anarchy

(C) Managing an issue portfolio. Leaders can identify a "portfolio" of issues they feel responsible for, and shepherd them through several stages to increase their likelihood of being worked on and resolved. This tool complements using relationships.

• More on Leading With an Issue Portfolio

(D) Forming enclaves. Leaders have the ability (and sometimes an obligation) to form "enclaves" within the larger organization. In these enclaves, key participants are shielded from the surrounding anarchy and supported in doing focused work. This is an example of the structural change path.

• More on this topic soon

I hope this will inspire others in our community to share their ideas and tools, because we all have a lot to learn and can use all the help we can get. I look forward to hearing everyone's ideas!

Endnotes

1. Matthew House, A Career in Organized Anarchy: Building Interpersonal Relationships in Higher Education, ACM SIGUCCS Annual Conference (2018).
2. Kenneth P. Ruscio, Leadership in Organized Anarchy, Public Administration Review (2016) (emphasis added).
3. Michael D. Cohen and James G. March, Leadership and Ambiguity: the American College President (1974), as cited in [1] above.

False dilemmas for architects

I was introduced to the idea of the false dilemma (aka false polarity, false dichotomy, or fool's choice) as a leadership concept by Alisa Hata some years ago. In political debate, a false dilemma is a common rhetorical device (e.g., "you're either with us or against us"). But false dilemmas also show up regularly when leaders (including architects), are asked to participate in decisions.

Once someone points this out to you, you may start seeing them everywhere. Here's a little collection of false dilemmas from my work as an architect. How would you respond to these?

Breadth or depth

• Q: "We don't have unlimited time to do analysis. Should we go for the most breadth or go deep in one area?"
• A: "Neither. All problem-solving moves naturally between breadth and depth. We'll need breadth for context, and we'll need to validate it with enough detail, and as we go we'll see we need depth in some but not all areas."

High level or detailed

• Q: "Senior leadership doesn't have time for details. We should keep the presentation very high level and not go into any detail."
• A: "It can be helpful to do both. Make the high level points quickly, but also pick one issue for more detailed explanation to help people understand what is behind each issue."

Top-down or bottom-up

• Q: "Should our management make this decision top-down, or should it be left to each team to choose bottom-up?"
• A: "Neither. We should help management gather input from employees and consider it in setting goals. Within those goals, teams should have latitude to meet local needs in the most sensible way."

Central or decentralized

• Q: "If groups start doing this themselves, it will become a decentralized activity! Wouldn't it be more efficient to do this centrally?"
• A: "Over time and across many activities, some of both. New activities are often best explored by a few groups with the biggest stake. There may turn out to be long-term benefit from centralization, and if there is, some groups will still have specialized enough needs that are most cost-effective to meet locally."

Standard or exceptional

• Q: "We have a standard process for this. If we start making exceptions to it, won't we lose the value of our standard?"
• A: "Probably not. Leaving aside very precise manufacturing processes, most business processes involve variation and judgment based on circumstances. Design and management of the process should account for this, within reason."

Waterfall or Agile

• Q: "For this project, should we iterate in an Agile way, or do traditional sequential planning, requirements gathering, design, and implementation?"
• A: "Some of both. Some up front planning and early discovery work will validate the scope, provide necessary context, identify priorities, and ensure a more successful project. But once enough good information is in place, the team should start designing and implementing iteratively."

Planned or unplanned

• Q: "It would take too long to plan this in enough detail to guide our work, and anyway the plan will change. Can we just identify one or two good next steps?"
• A: "Some of both. In order for the organization to stay on track over time, it should have a long-term vision, choose a high-level strategy, and maintain an evolving roadmap. But it is equally important to start working on short-term wins and learning from them. And certainly not everyone has to be involved in planning to the same degree."

Buy or build

• Q: "We'll have to decide whether to buy or build the solution. That will result in a totally different project and team."
• A: "To some degree. Unless the problem/solution is trivial, both approaches will require very similar planning, analysis, design, testing, integrations, change management, and even operations activities. Some of the detailed work and essential technical skills will certainly be different."

Secure or unsecured

• Q: "Should we design the solution to this security standard, or leave it unsecured?"
• A: "Neither. Just designing to a standard isn't enough to consider something secure. There is always a risk, so security is always a risk calculation. Work with the stakeholders to define the appropriate amount of effort to mitigate different kinds of risks."

Compliant or non-compliant

• Q: "We have to do it this way, even though it's costly, or the enterprise won't be legally compliant."
• A: "Maybe -- let's find out more. Like security, compliance is risk management; we'll never avoid all compliance risk, so we should prioritize. And do the users really need to rely on this particular solution for compliance? Also, the regulator may not have expected this outcome. What was really intended and what other approaches would be acceptable?"

Completed or failed

• Q: "It doesn't look like we can complete this work as scoped -- it turns out it won't be worth the effort. So the project should be considered failed."
• A: "Well, not exactly. Learning is also a return on investment in a project, if we apply what we learned. Based on what we've learned, we can better define successful future work. Continuing to do something with no further value would have been a failure."

In each case the catch is that taking into account more possibilities will probably take more effort and time. It may change the intended outcome of the meeting you're in or even the scope of a project. The stakeholders may not respond well initially. Deciding when to push back on a false dilemma is one of the judgments involved in being an architect or leader.

You can run but you can't hide -- Why we still do as-is analysis

"Do we really need to do as-is analysis?" I seem to hear this more and more often from project sponsors, project managers, or even management consultants. Just recently, for example, I was told by an associate from a Big Four consulting firm that there was no need to document current state as part of a business and IT transformation program on the order of $200 million, with hundreds of participants across the institution.

It always catches me a bit by surprise that this is even a question. From my professional experience, I start from the assumption that as-is analysis is essential, and the more so the bigger the effort. Clearly, however, I need to be able to justify this position better -- here's an attempt.

I'm confident that in any project, solid understanding of the current state will eventually become important. It is a question of when and how, not whether to do as-is analysis. I've seen projects defer this work, but sooner or later it always comes back.

At the latest, as-is analysis will become essential at one of the following points in a typical project. It might be useful to ask, do you think you will need to:

• Propose a business case, problem statement, or value proposition that is compelling because it is well grounded in current issues?
• Choose between alternative solutions by comparing impacts relative to current state?
• Prioritize increments of work (e.g., features in an MVP) based on what work will most address pain points (i.e., provide value) in current state?
• When part of an implementation turns out to be unexpectedly challenging, design a workaround based on current process?
• Validate that the design is still relevant to the organization (including changes that have taken place) and consistent with the business case and priorities?
• Plan change management based on how different users are impacted by changes from current state?
• Prepare to train or support users based on knowing how they will need to work differently compared to current state?

If the team answers yes to any of these -- and I can't imagine a team saying no to all -- then some as-is analysis is going to be needed.

By skipping the opportunity to engage groups of stakeholders in as-is analysis, a project may also miss out on:

• Helping stakeholders get to know and respect each other's concerns so they can collaborate better later
• Helping the project team build shared language and understanding of the business that helps them work together
• Digging into root causes that could significantly change the initial stated project scope

So where does the drive to avoid as-is analysis come from? I think it is based on misunderstandings of current methodologies that may appear not to require as-is analysis. Teams applying these methodologies may even feel pressured by their management to demonstrate that they can work faster by skipping "traditional" analysis work that doesn't appear, in itself, to create value.

Agile: Poorly understood or implemented Agile practices create a lot of misunderstandings. A common one is that business analysis, including as-is analysis, is no longer needed. In mature Agile teams, this work does in fact occur in several places: the Product Owner role brings deep as-is understanding to their prioritization of work for the team; the team does "just in time" as-is analysis to define and execute on user stories; and/or the project team does story mapping (or other up-front planning) that is rooted in understanding current state compared with the proposed functionality. The as-is analysis may be distributed differently, but as always, the more clearly the whole team understands current state, the faster it can work -- and without that, an Agile team is no more likely to create value than any other team.

Lean: Here again the highly iterative approach and the emphasis on always striving toward a future vision can make it look like as-is analysis is not happening. But in practice, every Lean effort I've participated in has started with convening stakeholders to map out their as-is process or value stream and identify pain points or opportunities to eliminate waste. That shared understanding of the as-is helps the team prioritize, work together, and communicate with sponsors going forward. The core Lean principles of Go See, Ask Why, Show Respect reflect this.

Design Thinking: Shifts in terminology can also obscure the real work. Design Thinking emphasizes terms like Empathize, Ideate, and Prototype over fuddy-duddy terms like "analyze" or "document". Now, I accept the spread of Design Thinking as a valid critique of and response to past experience: wherever teams in the past failed to think about and engage their customers, work collaboratively, or tap into people's creativity, that was certainly a mistake. Nevertheless, teams using Design Thinking are definitely doing as-is analysis when they interview people, define personas, identify challenges, and refer to the current state to assess feasibility of ideas.

SaaS: Software as a service products are often marketed with the promise that they deliver "out of the box" business processes that represent best practices and enable an organization to simply adopt higher maturity, more automated processes. This is often paired with a prototyping approach in which stakeholders are asked to give input on working software as a way of skipping formal requirements analysis and documentation. In my experience, this just means the as-is analysis is being shifted around or deferred.

In responding to prototypes, stakeholders are effectively being asked to bring their knowledge of the as-is and do an impact assessment or fit-gap on the fly. If there is not already strong shared understanding of the as-is, the prototyping sessions will raise lots of questions that the project team will then need to follow up on -- again doing as-is analysis, just later. And while prototyping sessions can guide the configuration, the project team still needs to do work such as change management and training that requires a shared baseline understanding of the as-is in order to be able to communicate well with a broad user base.

So the next time a team says they're not planning to do as-is analysis, please dig into what is really meant. Most likely the team has simply adopted another way of naming and doing this work, and it's good to just clarify how the shared understanding of current state will be created and carried forward in that way of working. The team may also be making a sound decision to defer deeper as-is analysis until it is really needed. But in the worst case, the team may think it can skip analysis in a way that will result in taking the project off track, doing expensive rework, or missing opportunities to collaborate.

Building muscles for change: Complexity and collaboration

Suppose you are a person of average health. You try to take the stairs; you're a little out of breath when you get to the top. You go to the gym for a while; the results are discouraging compared to the effort. The muscles in your body are optimized for your current lifestyle, which doesn't require a lot of physical activity.

Now suppose you decide to run the Boston Marathon. Do you have the potential to do it? Almost certainly -- but you know it'll take lots of training. You'll need to build the discipline, the muscles, the stamina, the techniques. And then you can do it.

Like our human bodies, our organizations have the potential to escape sabre-toothed tigers with split-second reflexes, or to bring down a woolly mammoth to feed the tribe. But do we have the current ability? Identifying the boundaries of an organization's current abilities and proposing a path for increased organizational capacity are core activities for business architecture.

Sometimes, though, we're surprised by the gap between our potential and actual ability. Maybe the problem is inherently different from what it seemed to be? Others have proposed that there are significantly different kinds of problems an organization might try to work on: [1]

• Simple problems that are routinely solved using known procedures (e.g., helping a customer at the help desk);
• Complicated problems that can be understood and overcome through analysis, expertise, and best practices (e.g., designing a house for a family);
• Complex problems that cannot be fully analyzed up front and require a highly adaptable, iterative, and experimental approach (e.g., starting a new business in a new market); or,
• Chaotic problems that are so fast-changing and open-ended that they are not subject to any repeatable approach (e.g., bringing about lasting world peace).

Further, others have proposed that some problems are wicked, meaning they are difficult or impossible to work on because of "incomplete, contradictory, and changing requirements that are often difficult to recognize" [Wikipedia]. This term was coined for large-scale social problems (poverty, sustainability, etc.), but some writers suggest (and I agree) that wicked problems can appear in large organizations or even large IT projects, with characteristics such as: [2]

• People aren't yet committed to working on the same thing; they don't agree what the right problem to solve is
• People aren't yet in enough agreement on right or wrong approaches to decide between alternative solutions
• People can't yet agree when enough has been done to address the problem
• The problem and solution are very interrelated with other similarly intractable problems
• Attempts to make progress are being judged as a total success or failure to solve the whole problem, rather than as a necessary experiment

(To avoid diluting the term "wicked problem" and its relevance for public policy, I want to emphasize that within organizations, while it is useful to identify apparently "wicked" internal problems, the purpose of doing so is to focus effort on transitioning the problem into one that is tractable. Unlike societies, organizations have a lot of control over the problems they choose and how they scope them.)

Applying the above concepts to my experience in higher education, I think large initiatives typically require a progression from wicked, to complex, to complicated, to simple, like this:

• Wicked - The problem is poorly defined and there is lack of consensus about how (or even whether) to approach the problem space; so we bring stakeholders together until we get to:
• Complex - The problem is scoped and people are committed to help solve it, but it is unique in our experience; so we try out different approaches until we get to:
• Complicated - We've identified (or created) expertise that makes the problem (or parts of it) directly tractable; so we work on it until we can implement:
• Simple - We have a solution that can be maintained by trained people in a well-documented, repeatable way.

In effect, every major change for an organization starts as a wicked problem -- for that organization, relative to its abilities -- until enough stakeholders can commit to work on something definite together; then it is a complex problem until people working together have tried enough approaches to to make it tractable; then it becomes a complicated problem that sheer resources and expertise can overcome in a predictable timeframe and budget. (If you agree with that, then an important corollary might be that time and budget are not predictable until people are collaborating effectively and the problem is transitioning from complex to complicated.)

Here's where collaboration comes in. In those early stages while the problem is wicked or complex, making progress requires people to build shared understanding and commitment -- often many people. Collaboration doesn't always work out, but it is still the best tool we have as human beings (it's how we reform a health care system or put a person on the moon). The basic reasons for this are two sides of the same coin:

• No single person has the capacity to maintain full understanding of the problem, design a balanced solution, implement it, and drive its adoption. Addressing complexity starts from admitting that no lone genius is going to solve the problem, and no individual heroic leader is going to transport us to the future.
• Whether the problem can be considered solved is inherently a matter of agreement and perception. There isn't an obvious right answer; there just might be an outcome that most people are satisfied with. Solving this kind of problem relies on bringing people along -- some as direct collaborators, and many others as willing participants in the solution.

Simply: without collaboration, an organization is likely to fail at big changes because it has neither involved enough people to arrive at a sound solution, nor convinced enough people for the proposed solution to be adopted.

Crucially, the degree of collaboration required to address the problem at hand may not yet exist in your organization. It is a muscle, and if it hasn't been used recently, it has atrophied. So the proposed problem comes with a meta-problem: the organization isn't ready yet to collaborate on the "real" problem.

This is pretty well understood in professions such as change management or management consulting. These professions bring practices such as organizational readiness assessment, maturity models, and specific competencies to help organizations build the muscles they need to start the marathon they really wanted to run.

As a business architect it is crucial to identify the need for this muscle-building and help management address it (because business architecture is management support). In my experience, initiatives that try to skip their "strength training" (or remain oblivious to it) become very costly. In the worst case I've experienced, it looked like this:

The organization hasn't fully agreed on the problem to be addressed (scope and commitment) but starts the project anyway. Existing capacity for collaboration is insufficient for key partners to contribute effectively, and the project isn't resourced (doesn't have the time and skills) to build up organizational capacity for collaboration. Key stakeholders start to disengage (because their input doesn't appear to matter), and so the project team starts to disengage (because it's too difficult to obtain relevant input). The project team turns inward to focus on its deadlines -- from here on, anyone not inside the team "bunker" is regarded as an obstacle, not a partner.

More complexity continues to surface. As the project continues to add detail without enough context, it diverges from the organization's needs. Senior managers no longer have enough insight or influence to guide the project from the outside. After several years of design and implementation, the project comes to a grinding halt as it becomes obvious to everyone, including the project team, that there is no way to validate the proposed solution, so it can't be adopted. The project has to be largely restarted with a new way of working together, revisiting all the major design decisions, taking it 50% or more over time and budget.

Obviously all your major projects can't be paused just because some business architect says the organization needs to build more muscles! Rather, the practical challenge is to build up enough organizational capacity for collaboration in the early days of a project (consider it like a project deliverable), so the capacity is there when it's needed.

What's involved in building up collaboration? There are many sources more qualified than me on this; I would encourage any business architect to look to other professions on this topic, and defer heavily to management on what collaboration practices have already worked or should be tried. A few key points I feel confident about stating:

• Collaboration means working together toward a shared goal; it is not just communicating better or being present in meetings.
• Collaboration relies on trust, which is built through working together on something, which results in growing to respect each other's strengths and constraints.
• Collaboration relies on motivation, which is heavily influenced by leadership, including role modeling collaborative behavior, rewarding collaboration, making it part of organizational culture long-term, and reinforcing shared goals.
• Collaboration can start small with tactical changes (such as managing each meeting toward a shared result and following up on next steps) -- but managers need to periodically assess whether all the small steps toward collaboration are adding up to the level of collaboration needed for the next phase of work.
• Collaboration can be accelerated by good facilitation, which provides external structure and support (like your personal trainer for the marathon, or the Scrum Master for your Agile team).
• Collaboration does not scale linearly. The effort it takes to get two people to collaborate closely is probably not 1/100th the effort it takes to get 100 people to collaborate at that same level. Because of this, other management tools such as hierarchies and delegation are also important in large efforts.

If you're an IT manager and this all sounds very theoretical to you, think about the last time you formed a new team to do a project or provide a service. How much time and effort did it take for that team to "form, storm, norm, and perform", compared to the actual design and implementation? If you're a non-IT manager, similarly, the last time you saw a major change happen, how much lead-up did it take to get the organization on board? I think you'll find that the cost of the necessary collaboration is quite significant, and it would be worthwhile to plan for it.

Those are my thoughts. In your work, what have you learned about complex and wicked problems and the degree of collaboration needed to address them? For each major initiative you're involved in, is the problem still wicked, or complex or complicated? I would love to hear from you in the comments or in email. Thanks for reading!

Endnotes:

1. See, for example: Snowden & Boone, A Leader's Framework for Decision Making (Harvard Business Review 2007); Nason, It's Not Complicated: The Art and Science of Complexity in Business (2017) as quoted in Kinni, The Critical Difference Between Complex and Complicated (MIT Sloan Management Review 2017).
2. See, for example: Wicked problem (Wikipedia); What's a Wicked Problem (Stony Brook University); Wicked Problems (wickedproblems.com); based on concepts originated in Rittel & Webber, Dilemmas in a General Theory of Planning (Policy Sciences 1973).

What is Architectural Thinking?

In his 2015 book Chess and the Art of Enterprise Architecture, Gerben Wierda makes the case that to be effective, enterprise architecture can't rely on handing off reference architectures and principles to others to read and implement. Rather, EA should promote collaboration and design skills that are widely held, using "our capability of cooperation to create the virtual enterprise architect that can handle ... complexity and unpredictability."

What would it mean for lots of people in an organization to do "architectural thinking"? Many concepts contribute to an architecturally sound design, but what makes thinking architectural? In our team we've been considering what to put in an introductory workshop that encourages architectural thinking by everyone.

As an opener for 2019, here's my personal take on what architectural thinking looks like, in five elements. There's even a mnemonic! (It's my first one, so be kind to it.)

Architectural thinking is:

• Sustainable
• Holistic
• Accountable
• Realistic
• Participatory

You can change the terms to suit your organization of course, but here's how I apply them:

Sustainable

"Sustainable (adj): Able to be maintained at a certain rate or level." [1]

Take responsibility for thinking about the long term. Strive for solutions that are sustainable for your organization, such as:

• A system that is cost effective to implement, operate, expand, and eventually retire
• An organization structure that can scale to accommodate likely growth
• A data model that is designed to be extensible

Many detailed principles and guidelines have been written in pursuit of sustainability, but I suggest that the important thing is for each person participating in a project to take ownership of sustainability on behalf of the organization. Pretend you are the CFO or CIO. How would you think about long term value, cost, and risk? Ask yourself whether what you are proposing will be, for example:

Cost-effective ... Easily adoptable

Reusable ... Extensible ... Adaptable

Secure ... Compliant

Thinking sustainably typically leads to decisions about tradeoffs. Building in flexibility and scalability has a cost; will it be worth it? The leading vendor's product is more expensive, do we really need it? These tradeoffs are difficult; digging into them and discussing them openly leads to better design decisions.

Sustainability may seem like a low bar for architectural thinking -- why not strive for amazing, innovative, groundbreaking solutions? Those can certainly be the starting point for open-ended design thinking and may turn out to be the most sustainable because of the very high new value they bring -- or not. Just leave time to consider their total cost of ownership like any other alternative.

Holistic

"Holistic (adj): Characterized by the belief that the parts of something are intimately interconnected and explicable only by reference to the whole." [1]

It is very common for an idea that would greatly improve one part of a system or process to have negative impacts on other parts. Take responsibility for seeing the big picture and the relationship of all the parts. Strive for solutions that increase connectedness across the organization and internally, such as:

• A system that integrates well in the enterprise and is also modular internally
• A business process design that considers the full value stream across organizational units
• A data model designed for reporting across subject matter domains

Thinking holistically usually adds information, complexity, and therefore effort. It requires balancing breadth and depth; it requires balancing the cost of further analysis with the benefit of knowing more about context that is likely to impact the solution. Ask yourself whether enough has been done to ensure that the proposed path will be, for example:

Aligned ... Integrated 

Well-understood ... Coherent ... Intentional

Complete ... Balanced

Holistic thinking typically challenges both the proposed problem and solution. The initial stated problem or goal often turns out not to be the highest priority or the right scope, once more context is considered. Likewise, the initial stated solution often turns out not to be the best available one, once all its impacts on the context are considered.

Working holistically also has a strong people dimension. To get to the right scope, problem, and solution, groups need shared language, concepts, and analytical tools. Architectural methods of all kinds exist to provide this foundation for people to work together on "systems thinking", dealing with complex problems and solutions.

Crucially, thinking holistically requires being willing to learn fearlessly and as a generalist. In any reasonably complex undertaking, no single specialist can provide the big picture that is needed -- but it can be achieved by a team willing to learn just enough from each other to see the relationships of the most important parts.

Accountable

"Accountable (adj): Required or expected to justify actions or decisions; responsible; able to be explained or understood." [1]

Take responsibility for enabling outcomes. Strive for solutions that are accountable to the goals of the organization, such as:

• A systems roadmap that best supports the long-term strategy of the organization
• A business process that is highly responsive to the needs of the customer
• A plan that is based on the best available data and has agreed-on measures of success

We've all seen some version of the "solution in search of a problem." A major challenge for any team is to agree on and stay focused on a set of goals all the way from ideation through the many twists and turns of design and implementation. At each stage, ask yourself whether what is being proposed is still, for example:

Value-driven ... Customer-centric ... Strategic

Measurable ... Testable ... Auditable ... Traceable

Transparent ... Documented

It can be surprisingly difficult to identify goals that are widely agreed on, clearly relevant, and also specific enough to drive design decisions. Lack of stated goals is never an excuse, however. It just means becoming responsible for helping others define or refine goals. This includes working with stakeholders, escalating issues to sponsors, or even identifying stakeholders and sponsors to establish accountability where it is missing.

Realistic

"Realistic (adj): Having or showing a sensible and practical idea of what can be achieved or expected." [1]

Take responsibility for charting a course that your organization can be successful in. Strive for approaches that are realistic for the organization as it works today or can feasibly work in the future, such as:

• A system that users can adopt and that IT can maintain with available expertise
• An operating model change that the organization can successfully bridge to
• A data management plan that builds in continuous improvement of data quality

Organizations progress over time and so do the solutions and changes they can adopt. At the same time, unpredicted changes occur. Think in terms of progressive maturity and iterative work that responds to changing conditions. Compare what has been done elsewhere, and consider past results in the same organization. Ask yourself whether what is being proposed is, for example:

Iterative ... Repeatable

Feasible today -- or achievable as a stretch
Benchmarked ... Researched

Bring realism to the role of architecture as well. In designing for the long term and the big picture, uncertainty about the future is an inherent part of the job, and there are diminishing returns to usefully reducing uncertainty.

Participatory

"Participatory (adj): Allowing people to take part in or become involved in an activity." [2]

Take responsibility for enabling groups to collaborate, gain new shared understanding, and  ultimately make better decisions. Strive to work in ways that help individuals and teams align and buy in to a shared approach, such as:

• A system developed with the active participation of business stakeholders and end users
• A business process change identified and prioritized by the participants in the process
• A data model conceptually validated by its future users 

Involving more people can seem daunting and undoubtedly adds near-term effort. Experience shows that the right level of participation ultimately saves time by generating better ideas, building a coalition that can overcome roadblocks, identifying conflicts that need to be resolved, and increasing adoption. At each stage, ask yourself whether your team's approach is, for example:

Accessible ... Understandable

Collaborative ... Interactive ... Diverse

Visible ... Communicated

This aspect of architectural thinking also brings us back to our starting point: as you practice architectural thinking, try to enable as many people around you as possible to incorporate architectural perspectives in their thinking as well.

Conclusion

In compiling notes for this post I was tickled that the mnemonic "SHARP" spells the last name of my favorite teacher and architect of recent years, Alec Sharp, author of Workflow Modeling. So this first post of 2019 is dedicated to Alec -- Happy New Year, sir!

Endnotes

1. Oxford English Living Dictionaries
2. Cambridge Dictionary

Distinguishing Business Analysis, Business Relationship Management, and Business Architecture

It looks like I'll be able to attend the Common Solutions Group meeting in January, and one of the topics there will be "BRM, BizArch, Biz Analysis - How are you connecting, documenting, and planning with campus partners."

To help get this conversation started, I'd like to offer some background for comparing the fields of Business Analysis (BA), Business Relationship Management (BRM), and Business Architecture (BizArch). I don't propose to resolve the active definitional debates between these fields, and I'm not here to offer "right" answers. I do want to help you compare these fields for yourself, so you can facilitate conversations in your organization that help you set up practitioners and teams for success.

As always this post is framed for higher education, and especially for IT organizations in higher ed.

Why is this even a topic?

IT organizations in higher ed (as in other industries) are challenged to align IT investments with business needs. The challenge is growing as the rate of change in the business of higher education increases, which also increases the opportunity for strategic IT. IT organizations view Business Analysis, Business Relationship Management, and Business Architecture as potential means to meet these challenges and opportunities.

Are we talking about the same thing?

The first common confusion I see is that the terms we're using can be used variously to refer to:

• A discipline: "We need to train more IT staff in Business Analysis skills so they can better understand and meet business needs."

• A role: "Our Business Relationship Manager met with the Dean's leadership team."

• A process: "This idea for an IT service came in through our Business Relationship Management process."

• A deliverable: "Sue will present the future state Business Architecture to the Executive Committee."

These are all distinct aspects for an organization to discuss and work on. To take Business Analysis as an example (you could apply the same thinking to the others):

• You may want a wide range of staff to acquire competencies from this discipline.

• You may want to hire or train staff as Business Analysts to build experience and specialize in this work.

• You may want to promote a shared process that helps teams consistently do a minimum of analysis that makes projects more likely to succeed.

• You may want to define standard deliverables to reduce Business Analysis planning in each project and more easily re-use past analysis.

Each aspect represents different goals for management. I'll suggest work to do on each aspect in the sections below.

The next level of confusion I see is that each field can be variously defined, and not all the fields are equally firmly defined. Let's take them one by one.

Business Analysis

The International Institute of Business Analysis (IIBA) defines Business Analysis as "The practice of enabling change in an organizational context, by defining needs and recommending solutions that deliver value to stakeholders." Wikipedia reinforces that this covers a wide range: "A research discipline of identifying business needs and determining solutions to business problems … includ[ing] … software-systems … process improvement, organizational change or strategic planning and policy development."

I consider this the longest-standing and most defined of the fields discussed here. A well-established professional organization defines the field and supports it with resources, certification [1], conferences, and local chapters around the world. The field is firmly rooted in the culture of IT; few IT managers would say they want to run a service or project without doing some analysis.

Nevertheless, some confusion can still arise:

• Because the field is well-established, it encompasses a wide range of jobs and career levels. Practitioners at different levels do significantly different work, sometimes difficult to discuss under one heading.

• In higher ed, the term Business Analysis is most used in IT and skews toward analysis for IT solutions; business units in the institution may feel that this version of Business Analysis doesn't describe the work they need done.

• Business Analysis work is done as part of many roles. IIBA acknowledges a wide variety of job titles that do some aspect of Business Analysis. Especially common in small teams, for example, are PM/BA and Programmer/Analyst type roles.

• Teams adopting contemporary frameworks such as Agile tend to think of Business Analysis as a shared responsibility and iterative work, rather than specialist roles and standard deliverables. Different views on the timing and content of Business Analysis may be unresolved in your organization.

To bring clarity in your organization I suggest:

• Review Business Analysis competencies, for example the competency model offered by IIBA for certification purposes, and identify those you care about.

• Propose which competencies you think should be held by which current and future roles in your organization (ideally as part of a larger workforce development strategy).

• For professional Business Analyst roles, define current and future career levels and paths -- it's not just one job, and position descriptions and compensation need to reflect that in order to retain and get the most from these sought-after professionals.

• For service and project teams, especially for Agile teams, build agreement in the organization on what Business Analysis work most contributes to success and reduces risk.

Business Relationship Management

ITIL Service Strategy defines Business Relationship Management as "The process that enables BRMs to provide links between the service provider and customers at the strategic and tactical levels." The Business Relationship Management Institute (BRMI) states that BRM "embodies a set of competencies ... to foster an effective, business value-producing relationship between business functions and their business partners."

I think of this field as more recent, but fairly defined. The well-respected ITIL ITSM [2] framework identifies BRM as a process area, and the BRMI defines the field for practitioners and offers certification.

But there is room for confusion:

• As a distinct, defined discipline, role, or process, BRM is probably new-ish to your organization and only as well understood as ITSM itself. Organizations adopting ITIL ITSM tend to start from operational processes and wait to formally define strategy processes later (including BRM).

• The work that BRM represents was historically done by IT managers (and others), with varying success. People in your organization may wonder why this topic needs distinct attention, and the discussion may raise insecurity or territorial conflicts.

• According to authors for BRMI, in many organizations BRM initially grows out of Customer Relationship Management or Demand Management. ITIL also acknowledges overlap with these areas. This can result in an organizational understanding of BRM that is less strategic than BRM has the potential to be.

• Similar to Business Analysis, your Agile teams may have their own views on how the work that BRM represents gets done.

To bring clarity in your organization I suggest:

• Propose whether your "flavor" of BRM is more tactical -- as in, "let's make sure we're being responsive to requests from the business and tracking changes they need to know about" -- or more strategic -- as in, "let's engage with business leadership at all levels to understand business strategies and revise our IT strategies and portfolio accordingly."

• Since you'll probably want some of both, propose who does what, and what their relationships are to other roles that also work with the business and guide IT work. For example, how do the people doing BRM relate to: senior IT managers with existing business relationships; business analysts defining outcomes with business stakeholders; and service owners scoping services based on perceived business needs.

• For service and project teams, build agreement in the organization on what your BRM practitioners will contribute and how they relate to team roles -- what are the accountabilities between, for example, a Business Relationship Manager and a Service Owner or Project Manager?

Business Architecture

The Open Management Group describes Business Architecture as "A blueprint of the enterprise that provides a common understanding of the organization and is used to align strategic objectives and tactical demands." In the TOGAF framework [3], Business Architecture defines a "Target Business Architecture that describes how the enterprise needs to operate to achieve the business goals." The Business Architecture Guild [4] calls for "An abstract representation of an enterprise and the business ecosystem in which it operates", used to establish "a recurring value stream" that provides the "ability to move from a strategic plan through solution deployment." I consider this field to be the least firmly defined of the three, especially in higher education. However, there is a professional organization with resources and certification. Business architecture is a frequent area of confusion because:

• As with BRM, the work that Business Architecture represents was historically done by others, mainly senior managers and some senior Business Analysts. Managers in particular may wonder why this field needs additional definition and work, may not be ready to collaborate with non-managers in this work, or feel that it challenges their own role. (I've seen otherwise fine Business Architecture work wasted because of this.)

• IT and business perspectives on Business Architecture tend to be quite different. IT tends to think of it as a means to better understand business needs so IT can respond appropriately. Business-centric practitioners tend to think of it as a means to plan and execute changes in the business (which may depend in part on IT services).

• Definitions of Business Architecture (as a field and even as a role) are often surprisingly vague about whether the intent is descriptive (to show the business "blueprint"), or transformative (to effect change in the design of the business), or both.

In my opinion, even spreading the term Business Architecture in your organization has little value until you work through these topics.
To bring clarity to your organization I suggest:

• Describe why you are housing Business Architecture in IT (or not).

• If it is housed in IT, Business Architecture could be variously intended to help the IT organization understand business strategies and needs; effect change in the business architecture of the IT organization (as a business unit); or be offered to other business units as a service to work on their business architectures. For you, is it some or all of that?

• If not, IT can still benefit from fostering Business Architecture work in the business -- for example, so that business units are better prepared to work with IT. Is that something your IT leadership wants to promote?

• Consider what "flavor" of Business Architecture work will be accepted and actively supported by your existing management (both IT and business managers). This is especially important for "transformative" Business Architecture to succeed. (See my related post, Business Architecture is Management Support.)

• Based on these decisions, if formalizing a Business Architect role, make sure job descriptions are clear about whether the role is descriptive, transformative, or both, and realistically describe its scope and how the work will be utilized. (See my related post, Defining a Business Architect Job Description.)

What are you ready for?

I can't say there is a direct maturity progression from Business Analysis to Business Relationship Management and Business Architecture, but I do think they build on each other in some ways, and you don't have to do grow them all at once. As a rough sketch I suggest:

• As quickly as possible, develop Business Analysis to the point where the IT organization (all of it, from portfolio management to service and project teams) can effectively gather, analyze, and drive work from well-defined business needs.

• Recognize that as they gain experience, your senior BAs can take you a long way, building stronger business relationships (as a partial substitute for BRM) and partnering with business stakeholders to work on strategies, goals, and changes to business capabilities and processes (as a partial substitute for Business Architecture).

• Ramping up more gradually, identify where BRM work is currently happening, what its challenges are, and where formalized BRM roles, practices, and process would be helpful and well received (or not). Make sure BRM work can hand off to effective Business Analysis practices (and other roles).

• Carefully consider whether and when you are ready to distinguish Business Architecture, perhaps first just as an analytical framework ("descriptive" mode), and more gradually as a practice that effects change ("transformative" mode). Business Architecture has the most ambitious potential scope out of these three fields, and is most likely to falter if scoped too broadly too early.

• As always, share what you find! I look forward to discussing these concepts and writing about them some more.

Endnotes

1. I mention certification because where certification is well-established, it means professionals have spent time defining their field, which provides a useful reference for you. I think the value of actually doing certification is situational -- many practitioners do just fine learning by doing, others benefit more from training and certification.
2. The IT Infrastructure Library IT Service Management framework -- see Wikipedia.
3. The Open Group Architecture Framework -- see Wikipedia.
4. In its Business Architecture Body of Knowledge Guide.