Three principles of Agile IT Org Design

I wrote an article for InformIT where I describe three principles of Agile IT Org Design that the book recommends and elaborates:

• Govern for value over predictability
• Organize for responsiveness over cost-efficiency
• Design for intrinsic motivation and unscripted collaboration

Essentials of scale-out management

I have previously stated that Agile scales by scaling-out rather than scaling-up. I expand on this in a presentation at an event in London. (slides)

Interview on InfoQ

http://www.infoq.com/articles/organization-design

"Sriram Narayan’s book provides a basis for reviewing and reshaping the IT organization to equip it better for the digital age."

Org Chart Debt - A Risky, Off-Balance Sheet Liability

Recently, Steve Blank, a luminary who developed the Customer Development methodology, a precursor to the Lean Startup movement, said that organizational debt is like technical debt in software but worse: “Organizational debt is all the people/culture compromises made to 'just get it done' in the early stages of a start-up.”

“Just when things should be going great, organizational debt can turn a growing company into a chaotic nightmare. Growing companies need to understand how to recognize and “refactor” organizational debt.”

He was talking about organizational debt incurred by startups at the time of hiring lots of people as it prepares to grow. On the other hand, Enterprise IT incurs organizational debt throughout the course of its existence.

“IT organization design is rarely thought out from a baseline of principles. The prevailing design is mostly a product of happenstance, mergers or acquisitions, people-retention compulsions, and the ideas of whoever is in charge at various levels in the organization. It resembles how software accumulates technical debt over time unless we periodically step back and reassess the design.” Agile IT Organization Design

And as with all debt that isn’t repaid on time, unaddressed organizational debt creates more problems. It results in silos, politics, unclear accountability, slow decision making and poor responsiveness to the market in general. Read further.

Agile Organization Structure

Some people mistake my book, "Agile IT Org Design" to be only about org structure. A couple of them even wondered if the book was a 250 page elaboration of Conway’s law. As if Conway’s law sums up everything there is to IT org structure.

Although I address it quite comprehensively, structure is but one aspect of organization design. Eight out of sixteen chapters in the book have almost nothing to do with structure. Only two (Ch 4. Superstructure and Ch 5. Team Design) are 100% about structure. The other chapters cover operational, cultural and political aspects of IT organization design. A re-org that only pays attention to structure is likely to be ineffective.

Do you scale Agile vertically or horizontally?

Most attempts to scale Agile struggle because the IT leadership attempts to scale up (vertical) rather than scale out (horizontal). Scale-up is old school management. It involves context-free management by numbers, standardized metrics, reports and dashboards and workforce motivation by targets and incentives. Inside IT, this approach usually ends up scaling supervision (not management) without improving IT performance. This is why we find managers functioning as supervisors and Scrum masters functioning as taskmasters in self-proclaimed Agile enterprise IT.

“Individuals and interactions” is not a marketing buzz phrase. It has to be taken seriously in the pursuit of agility. The organization has to be designed for intrinsic motivation and unscripted collaboration. Chapter three of Agile IT Org Design elaborates on this. Individuals respond to an environment of autonomy, mastery and purpose. Leadership balks at autonomy because it worries about loss of alignment with strategy and weakening of accountability. Chapters six and seven of the book explain how to allow autonomy and yet preserve alignment and accountability (using decision records, for example). This is the basis of scale-out management.

Team Design

Here's a free sample chapter on "Team Design" from my book. This chapter explores the following:

"Before asking how I can make my current set of teams work better together, ask whether they are the right configuration of teams."

However, middle managers usually shy away from asking the latter question because they feel it is beyond their pay grade. Nevertheless, it is necessary to address this question if we are to scale agility and make IT more responsive as a whole.

Agile over Lean

“That is, while there is value in the items on the right, we value the items on the left more.”

A number of reviewers suggested (in good faith) that I use “Lean” instead of “Agile” in the title of my book (Agile IT Organization Design) in order to improve its marketability. Apparently, Lean is in whereas Agile is jaded. However, the strong people-orientation credentials of Agile are core to the solutions I propose. I’d like to expand on this and some other reasons in this post which is also meant as a defense of Agile over Lean when it comes to organizational transformation in digital and tech organizations.

    Lean is great. Lean techniques emphasize waste avoidance, limiting work-in-progress, continuous improvement and caring about whole value stream optimization. I advocate these things in various chapters in my book. However, organizational agility needs more than process optimization.

People Orientation

When taken beyond development teams to organizational levels, the Agile principle of “individuals and interactions over processes and tools” actually has a bearing on the org chart, and the metrics and tooling regime. This may not be evident at first but I go to great lengths in the book to bring it out. For example, a matrixed IT organization demands more process in order to co-ordinate between the arms of the matrix than a product or capability centric org structure. Similarly, the Agile principle of “adapting to change over following a plan” has a bearing on how governance teams function, not just Scrum masters. Agile thus provides a sound base of attitudes upon which Lean techniques can be exploited.

    People orientation is an aspect of an organization’s culture. In his excellent Agile Survival Guide, Sahota examines Lean and Agile through the lens of Schneider’s culture model. This model classifies an organization’s culture on two axes—content (what-is vs. what-might-be) and process (people-first vs. company/policy-first), and names the four possible combinations as competence, control, collaboration and cultivation. Sahota argues that collaboration and cultivation cultures are Agile-friendly whereas control and competence cultures are Lean/Kanban-friendly. I find Sahota’s argument convincing and from this perspective, the overall approach is my book is clearly Agile.

Software Development isn’t Production

This is another reason why I think Agile has to be the foundation rather than Lean. Software development is a design process rather than a production process whereas Lean has its roots in the Toyota production system. If we are to draw parallels to manufactured goods, software development resembles the product development phase more than the manufacturing or production phase. Chapter three of Agile IT Org Design explains further.

    Applying Lean manufacturing principles wholesale to software development runs the risk of planting false metaphors of production in the minds of IT managers. In the book, I argue that the average IT manager’s fixation over predictability and relative disregard for actual benefits (value) partly stems from this false metaphor. Metaphors are powerful because they can be generative and therefore it is important to get them right.

Data-Driven Illusions

Advocacy for data-driven decision making is at an all-time high following the success of the Lean Startup book and its numerous offspring. However, it is somewhat naïve to believe that a culture of HiPPO (Highest Paid Person’s Opinion) decision making can be overcome by prescribing new formulae like cost-of-delay, for example. Prioritization is a political exercise in any organization of more than say, ten people. Students of history, geopolitics and current affairs will agree that politics trumps data.

    Formulaic, data-driven solutions for better decisions operate on the wrong side of “individuals and interactions over processes and tools”. In the chapters on accountability and metrics in Agile IT Org Design, I describe solutions that acknowledge politics, that continue to stand by “individuals and interactions”, and that are data-informed rather than data-driven.

Label Abuse

Some claim that “Agile” is a much maligned label and therefore it makes sense to move on to less discredited labels like “Lean”. But Lean has its own share of misinterpretations and misapplications. This post, for example, argues well that “Lean” has become synonymous with efficiency and layoffs. People tend to think of Lean as being the opposite of fat. It isn’t widely understood that capital-A Agile and capital-L Lean have only a limited semantic overlap with their common English counterparts “agile” and “lean”. Otherwise interesting debates have been framed poorly as “Lean vs. Fat”.

    There will always be cases where people blame their wrong extrapolations on the book. The Lean Startup devotes less than three pages to describe product/market fit. However, this doesn’t stop people from claiming that they failed to achieve product/market fit despite following the book’s advice. Label abuse is no reason to jump ship.

Agile is liberal, SAFe is conservative and their marriage will be rocky

Do we need a Big-Agile framework? This was the topic of a recent discussion on a LinkedIn group. Scrum, for example, may be viewed as a small-Agile framework. It suggests (prescribes?) rituals, practices, metrics and objectives for a development team. Many teams have adopted Scrum and realized its benefits. Then again, it is common to find people who wield Scrum certifications without a real understanding of software development. They do more damage than good on teams.

So can we extrapolate this to say that frameworks aren’t good or bad—it’s how you use them? Not so quickly. I’ve argued before that nothing is value-neutral. Later, Steve Yegge wrote an interesting essay arguing that different software methodologies espouse different value systems. “Embrace Change” is a liberal value. “Reduce uncertainty” is conservative.

A big-Agile framework like SAFe™ is conservative (hence “safe”). It may have won over conservative IT governance people but it will struggle to make IT truly responsive because it is incompatible with liberal Agile values. An alternative big-Agile approach I propose in my upcoming book is liberal and less prescriptive than a framework.

Am I implying that liberal is good and conservative is bad? No. It is a question of what is fit for the times. Change in technology and business is the order of the day. Embrace change or be safe and sorry.

New Book: Agile IT Organization Design

Update 9th June: Download the free sample chapter on Team Design

In order to scale Agile, we need a scalable yet Agile-friendly organizational architecture. That's the subject of this book.

The idea for this book germinated in May 2013 while I was preparing for a talk at DevopsDays, Berlin. My main argument in the talk was that while DevOps was great, it didn’t go far enough to influence business outcomes. However, when the rationale behind DevOps is applied to constituencies in addition to development and operations, we end up with a true cross-functional team that can be truly responsive to the market.

Within IT, Agile is commonly misconstrued as only applicable to the development team. So teams end up adopting continuous integration or even continuous delivery but it fails to improve overall IT and business responsiveness. If we are to influence business outcomes, we need to apply Agile principles and values to how we design teams, how we make important decisions, how we fund IT development etc. This is what I refer to as an Agile Organization Design—the stuff outside of team level engineering and process agility.

Software is eating the world and IT is increasingly strategic for every business. In the push to go digital, overall IT and organizational agility matter more than ever.

With the help of real examples, I cover a wide range of issues that influence organizational agility:

Chapters called Superstructure and Team Design address structural issues. A chapter on Accountability addresses political issues. Chapters called Alignment, Projects, Finance, Staffing, Metrics and Tooling address operational issues. Chapters on Norms and Communications address cultural aspects.

A rough cuts version is now available on Safari.