Kurt Bittner

Organizations who don’t understand why they want to become Agile also often take the wrong path to get there. Agility requires empowering teams and helping them make decisions on their own, learning from their experiences as they go. They must organize themselves, yet they often have Agile practices forced upon them. I’ve personally encountered many teams who didn’t understand why they were on an Agile journey, didn’t really want to be on the journey, and weren’t particularly happy about it. No good comes from those situations, and the result is usually a team going through the motions, telling management what they think they want to hear rather than really embracing Agile principles. Agile transformations can't be planned Management, for their part, are equally culpable in this comedy of errors: they don’t really understand what Agility means or demands. They can be motivated by mistaken assumptions such as thinking that Agility is a means of improving efficiency rather than a means of achieving better outcomes that cannot be fully understood at the onset. Convinced that Agile Everywhere must be a good thing, these managers want a plan for rolling out Agile, with milestones and progress measures that they can use to manage the roll-out. The Agile adoption is planned like a military campaign, with great speeches and slogans, sometimes even complete with banners exhorting teams on to victory. “Becoming Agile” is a strategic objective, like a hill that troops must take on the path to victory. The military metaphor for Agile adoption is flawed for many reasons, not the least of which that it applies a Waterfall process for adopting an empirical approach to delivering business value. It ignores the essential importance of team self-organization in a way that actually prevents teams from ever self-organizing. It discourages empowerment from the very beginning. Agile practices are not like a new version control tool that can be rolled out to one team after the other, in lock-step execution of a plan. We need a different metaphor to help us think about the problem. Becoming an Agile organization requires a cultural phase change In physics, phase changes occur when matter changes from one state to another, like when a pond changes from liquid water to solid ice. No one would talk about planning which molecules are going to change phase in what order; in fact the process is unpredictable. What happens is that, when the temperature of the water is just right, a small non-conformity causes a crystal to start to grow. Sometimes this happens in a single place, and sometimes in a number of places at once. The crystals grow and spread naturally, depending on local conditions. Changing a culture requires an approach more similar to encouraging a pond to freeze than planning a military campaign. The right conditions for the change have to be in place; these conditions can be evaluated by examining the beliefs of the product owners, business leaders, and delivery team members responsible for a product. When the beliefs of these people support experimentation and learning, they are open to trying new things and working in a new way. These product teams are capable of making the changes. When their beliefs are not supportive of the change, it is like having impurities in pond water that prevent it from freezing; forcing them to change is ineffective. Instead, you should work on changing their beliefs, perhaps by helping them to understand that a predictive approach to product delivery will not help them achieve the results they want. This is a long path, however, and if you are trying to change the organization, you are probably better off finding some other product team that is ready for the change than investing time in changing the beliefs of people who are not yet, and may never be, ready. Agile organizational change is opportunistic Working opportunistically, using the metaphor of the freezing pond, means focusing on helping parts of the organization that are ready to change, and that need to change to be successful. Agile practices will help them thrive. But other parts of the organization are nowhere near the critical phase transition, and no amount of energy applied to make them change will produce more than superficial conformity. Beyond finding product teams ready to change, what can be done? Well, every part of the organization can benefit from improving their engineering practices, and reducing their technical debt. Senior leaders can be helped to understand why an empirical approach may be essential for their organization’s survival, just as Jeff Immelt came to see that Agile software delivery was essential to the future of GE. Leaders have a strong influence over the beliefs of an organization, and they alone usually have the ability to help their organizations see things in a new way. What they usually need is to be shown proof that an empirical approach delivers innovation where the predictive approach won’t, usually by trying it on a product in a new market, or on a product that has nothing to lose by experimenting with a new way of working. These are the pockets of non-conformity where the cultural phase change starts and from which it spreads. Find and nurture these, and then look for other similar opportunities while building a supportive Agile ecosystem.

I’ve spent a lot of time looking at how organizations are using DevOps to improve their software delivery cycle time, their ability to innovate, and their ability to improve quality. I’ve heard some people go so far as to say that DevOps has replaced Agile, but I don’t think that’s true. If anything, DevOps reinforces and supports Agile, and Agile is still at the heart of DevOps. The essential features of Agile - working in small increments (batches), delivering working software frequently based on business priorities, working in small self-managing cross-functional teams, and measuring progress by working software - are essential for DevOps to work. In my view DevOps makes explicit what many Agile teams had already learned but may not have formalized. DevOps also spreads Agile principles to other parts of the organization, especially Ops. In truth, Agile and DevOps are symbiotically and inextricably linked. Agile needs automation to really boost velocity The Agile Manifesto says Agile teams value working software over comprehensive documentation. This means that they need to build and test software. A lot. All of the time, in fact. Ideally, they need to build every time they commit a change to the source code repository. But builds are just the starting point - they also need to test every time they make a change. Not just unit tests, but functional tests, regressions tests, performance and scalability tests, all driven from APIs. And not just in simple environments, but ones that look as much like production as possible. When they do this they deliver better code. To do this, they need to practice trunk-based development. They need continuous integration automation that drives not only the build process but a robust API-based testing process, every time code is changed. They also need to have robust test environments available on demand, which means using infrastructure as code or other environment provisioning automation. And at some point, when they are fully responsible for supporting the application in production, they need release automation tools to reliably deploy code. Some people have taken the Manifesto’s statement that Agile teams value Individuals and interactions over processes and tools to mean that tools aren’t important. People are most important because they are the ones who actually create software, but in practical terms those people need tools. The gap between “potentially releasable” and “actually released” can be vast Demonstrating working code in a demo at the end of every sprint is essential. Demos are not the same as the real world, however. Demos don’t test real workloads in real-life threat environments. Demos don’t have to play nice with other applications. Demos don’t have to be fault tolerant or self-healing. Demos in a test environment are necessary but not sufficient. Scalability, security, reliability, and maintainability are essential. Demos can focus too much on the look and feel of the application and not enough on its real-world characteristics. Agile teams who support their applications in real world environments learn this in a hurry. Realistic production-like testing environments are a good first step. Advanced deployment techniques like blue-green deployments and canary deployments, deployments to subsets of the user community, provide insights that demos never can. Testing the deployment processes themselves by frequently deploying to real-world environments helps to remove inconsistencies that are significant sources of production incidents. Agile teams need to adjust their definition of done to eventually encompass deploying to real customers - until they do they may be fooling themselves about the quality and value they think they are delivering. Product Owner and stakeholder feedback is great but real customer feedback is the true test of value delivered The Product Owner is the single authority on what needs to be built and the arbiter of “done”, at least until real customers speak. Product Owners, even great ones, are people like the rest of us, with their own biases and blind spots. Adding other stakeholders to demos can help by expanding perspectives, but they, too, have blind spots. The ultimate arbiter of value is the customer, and the faster a team can deliver to real customers, the faster they will confirm or reject the theories they have about what those customers really need. This is not a criticism of the Product Owner’s role, or the opinions of stakeholders, merely a reflection of reality. Application architecture matters Performance and scalability matters. Security matters. Services, microservices, and APIs especially matter. The more loosely coupled an application, the easier it is to change one part without that change breaking lots of other parts. APIs can be regression tested as part of the Continuous Integration process. Minimizing code dependencies also has a pay-off when tackling larger applications: coordinating work with other teams through shared APIs is a lot easier than sharing data structures. A lot of complex program management and team coordination overhead can be avoided when team dependencies are mediated through APIs. Techniques like service virtualization (otherwise known as mocking and stubbing) can be used to simulate commitments while the other team works on delivering the new API. One of the reasons that legacy application code is so hard to release frequently is the the applications tend to be monolithic, sharing data structures and databases with other applications. They are hard to change because every change has unknown side effects. Getting this code back under control will take years, and it may simply be easier to rewrite it. But modern applications don’t have to be this way, and Agile teams need be work hard to see their applications loosely coupled, lest these applications become the legacy applications of tomorrow. Small batch size is really important for fast feedback cycles and reducing release risk The smaller the release, the fewer the dependencies. The fewer the dependencies, the fewer things there are to go wrong. If you want to reduce the risk of a release, reduce the size, don’t double-down on control processes that actually increase release size by creating a process so onerous that people will do anything to avoid it, like releasing less frequently. Smaller releases makes coordination easier. They make user adoption easier because there are fewer changes or new features to learn. Smaller more targeted releases make it easier to evaluate their impact because their are fewer things to measure. This makes feedback faster, which reduces waste and helps teams to better scope future releases. Smaller, more frequent releases are good. They are good, but only if a team is continuous building and testing their software. They are good only if release processes are simple, streamlined, and predictable. They are good only if applications are loosely coupled and don’t create a cascading series of problems when something fails. Agile teams need DevOps practices, and DevOps needs Agile’s teaming model and connection to business value to drive better results DevOps practices help Agile team to automate the mundane and routine, letting their focus on writing high quality modular, loosely coupled code that delivers the right solution. DevOps needs Agile’s business-driven (through the Product Owner) backlog and pipeline management capabilities. It needs the Agile teaming model to create high-performing cross-functional teams. Agile teams need to add Ops and Security skills to their repertoire as they increasingly take on responsibility for managing applications in production. Agile and DevOps are not really separate things. Agile has always embraced excellence in engineering principles. Having to deliver and support applications in production expands that focus. Doing so at significantly faster cycles means that automation is now an even more important part of those engineering practices.

Agile approaches, including Scrum, are empirical approaches to delivering software and business value. It is ironic, then, that the biggest impediment to adopting an Agile approach is the culture of the adopting organization. The Cambridge English Dictionary states that culture is "the way of life, especially the general customs and beliefs, of a particular group of people at a particular time.” The beliefs of the people in the organization, and especially its leaders, are powerful predictors of whether that organization will successfully embrace Agile principles.   Change is painful. The change that we ask organizations to make when they embrace Agile principles can be deeply unsettling. We ask the people in the organization to potentially see that everything they have come to believe about planning and managing software delivery, and even business value delivery, is wrong. They may believe that the best way to deal with uncertainty is to have a more precise plan. They may believe that resource specialization is the best way to manage scarce skills, and they believe that career paths based on specialization will best guarantee their economic success. They may believe that they will get the best results if they reward individual success and punish individual failure. Organizations that embrace Agile principles reject all of these beliefs, and many others, as false. When we ask people to embrace Agile practices, we are asking them to abandon a world view that, while not perfect, may seem to them better than the next best alternative. At the core of our mission to help them, we must show them that Agile is not just different but better. We are only successful in that mission when their beliefs change. In order for us to do this, there have to be cracks in their world view that can let new ideas penetrate. I have found that the following beliefs establish the possibility that the organization is open to change. The greater the degree to which the people in an organization affirms these beliefs, the more open they will be to embracing Agile principles. Conversely, when those people do not affirm these beliefs, their organization’s cultural inertia may be too strong to overcome. Belief #1: “If the organization does not change, it will cease to exist.” There is nothing like imminent demise to make an organization willing to try something new. Plenty of organizations find themselves in this position today; upstart digital innovators are breaking old business models at a pace that traditional organizations cannot match. When an organization’s leaders believe that their current culture cannot produce the innovative products and services that the market demands, they are open to changing anything and everything. The problem that many organizations face is that they don’t see that the world has changed around them until it is too late, if they even see it then. They can powerfully insulate themselves from seeing the truth. When they do finally see it they can succumb to despair or finger-pointing. Neither are productive. They must come to believe that Agile is a way out of the crisis in which they find themselves in order for an Agile transformation to be successful. Belief #2: “The path forward cannot be planned or predicted without experimentation.” Even when they believe that they are in trouble, people may still cling to the belief that traditional, predictive planning will save them. They may believe that if they can just deliver the competitor-killing new product that they have on the drawing boards, they will be fine. The problem with this belief is that when markets change, assumptions change along with them. Traditional planning approaches are chock-full of assumptions about markets, features, and customer needs that have never been tested or validated. As long as the organization believes that “the business” knows what customers want, they just need to deliver it faster, they will continue to cling to traditional delivery models.   If they believe that they already know what they need to deliver, they will see no benefit in working iteratively, delivering working software, measuring the result, and refining their definition of the right solution. They will believe it reasonable to expect a detailed plan on when those requirements will be developed and tested. They must first come to believe that their understanding of customers is imperfect, that they really only have theories about what customers really need, and that they need to form and test hypotheses about those needs in order to achieve success. Once they believe this they are ready to embrace an empirical approach. Belief #3: “Delivering business value is more important than individual employee utilization.” Knowledge work in the 20th century came to be characterized by functional separation into roles and job categories, based on the assumption that work is specialized and that the best way to reduce cost is to keep expensive resources as busy as possible all the time. This usually means that people work on lots of different projects at once to minimize idle time. Since almost everything modern organizations do requires people to work together, the result of this is that everyone waits on someone else to get things done. More multitasking means more waiting on others, which leads to more multitasking and more waiting. The result is that delivering even the simplest change can take a very long time due to waiting and hand-offs. Delivering value to customers, faster, means reducing wait time, which means reducing role specification and dedicating resources to focus on product delivery. The Scrum team does this by having dedicated multi-skilled resources that reduces their dependence on resources external to the team. This minimizes wait time and improves time-to-market. Until organizations value time-to-market more than employee utilization they will be unable to improve their ability to deliver value quickly. Belief #4: “The organization’s leaders believe that their teams are capable of making decisions and doing the right thing.” Traditional organizations reward and promote based on the ability for individuals to get things done. They reward initiative and results. They celebrate “heroes” who single-handedly rescue troubled projects from disaster. The underlying belief is that teams are incompetent on their own and without a strong hand guiding them they tend to be hapless. This, unfortunately, tends to be true in organizations who punish mistakes and does not encourage learning and innovation. Leaders in these cultures use phrases like “accountability”, and want to have “one throat to choke”. Unfortunately, when the work is complex, these “heroes” usually cause more damage than they fix. Delivering software, or designing complex physical products that involve software, is a complex team activity, not the result of individual heroic acts. When the work is complex enough to require a team of professionals to deliver the product, managers must trust that the team is capable of making important decisions. Managers can set goals, ask questions, and clear impediments, but they must accept that they don’t understand the work and can’t effectively direct it. This realization can be an emotional punch in the gut, and is one of the hardest transitions a manager has to make when adopting an Agile approach. When leaders believe in teams, they are tolerant of mistakes. They ask questions that help the team to learn from empirical data. They use retrospectives at multiple levels to help the organization learn and improve. They DO NOT look to lay blame or punish mistakes. They DO help the team clear roadblocks and make sure they have the resources they need. Belief #5: “The organization’s leaders believes that innovation is more important than efficiency.” An organization that values efficiency over innovation believes that it knows its customers and that it is building the products those customers want. It also believes that it is doing it better, at least in some important dimensions, than its competitors. An organization that values efficiency over innovation believes that it will differentiate itself through lower costs, or it believes that its prices are good enough and it simply wants to increase its profit margin. An organization that thinks efficiency is more important than innovation will never successfully embrace Agile principles. An organization that values innovation over efficiency does not yet know what its customers want, or it seeks a new set of customers with unknown needs. It also does not yet know what it needs to do to meet those unknown needs. It relies, instead, on an empirical approach, on forming hypotheses about what customers need and what will satisfy them. It values fast delivery of high quality solutions, and it values and acts on feedback. Agile approaches help organizations systematically create solutions that test these hypotheses and adapt based on results. Recommendation: Understand your organization’s belief system and choose battles carefully   Large organizations are complex societies with sub-cultures and pockets of non-conformity. Even small organizations can have sub-cultures. If you’re trying to introduce Agile practices into an organization, focus on finding small enclaves where the traditional belief system is being threatened by forces from the outside - an emerging market segment, or one which is threatened by digital competitors. Build Agile success here first.  Success will get attention from other parts of the organization who are also similarly threatened and ready to change. Not all these beliefs may be fully in place, but there need to be signs that the beliefs are starting to change in an Agile-friendly direction. Beliefs can be influenced, nurtured or dampened, but they are ignored at one’s peril.

I’ve been working in software for a long time, and for most of that time I’ve been trying to find better ways of delivering working software. I’ve had the good fortune to start my career, in the early 1980’s, at the beginning of a real revolution in the way software was developed and delivered. For the first time, computers became widely available, demand for software exploded, and a period of tremendous innovation and creativity in software development and delivery began and it has never really stopped. It has become a cliche to say that every company has become a software company, but it’s true. Software has gone from merely supporting the business to being the core means by which industry leaders deliver value to their customers. It is not hyperbole to say that organizations who cannot become great at delivering software will simply cease to exist. The business leaders who understand this are changing entire companies and industries. When Jeff Immelt, CEO of GE, regards software delivery as key to GE’s ability to innovate and thrive, you know something remarkable has happened. However, the problem is that many organizations still struggle with the profound transformation that they must undertake to become truly great at quickly delivering high-quality experiences through software. They want to transform, but transformation is hard, and with so much at stake they can’t afford to make big mistakes. They are, in fact, afraid of making any mistakes, and that fear paralyzes them. I have seen what can be achieved when organizations become truly agile enterprises. I have also seen enough organizations in the throes of transformation to know how hard it can be. I joined Scrum.org because I think it is has the best potential to help organizations solve this challenge. I am equally excited to have the opportunity to work with Ken Schwaber. He is one of the foremost leaders in the Agile community as a signer of the Agile Manifesto, the co-creator of Scrum, founder of the Agile Alliance and Scrum.org, and someone who has the passion for improving the profession of software delivery. I now get to work side-by-side with Ken to continue on his vision of bringing Agile and Scrum to the enterprise. I believe that Scrum.org has the best opportunity to help organizations fulfill the full potential of Agile. It has great credibility in the Agile community for delivering high quality thought leadership, training , and certifications. It also has a great community of experts and practitioners who are already helping organizations at many levels. I bring to this my own experience in helping large organizations transform. I have also learned a lot in my years covering Agile and DevOps as an industry analyst; I’ve seen what leading organizations are doing, and I have helped other organizations learn from the experiences of others. I plan to continue and build upon that work, working with my new colleagues and new community at Scrum.org. Our timing could not be better. Becoming an agile enterprise is on the agenda of every C-level executive, and yet they struggle with the change. They are looking for help and guidance, and partners who can help them make the change. Agile has always been popular at the practitioner level, but many teams became frustrated by the lack of executive support. We can help execs see that agile is not just a better way to develop software but a better way to run their businesses, and then help them adapt and thrive in a period of unprecedented challenge and opportunity.