You’ve probably heard a lot about the Agile Methodology and may even be practicing it. In our organization, Cisco CX Engineering and Product Incubation, we are entering year two of a transformation that includes the adoption of Agile processes across a large global team. Agile is being mapped to business processes, engineering processes, and onwards. We have seen improvements, as Agile helps define in clear terms the definition of timelines and requirement organization. Yet, with any new process adoption, there will be hiccups along the way. In fact, software development and Agile can be at odds.
What?! Why would I say this? Software developers are taught to think through all use cases, including possible problems, inputs, outputs and design for long term use. Agile instructs developers to deliver what is defined in requirements, not what the developer believes is needed. The methodologies of “complete solution” vs “what is needed” present a conundrum that developers must resolve.
So, how does a developer resolve these two methodologies?
Let’s walk through a quick example to explain how our seasoned developer, Anne, will handle what looks like a simple requirement. (Note: I am keeping this example very simple for brevity.)
Requirement: Develop a logging solution for existing services that sends logs to a central server in JSON format.
Background: Currently, services are written in Java and Python and there is an effort to investigate other languages that could be chosen to replace Java.
Approach 1: Design Java and Python libraries from scratch that provide a consistent API.
- Consistent library across both service languages
- More development required to provide all features and shake out bugs
- Maintainers of the library can easily debug and add new features
- Service developers must wait for their respective library to be coded and debugged
Approach 2: Design two libraries that utilize existing libraries in each language.
- Same as approach 1, provides consistent library
- Less development than approach 1, but there could be inconsistencies in capabilities of the supporting libraries
- Maintainers of the library can easily debug the wrapper, but will struggle to debug supporting libraries
- Service developers must wait for their respective library to be coded and debugged
Approach 3: Utilize existing libraries for each language and utilize a log scraper.
- No library to develop or to maintain
- Service developers can start using the chosen libraries immediately
- Logging team must learn to use a log scraper
- Log scraper is independent of service language
From the three possible approaches above we can see that each have their own benefits and drawbacks. In a strict execution of the Agile methodology, you should only develop what is required and nothing more. With that in mind, Anne should choose either approach 2 or 3. Approach 1 requires more development and should not be considered. Some would argue Approach 3 satisfies all the requirements and is future proof.
As noted, the above example is contrived, and I didn’t give you the full story. What if the requirements included that the logging API must run on a spaceship with limited memory and custom communication mechanism to the central server? Or if there is strict restriction on open source libraries? Each of these two requirements would have changed the choice. This is where management will lean on Anne to understand the requirements and choose a solution that meets both Agile and smart engineering. For example, if we had learned that Anne was providing the solution for a spacecraft, approach 1 would most likely be the best solution because of tighter requirements imposed on a space platform.
You can see from this example that software developers struggle with their natural instincts to deliver a “complete solution” compared to the “what is needed” Agile methodology. While Agile is an industry best practice, it can be at odds with the training of developers and their natural instincts. Agile development works because it sets guardrails to help limit the over-engineering instincts. Anne is still responsible for advocating that sometimes the minimal is not always enough. Her knowledge and experience should guide her in advocating for the best solution based on time, budget and requirements.
Companies that continue to adopt Agile methodologies in attempts to achieve effective and accelerated software releases should also factor into their process the experience and intuition of their software developers. Agile is not a “one size fits all” approach, human input is valuable to the process, and must be heard and incorporated for the best possible outcome.
Photos above (taken before covid): CX Engineering & Product Incubation leaders participate in team building and workshop on agile development process.
A topic for another day will be how requirements (or lack thereof) are defined can reduce or intensify confusion. I will cover approaches that companies follow to green light a project, communicate the overall project goals and how this process can help or hinder the requirement definition process.
Do you believe that Agile is at odds with good software development? Share your thoughts and or learnings below on the challenges and or solutions you have used to blend Agile with software development.
Super insightful Scott! Completely agree with you – in general, Agile frameworks go against our core instinct of expecting order in times of chaos. This is why, as you eloquently put it, human interactions are so critical. Human connections help us deal with the volatility and ambiguity that surrounds us as a product development organization. Markets change, customers’ needs change, competitive pressures guide us more than perhaps they should, etc – the ability to sense what’s going on around us, how we respond to those things, and our ability to adapt and scale/shrink as needed is what Agility is all about. It’s much less about what we need to do and far more about how we think about what we need to do. Kudos for authoring this and sharing! #reset your mindset #agility
Never heard or believed there was a conflict. But the article articulates this well. Thanks for sharing this.
Hello Guru,
Thank you for the comment. One would hope it isn’t a strong conflict, but we have all worked with engineers, maybe we’re even one of those, that want to have all the details before they start. Some in the Agile world would believe you can just run and don’t need to plan ahead. But as you know, building software that is going to be used long term has to have some foresight but also a balance of incremental work that Agile proposes. It is our job as leaders to balance the future and incremental forces to produce a timely, secure and effective product.
Scott,
Great article. I understand that the example you used is simplified. Some of the issues you mentioned Anne having to deal with could be solved by allocating enough time in discovery. Maybe a longer sprint zero would help. Most of the problems I’ve seen with Agile stem from incorrect implementation of the methodology.
However, I agree that Agile is not the answer to every problem. Even in cases that seem clear-cut, care and thought needs to be put in to identify the framework that would work best for the team, and more importantly, knowing when to stop/switch/modify it.
Zain,
I agree with you, but a point I have an issue with is the thought of Sprint 0. Sprint 0, to me, was something created to do pre work as opposed to just making it part of the sprint cycle. Honestly, Sprint 0 feels like we weren’t sure what to do with all the architects and the work they previously did prior to the “real” work was started, the high level design, requirement gathering phase of waterfall.
This is where I would argue that in general you set up sprints and you choose a length and go with it. If the first three sprints are business, platform and other architectural work to set the stage for the developers to work on, so be it. You would hope also that the developers could start working as well, just some of the work will be set for a later sprint.
My sense is that the problem being described here is really dogmatism.
‘Agile’ is being used monolithically as if it’s a set of rules to be followed.
The manifesto refers to values and principles. Principles require thought in their application. That’s both the challenge and the benefit.
In this example, the principle is to build no more than is required to satisfy a need. But there may be sound architectural reasoning why extending the solution further makes good sense. Good sense should translate ultimately back to business rationale – avoid risk, reduce future costs etc. the intent of agile is that there would be collaboration between those asking (there is no product owner in ‘agile’) and those fulfilling to collectively determine the best solution.
So the problem is either applying some predetermined meaning of agile as a rubber stamp template. Or it’s in following any agile pattern unquestioningly or one-sidedly.
Cisco, I’d start by baselining your principles, one of which sounds like should be ‘balance architectural imperatives when minimizing MVP scope. Secondly I would instill in the organization the autonomy and freedom to determine locally how to best apply the principles.
Dan,
You have hit the proverbial nail on the head. I don’t think I could have summed it up any better. What you usually see from organizations is they apply some process following the guidelines to the letter. There isn’t enough thought given, as you said, the principles and values and then apply those to the best you can in your groups. I have seen you get better results of instead of just saying here is a new process, let’s follow it. Instead teach what the new process in theory gives you, and then evaluate with your leaders is it a wholesale change or incorporation.
What I think leaders and even implementers need to understand is, everything has inertia. A development group operating at some level has inertia. Most of the times these changes to process are not just improvements, they are 90 to 180 degree changes in direction. And like any rolling boulder, you don’t just change its direction, this is not high school physics. That boulder may slow down, may stop, or it will be a gentle curve to go the new direction. But you almost never get the velocity of the team to go from X to Y without some level in loss of inertia.
I plan to write future article where we tease out some of these rationales for Agile and what do they mean when they say
Faster time to market
Reduced ROI
Higher Customer Satisfaction Ratings
etc
Also agree with your point and especially the importance of purpose.
I’ve done some articles on linkedin about this – arguing that mission, context, and values/ principles should determine which practices to apply. (MVC=P) The mission part is the why. Adopting agile as its own outcome definitely provokes that inertia. Engaging smart people in a clearly-articulated mission by providing principles they can apply to their context to determine how to work is a way of avoiding this struggle.
good informaton