Universal Release Criteria 2.0–A Disruptive Quality Management Framework

Contributions from Munir Palla and Satya Pradhan.

Quality cannot be an afterthought! It takes strategic planning for meticulous implementation. The more time we invest in proactive thinking, tooling, Shift Left approaches, quality governance, and a process-driven culture, the more money and time we can save across the life of the product. Cisco’s Universal Release Criteria (URC 2.0) represents one such disruptive quality management framework that has been developed and adopted. The following criteria will help define a comprehensive set of quality goals, metric standardization, and process governance across the full development lifecycle from product development requirements to releases. The IOS-XE product fully adopted, implemented, and proved the URC 2.0 development process very effective.

Having worked in the software industry for more than two decades, I have gained experience in all aspects of the software development life cycle, process, metrics, and measurement. As a programmer, I designed and developed complex systems. I led the software development strategy and process for CI/CD pipelines, modernized it, and automated it. Also, I led the quality journey for various types of software releases for businesses to grow to scale. Given my background, I feel qualified to share my thoughts on URC development processes that helped us transform traditional development processes to more modern, automated, and streamlined counterparts.

Limited vision in quality goals and unchecked negative behaviors directly impact product quality. Resistance to upgrade requests from customers, lack of scaled environments to test deployments, and delays in early adoption really plague projects. Short-sighted quality goals impact deployments in the following manner:

• There is excessive focus on backlog issues with inadequate focus and management of incoming defects.
• Critical defects are prioritized exclusively.
• Inadequate or negligible attention is paid to proactive defect prevention.

URC 2.0 is the brainchild of cross-functional quality specialists with representatives from operations, development, testing, quality, and supply-chain organizations. The main objective of this process innovation is to “address defects found in a release” that “will have to be fixed in the same release” by bringing “no technical debt” forward. Initially, this simple rule placed tremendous pressure on teams, but within a couple of years, everyone willingly embraced this cultural shift!  The results of URC 2.0 can be summarized as follows:

• Provides a failproof framework for release quality management.
• Outlines comprehensive release quality criteria for product development.
• Transforms the departmental culture within software and hardware teams.
• Prevents defects and reduces escape conditions during the product development lifecycle.
• Fosters trickle-down innovation, enhances development practices, and furthers tools automation.

Shift Left techniques enjoy the benefits that come from URC’s quality algorithm innovations and the processes such as the following:

• Manages the incoming defect process using escape analytics and Raleigh’s curve.
• Enhances the algorithm to help address incoming, backlog, and bug disposal trends.
• Prioritizes age-based bug fixes to address high-risk defects.
• Simplifies operational management of the bug backlog.
• Reduces last-minute code churn.
• Operationalizes escape reduction.
• Targets addressing of security vulnerabilities at the right level.

The URC 2.0 Framework

The URC framework is based on four basic tenets of quality management:

• Prevention
• Identification
• Evaluation
• Removal

A set of metrics is identified for each tenet to measure and drive quality. The objective of each phase and
corresponding metrics are described in the diagram below.

To measure the effectiveness of activities in each framework, use a set of metrics to achieve very specific objectives per category. From an execution viewpoint, all URC 2.0 metrics can be grouped into categories: defects, bug evaluation, late code churn, security, testing, and parity. This grouping with associated goals are published and evangelized throughout the release cycles. Executives are briefed to ask the right questions to prevent the release of inferior code quality to customers. This discipline raises the bar for teams, encourages them to work hard to meet goals, and helps them proactively address challenges. In this way, teams are able to confidently stand behind the quality of their release offerings. Here are some categories to help address code improvement goals:

Important URC Framework Criteria

To enhance tracking clarity, the URC 2.0 framework provides the following critical and grouped elements.

• Track Defects–Address incoming and legacy issues based on severity and resolve issues within the same
• Use Metrics–Focus on execution quality and pass rate. Adhere to strict goals and periodically measure
• Ensure Release Defect Parity–Ensure that all fixes from prior releases are incorporated into the current release.
• Security Vulnerabilities–Address all known issues before general release availability.
• Control Late Code Churn–Set aggressive mean time to repair goals for all internally and externally found

The URC framework defines some critical terms for all releases: URC window and URC backlog.

• URC Window–The time between the “URC Freeze” date of the prior release and the “URC Freeze” date of the current release is considered the window of opportunity to make a difference.
• URC Backlog–When implementing URC 2.0 for the first time, address defects that fall into these
  major categories:
  • All defects that are applicable to the current release, irrespective of where or when they were found.
  • All URC defect fixes that must be carried forward from the previous release to maintain functional parity amongst releases.
  • Pick a start date from the previous window to begin your URC window for the current release. This will help serve as a reference point to address critical defects.
• URC Freeze–Select a date when you stop working on your URC backlog. Ideally, this date must be after the completion of feature test for the release. The frozen URC backlog must be addressed and brought down to zero within three weeks of the freeze. This date must coincide with the critical defect cut-off date for the release. No bug fixes, immaterial of their severity, should be added to the current release after this critical defect cut-off date.
• Final Code Validation–Conduct final testing and release readiness checks before the release is shipped to

Proof of URC 2.0 Success

Witness the “defect mountain Shift Left”. It illustrates the success of adopting the URC 2.0 framework. It also clearly demonstrates Cisco’s emphasis on software quality before general release and serves as the best representation of all the hard work.

The unique and rewarding digitization and release quality journey makes a significant difference for our developers, customers, and to our business. It ensures our ability to deliver features and functionality with speed and at scale. With the combination of our DevSecOps digital transformation (captured in my blog, “How do you gauge software quality before deployment?”) and the URC 2.0 Framework, we will be able to elevate the quality of our software
offerings! Customers will enjoy receiving software that is error-free and impactful right off the bat!

Related Resources–Cisco blogs:

• How do you gauge software quality before deployment?
• Digital Transformation Enabling Automated DevSecOps Success
• The We’re Listening Blog Series: Software Quality – We Heard You!

 

Subscribe to the Networking blog