%20small.jpg){kind=small}
A framework for a faster, smarter, and more consistent organization.
Every organization collects numerous KPIs and metrics to report progress and incentivize roles. Generating huge reports with an overwhelming number of KPIs often indicates missing structural transparency and a lack of management understanding regarding project status. Additionally, it often leads to an information delay between the report's creation and its use by management.
Hence, it is crucial to define metrics with a clear purpose to fully understand their implications. Generally, a metric is a direct or indirect quantification of a particular incentive to increase the measurability of an outcome [1]. Metrics are linked to a particular role and often directly influence its behavior. Therefore, it is important to understand the expected behavior change and how this contributes to team and organization objectives. Keep the following criteria in mind that every metric should satisfy.
Is objectively measurable over time
Has a strong causality to expected behavior (incentive and accountability)
Is directly controllable by incentivized role
While many organizations focus their KPIs on products or services, our approach in this article focuses on the organization’s capabilities. Incentivizing an organization independently of a specific product or service supports a long-term learning culture which, in turn, forms the basis for a continuous improvement mindset.
When considering such an accountability and incentive-based framework across your delivery organization, every metric must be defined in terms of:
Who will be incentivized?
Why is this metric used to measure/ visualize the incentive?
Is the metric measuring the capabilities of the organization or the progress of a project, product, or service?
Figure 1: Applicability dimensions and Spark Mind's approach to metrics
For every role, each accountability must be measured by a metric that not only contributes to an improvement of a particular role’s objective(s) but also positively contributes to team-level goals and organizational objectives. Whenever a metric can be optimized by a role without having a positive impact on team and organization, that metric should be discarded.
“Not everything that can be counted counts, and not everything that counts can be counted.”
Albert Einstein
Why are metric formulation and tracking so crucial?
The organizational structures, roles, and accountabilities must be simplified and standardized to enable an organization-wide incentive system. In every delivery organization, different knowledge areas, expertise, and skills are required, which we segregate into three distinct authorities:
Governance: this authority is accountable for defining the “way we work,” the collaboration approach, practices, and reporting, and is responsible for managing the resolution of impediments, risks, and dependencies while continuously realizing organization-wide improvements.
Business Content: this authority understands the customer and their respective business needs, formulates business requirements, and prioritizes efforts for the delivery organization.
Solution Design: this authority has all the skills and knowledge required to design, build, test, and operate a solution that meets customer needs and fulfills high-quality requirements to ensure a stable development and production environment.
We strongly believe that every organizational progress is based on these three distinct authorities [2], like separating a state’s power into a legislative, an executive, and a judicial branch. These authorities are strictly separate, and no person should simultaneously be part of two authorities. Each authority has a dedicated set of accountabilities, where metrics incentivize each authority and its respective roles. Such a guiding incentive system forms the basis for a learning organization.
Figure 2: Three distinct authorities with respective roles on the team level and at scale.
The three authorities can only be successful in combination when collaborating and contributing with their specific accountabilities in mind. Therefore, it is imperative that every accountability per authority has dedicated metrics to incentivize the behavior that ensures the respective roles contribute to the overarching success of the organization. These accountabilities spanning the three authorities can lead to positive conflicts, where a resolution satisfying interests across all authorities usually improves the organization’s problem-solving capabilities and, finally, the quality of the product.
This article highlights significant accountabilities per authority with selected key metrics.
Governance Authority
The key accountabilities of the governance authority are to improve and ensure a high-velocity delivery organization that continuously and consistently delivers high-quality software, often referred to as the principle of flow. Therefore, measuring high-velocity in a delivery organization contributes to three organization goals. First, reducing the amount of invested work without realized gains (development work for which no business value in a production environment has been realized). We call this organization ability business agility, providing flexibility for the business content authority to pivot on their requirements based on market demand, changes in the economy, and critical customer feedback while minimizing the waste of invested effort. Second, increasing the consistency by which product increments can be estimated, developed, and released to production. We call this organization ability predictability, improving the trust between all authorities to meet committed PI objectives and sprint goals. Finally, the governance authority is responsible for establishing a resilient delivery organization that can respond and adapt to incremental changes and sudden disruptions.
Business agility, predictability, and resilience depend on a stable, maximized value creation where output can be continuously and sustainably delivered. In other words, the goal is to maximize the work completed over a certain time without negatively impacting the organization. This is commonly measured by Little’s Law, dividing work in progress (WIP) by the cycle time. WIP is defined as all active/ open tasks, and cycle time is typically measured as the time between the start of the implementation of a task and the moment it is completed. The cycle time should be minimized to maximize work completed, which can be achieved by keeping people focused on a particular task until it is completed to avoid context switching and distractions.
Flow can generally be measured by an organization’s average cycle time and lead time. A task’s lead time generally measures the age of a task from the moment it is created to the moment it is completed. In other words, the time the task is waiting in the backlog or in status “ToDo” plus the cycle time. The goal is to minimize both metrics over time, as seen in the formulas below.
To successfully establish a high-velocity organization with a constant flow of business value creation, the following important accountabilities play significant roles and should be tracked using respective metrics:
Get things to done – Limit work in progress and reduce waiting times to maximize focus of every person on one or a minimum set of activities. Then, get the activity(s) completed as quickly as possible.
Track, remove, and/ or escalate impediments – Transparently track and remove impediments as quickly as possible. If necessary, escalate impediments to achieve resolution.
Track and minimize risks and dependencies – Transparently track, resolve, or minimize dependencies and risks. If necessary, escalate before they become impediments.
Ensure single source of prioritized requirements, decoupling, and single funding source – Establish a single source for backlog inputs and define a single point of contact providing these requirements.
Ensure capacity-based funding and prioritized business initiatives (on portfolio level) – Instead of project funding/ project thinking, establish product-oriented thinking with capacity-based funding.
Establish and ensure cadence alignment (cycles/ iterations) and time-boxing – Align cadence across the delivery organization to ensure synchronized and time-boxed cycles.
Establish and ensure business content alignment – Formulate PI objectives and sprint goals during refinements and planning meetings. Subsequently, challenge them during reviews and system demos.
“An organization's ability to learn, and translate that learning into action rapidly, is the ultimate competitive advantage.”
Jack Welch
Business Content Authority
The business content authority has the key accountability for maximizing the business value created by the delivery organization. It does so by improving two factors. First, it continuously formulates small/ minimal sets of requirements that generate business value, often referred to as minimum viable product (MVP), and tests them with respective customers as quickly as possible. The goal is to minimize time-to-market for any new business functionality. Second, it increases the frequency of customer feedback to validate assumptions, applicability, and customer demand as quickly as possible. Continue with the MVP (solution) if the hypothesis is confirmed, or pivot (solution) if the hypothesis was rejected by the customer. The goal is to continuously validate and improve the product-market fit using small MVPs and short cycles.
Thus, the business content authority is incentivized to create small product increments in short cycles and quickly validate them with the customer that will regularly use the product. This small batches approach should significantly reduce the potential loss of time and the number of required product amendments.
Figure 3: The three distinct authorities with specific, significant accountabilities.
Minimizing the time-to-market can be measured as the time from defining a minimal set of user stories called MVP to customer validation.
Optimizing product-market fit can be measured by maximizing the customer's acceptance ratio of product increments based on working software.
A customer satisfaction score is another option for measuring the business value delivered to the customer. Generally, this is achieved by sending questionnaires to customers. The faster product increments are released, and the better the product meets customer expectations, the greater customer satisfaction usually is. The main drawback of such questionnaires is the sizeable effort required to formulate or amend the questionnaire to specific situations.
To maximize business value generation, the following accountabilities have significant impacts and should be tracked using respective metrics:
Define and communicate product vision and product strategy – The product vision and strategy are understood and transparently accessible to everyone within the organization.
Ensure business objectives are understood and business value is formulated for each feature – Business objectives are formulated for every feature in terms of cost of delay and confirmed by relevant stakeholders.
Establish and continuously ensure high-quality product backlog – The product backlog is always DEEP (detailed appropriately, emerging, estimated, prioritized).
Communicate business priorities and agree to objectives with the highest business value per cycle – Business priorities are transparently available in the form of a prioritized product backlog, and business objectives are based on team capacity and agreed upon during relevant planning meetings.
Ensure completed tasks meet all requirements and acceptance criteria – Every feature or story is accepted or rejected during review and system demo based on formulated acceptance criteria.
Align with the customer and foster short feedback loop cycles – Customer feedback should be obtained frequently and as quickly as possible.
Solution Design Authority
The solution design authority is responsible for the product's design, implementation, and operation and has two key accountabilities. First, the authority is accountable for maintaining stable products in production with low operating costs. Second, they are accountable for creating high-quality product increments and releasing increments continuously to production.
Maintaining stable products in production is an important factor in customer satisfaction. A product is only good if it runs without major interruptions or incidents. Hence, a key metric to measure stability is the time the production system is available and fully operational.
The time a system is available and fully operational is heavily influenced by additional metrics. The number of critical production incidents and escaped defects should be minimized and serve as a quality measure of the product. Additionally, the time required from the development team to maintain a stable production environment should be minimized, or there is a risk of a hidden unstable and poorly operational system.
The second accountability is focused on how product increments are built, deployed, tested, reported, and released. If done manually, these activities can consume a lot of knowledge workers’ time who could otherwise focus on value-generating efforts. Furthermore, with manual execution, quality can be substantially reduced over time (e.g., manual regression testing is usually not executed continuously across the entire functionality). Hence, it is key for the solution design authority to establish a DevOps pipeline [3], thereby minimizing manual effort and, thus, time to bring code to production.
Furthermore, the development team should improve their ability to estimate realistic work efforts and commitments for PI cycles and sprints that can be repetitively delivered. This ability contributes to the delivery organization’s predictability. It can be measured using the cycle velocity with the goal of minimizing the deviation from the historical velocity per cycle.
To achieve a high quality product in a stable production environment and continuously deliver small, high-quality product increments, the following accountabilities have a significant impact and should be tracked using respective metrics:
Properly understand, refine, and estimate features/ stories – Participate in relevant practices and collaborate with POs to deepen understanding and estimate as accurately as possible.
Estimate complexity of work based on relative effort estimation and consistently deliver – Estimations should be consistent over time to support predictability in achieving product increment (PI) objectives and sprint goals.
Whole team creates PI objectives and provides sprint goals – Participate in PI and sprint planning meetings to define PI objectives and sprint goals.
Adhere to formulated DoD and validate all acceptance criteria – Before bringing a new feature to a system demo, check if DoD and acceptance criteria are validated.
Execute sprint reviews and PI system demos based on working software – Respective developer presents working software during system demos and sprint reviews to relevant stakeholders.
Implement and integrate changes in small batches continuously – Changes are implemented incrementally and iteratively.
Conclusion
In summary, as seen in Figure 3, products evolve around three distinct authorities: governance, business content, and solution design, each with dedicated accountabilities incentivized by clearly defined metrics. The effective interaction of the three authorities ensures a stable and high-velocity organization that continuously and consistently delivers high-quality products to its customers.
“Progress cannot be generated when we are satisfied with existing situations."
Taiichi Ohno
Spark Mind can support establishing a learning organization by implementing our incentive framework that links every authority to a set of accountabilities, measured and tracked by a broad set of pre-defined metrics. We have the knowledge, skills, and expertise to support your transition toward a faster, smarter, and more consistent delivery organization.
Download the PDF version of this article here:
_______________________________________________________