Navigating Enterprise Scaled Agility by OKR and KPIs

“We cannot improve what we don’t measure!”, you have probably seen and heard a variety of versions of this quote, and it is essentially true for the simple fact that:

You must have established an understanding of the initial (or current) state of that specific factor to compare it with the next (or future) state of it and get an idea towards the trend it has followed. Is it better, worse, or  unchanged?

There is a common confusion about the hierarchy of elements and their relationship when it comes to making sense of the observations and using the gathered measured values to build a strong navigation and status control mechanism for an organization. Here I would like to start with some clarifications:

Measures

Measures are numerical values obtained by taking measurements showing the amount or degree of something. They are produced through direct mapping of observations to numbers.

Metrics

Metrics (things you want to observe and measure)  in support of your business-as-usual and would be of value to your organization. They measure the performance of different parts of your business against a benchmark and are driven by a tactical team or initiative in the organization.

Metrics are static values extracted and organized by activity or process and provide information that can be historically tracked and  ingested (and have more of post-mortem value than future outlook). Individually they do not provide many insights to the organization.

Metrics are of quantitative nature and are measured out of the measurement data that is gathered and are used as the base for KPIs.

KPIs

KPIs (or Key Performance Indicators) are the selected “Key” Metrics – usually limited to 5-8 metrics - that are very important to you and are supporting some important objective that your organization wants to manifest.

KPIs are of  quantitative nature and are always presented as percentages, average or ratios and they measure the progress of the most critical objectives of your teams or your entire business and are driven by strategic executive decisions and encompass multiple areas or teams in your organization.

KPIs are designed to provide comparative insights, incorporating organizational goals, are forward looking and of a dynamic nature and as a result they can show performance changes over time.

KPIs work towards establishing a holistic view of the performance of different areas of your organizations. They measure accomplishments (not just work that is completed).

KPIs are action-focused, output oriented tools in achieving symptomatic  goals, monitoring the steady-state, and comparing them against benchmarks. They can trigger defined actions in response to metrics hitting a threshold.

KIPs are measured on an on-going basis with their results feeding the organization’s quarterly, semi-annual, or annual OKR. In many organizations, KPIs reflect revenue targets and growth of various business metrics.

Many organizations use the SMART approach to identify and establish KPIs.

SMART stands for:

• Strategic & Specific:
  • What are we trying to measure?
  • Why is that important?
• Measurable:
  • How much change is needed?
  • How many actions or cycles would be needed?
  • How do we know we have succeeded?
• Actionable:
  • Is it feasible for us to perform this measurement?
  • Can we commit to achieving this activity?
  • Is the expected outcome reasonably believed to produce value for us?
• Relevant:
  • Would this activity provide meaningful outcome for the teams or management?
• Time-based
  • What is the timeframe for completing this activity?

OKR

OKR stands for Objective and Key Results, and is a collaborative framework established at enterprise (or departmental or group) level to assist in achieving a set of “ambitious and inspiring” goals. It is essentially a goal-setting tool to setup objectives and connect the required KPIs to measure the progress, and performance during the progress and the end results.

OKR acts as the navigation system toward the objectives, like a GPS, while KPIs act like the gauges in a dashboard that show how the functions in the organizations are working while the organization is moving towards achieving the goals.

OKR is an outcome-focused, growth-oriented, qualitative tool used in achieving structural goals. They are feature-focused and directional.

OKR is essentially a strategy-in-a-nutshell for the organization’s Value Streams, helping connect short and mid term priorities to the reason behind them.

Objectives should be aggressive and bold, motivating the teams towards some important accomplishment, and compared to KPIs, posses much higher levels of flexibility.

Objectives are like a mission statement, but with a shorter life-span. A great Objective inspires the team, it is not easy to achieve, but also not impossible to accomplish in a set time frame by the teams.

Some main questions, that can assist in narrowing down the Objective’s range at this level would be:

• What are the core value propositions that define our products and services?
• What differentiators do our products and service have compared to our direct and indirect competitors?
• What are the essential customer problems that our Value Streams are addressing right now?
• Which ones do we want to address in the future?
• What alternatives could our users have to solve their problem?

Arriving at a high impact set of OKRs is an incremental practice through iterations and organization’s strategic themes should be able to assist in identifying the gaps in the current state and set the priorities for achieving the future state and bridge the distance between the strategy and implementation.

Key Results are mini-goals which contribute to the Objectives. They may map one-to-one, or you may have multiple Key Results, each serving certain parts of an Objective, aggregating towards what needs to be satisfied.

An organization usually has two to five Key Results for each Objective, and they can be based on anything you can measure. In fact, Key Results take the Objective’s motivational language and quantify it.

Key Results are created by asking a set of relevant questions, such as:

1. How would we know if we have reached the Objectives?
2. How does success for our product or service look like at the end of this iteration?
3. What should have changed when we get there?

Answering these questions can be difficult if the teams supporting Value Streams lack tangible guidance, so the Objectives need to convey the needed clarity and visibility.

OKRs are commonly reset every three months, but when the Objectives are particularly large and ambitious some of them may carry on to the next quarter. Most teams reset the slate clean and start over again.

The duration of three months is most preferred because it is not too short to achieve anything significant, and yet it is not too long for the teams to be away from the market changes and chasing Objectives that may have lost value due to aging.

For example:

• Objective: Establish a Mobile App Store by end of June
• Key Result #1: Customers can search and browse the items in the mobile app and select and add them to cart.
• Key Result #2: Customer should have secure checkout and payment service with both Visa and Mastercard.
• Key Result #3: Min of 50 selected customers should have fully tested the service and provided feedback on their shopping experience by May 1st.

For organizations following the Scaled Agile Framework, Objectives are supported by the Features brought to teams during the Program Increment (PI) Planning, and the Key Results are the User Stories (or more accurately, the Acceptance Criteria of the User Stories).

For smaller companies running individual teams, Objectives may roughly map to their Sprint goals (probably stretched over multiple Sprints), with the User Stories (and their Acceptance Criteria) covering the Key Results.

Three important success factors for OKRs are:

1. Setting them based on Priorities, not on what will come next:

Focus should be centered on what impact the value delivery pipeline should experience. Generating more of the preferred Business Outcome is the main priority which will drive other priorities as it cascades down the value streams.

2. Leading to more decentralized decision making and teams self-organization:

Teams should experience more ownership and self-driven commitment and less hand-off from the management, so they would be afforded the needed sense of self-drive and accountability towards achieving the Key Results.

3. Stronger organizational alignment towards the OKRs:

The lines of business and the internal clients should first focus on the set of target Key Results that the organization (or the group) tries to achieve, and then create the matching set of work requests that would manifest them. This will establish the needed correlation between the targets and the efforts in obtaining them and not the other way around.

OKRs’ Practical Considerations

Based on my experience of the past decade, I would like to recommend the following steps and considerations to enable OKR to support and boost the Agile environments.

When setting the Objectives and Key Results:

• Set the Objectivesthat map to your organization’s large goals, with the highest positive impact, and make them visible and crisp and understandable for your teams, to help them share your vision and participate in defining their divisional (cascade) OKRs and their needed Key Results.
• The Objectivescan have different priorities and can address a mix of short, mid, and long term needs, if the longer duration targets can be broken down into incremental progressive steps (with distinct deliverables at the end of each increment). This puts the OKR in alignment with the Lean Agile delivery teams.
• The Objectivesmay lead to higher revenue, lower risks, learning new ways or expanding on existing ones and any combination of these outcomes. It is not always measured by revenue!
• The Objectivesmay be positioned somewhere on the line connecting the two extreme poles of High Predictability versus High Innovation. The first is based on well-rooted setting of OKR to follow a tight, feasible and achievable set of Objectives with calculated value addition, while the second is structured for high innovative exploration in search for new ideas, market discovery and ways of value delivery.
• OKRs definition for Value Streamsshould be a collaborative practice, including all cross-functional team members to make sure it is done is based on the capabilities and commitments of the people who are doing the work. A workshop with proper facilitation would guide the teams through the process, and helps domain experts focus on the Objectives. All relevant inputs should be presented by their owners (to avoid any misinterpretations ) and considered and incorporated into the decision making on the final list of Objectives and their Key Results.

• Every three month reset the OKRs. This should be easy for the short and mid term goals. For long term ones do a re-assessment against the new market status and predictable upcoming trends to change course or make the needed re-calibration to maximize the impact from the changes.

• Have enough flexibility to freeze the work on the OKRand modify them on the go if the market conditions take a sudden and radical shift which would open up opportunities or pose threats to your organization’s livelihood or business. OKRs are live and active goal targeting mechanism that would need to adjust when the return on the changes would be expected to outweigh the cost of stopping-modifying-starting the work for the teams.

• Do not limit yourself – or your teams – by forcing a single approach or tool. Allow your teams to contribute to that decision making, in order to benefit from the power of their brainstorming and contribution. You should always consider the fact that these teams are the ones to actually achieve the Key Results, and the more they are involved in the level-setting process and tooling, the better they will own and commit into using them towards achieving the Key Results. Best – starting – OKRs would be the ones that will reduce your time-to-market through an optimized continuous delivery pipeline.

• As OKRs would cascade in many organizations, we should try to keep the number of cascades to a low number to reduce the amounts of hand-offs and the resulting ambiguity. We also should try to set the strategic level OKRsas a guidance and not as a solid constraint above the group level OKRs to avoid pushing the teams towards focusing on the Output (e.g., completion of a software component),  instead of the Outcome (e.g, market share expansion, profit generation, customer satisfaction).

• When setting up the Key Results, make sure they are small enough to be delivered by your teams within the time frame you are allocating with the consideration that they need to follow an optimized orchestration, to be able to aggregate towards achieving the target Objectives, and do all that by the Lean Agileteams (or SAFe Agile Release Trains).

• Use a time-boxed, hypothesis-drivenapproach for your Key Results to ensure they are properly paced and follow a good solution structure.

• Make sure the organization’s strategic roadmap and OKRs are not creating parallel (shadow) value delivery pipelines.

• Put your best effort to setup the Key Resultsindependent of each other, so if one of them runs into an obstacle or even fails, it would not domino into others and crash the OKR. They key factor is to design the target Key Results in a manner that will limit their blast radius so if they fall apart the damage will be controlled.

• To properly align with the Lean Agile mindset of exploration and learning from the mistakes, make sure that the teams have the psychological safetytowards the cultural norm accepting that failing is part of the learning process and it is okay to run into issues. As they say, “If you are not failing, you are not trying!”.

• OKRs supports incremental improvement, so we should avoid any attempt at one-shot transformations and re-creating the organization overnight.

• OKRs must enable more transparency about progress and success and enhance collaboration within the organization.

• Don’t forget to collect feedbackfrom your teams on the quality of the OKR setup, relevancy, and contribution of the used KPIs in monitoring and assessing the progress alongside the work. This provides you with a wealth of real-time and live view into how effective and impactful your OKR will be.

• Set your OKRsa few week before the next quarter and make sure to setup an Inspect and Adapt workshop at the end to go through the OKR process, the successes and failures, their root-cause analysis, and the needed remedial works. The corrective actions (to fix the issues and preventing them from happening again) or to amplify the synergies to make repeatable success, should be part of the OKR for the next quarter.

• Allow a preparation periodfor the teams to align to the new OKR set before the next quarter begins. Some teams may need a bit longer to get their teams resourced and trained and their systems ready before the work starts.

Outcome vs. Output

When designing the OKRs, there is always a chance of confusing Output with Outcome. An Outcome describes a measurable change in human behavior that creates a desirable Impact. A more pragmatic example would be how we influence the shopping habit of our customers, so they would buy more or use more of our services.

An Output is a change in the state of something (like the launch of the web site), as a result of our activities. A series of Outputs can be orchestrated to create an Outcome.

Examples of Output:

• The shopping cart section is completed.
• 500 regression test case scenarios are passed.
• We have complete features A, B and E and pushed to production.

Example of Outcome:

• Customer satisfaction with the web site has improved from 62% to 87%.
• Our market share has grown to 25%.
• More than 90% of our customers use the mobile app.

When designing our OKRs, we need to focus on the Outcome, as the ultimate result we are trying to achieve.  We should also avoid the myth that OKRs would automatically define their Outcomes. They need to be defined, discussed, fine tuned and turned into Objectives in a collaborative effort with the teams.

OKR and KPIs in Scaled Agile

Depending on how the management envisions – and actually implements - the OKR, they can either elevate the Agile environment to a much higher level of productivity, predictability and optimized value delivery, or end up suppressing the Agile values and hamper teams’ flexibility and market responsiveness trough forming a command-and-control dictatorship where the teams would be deprived of their sense of innovation and exploration and be pushed into a rigid, preplanned work formation, which would essentially create a hard come back of Waterfall approach with all of its problems and deficiencies.

Scaled Agile Framework (SAFe®) proposes a comprehensive set of KPIs that can measure and visualize a wide variety of performance and progress indicators at team, program, and enterprise level. SAFe’s newest KPI model puts focus on Business Agility, which is a compound KPI structure measuring multiple elements in Organizational Competency, Flow of Value through the enterprise and the Business Outcome, resulting from the other two.

Breaking down the constructs of these main KPIs, will reveal a large arsenal of 2^nd level KPIs which visualize the quality of the flow (its distribution across different activities, speed, volume, efficiency, and predictability). It will also expand into envisioning multiple competency levels, from team level all the way up to the enterprise portfolio. The break down can continue all the way down to the teams where insights into individual teams’ performances and efficiencies are brought to light.

For a detailed list of these metric, their sub-metrics, and their correlations you can visit their website.

At the time of this writing, SAFe® does not provide a proposed OKR system, but a complete structure for Objectives setting (and Key Results) can be established with its comprehensive process model, especially through the top-down (Portfolio Backlog to Agile Release Train’s Objectives set for each Program Increment) and their aggregation upward across the teams all the back to the Portfolio level KPIs.

Conclusion

OKR and its supporting KPIs act like the navigation system and indicators’ dashboard for an organization. Without them , we cannot move towards a destination and have no way to tell how good we are moving there.

OKRs have a multitude of benefits, but should be handled with care so they would not turn into a device of corporate dictatorship and top-down tyranny.

OKR does not come with a pre-define solid set of steps to follow and there is no written book of rules for it. Every organization need to experiment with the best practices that should be tailored to their needs and over time get incrementally improved into a stronger and more effective system.

In the currently accelerating, fast paced, ever changing and extremely competitive market landscape of the first half of the 21s century, proper and effective use of OKR is an absolute contributor to an organization’s survival and growth into the upcoming years.