Measuring Change Adoption Without Measuring Impact Is a Trap
The Measurement Problem
Traditional change management metrics focus on activity: training completion, licenses issued, adoption rates. But these don't show whether change is delivering value.
Activity vs. Impact
|
Activity Metrics |
Impact Metrics |
|
Training completion |
Behaviour change |
|
Licenses issued |
Active usage |
|
Communications sent |
Message retention |
|
Survey participation |
Sentiment change |
|
Adoption rates |
Business outcomes |
What to Measure
1. Awareness
Are employees aware of the change?
- Communication engagement
- Survey awareness scores
- Message retention
2. Preparedness
Are employees prepared to adopt the change?
- Training participation
- Training effectiveness
- Help desk metrics (tickets, escalations)
3. Adoption
Are employees using the new tools or processes?
- Usage and utilization reports
- Compliance and adherence reports
- Behavioral observations
4. Impact
Is the change delivering business value?
- Benefit realization
- ROI
- Quality metrics
- Customer satisfaction
The Outside-In Mindset
Otto recommends a change manager focuses on two outside-in outputs :
Awareness: Measuring if employees understand the change
Preparedness: Measuring if employees have the knowledge and ability to make the change and sustain it
Measuring Performance
Zendesk's Dana Otto recommends the following approach:
1. Focus on Two Main Outputs: Awareness and Preparedness
Make sure employees understand the change and have the tools to adopt it.
2. Quantify Qualitative Data
Frame qualitative questions in employee surveys on a scale to quantify responses. "Measuring change management is one of the most difficult parts of the process because you're measuring people and their emotions, which is hard to quantify" .
3. Ask Managers to Hold Teams Accountable
A major component of measuring change is assessing if people are doing their part to meet project goals .
4. Measure if the Business Is Prepared
Evaluate if the business is ready to move from its current state to the desired future state .
5. Leverage Technology to Track Communications
Use tools to track if communications are impactful .
6. Incorporate Feedback Early On
Foster a feedback loop between impacted groups and business leaders early on .
7. Measure if the Change Stuck
Continue measuring to see if people continue to incorporate the change over time .
Conclusion
Measuring change success requires moving beyond activity metrics to impact metrics. By focusing on awareness, preparedness, adoption, and impact, organizations can understand whether change is actually working.
Action Items for Your Organization
- Define success metrics for your change initiative
- Measure awareness and preparedness
- Track adoption over time
- Measure business impact
- Use outside-in metrics (employee perspective)
- Continue measuring after rollout
