Why performance data isn't enough — and what real impact looks like

You can hit every target — and still miss the point.

It's a familiar pattern for anyone working in social impact, development, or mission-driven organisations. We get so good at measuring what's easy to count — budgets, deadlines, deliverables — that we risk losing sight of the very thing we set out to do: create meaningful, positive change.

That's the fundamental difference between performance data and impact data.

And too often, we don't even ask for the latter. We build entire reporting systems, strategies, and dashboards without ever digging into whether our work truly made a difference. Impact data isn't deprioritised — it's often not invited to the table at all.

The Illusion of Control

Let's start with performance data. This is what most organisations track rigorously:

  • Were activities delivered on time?
  • Did we stay within budget?
  • Were all contractual KPIs met?

These are important. They build trust with funders, ensure operational discipline, and make it possible to scale. As Harry Hatry (1999), one of the pioneers of public sector performance measurement, wrote: “Performance measurement is essential for improving services, enhancing accountability, and ensuring cost-effectiveness.”

But Hatry also warned that an exclusive focus on outputs can lead to a false sense of progress — where success on paper does not match the reality on the ground.

Performance data shows that the engine is running. But impact data tells you if you're headed in the right direction.

Impact Is Relational, Not Just Technical

Impact data is harder to capture. It's more qualitative, more contextual, and usually more delayed. But it's where the real answers live.

Impact data forces us to ask:

  • What actually changed for the people we serve?
  • Did this programme lead to increased income, skills, wellbeing, dignity?
  • Are those outcomes sustainable — and are they valued by the people experiencing them?

As Chris Roche (1999) argues in Impact Assessment for Development Agencies, “impact is not just about technical success or delivery; it is about power, participation, and lived experience.” Measuring impact is about getting outside of our own systems and looking critically at what those systems produce for others.

This is uncomfortable terrain for many organisations — because it challenges the tidy logic of inputs and outputs with the messy reality of people and change.

But if we are not measuring what matters to beneficiaries, are we really measuring impact at all?

Rethinking Accountability

This is where the distinction becomes more than just semantics. It's a question of who we are accountable to.

  • Performance data = internal accountability: to funders, to contracts, to plans.
  • Impact data = external accountability: to communities, end users, society.

This idea is central to Michael Quinn Patton's (2008) concept of developmental evaluation. He encourages organisations to embed evaluative thinking into their work — not just to prove success, but to learn, adapt, and evolve alongside complex challenges.

In Patton's framework, impact data becomes less about judgment and more about learning. It gives permission to experiment, to pivot, to admit when things aren't working and to do better.

That's the kind of culture we need in organisations that claim to be mission-driven.

Strategic Fit Matters

Even with the will to do better, organisations often struggle with how to measure impact in a way that is useful, proportionate, and aligned with their purpose.

This is where Theories of Change (ToC) come in. A Theory of Change is more than a planning tool — it articulates an organisation's desired long-term impact and maps out the pathways, assumptions, and preconditions required to achieve it. It connects strategic vision to implementation reality. Crucially, it moves beyond inward-facing organisational growth and puts external ambition at the centre.

As Weiss (1995) wrote, “A Theory of Change outlines the causal pathway from inputs and activities to intended impact, making assumptions transparent and enabling evaluators to test them.” It helps organisations clarify not just what they do, but why it matters — and for whom.

ToC is essential for deciding what to measure and why. If your theory of change centres on improving the agency and livelihoods of young people, your data strategy should include metrics around dignity, resilience, and economic independence — not just job counts or cost-per-placement.

Building on this, Ebrahim and Rangan (2014) argue that measurement systems must fit both an organisation's level of control and its scope of ambition. This is where ToC adds critical nuance:

  • An output-focused ToC will likely prioritise performance data.
  • An outcome- or systems-oriented ToC demands investment in longer-term, more complex impact data.
  • And a transformational ToC — seeking social or structural change — requires a focus on contribution over attribution, and on learning over simple validation.

Too often, organisations adopt lofty theories of change but then build performance frameworks that only measure near-term efficiency. The result is a strategic disconnect: the vision says one thing, the numbers say another.

By rooting measurement in a clearly articulated, well-debated Theory of Change, organisations can navigate this complexity with greater integrity. They can make informed trade-offs, communicate transparently with stakeholders, and learn faster from what the data reveals — not just about performance, but about purpose.

Bringing It Back to Practice

At The BUSY Group and the Challenge Fund for Youth Employment (CFYE), we're navigating this exact balance.

We track performance data meticulously — financial milestones, contractual outputs, delivery deadlines. But increasingly, we are building systems that also capture:

  • Long-term employment outcomes
  • Gender-inclusive job quality improvements
  • Youth satisfaction and voice
  • Stories that reflect both hard data and human experience

And here's the key: impact data is not just for your MEL team. When intentionally integrated into organisational learning, impact data becomes a strategic resource — activating feedback loops, surfacing inconvenient truths, and compelling leadership to think beyond internal targets and toward real-world relevance.

When leadership engages deeply with both quantitative and qualitative impact data, it shifts the focus from “how are we performing?” to “are we truly helping?” It forces management to lead with a more externally focused view — grounded in the lives, choices, and perspectives of the people we exist to serve.

In short: impact data drives smarter learning, more meaningful decisions, and stronger alignment between values and action.

We are learning to treat impact data not as an afterthought, but as a strategic driver of decisions, accountability, and continuous improvement.

It's a shift that requires investment — in tools, skills, and mindsets. But it's worth it. Because if your purpose is impact, your data should reflect that.

Not Just a Once-a-Year Report

Let's be clear: this is not just about publishing a polished impact report once a year.

If your impact strategy lives in a PDF, you're doing it wrong.

Annual reports can be important artefacts — but they are retrospective, static, and often curated for external audiences. What we need instead is a living, breathing relationship with impact data. One that informs day-to-day decisions, challenges assumptions, and drives continuous learning across the organisation — not just within the MEL team or the comms department.

Embedding impact data into the regular rhythm of operations changes everything. It influences:

  • Which projects get resourced and scaled
  • How frontline teams adjust their approach
  • What senior leaders prioritise in boardrooms
  • And how strategy evolves in real time

It also empowers staff at all levels to ask: “Are we doing the right things?” — not just “Are we doing things right?”

In high-performing, mission-driven organisations, impact is not an output. It's an operating principle. Something wired into team conversations, leadership decisions, and the stories we choose to tell — every day, not just once a year.

Making Trade-Offs With Eyes Wide Open

Let's be clear: this is not about throwing out performance data. It's about bringing it into partnership with impact data — so that both can inform decision-making with balance and integrity.

Because here's the reality: the most impactful project might not be the most profitable one.

And sometimes, the projects that deliver the cleanest outputs may not deliver the deepest change.

These tensions are real — and they're not going away. But they can be navigated well if organisations are willing to do three things:

  1. Be transparent about the trade-offs. Don't hide them — name them.
  2. Make values explicit. Clarify how decisions are made when impact and performance metrics pull in different directions.
  3. Decide actively, not passively. Too many organisations let the default be the decider: “it's cheaper,” “it's faster,” “it's easier to report.” But what if the deciding factor was what mattered most — not just what measured best?

When impact and performance data sit side by side, they don't dilute each other. They deepen the conversation. They force leadership to lead — not just manage. And they make space for decisions that are not only defensible, but deeply aligned with purpose.

That's not just smart governance. That's integrity in action.

Final Thought

If you're only measuring performance, you're only managing part of the story.

Let's elevate the role of impact data — not just to prove what we've done, but to improve what we do next.

What's one indicator in your organisation that genuinely reflects whether your work is making a difference? Let's compare notes. The world doesn't need more perfect reports. It needs more honest ones.