A Metric Is a Signal, Not a Lever

Teams arguing about metrics is a good thing. It’s one of the healthier arguments a team can have - and it tends to surface a misunderstanding I think is close to universal.

A very common objection to moving a metric further down the funnel is that it would mean it’s “out of our control”, because now we’d be measuring something external forces influence as much as we do. It’s a thoughtful objection, but it’s backwards.

The goal of a metric is not a number you can control. The goal is a signal.

What Metrics Are For

Metrics exist to answer one question: are we getting closer to our vision?

Not “are we busy”, not “are we shipping”, not even “are we hitting our targets”. It’s is the thing we exist to do actually happening in the world?

That means a good metric has to be able to tell you no. It has to be able to go the wrong way. If your mission isn’t working - if the market has shifted, if your approach is wrong, if something upstream broke - the number should fall, and you should be glad it fell, because your signal is working.

A metric that can only go up is a participation award.

The Control Paradox

The more direct control a team has over a metric, the less valuable that metric is as a signal.

This feels wrong, because control feels good. If I can move the number by doing more of my job, then my job is clearly valuable, and the number proves it.

But think about what you’ve actually measured. You’ve measured activity. You’ve built a metric that says “we did a lot of the thing we do”. It can’t tell you whether the thing you do matters, because it goes up whenever you do it, regardless.

In Goodhart and Good Hearts I wrote about how a measure stops being useful once it becomes a target. A fully controllable metric is Goodhart’s Law pre-installed - it was never really a measure to begin with.

A Worked Example

Say your team owns notifications, and you select a metric like “notifications sent”.

You could send a thousand low-value notifications tomorrow. Ok, well how about “notifications opened” - that implies they have some value? Some fraction of people will tap on them - or you could set them to auto-open. Your metric goes up.

Have you done anything valuable? No. Has your metric told you that? Also no.

Now say you turn off a spammy notification that’s been annoying users for months - genuinely the right call. Your metric drops - it just punished you for doing the right thing.

Move the metric one step further from your control. How about measuring users who come back from a notification and then actually do something?

Spam doesn’t bring people back to do anything (it mostly just gets notifications turned off), so the spam strategy stops working. Retiring a notification that was never driving anyone costs you nothing. The metric has started to distinguish between activity and value.

But you’ve lost a little control. If the notifications are leading to poor features owned by other teams, or a memory-hogging Android implementation that crashes on cheaper devices, then your number goes down too.

The fact that another team’s product experience affects your number isn’t a flaw in the metric - it’s a true fact about the business that you’d otherwise be ignoring.

A metric that’s purely within each team’s control lets everyone succeed while the business fails. That isn’t independence - it’s irrelevance.

Check the Mission

A practical test: read your team’s mission statement, then read your metric, and ask whether the second actually measures the first.

Your notifications team’s mission probably says something like “help people stay on top of what matters to them”. It almost certainly doesn’t say “send as many notifications as possible”. If your metric is a count of your own output, you have a metric that says nothing about your mission - and you can win the metric and fail the mission every single quarter.

The vision is never to move a specific number. If it were, you’d just pick a number you control and move it. The vision is a change in the world, and the metric is the instrument you use to check whether it happened.

For the Teams Arguing About Theirs

If you’re in this argument right now, here’s what to push for:

• A metric that can go down when you’re doing something wrong.
• A metric that doesn’t go up when you do something cheap and valueless.
• A metric that’s far enough from your direct control that it reflects value, not activity.
• A clear line from the metric back to the mission statement.

If that sounds uncomfortable, it should - a metric you can’t be embarrassed by isn’t a commitment. And picking the easiest number to measure is just the same mistake from a different direction.

So when someone says “but that’s outside our control” - agree with them. That’s the point. You’re not trying to hold the number.

You’re trying to hear what it says.