The checks. Each one is a way a RICE score can look rigorous while resting on a number nobody really estimated.
The problem
Reach × Impact × Confidence ÷ Effort produces a confident-looking order from four numbers, most of which were estimated in a meeting by people who wanted the meeting to end.
The failure is not bad arithmetic. It is a backlog whose order is set by one of the four inputs while appearing to rest on all of them — or one so fragile that a single re-estimate reorders the top.
What it does
A backlog
Eight things, scored the usual way.
Five checks
Asked of the inputs as a whole, not of each item.
A shuffle
Every guess nudged a notch, to see whether the order survives it.
A verdict
A decision, or a coin flip dressed as one.
What it showed
Checks that fired
4 / 5
On an eight-item backlog scored the ordinary way.
Confidence against final order
0.77
It is doing most of the ranking. Worth knowing which number you are really sorting on.
Two of the findings are the kind nobody looks for. Reach is identical on every item, so it cancels out of the ranking entirely and is decoration rather than input. And confidence — the one field meant to carry doubt — is pinned at 95% or above on five of eight items.
Put together: the order is being set by a field that was supposed to express uncertainty and has instead been filled in to avoid expressing any. The arithmetic is perfect and the ranking means very little.
What it cannot do
It checks the shape of the inputs, not their truth. Eight well-distributed numbers that are all wrong will pass every check.
The stability shuffle tells you whether the order is robust to re-estimation. It does not tell you the order is right.
The long version of the argument behind it is How to hire a PM now.