Productivity, checked · Part 7 of 8
Feedback: how to read the gauges without fooling yourself
Daniel Kahneman once taught flight instructors in the Israeli Air Force that rewarding good performance works better than punishing mistakes. A senior instructor objected. When he praised a cadet for a clean manoeuvre, he said, the next attempt was usually worse. When he shouted at a cadet for a sloppy one, the next attempt was usually better. Praise didn't work; shouting did. He had years of experience to prove it.
He was describing something real and explaining it wrongly. A cadet who flies an unusually good manoeuvre has probably had some luck, and the next attempt will usually be closer to their average, praise or no praise. The same goes in reverse for an unusually bad one. Kahneman had the instructors throw coins at a target twice. Those who did best the first time mostly did worse the second, and nobody had shouted at anyone.
This part is about the last station on the line, the one that tells you whether the other five are working. It sounds like the easy one: count something, look at the number, adjust. The flight instructor shows why it isn't. Feedback can make you better, and it can just as easily make you confidently wrong.
Count results, not motion
Most of what is easy to count is activity: hours logged, emails sent, tickets closed, lines of code written. Accelerate (Forsgren, Humble & Kim 2018) makes the case that these are the wrong things to watch. They measure motion rather than results, they are easy to game, and pushing utilisation towards 100% makes queues explode (~85%; the queueing maths is in Flow).
The book's alternative, built from the State of DevOps surveys, is four measures of software delivery: how often a team deploys, how long a change takes to reach users, how quickly service is restored after a failure, and how often changes fail. These "DORA metrics" became the industry standard, and they hang together as a coherent picture of delivery performance. The most interesting finding is that speed and stability didn't trade off. The best teams were faster and broke things less (~70%). The caveat: these are self-reported surveys from self-selected enthusiasts, so they show that good teams have both, not that speed causes stability.
The authors later went further. The SPACE framework (Forsgren, Storey et al. 2021) argues that no single number captures developer productivity, and recommends at least three of five dimensions (satisfaction, performance, activity, communication, and efficiency or flow), with at least one that asks people how things feel ~70 %. It is a framework rather than a tested model, but its core point travels well beyond software.
For a single person the translation is simple: ask what the week produced, not how busy it was. A thesis chapter drafted, a patient discharged safely, a customer who renewed: those are output. Everything else is input in disguise.
Every number needs a rival
In 2016 it emerged that employees at Wells Fargo, under pressure to hit targets for new accounts per customer, had opened millions of accounts that customers never asked for. The bank fired about 5,300 people and paid a $185 million fine. The account count went up. The thing it was supposed to signal, customers who wanted more from the bank, went down.
This is Goodhart's law, in the anthropologist Marilyn Strathern's popular phrasing: when a measure becomes a target, it ceases to be a good measure. Economists have a formal version. Holmström and Milgrom (1991) showed that when you reward only the part of a job that is easy to measure, effort flows away from the parts that are hard to measure. Nobody needs to be a villain for this to happen; they only need to respond to what is counted.
Andy Grove's fix in High Output Management is to pair every indicator with its counter-indicator: measure an effect and the thing that would suffer if someone over-optimised it (~85%). Count output, and also count defects. Track speed, and also track rework. The DORA metrics are built this way, two for speed and two for stability, which is why gaming one tends to show up in another. Grove also prefers leading indicators, which warn of trouble before it reaches the output; the hard part, which no study solves, is finding one that really leads.
For a person, a pair might be "pages written" and "pages kept after a week", or "tasks closed" and "tasks reopened". One number can be flattered. Two numbers pulling in opposite directions are much harder to fool.
Noise in the signal
Back to the flight instructor. Regression to the mean is not a psychological quirk but arithmetic: whenever performance has a random component, an extreme result tends to be followed by a less extreme one (~95%). The trap is that the follow-up result arrives right after whatever you did in response, so you credit or blame your response. Shout after a bad week and the next week usually improves. Celebrate a great week and the next one disappoints.
Small samples make it worse. Kahneman's example: the counties with the lowest kidney-cancer rates are mostly small and rural, and so are the counties with the highest. Small counties are simply too small to produce stable averages, so they land at both extremes. A single person's week is a very small county. One brilliant Tuesday or one wasted Thursday says little, yet it's tempting to build a new system on either.
The same noise is why decisions should be judged by their process rather than their outcome. Under uncertainty a good decision can turn out badly and a reckless one can get lucky, yet people judge by results anyway. Baron and Hershey (1988) showed this outcome bias, and a preregistered replication with 692 participants confirmed it, even among people who said outcomes shouldn't count (Aiyer et al. 2023). Whether reviewing your process actually improves later decisions is much less tested. The tools for making decisions in advance live in Choose; the point here is simply that a result is a noisy grade on a decision, not a verdict.
Loops that actually teach
Peak (Ericsson & Pool 2016) is a book about deliberate practice, which belongs to Leverage. But it makes one point that belongs here: without feedback, practice doesn't improve anything. Purposeful practice, in the book's terms, needs a specific goal, full focus, immediate feedback and work just beyond what is comfortable. Remove the feedback and repetition turns into rehearsal of whatever you already do.
The most sobering evidence comes from medicine. A systematic review of 62 studies found that over half reported doctors' performance declining with years in practice, and only one found improvement (Choudhry, Fletcher & Soumerai 2005) ~70 %. The caveats are real: most studies measured knowledge and adherence to guidelines rather than patient outcomes, and older doctors trained under older guidelines. Still, the pattern fits a broader rule. Kahneman and Klein (2009) argue that expert intuition can be trusted only where the environment is regular and feedback is fast and clear, as in chess or firefighting, and not where it is slow or murky ~70 %. Twenty years of experience can be one year repeated twenty times if nothing ever tells you which of your calls were wrong.
Feedback can also make things worse. A large meta-analysis by Kluger and DeNisi (1996), covering 607 effect sizes, found that feedback improves performance on average, but over a third of feedback interventions decreased it, especially when the feedback drew attention to the person rather than the task. "Your report missed the deadline because the data arrived late" helps. "You're not a details person" mostly doesn't.
The simplest loop of all, watching your own progress, holds up well. A meta-analysis of 138 randomised studies with nearly 20,000 participants found that monitoring progress improved goal attainment, with larger effects when progress was physically recorded or made public (Harkin et al. 2016). Most of the goals were about health, but the mechanism is general. This is the part of Atomic Habits' advice to track your habits that rests on solid ground.
The review
David Allen calls the weekly review the critical success factor of Getting Things Done: without it, the lists go stale and the system rots. It's a plausible claim, and almost untested (~50%). The closest evidence is about reflection on learning. In a study of call-centre trainees at Wipro in India, those who spent 15 minutes at the end of each day writing about what they had learned scored about 23% higher on the final test (Di Stefano et al. 2014). That supports the habit of looking back more than Allen's specific ritual.
A review has a hidden enemy: memory. Kahneman's research on the "remembering self" found that people judge a past experience mostly by its most intense moment and its end, and barely by its length ~85 %. In a trial with 682 colonoscopy patients, a gentler ending made the whole procedure remembered as less unpleasant. A week reviewed from memory is a week reviewed by its worst meeting and its Friday afternoon. Written notes from the week itself are the antidote. So is the reverse use: how a work session ends shapes whether you want to return to it, which is a decent argument for stopping on something that went well.
The 37× problem
Atomic Habits (Clear 2018) offers the most quoted number in the genre: get 1% better every day and after a year you'll be about 37 times better. The arithmetic is correct: 1.01 to the power of 365 is about 37.8, and 0.99 to the power of 365 is about 0.03 (~95%). As a description of how people actually improve, it is wrong (~15%).
Real learning curves have the opposite shape. The power law of practice (Newell & Rosenbloom 1981) describes large gains early and shrinking returns later. Your first week of learning to type, cook or write code brings more improvement than your hundredth. Clear's related idea, the "plateau of latent potential", in which results stay invisible for a long time and then break through, has the curve backwards for most skills (~30%). Some outcomes have thresholds, but there is no general law of hidden progress.
The book gets one thing right: small, repeated improvements add up. They add up the way practice does, quickly at first and then slowly, not the way a savings account does. The practical danger of the 37× story is in how you read your own feedback. If you expect steady compounding, a slowdown in month three looks like failure. It's usually just the normal shape of learning.
What doesn't work
- "1% better every day = 37× in a year." The maths holds; real progress follows a curve of diminishing returns (Newell & Rosenbloom 1981). (~15% as a description of progress)
- The "plateau of latent potential." Skill gains come early, not after a long invisible wait. (~30%)
- Judging a decision by how it turned out. Outcomes are noisy; people still do it, even when they say they shouldn't (Aiyer et al. 2023). ~85 %
- Feeling productive as a measure. Developers using early-2025 AI tools felt about 20% faster while taking 19% longer (METR 2025; see Leverage). ~85 %
- Single activity metrics. Lines of code, hours and utilisation get gamed and pull effort away from what isn't counted (Holmström & Milgrom 1991). (~85%)
Try this
- Write down what the week produced, not what it consumed. Two or three results, in a sentence each, every Friday.
- Pair every number you track with its rival: output with rework, speed with errors, pages written with pages kept.
- Don't change your system after one great or terrible week. Look at four to eight weeks before deciding something works or doesn't.
- Record progress where you can see it, on paper or a shared board, rather than trusting your memory of the week.
- When you review a decision, grade the process separately from the result: what did you know, what did you weigh, what would you do again with the same information?
7 claims checked
The evidence behind this article
Every claim above, rated by the chance it holds up against the research. Open one for the sources and their exact words.
- Aiyer, Kam, Ng, Young, Shi, Feldman (2023). Outcomes Affect Evaluations of Decision Quality: Replication and Extensions of Baron and Hershey’s (1988) Outcome Bias Experiment 1 · Replication
- Allen (2001). Getting Things Done · Book
- Baron, Hershey (1988). Outcome bias in decision evaluation. · Study
- Choudhry, Fletcher, Soumerai (2005). Systematic Review: The Relationship between Clinical Experience and Quality of Health Care · Meta-analysis
- Clear (2018). Atomic Habits · Book
- Di Stefano, Gino, Pisano, Staats (2014). Making Experience Count: The Role of Reflection in Individual Learning · Study
- Ericsson, Pool (2016). Peak · Book
- Forsgren, Humble, Kim (2018). Accelerate · Book
- Forsgren, Storey, Maddila, Zimmermann, Houck, Butler (2021). The SPACE of Developer Productivity · Source
- Grove (1983). High Output Management · Book
- Harkin, Webb, Chang, Prestwich, Conner, Kellar, Benn, Sheeran (2016). Does monitoring goal progress promote goal attainment? A meta-analysis of the experimental evidence. · Meta-analysis
- Holmström, Milgrom (1991). Multitask Principal-Agent Analyses: Incentive Contracts, Asset Ownership, and Job Design · Study
- Joel, Rush, Barnes, David (2025). Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity · Study
- Kahneman (2011). Thinking, Fast and Slow · Book
- Kahneman, Klein (2009). Conditions for intuitive expertise: A failure to disagree. · Source
- Kluger, DeNisi (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory · Meta-analysis
- Newell, Rosenbloom (1981). Mechanisms of skill acquisition and the law of practice · Study
- Redelmeier, Katz, Kahneman (2003). Memories of colonoscopy: a randomized trial · Study