Better Product Decisions Need Bayesian Thinking
The analysis placebo is a comforting lie—the feeling of control we get from a dashboard full of vanity metrics that don’t actually change a single decision. To move from being a "mercenary" building features to solving problems, we need a formal mechanism for changing our minds: Bayesian inference.
Bayes is fundamentally about epistemic mastery—the ability to quantify what we don’t know and update that belief as high-integrity signals arrive. In the Bayesian world, probability is not a fixed property of the universe, but an expression of our own personal state of ignorance.
Step 1: Establish the Prior (The Outside View)
High-integrity product discovery begins by establishing a Prior—our baseline belief before we see any data.
If you were absolutely truthful with your organisation’s success rate, you might say that 40% of your investments (HTMA in project management, Hubbard) yield the benefits within the budget constraints you set out to accomplish. The remaining were either failures or generated modest underwhelming results.
This 40% is your skeptical starting position, your clear-eyed estimate of success without the hopeful thinking.
Step 2: Identify the Signal (The Test)
Every discovery activity is a "test," not the event itself. To update your prior, you must identify a signal with a strong likelihood ratio—asking:
"How much more likely am I to see this evidence if I’m succeeding versus if I’m failing?".
You define decision-driven metrics (like a stable problem definition) rather than tracking clicks or downloads. My favourite is whether or not a team can ship a small slice of something to a production environment within a timebox. Whilst this is evidence of delivery capability, a balanced set of signals also needs something to show signs that a user wants what you have.
Step 3: Update the Posterior (Small Data Mastery)
You do not need "Big Data" to move the needle; you need Small Data Mastery. By applying the Rule of Five, you can achieve significant uncertainty reduction with minimal effort.
The Update: Let’s say you conduct just five problem interviews. If all five potential users consistently confirm they face the exact pain point your idea solves, you have a powerful signal. You might estimate the likelihood of seeing results like this strongly connected to a successful outcome. This evidence "out-shouts" your initial 40% skepticism, causing your Posterior credence—your updated belief—to leap toward a high-integrity commitment.
Why This Sharpens Focus
By adopting the shared language of probability and ranges, we create a single hierarchy that aligns the organisation top to bottom. When stakeholders are on the same page regarding risk, we stop over-measuring easy variables (like exact dev costs) and focus on the "swing variables" (like adoption rates or delivery capability) that actually determine the impact of an idea.
Uncertainty is not an excuse for indecision; it is a variable we can price. Use the math of Bayes to ensure every dollar spent on discovery is an investment in decision integrity.
Try it yourself using my online calculator: https://mkzer0.github.io/bayes/