How to Make Evidence-Based Decisions

Managers ask "What's the probability this closes?" when they SHOULD ask "What evidence do we have that this buyer can actually purchase?" Those are completely different questions. One is guessing. The other is qualifying. The good ol’ pipeline review sounds like this: "How's the Microsoft deal looking?" "Great, had a good demo last week. They're really interested. I'd say 70% to close." Based on what? A feeling? Their enthusiasm level? The number of follow-up questions they asked? Folks should really start structuring evidence-based pipeline reviews. Try examining four things: 1. Purchasing authority evidence: - Has this person bought software like this before? - What was the process and timeline for their last similar purchase? - Who else was involved in that decision? - What budget threshold requires additional approvals? 2. Internal alignment evidence: - Have they shared our proposal with their team? - What feedback came back from those conversations? - Who asked questions and what were they? - Has anyone expressed concerns or objections? 3. Urgency evidence: - What specific business event is driving this timeline? - What happens if they don't solve this problem by their stated deadline? - Is this tied to budget cycles, project launches, or compliance requirements? - Who gets in trouble if this doesn't happen? 4. Decision-making process evidence: - What steps remain in their evaluation process? - Who reviews contracts and how long does that typically take? - What other vendors are they considering and why? - When do they need to make a final decision and why? Think about it. A traditional pipeline review sounds like: "They love the product and want to move forward. The champion is pushing internally. Should close next month." Meanwhile, an evidence-based pipeline review sounds like: "Champion confirmed budget approval authority up to $100K. Shared proposal with IT director who asked about security protocols - we're scheduling that call Wednesday. Legal review typically takes 2 weeks. Final decision needed by month-end to start implementation before Q1 planning cycle." Which sounds better to you? Exactly. :) Most sales forecasts fail because they're built on enthusiasm metrics instead of buying evidence. Prospects can be excited about your solution and still never purchase it. Start changing up your questions: - Instead of "How confident are you?" ask "What evidence do we have?" - Instead of "What's the close probability?" ask "What could prevent this from happening?" - Instead of "When will they decide?" ask "What specific event triggers their decision?" Your forecast accuracy will improve when you stop predicting outcomes and start evaluating evidence.

Stop having opinions. Start having hypotheses. Everyone in business has opinions. But the best executives work with hypotheses. An opinion is a point of view, sometimes even backed by data. “We’re losing customers because we’re more expensive.” It sounds logical. You check the numbers, and yes, you are more expensive. But that doesn’t prove that price is the reason they leave. That’s just a correlation. A hypothesis, on the other hand, is testable. “If price is really driving churn, then the customers who pay the highest premium should be the ones leaving fastest.” See the difference? The opinion describes the symptom. The hypothesis seeks the cause and tells you what data will confirm or kill your assumption. That’s what makes hypothesis thinking so powerful: It forces you to move to test. From “I think” to “let’s check.” From debating opinions to discovering truth. Here’s how to apply it like a master 1. Start every discussion with an “if–then.” “If X is true, then we should observe Y.” It makes your thinking structured and measurable. 2. Define what would make you change your mind. Don’t just say “we’ll look at data.” Be specific about what evidence would disprove your idea. 3. Refine fast. Good consultants don’t cling to their first hypothesis. They update it every time new facts appear. In short: Opinions sound smart. Hypotheses make you smarter.

One of the ways that Jeff and the S-Team instilled operational excellence at Amazon was through disciplined, data-based decision-making. Most CXOs don't have a method to ensure their organizations make high-quality decisions. Below is my take on a set of principles and processes to operationalize good decision-making: 1. Timely - "Last Responsible Moment" (LRM): The concept of LRM emphasizes understanding the latest date by which each decision must be made to keep a project on track.  Early decisions can lead to mistakes, and late decisions make it harder to meet operating goals.  Forcing the organization to determine the last responsible moment improves its understanding of the decision, and it also spreads out the time between decisions. 2. Differentiated - One-Way vs. Two-Way Doors: The idea is simple: a two-way door decision is one where, if you walk through the door and don’t like what you see, you simply turn around and go back through.  Two-way door decisions are reversible and can be made quickly without extensive analysis, enabling greater operational agility.  One-way door decisions, on the other hand, are either irreversible or very expensive to reverse.  These should be made slowly and with great care. 3. Informed Truth Seeking: Decisions should be made after a period of dedicated data gathering, analysis, and truth seeking supported by a clear and concise business narrative. High-quality analysis includes objectively exploring multiple courses of action and recommendations based on costs and benefits. 4. Debate: In the words of Peter Drucker, a decision is a judgement, not a choice between right and wrong.  To understand an issue, a robust debate between high-judgement leaders offering different viewpoints is required.  Corporate cultures that encourage open, data-based debate excel at this. 5. Consistent Forum: Decisions of consequence (one-way doors) should be made in the consistent forum. At Amazon, this meant reading a narrative at a meeting with Jeff and the S-team. The decision(s) would be made in the meeting with all of the relevant people present. The decision wouldn't be reversed by a subsequent conversation with the CEO. 6. Detailed:  The details of any decision matter a lot. The documentation used to make a decision should include all relevant implications and details: costs, personnel, timeline, and detailed features. This enables alignment with the CEO and allows teams to move fast once a decision has been made. 7. Experienced Leaders:  The only way to get good at decisions is to make lots of them and to be held responsible for the consequences.  We all learn more from mistakes than from success.  This requires an organizational structure and culture of ownership (not an ambiguous matrix), as well as a willingness to fail. Leaders – what are your thoughts on my list? What would you edit, add, or subtract??

When a new product or feature launches, one of the toughest questions is: Is it actually working? In a recent post from Meta’s analytics team, the team introduces the “ladder of evidence”—a simple way to think about how confident we can be when measuring product effectiveness. It’s a helpful reminder that not all insights carry the same weight, especially when real decisions and resources are involved. - The core idea is that evidence comes in different levels of rigor. At the bottom of the ladder are observational signals—trends, correlations, and before-and-after comparisons. These are easy to access and often directionally useful, but they can be misleading because they don’t establish causality. As you move up the ladder, methods become more robust. Quasi-experiments and A/B tests help reduce bias and get closer to true cause-and-effect. The tradeoff is that these approaches are more complex, time-consuming, and not always practical. - What’s especially insightful is that this isn’t framed as “good” vs. “bad” analysis. It’s about knowing where you are on the ladder and making decisions accordingly. Sometimes quick directional insight is enough; other times—especially for high-stakes decisions—you need stronger evidence. The “ladder of evidence” is a powerful mental model for anyone working with data. It pushes us to be clear about how strong our conclusions really are—and to stay humble about their limits. In practice, it’s about matching the level of evidence to the importance of the decision. Because in data science, the goal isn’t just to measure impact—it’s to measure it with the right level of confidence to make the right decision. #DataScience #DecisionMaking #ABTesting #CausalInference #SnacksWeeklyonDataScience – – –  Check out the "Snacks Weekly on Data Science" podcast and subscribe, where I explain in more detail the concepts discussed in this and future posts:    -- Spotify: https://lnkd.in/gKgaMvbh   -- Apple Podcast: https://lnkd.in/gFYvfB8V    -- Youtube: https://lnkd.in/gcwPeBmR https://lnkd.in/giKyY_eq

Safety Innovation Advent: Day 6 - Practice research to inform your practice In the lead-up to Christmas, I’m sharing an insight, activity or practical tip each day to help you innovate in health and safety. Today’s tip: Research, critique, and summarize the evidence for your health and safety processes Today’s tip is inspired by Punk Rock Safety Episode 19, where Ben Goodheart, Ph.D. David Provan, and Ron Gantt remind us of the importance of researching the evidence for our practice. As health and safety professionals, we should strive to be evidence-based practitioners, grounding our work in solid research. Yet, when working with cutting-edge innovation; where evidence is often lean (or not very good), we also need to experiment, conduct ethnographic research, and creatively draw on existing knowledge to drive progress. How to start: 1️⃣ Identify a safety challenge: Pick a topic relevant to your work, such as: - The effectiveness of Take 5 risk assessments. - Best practices for incident investigation methodologies. - Improving learning and engagement in e-learning packages. Choose our own 2️⃣ Search for evidence Go to Google Scholar or a trusted research database and explore recent studies or reviews. 3️⃣ Critique the research Evaluate the studies critically: Are the methods robust? Look at sample size, methodology, and validity. Are the conclusions practical? Assess how they apply to your field. What are the limitations? Note gaps in evidence or research contexts. 4️⃣ Summarise your findings Write a brief summary of your learnings, focusing on: - Key takeaways and practical applications. - Gaps in research or opportunities for experimentation. - How you might use these insights to inform or adjust your approach? Why this matters for safety innovation: Research helps us uncover trends, patterns, and gaps, driving smarter decision-making. By practicing research, we strengthen our ability to innovate while staying grounded in evidence. I’m currently exploring the state of the science for fatigue management and how wearable technology can help us better understand sleep and fatigue, blending research with experimentation to guide solutions. 🔧 Pro Tip: Share your findings with your team and the broader safety community. By sharing what you’ve learned, we can build our collective knowledge base and foster innovation in health and safety, together. Stay tuned for more practical tips in this series by following my profile and the hashtags #SafetyInnovationAdvent #SafetyInnovation #SafetyTech.

Decision-making can be daunting. Strategy fundamentally involves making choices—determining where to focus, what trade-offs to accept, and how to allocate resources to achieve desired outcomes. The brutal fact: We don’t always have all the answers, but decisions can’t wait until we fully understand every detail. One of the decision-making frameworks I keep in my pocket - 𝐂𝐲𝐧𝐞𝐟𝐢𝐧 𝐅𝐫𝐚𝐦𝐞𝐰𝐨𝐫𝐤 It is based on the nature of the situation (complexity) and the level of predictability. 𝟏. 𝐊𝐧𝐨𝐰𝐧 𝐊𝐧𝐨𝐰𝐧𝐬: 𝐓𝐡𝐞 𝐏𝐫𝐞𝐝𝐢𝐜𝐭𝐚𝐛𝐥𝐞 ➡️ What it means: You know what's happening and understand it fully. ➡️ Example: A customer support team spends hours every day categorizing and assigning incoming tickets manually, following a consistent set of rules. ➡️ How to decide: 1. Sense: Understand the facts of the situation. 2. Categorize: Match it to a known framework or pattern. 3. Respond: Apply a straightforward solution, as the answer is often obvious. 🔔 Key Tip: Stick to tried-and-true methods for efficiency and consistency. 𝟐. 𝐊𝐧𝐨𝐰𝐧 𝐔𝐧𝐤𝐧𝐨𝐰𝐧𝐬: 𝐓𝐡𝐞 𝐁𝐥𝐢𝐧𝐝 𝐒𝐩𝐨𝐭 ➡️ What it means: You know there’s a problem but don’t fully understand it yet. ➡️ Example: User churn rates are high, but the reasons behind it are unclear. Analytics show patterns, but they don’t provide definitive insights into why users are leaving. ➡️ How to decide: 1. Sense: Gather all relevant data and inputs. 2. Analyze: Use expert opinions, tools, or detailed studies. 3. Respond: Choose the best course of action from multiple viable options. 🔔 Key Tip: Don’t rush. Use analysis and expertise to guide decisions. 𝟑. 𝐔𝐧𝐤𝐧𝐨𝐰𝐧 𝐊𝐧𝐨𝐰𝐧𝐬: 𝐓𝐡𝐞 𝐔𝐧𝐜𝐞𝐫𝐭𝐚𝐢𝐧𝐭𝐲 ➡️ What it means: There are things you don't realize but could understand if you investigated. Patterns exist but are not obvious upfront. ➡️ Example: Discovering unconscious biases affecting hiring dynamics. ➡️ How to decide: 1. Probe: Conduct safe-to-fail experiments to uncover hidden factors. 2. Sense: Observe the results to identify emerging patterns. 3. Respond: Adapt based on the insights gained. 🔔 Key Tip: Be open to exploration and learning; flexibility is crucial. 𝟒. 𝐔𝐧𝐤𝐧𝐨𝐰𝐧 𝐔𝐧𝐤𝐧𝐨𝐰𝐧𝐬: 𝐓𝐡𝐞 𝐔𝐧𝐞𝐱𝐩𝐞𝐜𝐭𝐞𝐝 𝐂𝐡𝐚𝐨𝐬 ➡️ What it means: You’re blindsided by events you couldn’t predict or prepare for. ➡️ Example: A sudden industry-disrupting technology or a global crisis like COVID-19. ➡️ How to decide: 1. Act: Take decisive steps to establish stability like emergency measures. 2. Sense: Identify areas of order or stability amid the chaos. 3. Respond: Gradually transition to a more manageable situation by creating structure. 🔔 Key Tip: Speed is critical; act first, then refine your approach. And when faced with 𝐜𝐨𝐧𝐟𝐮𝐬𝐢𝐨𝐧 (𝐝𝐢𝐬𝐨𝐫𝐝𝐞𝐫)—a completely unclear state— break it down into smaller parts and assign each to its appropriate category for clarity. Have you used this framework? #productmanagement #strategy

Bayes’ Theorem is a powerful mathematical framework for learning from data, whether in machine learning or everyday decision-making. It provides a systematic method to update beliefs and improve predictions as new evidence becomes available. By treating probabilities as measures of certainty, it helps refine our understanding over time based on the information we receive. For instance, suppose the weather forecast predicts a 30% chance of rain (your initial belief). Stepping outside, you notice dark clouds (new evidence). Incorporating this information, you revise the likelihood of rain to 70%, resulting in a more accurate prediction. This process of updating beliefs based on new evidence is central to Bayes’ Theorem and enables better decision-making. Here’s how you can apply Bayesian thinking in daily life: 1. Avoid Availability Bias: We tend to overemphasize recent or easily accessible information, neglecting older, potentially more relevant data. Bayesian thinking helps balance new evidence with existing knowledge to avoid skewed conclusions. 2. Focus on Differentiating Information: Not all new data is equally valuable. If evidence supports multiple hypotheses equally, it adds little insight. We should prioritize data that helps distinguish between competing possibilities. 3. Recognize Costly Signals: Costly signals are actions or behaviors that convey valuable information but require a significant investment of time, effort, or money. These signals are more trustworthy because only those who possess the desired quality are likely to pay the cost. By identifying and evaluating these signals, we can better assess credibility and make more informed decisions in situations where trust and accuracy are crucial. By adopting these principles, we can better process information, update beliefs, and make more informed decisions in all areas of life.