Dear CIO,

This article is a thought exercise based on a discussion of the work my friend Elliott Clark has been doing. 

Counterfactual Regret Minimization (CFR) is a framework that is used to solve imperfect-information games like poker. It uses regret matching and self-play to compute a Nash equilibrium (More on this later). This framework originally comes from game theory and computer science, not from organizational development, but I am interested in translating this into organizational theory. Now, I am not going to suggest that companies can simply import a poker solver's mathematics and use it as a management system, as organizations are too messy, too political, and too difficult to simulate for that kind of direct translation. I am intrigued by the underlying idea, though.

Could organizations learn more effectively from failure if they treated counterfactuals not as stories about what would definitely have happened, but rather as probabilistic estimates of what might have happened?

Best Regards,
John, Your Enterprise AI Advisor

A Thought Exercise on Counterfactual Regret and Organizational Learning

Counterfactual thinking has a bad reputation in organizations, and often for good reason. It is the language of “if only.” If only we had delayed the launch. If only we had chosen the other vendor. If only leadership had listened earlier. If only… you get the picture. When it is used this way, counterfactuals can become harmful because they encourage rumination, hindsight bias, blame, and the comforting illusion that the path not taken would certainly have produced a better result. After a failure, everyone magically seems to know what should have been done.

However, what if counterfactual thinking itself is not the problem? Perhaps the problem is how casually and confidently we use it. A counterfactual becomes dangerous when it is treated as fact. Saying, “If we had delayed the launch, the failure would not have happened,” sounds reasonable, but it claims knowledge we do not possess. We cannot visit the alternate history in which the launch was delayed. In fact, we have no clue about what new problems, costs, or opportunities might have emerged.

Because of this, I believe a counterfactual could become more useful when it is treated as a probability. Instead of saying that delaying the launch would have prevented failure, we might say that, based on the available evidence, delaying the launch could have reduced the probability of a major incident from 40 percent to 15 percent. At the same time, it might have increased the probability of missing the market window from 10 percent to 35 percent. With this framing, a counterfactual is no longer regret disguised as certainty, but rather an attempt to learn. This distinction matters. Counterfactual thinking should not be used to rewrite history. It should be used to improve the probability of making a better decision when a similar situation appears again.

From Human Regret to Mathematical Regret

An interesting model for this comes from poker-playing artificial intelligence. Counterfactual Regret Minimization (CFR) is an algorithm for learning strategies in games where players must make repeated decisions without possessing all the relevant information. It was introduced in 2007 by Martin Zinkevich, Michael Johanson, Michael Bowling, and Carmelo Piccione, with their research showing that a system could move toward a balanced and difficult-to-exploit strategy by calculating regret at individual decision points.

Poker acts as an ideal environment for this problem because players never have complete information. You know your own cards, but you do not know your opponent’s cards. At each play, you must decide whether to fold, call, or raise while reasoning from probabilities rather than certainty. CFR repeatedly simulates the game. At each decision point, it compares the action selected with the actions that could have been selected. It asks how much better or worse each alternative might have performed, and the difference is recorded as regret.

Actions that repeatedly appear better than the current choice accumulate positive regret and become more likely to be selected in future rounds. Similarly, actions that consistently perform poorly become less likely, and over many iterations, the strategy becomes more balanced and less exploitable. The poker system described in The Range Reader uses this same fundamental idea. It breaks the game into small decisions, measures the regret associated with the actions not taken, and adjusts future play in proportion to that regret.

It is important to note that this is not regret in the emotional sense. The computer does not feel bad about calling when it should have folded. In these models, regret is the difference between the expected value of the chosen action and the estimated value of an alternative. That distinction may offer something useful to organizations.

Organizations Also Operate With Hidden Information

Organizational decisions resemble poker decisions more than most leaders would probably like to admit. Leaders rarely know how customers will respond, whether a deadline is realistic, which technical dependency will fail, or whether a new technology will create more value than disruption. Every day, organizations make decisions under incomplete information, yet they often evaluate those decisions as though the outcome should have been obvious from the beginning.

For instance, in most organizations, when a project succeeds, the original decision is celebrated as wise, and when the project fails, the same decision is treated as foolish. This is outcome bias, and it confuses the quality of the result with the quality of the decision. A good decision can produce a bad result because of uncertainty, and a poor decision can produce a good result because of luck. Here is where CFR can change things. A CFR-inspired approach would evaluate a decision using the information available at the time. It would then compare the chosen action with the genuinely available alternatives with the goal of updating the organization’s future decision strategy.

A Product Launch That Failed

Imagine that a software company launches a major product update, and shortly after the launch, the organization experiences several serious production incidents. Customer support requests increase dramatically, engineering teams become exhausted, and an important customer leaves. In a traditional postmortem, leaders might produce a straightforward conclusion: “We should have delayed the release.”

While that conclusion may be understandable, it contains an unsupported counterfactual. No one can know with certainty what would have happened if the release had been delayed. The additional testing might have found the defects, but it might also have missed them. The delay might also have reduced customer impact, but it could have caused the company to lose an important market opportunity.

A probabilistic counterfactual review would begin by reconstructing the actual decision point. Suppose the organization had four realistic options. It could launch the complete release on schedule, delay the complete release for four weeks, launch a reduced version on schedule, or release the full version to 10 percent of customers and expand gradually. The organization could use prior release data, defect rates, deployment history, customer research, operational capacity, and expert judgment to estimate the likely outcomes of each option.

The estimates might look something like this:

Again, these numbers are not facts about an alternate universe. They are estimates, and they should be accompanied by confidence ranges, assumptions, and an explanation of the evidence used to produce them. The purpose of this probabilistic counterfactual review is to recognize that, given the available evidence, it appears to have offered a more favorable distribution of possible outcomes. The organization’s greatest regret might not be that it launched too early, but rather that it treated the launch date and the launch scope as a single decision. That is a much more useful lesson.

The organization can now update its future operating policy. When defect uncertainty is high, it can increase the probability of choosing a gradual rollout. When market timing is critical, it can favor reducing scope rather than delaying the entire release. When rollback capability is weak, it can limit customer exposure until the system has been tested in production. This resembles regret matching. The organization is increasing the likelihood of a gradual release under conditions where that choice appears to outperform the alternatives.

Why the Mathematics Matters

Most organizations already conduct retrospectives, postmortems, and after-action reviews. The problem is that these discussions are often dominated by narrative. I will admit that narratives are powerful, but they are also easy to manipulate. In organizations, the most senior person may tell the most convincing story, and participants may unconsciously defend their earlier positions. In these narratives, the final outcome may make one alternative appear inevitable.

Probability forces a little more discipline. Instead of saying, “The launch was obviously too risky,” the organization must ask, “What did we believe the probability of failure was before the launch?” It can then compare those earlier estimates with what it knows now. Perhaps leadership believed the probability of a major incident was 5 percent, while the engineers closest to the work believed it was closer to 35 percent. The failure may therefore reveal more than a technical problem. It may reveal a problem in the way information moves through the organization.

Instead of saying, “A limited rollout would have prevented the incident,” the organization might conclude, “A limited rollout probably would not have prevented the defect, but it likely would have reduced the number of affected customers by 80 to 95 percent.” That is a more honest and useful counterfactual.

Mathematical language does not make the estimates objectively true. Organizations can still attach false precision to weak assumptions. The value, though, comes from making those assumptions visible, challengeable, and revisable. A useful counterfactual should identify the alternatives that were actually available. It should estimate the probability of the most important outcomes. It should explain the evidence behind those estimates and express how confident the organization is in them.

Regret Without Blame

The most important lesson from CFR may be that regret does not require blame. In a computer algorithm, regret is information. It identifies an action whose probability may need to increase in future play. Organizations could treat regret in the same way, asking, “Who made the wrong decision?” However, the better question would be, “What features of our system made this decision more likely than the alternatives?”

Perhaps the team lacked the authority to reduce scope or that incentives rewarded hitting the announced date more than protecting customers. Perhaps risk information was available but fragmented across departments, or instead leadership interpreted confidence as competence and uncertainty as weakness. The individual decision may have been completely understandable within the system that produced it.

A mature organization can use counterfactuals not to punish the decision maker, but to improve the decision environment. That means improving how options are generated, how uncertainty is communicated, how probabilities are estimated, and how reversible decisions are distinguished from irreversible ones.

Avoiding the Hindsight Trap

There is still a major limitation. Poker algorithms can simulate similar situations millions of times, but organizations cannot rerun an acquisition, restructuring, product launch, or strategic transformation under identical conditions. Organizational counterfactuals will always contain uncertainty. For that reason, the goal should not be to calculate one definitive answer. The goal should be to describe a range of plausible outcomes.

Instead of saying, “Option B would have saved the project,” the organization might say, “Our current evidence suggests that Option B had a 60 to 75 percent chance of reducing the delay. Our confidence is low because we have only two comparable projects.” This prevents mathematics from becoming theater. It also creates an opportunity for experimentation. The next time a comparable decision appears, the organization can test Option B on a smaller scale, and the new evidence can then update the original probability. The process becomes straightforward: decision, outcome, counterfactual comparison, probability update, and next decision.

From Lessons Learned to Strategies Updated

Organizations often end postmortems with a list titled “Lessons Learned.” Unfortunately, many of those lessons never produce changed behavior. The organization records what happened, but it does not alter the probability of what it will do next. Counterfactual Regret Minimization suggests a stronger standard. Here, a lesson has not been learned until it changes the strategy, and if an organization repeatedly concludes that smaller releases would have reduced risk but continues rewarding large launches, it has accumulated regret without minimizing it.

If teams repeatedly discover that earlier customer feedback would have changed the product, but funding still requires a complete annual plan, the organization has documented the alternative without making it more likely. The organizational adaptation of CFR is therefore more than another retrospective technique. It asks whether learning has actually changed future decision probabilities.

The Roads Not Taken

Ultimately, I do think that counterfactuals deserve some of their bad reputation. They can trap people in the past, produce blame, create false certainty, and generate stories in which every failure appears avoidable. They can also encourage leaders to judge yesterday’s decisions using today’s information. That said, abandoning counterfactual thinking would also mean abandoning one of our most important mechanisms for learning.

We learn not only by observing what happened. We also learn by carefully comparing it with what might plausibly have happened under a different choice. The key is to replace certainty with probability. The right question is not, “What would have happened if we had chosen differently?”

The more honest question is, “Given what we knew then and what we know now, how should this experience change the probability of the choices we make next?”