Dear CIO,

I keep coming back to Mark Schwartz's The Art of Business Value as I watch organizations navigate the current wave of AI adoption. Schwartz was writing about Agile, software delivery, and the relationship between technology teams and business leaders, but many of the ideas in that book feel even more relevant today than they did when it was first published. If anything, the rush to adopt AI has exposed how persistent some of our old management habits really are. We continue to search for simple metrics that can tell us whether we are succeeding. We continue to build dashboards that create the feeling of control. And we continue to believe that if something can be measured, it can be managed.

Best Regards,
John, Your Enterprise AI Advisor

Why AI success is about value, not usage

I keep coming back to Mark Schwartz's The Art of Business Value as I watch organizations navigate the current wave of AI adoption. Schwartz was writing about Agile, software delivery, and the relationship between technology teams and business leaders, but many of the ideas in that book feel even more relevant today than they did when it was first published. If anything, the rush to adopt AI has exposed how persistent some of our old management habits really are. We continue to search for simple metrics that can tell us whether we are succeeding. We continue to build dashboards that create the feeling of control. And we continue to believe that if something can be measured, it can be managed.

The metrics themselves have changed over time, but the pattern has not. During the Agile era, organizations became fascinated with velocity, story points, feature counts, and ROI calculations. Today, the conversation revolves around token consumption, prompt volume, active users, AI-enabled workflows, and adoption percentages. Different era, different vocabulary, same underlying temptation. We are still trying to reduce complex questions about value, learning, and organizational effectiveness to numbers that fit neatly on an executive dashboard.

That is why Jensen Huang's recent comment about a $500,000-a-year employee spending $250,000 in AI tokens is so interesting. There is an important idea behind the statement. Highly skilled knowledge workers should be using powerful tools to amplify their capabilities. AI can help people evaluate more alternatives, automate routine work, test ideas faster, and spend more time on higher-value decisions. Used well, these tools can dramatically expand what an individual or team can accomplish.

The problem begins when that observation gets translated into a measurement system. It is a short step from recognizing that AI can increase effectiveness to assuming that more AI usage automatically creates more value. History suggests otherwise. Eliyahu Goldratt captured the problem decades ago when he said, "Tell me how you measure me, and I will tell you how I will behave." Once organizations begin treating token consumption as evidence of success, people inevitably start optimizing for token consumption. If dashboards reward usage, usage will increase. If leaderboards rank teams by AI activity, teams will generate more activity. Employees quickly learn how to look productive according to the metric, whether or not they are actually improving outcomes.

We've seen this dynamic play out repeatedly. When organizations measured lines of code, they got more lines of code. When they measured story points, they often got story point inflation. When collaboration became a proxy for effectiveness, calendars filled with meetings. AI is not exempt from the same forces. If we choose token consumption as the primary indicator of progress, we should not be surprised when organizations generate more tokens. What we should be asking is whether those tokens are helping people make better decisions, reduce waste, improve customer outcomes, or create new capabilities. The metric alone cannot tell us that.

This is where Schwartz's argument becomes especially important. One of the central ideas in The Art of Business Value is that business value is not a fixed quantity waiting to be discovered through accounting. It is not a universal metric that can be calculated once and optimized forever. Rather, value is something organizations learn about through experimentation, feedback, and adaptation. Leaders form hypotheses about what will help the organization achieve its goals, and teams test those hypotheses through delivery and learning. Over time, the organization's understanding of value evolves.

That perspective feels particularly relevant in the age of AI because so many organizations are approaching adoption as a deployment problem rather than a learning problem. The first question often becomes, "How do we get everyone using AI?" A better set of questions might be: What outcomes are we trying to improve? Where are our bottlenecks? Which decisions are taking too long? What work creates unnecessary friction? Where are feedback loops slow or ineffective? What new capabilities become possible if we use AI effectively? These questions move the conversation away from technology usage and toward business outcomes.

None of this means adoption metrics are useless. Organizations should absolutely track usage, spending, experimentation, and demand. Those indicators can reveal where learning is taking place, where additional training may be needed, and where governance risks might emerge. The mistake is treating those indicators as proof that transformation is occurring. A dashboard can tell us that people are using AI. It cannot tell us whether the organization is becoming better at creating value.

That distinction becomes even more important when viewed through the J Curve of AI adoption. Real transformation rarely follows a smooth upward trajectory. Early in the process, costs often rise before benefits appear. Teams spend time learning new tools, redesigning workflows, building guardrails, and discovering where AI helps and where it does not. Productivity may temporarily decline. Experimentation may create visible expense before it creates visible value. To a leader focused exclusively on dashboard metrics, that period can look like failure. In reality, it may simply be the cost of learning.

The challenge is that dashboards are often very good at measuring activity and very poor at measuring learning. They can tell us how many prompts were submitted or how many tokens were consumed. They cannot easily tell us whether teams are making better decisions, shortening feedback loops, reducing operational risk, or discovering more effective ways of working. Those outcomes emerge over time and often require judgment rather than measurement alone.

This is also where I think organizations are repeating one of the mistakes Schwartz criticized in traditional IT management. The old model treated business leaders as the owners of value and technology teams as delivery organizations. Agile and DevOps challenged that assumption by encouraging technology teams to become active participants in value discovery. Yet many AI initiatives seem to be recreating the same dynamic. Executives declare AI a strategic priority. Centralized teams purchase tools and establish adoption goals. Dashboards track compliance. Employees are instructed to use AI and demonstrate productivity gains. Meanwhile, the people closest to the work are often excluded from defining what value actually means.

That is not a transformation. It is a modern version of the same command-and-control thinking that Agile was intended to replace. If AI is truly strategic, then teams need more than access to tools. They need context, decision-making authority, opportunities to experiment, and mechanisms for sharing what they learn. Most importantly, they need a meaningful role in defining what successful outcomes look like. AI cannot be something that happens to teams. It has to become something teams use to discover better ways of creating value.