Dear CIO,

Better specifications matter, but the real opportunity lies in improving how we develop them and how we gather the evidence needed to determine what is ready for a production pilot.

In The New Bottleneck Is Ideation, I argued that AI is exposing a part of the software value stream we have neglected for years. While we spent enormous effort improving the Commit → KaChing, we have not brought the same discipline to Aha → Commit. If we want better results from AI, we need better specifications, and I see this conversation is currently happening. It is getting us closer to that underlying problem, but I also think it stops one question too soon.

Best Regards,
John, Your Enterprise AI Advisor

Better specifications begin with better decisions

What process produces a better specification?

Where did the requirements come from? Which assumptions have been challenged? Who agreed that the proposed outcome would be valuable? What happens when two stakeholders disagree? What evidence would convince us that this idea shouldn't receive any more investment? A specification can be clear, detailed, and completely wrong about what matters. What’s worse is that with AI, these misunderstandings can then be implemented with remarkable efficiency. We should improve the spec, but we should also improve the system that creates it.

Start the analysis at the spec. Then work backward.

When an AI-generated system produces a disappointing result, it is tempting to blame the prompt, the model, or the specification. Sometimes, though, the specification is where an earlier problem finally becomes visible. An ambiguous requirement might reflect a missing stakeholder conversation. A contradictory requirement might reflect an unresolved policy disagreement. A requirement that changes during implementation might reflect new learning or information that was available all along but never reached the team. These are different problems, and a better template will not solve all of them.

Specifications may be a starting point for bottleneck analysis, not because they're always the bottleneck, but because they connect what the organization believes to what the system is expected to do. Take a requirement and trace it backward. What service need does it serve? What observation supports it? Who supplied that observation? Is it an established constraint, a stakeholder preference, or an untested hypothesis? Where did it wait before becoming actionable?

Then trace it forward. What behavior will implement it? What will verify that behavior? What evidence will show whether the result actually improves service and creates value? Who will decide whether that evidence warrants a production pilot? This moves us one step further, by examining a value stream rather than editing a document, and this distinction matters. The spec might be blocked because its author lacks time. It might be blocked because three departments have not agreed on authority. Alternatively, it might be waiting for an experiment that would resolve a genuine unknown. While it might be tempting to prescribe faster writing for all three, they might not be the solution we need for each group. 

Requirements should come from understanding the work

There’s another approach that starts with discovery to actually understand the work before choosing what to automate. While that sounds obvious, it is also easy to bypass when a working prototype is only a prompt away. Discovery should capture more than a list of requested features. It should preserve the stories of how work actually happens: considering who makes a decision, what information they use, which systems they depend on, and what happens when the normal path breaks down.

Semantic Storytelling, created by Mark Burgess, provides one lens for capturing that context. Wardley Mapping provides another for connecting user needs to the capabilities that support them. Together, they help us ask whether to build, buy, reuse, or investigate a capability further. The output isn't more meeting notes, but rather a scoped opportunity, a baseline business requirements document, and an explicit backlog of what we still don't know.

Stakeholder attestation also matters here. Recording who contributed a requirement, its context, and any unresolved disagreement gives us something to inspect later. Attestation does not make a statement true, as it makes its origin and accountability visible. For example, consider an agent-assisted clinic booking workflow. “Make appointment booking faster” here is an aspiration, but not yet a useful specification. Where does the current delay occur? Is it identity verification, eligibility checking, finding an appropriate slot, or recovering from an exception? Are staff repeatedly entering information, or waiting for another system? Could improving an existing interface solve the problem without adding an agent? If we don't understand these questions, we may automate the visible activity while leaving the real constraint untouched.

Preserve the reasoning, not just the requirement

One of the most useful concepts is a Crystal Ball: a navigable record of project knowledge that persists across discovery, development, and operation. The name here matters less than the connection between evidence and decisions. A requirement should not lose its history when it moves from a workshop into a specification. We should be able to see why it exists, which policy or observation supports it, and what depends on it. When an assumption changes, that connection should identify the contracts, tests, and releases that need review.

In this case, an agent process can help organize that material by surfacing conflicting statements, identifying missing evidence, and generating alternatives for people to assess. Again, though, an inferred relationship is still a candidate relationship. A plausible explanation is not a stakeholder decision. This is where the process can produce a better spec, not by asking a model to make the document sound more complete, but by giving the team a better account of what is known and what remains uncertain. Some unknowns must be resolved before you allow consequential behavior. Others can become questions for the next experiment. The discipline is deciding which is which.

Contract-Driven Development: Making Behavior Verifiable, Not Value Certain

Contract-Driven Development (CDD) provides a disciplined approach to connecting what a system is intended to do with how its behavior will be verified. It treats a versioned, machine-readable contract as a shared reference for specification, implementation, and verification. Rather than managing requirements, development, and testing as disconnected activities, CDD establishes a common set of expectations that can be examined throughout the software lifecycle.

The contract describes the system's goals, scope, expected states, state transitions, invariants (conditions that must always hold), dependencies, and authority boundaries. Its corresponding verification record documents the claims, assumptions, checks, and acceptance evidence associated with each contract version. This creates traceability between intent, implementation, and observed outcomes. The objective, therefore, moves beyond whether software executes successfully, and to whether its behavior conforms to explicitly defined requirements and constraints.

This approach connects to formal methods and formal verification, disciplines that have existed for decades. Formal methods use mathematical representations of system behavior to reason about properties such as safety, consistency, and correctness. Techniques including model checking, theorem proving, and formal specification can establish whether particular properties hold within a defined model and its assumptions. For example, a scheduling system might specify that no two confirmed appointments may occupy the same exclusive resource and time interval. Model checking can explore possible state transitions to identify when this invariant might be violated.

However, formal verification does not universally guarantee correctness. A proof establishes a property relative to a particular formal model, assumptions, and implementation relationship. If the model omits an important real-world condition, or if the deployed implementation does not faithfully correspond to the verified design, the guarantee may not extend to actual operation. This distinction matters most when probabilistic AI components interact with deterministic business systems.

Another useful connection is Promise Theory, developed by Mark Burgess, which offers a different perspective on how distributed systems should express responsibilities and expectations. Promise Theory describes systems as collections of autonomous agents that voluntarily declare the behaviors they intend to provide. Rather than assuming a central authority can compel every component to behave correctly, it focuses on explicit promises, the agents responsible for keeping them, and the conditions under which other agents can depend on them.

For Contract-Driven Development, a contract can specify what a component is expected to do, but it cannot, by itself, guarantee that the component will fulfill that obligation. Promise Theory encourages us to identify who is making a behavioral commitment, what dependencies that commitment requires, and how other components can assess whether the promise has been kept. Formal verification and Promise Theory therefore offer complementary perspectives. Formal methods establish whether specified properties hold under defined assumptions, while Promise Theory explains responsibility, autonomy, and dependency relationships among interacting components.

Consider an AI-assisted clinic scheduling system. The requirement "do not double-book" should not remain an instruction buried in prose or an LLM prompt. It should become an explicit invariant with an identified enforcement boundary and independently observable evidence. The language model may propose an appointment, but the authoritative scheduling service must inspect the current state, enforce the relevant constraints, and either commit the reservation atomically or reject it. If the slot becomes unavailable after the agent offers it, the scheduling service must not rely on the agent's earlier understanding. Viewed through Promise Theory, the scheduling service commits to preserving booking consistency, while the AI agent acts as a requester that depends on that commitment. 

Doable, valuable, and production-ready are different questions

This method also separates proof of concept, proof of value, and pilot.  That separation matters because it prevents one attractive demonstration from answering questions it hasn't actually tested.

A proof of concept asks: Is it doable? A POC can be a virtual demonstration, outside the stakeholder environment. It explores the mechanism against known constraints. Normal behavior matters, but so do refusal and changed-state scenarios.

A proof of value asks: Does the experience meet the goal? Stakeholders operate the solution in their non-production environment. It can cover only part of the business requirements and may mock an external system where access is unavailable.

Before that trial begins, the team should agree on the current-process baseline, the desired improvement, the evidence to collect, and the person accountable for the decision. Otherwise, we risk finishing with a demonstration everyone liked and no basis for deciding whether to continue. For the sample clinic workflow, that means looking beyond whether an appointment was booked. Did representative tasks succeed? Was quality preserved? What happened to total staff effort? What did a successful outcome cost once you included review and rework? Were necessary refusals understandable, and were unnecessary blocks making the experience unusable?

A non-production trial with mocked dependencies cannot establish the cost or behavior of the complete production system. Those limitations belong beside the findings, not in a footnote nobody reads. 

A pilot is production with limited users. The approved business requirements are fully implemented, and the required business systems are connected. However, limited users do not mean incomplete functionality within that agreed scope. That step requires more than enthusiasm about the proof of value. It requires scoped authority, security and governance, operational ownership, monitoring, support, and a recovery path.

The stages should form a learning loop, not a conveyor belt. A proof of value can send us back to discovery. Likewise, a refusal can reveal a misunderstood workflow. Alternatively, a policy change can require revising a contract. Here, an idea can stop without having failed as an experiment.

Pilot allocation is an investment decision

Once building becomes easier, we can create more candidates than we can responsibly take into production. Pilot capacity is not just developer availability, as it includes integration specialists, security reviewers, operational owners, business participants, and the attention of the people who will use the system. Those resources can remain scarce even when generating code becomes cheap. That makes pilot allocation part of the bottleneck conversation. We should not allocate a production pilot simply because a team produced the most convincing demo, nor should every successful proof of concept automatically graduate to one. 

Consider two hypothetical candidates. One has an impressive demonstration but an untested value hypothesis and several unresolved integration dependencies. The other is less spectacular, but stakeholder trials show a useful improvement, its limitations are understood, and an operational owner is ready to support it.

The first might deserve another small experiment. The second might deserve the next production pilot. Treating both as “AI projects ready to advance” hides the difference in what we actually know. The allocation question isn't just, “Which idea looks most promising?” It is, “What does the evidence justify funding next?” 

For a pilot, that means examining demonstrated user value alongside expected production cost, unresolved risk, and readiness. It also means stating what the pilot will teach us that a cheaper non-production experiment cannot, and what other work we will delay by committing scarce capacity.

An important opportunity with unresolved questions should not necessarily be rejected. It may need a different next investment. Fund the learning needed to make a responsible pilot decision rather than using production as the default place to discover everything. This is a practical extension of the deck's staged approach: allocate resources to the next justified decision, not to the momentum of the project.

Measure how the organization becomes ready to decide

In my original article, I argued that we should measure the first ten inches of the ruler, not just the final two. Specification gives us a concrete place to begin observing that work, but how long does it take to turn an identified problem into a testable value hypothesis? How much of that time is active learning, and how much is waiting for access, ownership, or a decision? Which unresolved assumptions prevent a reviewable contract? How long after feasibility is established do we obtain evidence of stakeholder value?

We should also inspect why specifications change. A revision driven by a well-designed experiment shows the learning loop works, yet a revision caused by a known policy that never reached the team is a different kind of rework. Counting both as “requirements churn” would hide the opportunity for improvement, and the same care applies to pilot decisions. Stopping a project can be a good outcome if a low-cost experiment showed it didn't deserve production capacity. More pilots launched is not automatically better performance.

The goal is not a higher document count, fewer revisions, or faster approval at any cost. It is less avoidable waiting and better evidence for the decisions that matter. If specification is the recurring constraint, improve the process feeding it. If that improvement reveals that access to stakeholders or production integration is now the constraint, follow the evidence there. Bottleneck analysis should not become loyalty to a particular artifact.

Better specs are an outcome of a better learning system

The other day, during my DevSecOps for AI workshop, a student asked me a great question. He said, "You're telling us to slow down, but all of my leaders are telling me to go faster. What am I supposed to tell them?"

I told him that we're not really talking about slowing down. We're talking about being smarter about where and how we accelerate. Think about it this way. After years of DevOps improvements, delivering an end-to-end Tier 1 service might have taken three months. Now, with AI, everyone seems to think we should be able to do it in two days. 

Why does it have to be two days, though? I am not arguing for a return to six months of requirements work before anyone can build anything. Instead, what if we went from three months to one week? That's an extraordinary improvement in speed and efficiency, and we might actually produce better outcomes because we've preserved enough time to learn, validate, and understand what we're building. The third path still applies: think enough to design a credible experiment, build it quickly, learn, and repeat. Specifications and contracts should make that loop more explicit, not replace experimentation with the hope that you can know everything in advance. 

My point is: Speed is not the objective. Better outcomes are. Speed is valuable when it helps us achieve them. The opportunity is to connect the pieces. Understand the work. Preserve the evidence. Make behavior and authority explicit. Test feasibility. Observe user value. Allocate production capacity when the evidence warrants it. Better specifications help AI build what we ask for. Better contracts help us define and verify what must hold. A better learning process helps us decide whether we asked for the right thing and whether it deserves the next investment. We have spent years improving the machinery that turns a commit into a running system. Now we need to improve the machinery that turns an idea into a justified commitment.