The Method

A pilot is a rehearsal, not a demo.

Most pilots are staged to succeed. A pilot that is built to prove the concept teaches you nothing about whether it will survive contact with the real organization.

Dakhalfani Boyd · · 7 min read

The pilot is one of the most trusted moves in transformation. Prove the concept small, learn what works, then scale it. Done well, it de-risks an expensive rollout. Done the way most pilots are done, it does the opposite: it manufactures a success that cannot be repeated and calls it evidence.

The distinction is between a rehearsal and a demo. A demo is staged to impress. It runs in favorable conditions, with the best people, the most motivated site, and unlimited attention from the program team. A rehearsal is staged to learn. It runs under conditions as close to real as possible, precisely so that the friction shows up while it is still cheap to address.

Why the demo pilot lies

A demo pilot is designed, consciously or not, to produce a yes. It picks the enthusiastic team, the clean use case, the site where the local leader is already a believer. It surrounds the pilot with hands-on support no full rollout could ever afford. Under those conditions, of course it works. The pilot succeeds, the program gets funded to scale, and then it meets the ninety percent of the organization that had none of those advantages.

The scaled rollout fails in ways the pilot never surfaced, because the pilot was engineered to avoid exactly the conditions that cause failure. The organization concludes that scaling was the hard part. In truth, the pilot lied, gently, by testing the idea only where it was guaranteed to work.

A pilot that succeeds only under conditions you cannot reproduce at scale has not reduced your risk. It has hidden it.

Design the pilot to find the friction

A rehearsal pilot inverts the design choices. Instead of the most willing site, it picks a representative one, including some of the skepticism and constraint the full rollout will face. Instead of unlimited support, it runs with roughly the support ratio that scaling will actually allow, so that the gaps in self-sufficiency reveal themselves now. Instead of the cleanest use case, it includes the messy edges where the real work lives.

The goal is not to make the pilot easy. It is to make the pilot honest. A rehearsal that struggles in instructive ways is worth far more than a demo that sails through, because the struggle is data about what scaling will require.

Instrument what will not travel

The most valuable output of a rehearsal pilot is a clear list of what worked because of conditions that will not exist at scale. The expert who sat with every user will not be there for ten thousand people. The novelty and attention will fade. The workaround that a small motivated team tolerated will not survive a large indifferent one.

Naming these explicitly turns the pilot into a design input for the rollout. Each item becomes a thing the scaled plan has to solve deliberately: build the self-service the expert used to provide, engineer the reinforcement that novelty used to supply, fix the process the workaround was hiding. A demo pilot skips this because it never encountered the limits. A rehearsal exists to find them.

The question a pilot should answer

A pilot should not be trying to answer whether the idea can work. Almost any idea can work somewhere, with enough attention. The question worth answering is whether the idea can work here, under the conditions we can actually sustain, for the people who did not volunteer.

That is a harder question and a more useful one. It is also the one that separates a pilot that de-risks a rollout from a pilot that sets one up to fail with false confidence.

Rehearse for the room you will actually play

Before the next pilot launches, be honest about which one it is. If it is built to prove the concept to a funding committee, it will likely succeed and teach you little. If it is built to rehearse the rollout under real conditions, it may look less impressive and will tell you the truth. Choose the rehearsal. The demo feels better in the moment and costs far more at scale.

Where this goes

This essay draws on the 5A Framework, the repeatable system BoydNorth uses to close the execution gap between strategy and outcomes.

← All essays in The Compass