
A small group got Copilot licenses months ago. A few were on the technology committee, a couple needed it for something specific, a few asked loudly enough. The instructions were roughly "try it and tell us what you think."
Now the renewal date is on the calendar, or someone in finance wants to know whether to buy more. And the honest answer to "so what did the pilot tell us?" is that people seem to like it.
That's the question I hear most from IT directors heading into a renewal or an expansion decision right now. We ran a pilot. Why don't we know anything?
Because the pilot was never set up to fail. If there's no result that would make you say "this didn't work," there's also no result that proves it did. A group of people trying a tool is a test drive. A pilot is an experiment with a question in it, and the answer is allowed to be no.
The story I heard over and over at TechCon365 Dallas earlier this month went like this. Buy Copilot licenses. Hand them to whoever asks. Give people an hour of training, if that. Call it a rollout. Then stare at the usage numbers.
It's common. When I polled 116 IT leaders in Seattle, 105 had Copilot licenses and 82 had no usage guidelines or training in place. Licenses go to whoever is curious or senior, nobody picks a job for Copilot to do, and the only data at the end is the usage report.
The usage report can't carry the decision. It tells you who opened Copilot and how often. It doesn't tell you whether anything about the work changed. Chad Doty, who led the Copilot rollout at Ecolab, put it plainly when we talked: "Our adoption numbers looked great, but it didn't really tell me anything." He also said "rolling it out is easy." The hard part is having something specific to point at when someone asks why you're keeping it.
There's a second problem. Curious volunteers are the wrong sample. They'll find a use for anything. When you expand to everyone else, the people who never raised their hand behave nothing like the pilot group, and the numbers you were counting on don't show up.
Four decisions, made before anyone gets a license.
Before the licenses go out, finish this sentence: "We will know this worked if ____ by ____." If you can't fill in both blanks, the pilot can't give you an answer.
Fair. Most people who own Copilot adoption have that job on top of another one. But this version is less work. One workflow instead of twelve. One team instead of a scattered group of volunteers. One page that says what you're testing and when you'll decide.
The vague pilot is the expensive one. It eats six months and leaves you exactly where you started, with a renewal decision and nothing to hand finance.
The other objection I hear is "what if it fails?" Then you learned something real for the cost of a small batch of licenses, and you learned it before you bought more. When a pilot like this comes back with a no, it usually means one of two things. The workflow was the wrong one, or people never got hands-on help with their actual work. Both are fixable, and now you know which one to fix.
The finance team's story in Dallas didn't end with the reconciliation. Other teams started asking how they did it. Nobody had to mandate anything. People started pulling licenses from them.
That matters for your business case. The expansion request now comes from the business, with a before and after attached, and IT isn't the one doing the selling.
It's also how you walk into a renewal with something to say. A usage percentage on its own tends to invite a cut. "This task got faster for this team, here's the before and after, and two other teams have asked for the same setup" invites a conversation about where to go next.
When we run Copilot Catalyst with a customer, this is where it starts: one group, their real work, weekly hands-on sessions, and a scorecard that tracks observable behavior change. You can do the setup above on your own, though. None of it requires buying anything.
Take the pilot you're running now, or the one you're about to start, and write that one sentence. If the blanks won't fill, pick one workflow this week, get the team that owns it to agree on a rough before number, and put the decision date on the calendar.
If you're heading into a renewal or an expansion decision and your pilot didn't give you an answer, reply or grab time with me. I'm happy to look at what you have and help you set up a version that does.
Stop Pissing Off Your Software Users! There's a Better Way...
VisualSP makes in-app guidance simple.