The chart looks healthy. Eighty-one percent of your licensed seats are green, the trend line points up, and the slide goes to the board on Thursday. Then a VP asks the finance team to rebuild the monthly variance memo with help from the model, and three of the eight people cannot produce a draft they would send. Both facts are true at once, because AI usage dashboards and AI enablement measure different things. One counts actions. The other asks whether the work got easier. The gap between those two questions is where most license budgets quietly go.
What the dashboard actually counts
Read the vendor’s own definitions before you read the chart. Microsoft’s Copilot usage report documentation says it plainly. Enabled Users counts unique people holding a license over the chosen window. Active Users counts those enabled people “who tried a user-initiated Microsoft Copilot feature, in one or more apps.” Active users rate is the second number divided by the first.
Tried. One feature. Once.
The docs are equally precise about what does not count, and the example they pick is telling: clicking the Copilot icon in the Word ribbon to open the chat pane is not active usage, while submitting a prompt is. The bar is a single prompt, anywhere in the suite, inside the window. The same page notes that audit-log data “isn’t intended to be used as the basis for Copilot usage reporting,” so the tidier numbers your security team can pull will not reconcile with the ones on the slide.
None of this is a vendor trick. It is published, documented, and almost nobody on the buying side reads it. The report answers the question it was built to answer, which is whether the licenses you paid for are being touched. It was never built to answer whether your team got better.
Three populations, two of them visible
There are three different counts in your company and only two of them appear in any console.
| Population | Definition | Who can see it |
|---|---|---|
| Enabled | Holds a license | Admin console, exactly |
| Active | Submitted at least one prompt in the window | Admin console, by its own definition |
| Capable | Can get a usable result out of their own real work, unaided | Nobody, until someone tests it |
The third row is the one your budget was approved against, and the only one with no telemetry behind it, because capability is a property of a person doing a task rather than a session hitting an endpoint. A log can tell you a prompt was sent. It cannot tell you the output was used, was right, or saved anyone a minute.
That is the shape of the problem, and the same arithmetic separates a seat invoice from a capability budget in the case for treating license spend and team capability as two different problems.
The 180-day window does most of the overstating
The single biggest source of inflation is a dropdown, not a metric. The Copilot report can be viewed over 7, 28, 90, or 180 days. Active-user rate is computed against whichever you pick.
A person who sent two prompts in May and nothing since is dormant by any working definition. On a 180-day view in October, they are green. Switch the same tenant from 180 days to 7 days and the number that reaches your board can fall by half without a single behavior changing. Ask whoever builds your slide which window they used; in most companies nobody has asked.
The user-level table makes the point more bluntly. Microsoft’s docs state it “shows all users who were licensed for Microsoft Copilot at any point over the past 180 days, even if the user later removed the license or never had any Copilot active usage.” Numerator and denominator both drift for reasons that have nothing to do with how your team works.
Two rules follow. Report a 28-day window as the headline, because a month is the shortest period that survives vacations and quarter-end. And report the count, not the rate: 54 of 150 is a sentence a CFO can act on, where 36 percent is one they can only nod at.
Depth metrics are still usage metrics
The honest answer to a shallow metric is usually a deeper one, and this is where the market goes next. Microsoft’s Copilot Dashboard in Viva Insights sorts its views into readiness, adoption, impact, and sentiment, with sentiment drawn from survey data you upload yourself. Third-party products go further, classifying activity into categories like drafting, research, summarizing, and coding to show where use is deep and where it is thin.
These are better instruments and worth having. They do not cross the gap. Prompts per user, active days, actions per feature, and category mix all answer how much and where, not how well. A person can submit forty prompts a week and accept forty mediocre drafts. The dashboard reads that as a power user.
Volume can also count the wrong work. CompTIA’s AI Skills Tracker, fielded in June 2026 across more than a thousand business and technology leaders, found 80 percent using AI tools several times a month while only 29 percent reported high familiarity, and more than half said business activities make up a fifth or less of their AI use. Some of what your telemetry counts is a person planning a vacation.
What a tested capability number looks like
Capability has to be tested, and when someone tests it the picture changes.
Section’s AI Proficiency Report, published in June 2026, assessed 5,026 US knowledge workers with a multiple-choice knowledge test and hands-on skill tasks covering prompting, building small automations, and writing assistant instructions. Not a survey of how confident people felt. Scored work. Two results from it belong next to your adoption chart:
- Workers at organizations that paired an agent rollout with training specific to it averaged 44 out of 100. Where the same tools went in without that training, the average was 25.
- Asked to define an AI agent in their own words, fewer than one in ten gave a directionally correct answer, and 82 percent missed it on a multiple-choice version. Meanwhile 69 percent reported their company had taken some action on agents, and 16 percent said they use an agentic tool at work.
The first bullet is the most useful number in this field for anyone writing a business case, because training moves a measured score rather than an activity count. Note what it is not: 44 out of 100 is still a failing grade, and a 1.8x gap in a cross-sectional sample is an association, not a promise about your tenant. Treat it as evidence that the lever exists.
Behavior shifts on a smaller dose. BCG’s 2026 AI at Work survey of nearly 12,000 employees and managers found 79 percent of people who got more than five hours of training were regular users, against 67 percent below five hours, while only 36 percent called their upskilling adequate. Five hours is one long morning, set against the 16.7 formal learning hours per employee per year that ATD’s 2026 State of the Industry report recorded. The constraint is not budget. Nobody scheduled the morning.
The leaders and the staff are reading different dashboards
The same Section research found 57 percent of C-suite respondents saying AI was widely integrated into workflows, against 18 percent of individual contributors: a 39-point perception gap inside the same companies.
Dashboards widen that gap rather than closing it, because of who looks at them. Leadership sees a tenant-level rate, which aggregates across roles and rounds away the variance that matters. The analyst who cannot get a clean draft out of a document she wrote herself does not appear as a red cell. She appears as one unit of green, having sent a prompt on Tuesday.
Role-level variance is where the signal lives. Recon Analytics, surveying more than 150,000 US paid AI subscribers between July 2025 and January 2026, found that among people with workplace access, ChatGPT converted 83.1 percent into active users while Microsoft Copilot converted 35.8 percent and Gemini a near-identical 34.0 percent. If a tool-level gap that wide exists across a market, a role-level gap at least as wide exists inside your company, and a tenant-wide average is built to hide it.
What happens when the activity number becomes the target
The predictable failure is to manage the metric. Trade press reported this year that KPMG built an internal dashboard tracking AI tool use across roughly 10,000 US advisory staff, with a 75 percent usage target and a view letting people compare themselves against peers. The same coverage noted that some developer tools and workflows were not fully captured, so the data was thinnest where use was most advanced.
Hold aside whether a usage target is good management. Look at what it does to your measurement: the moment a number is tied to standing or pay, it stops being a measurement. You will hit 75 percent, you will not know what you bought, and the people already fluent in untracked tools will show up as laggards.
That is the cost of asking an activity log to do a capability assessment’s job, and it is cheaper to notice early than to unwind later, for the same reason laid out in what a cheap AI enablement vendor actually costs you.
Four ways to measure AI enablement instead of activity
None of these needs a new tool. All four are runnable this quarter.
1. A timed task on your own documents. Pick one recurring deliverable per team: the variance memo, the client update, the weekly pipeline summary. Have each person produce it with the model, on the clock, using real inputs. Score minutes taken and whether a manager would send the output without a rewrite. That is a baseline and a pass rate. Repeat after the program.
2. The eight-week retention check. Run the same task eight weeks later with no warning and no refresher. Capability that fades when the facilitator leaves was never capability. This is the number vendors never report, and the one that separates a session from a program.
3. Named workflows, not hours saved. Self-reported time savings inflate and will not survive finance. Count something countable: how many recurring tasks, by name, moved to an AI-assisted method and stayed there for a full cycle. Three named workflows per team beats an unverifiable claim about hours, and it is the shape of evidence that holds up when you underwrite the budget so a CFO signs it.
4. The replacement question. Ask one question in your next pulse survey: if this tool were removed tomorrow, how would your Monday change? Specific answers mean the habit exists. Vague answers mean a seat is being paid for twice, once in license fees and once in the hour nobody saved.
Run these alongside the dashboard, not instead of it. Keep the console for breadth, licensing hygiene, and spotting dormant seats worth reclaiming. Use the four above for the question it cannot reach.
How to pair the dashboard with an AI enablement program
The sequence is short. Pull the 28-day active count and the per-role breakdown, so you know which teams are nowhere near the tool. Run one timed task per team for a baseline about capability rather than clicks. Then train against the gap that baseline exposes, on the team’s own documents and its own reporting cycle, which is the only context in which fluency transfers. Vendor material is written for every customer, so it cannot be about yours.
Market rates for facilitated sessions run roughly $2,000 to $15,000 each depending on format, which puts a multi-cohort rollout at $150 to $400 per trained person. Measure the same task at the end and again eight weeks later. The dashboard then becomes a maintenance instrument, watching for drift in a population you have measured properly once. The full sequencing is laid out in our AI enablement playbook.
If your active-user rate looks fine and you are not sure what it is telling you, that is the conversation we have on a free AI-readiness call. Book one here and bring the chart.
Frequently asked questions
What do AI usage dashboards actually measure?
License assignment and activity. In Microsoft’s Copilot usage report, Enabled Users counts license holders, Active Users counts those who tried a user-initiated Copilot feature at least once in the selected window, and the active users rate divides the second by the first. They are breadth instruments, not capability instruments.
Why does our active-user rate look high when adoption feels low?
Most often the reporting window. Active-user rate can be computed over 7, 28, 90, or 180 days, and a 180-day view counts anyone who sent a prompt within the last six months as active. Check which window your slide uses before concluding anything.
What counts as an “active user” in Microsoft Copilot?
An enabled user who performs an intentional action on an AI feature. Microsoft’s documentation is specific that opening the Copilot pane in Word does not count while submitting a prompt does. One prompt in one app inside the window is enough.
Can a dashboard tell me if AI is saving us time?
No. It tells you prompts were sent and in which apps. It cannot tell you whether the output was used, correct, or faster than the old method. Time saved has to be measured on a defined task with a before-and-after, not inferred from a log.
Is depth-of-use a better metric than active users?
Better, still not enough. Prompts per user, active days, and category mix tell you how much and where, not how well. Someone can submit forty prompts a week and accept forty weak drafts, and the dashboard reads that as a power user.
How do I measure AI capability instead of AI usage?
Test it. Give each person a timed task on a real recurring deliverable, score minutes taken and whether a manager would send the output unedited, then repeat the same task eight weeks later with no refresher. That gives you a baseline, a pass rate, and a retention figure.
Should we set a company-wide AI usage target?
Be careful. A target tied to standing or pay converts your measurement into a performance game, and usage data is already incomplete for people working in tools your telemetry does not cover. Set targets on outcomes you can inspect, such as named workflows that moved and stayed moved.
Does more training change these numbers?
The best available evidence says yes, with limits. Section’s 2026 proficiency testing of 5,026 US knowledge workers found an average of 44 out of 100 where a rollout was paired with training specific to it, against 25 where it was not. BCG found 79 percent regular usage above five hours of training against 67 percent below. Both are associations across survey populations, not projections for your company.
How often should we review AI usage data?
Monthly on a 28-day window for licensing hygiene and dormant-seat recovery, quarterly for the capability measurements. Weekly review of an activity chart produces noise.
Who should own this inside a 50 to 500 person company?
Whoever signs both the license renewal and the training budget, usually a COO, CFO, or VP of Operations. The per-active-user figure and the capability baseline sit in different departments at most companies, which is why nobody computes the ratio between them. Worth closing before the next renewal, as covered in the difference between AI transformation and enablement.
Key takeaways
- AI usage dashboards report two populations, enabled and active. Your budget was approved against a third one, capable, which has no telemetry behind it.
- “Active” means one prompt in one app inside the reporting window, and that window can be set to 180 days. The dropdown inflates your rate more than any metric does.
- Depth metrics improve the instrument without crossing the gap. Prompts per user and category mix measure how much and where, never how well.
- Tested proficiency moves with training: Section measured 44 out of 100 where tool-specific training happened against 25 where it did not. Both figures are low enough to be sobering.
- Measure capability with a timed task on your own documents, an unannounced eight-week retest, named workflows that moved, and the replacement question. Keep the dashboard for breadth and dormant-seat recovery.
Arthur Wandzel
