Key Takeaways

• The heaviest AI user in your company is probably not your best AI user. A well-scoped prompt costs less than a badly scoped one.

• Meta stopped using token counts to evaluate engineers in September 2026, after people ran up consumption to climb an internal leaderboard.

• Four signals do hold up. Cost per unit of work falling. Model choice matched to task difficulty. Depth on a few workflows rather than novelty across many. Use that survives past week twelve.

• Metadata cannot tell you whether the work was any good. It tells you where to look. A manager still has to look.

• Guickly, the AI usage visibility and control platform, attributes AI usage and cost to team, tool and user from metadata alone. No prompts, no responses, no reading over anyone's shoulder.

Does high AI usage mean someone is good at AI?

No. The most useful evidence for that arrived this month.

In September 2026 Meta told engineers it would no longer use AI adoption dashboards or token counts to evaluate impact. The Information reported the internal memo on 8 September. Engineers had been running up token consumption to climb an internal leaderboard, a practice that acquired the name tokenmaxxing.

Meta is not abandoning measurement. It is still building a central dashboard and budgets for 2027. What it abandoned was one specific idea: that consumption is evidence of contribution.

The failure was predictable. When a measure becomes a target, it stops being a good measure. Build a leaderboard out of token counts and people will optimise the leaderboard. After that the token count describes the leaderboard and nothing else.

There is a second reason usage misleads. It has nothing to do with gaming.

Getting good at AI usually makes you cheaper. A precise prompt returns a usable answer in one pass. A vague one returns three drafts, a clarification and a retry. Someone who has learned to scope a task well produces the same output for a fraction of the tokens. On a usage leaderboard, that person looks like they are barely working.

Why does the heaviest user look like the best user?

Because AI cost inside a company is extraordinarily concentrated. The top of the list is very visible and very easy to misread.

Bain's June 2026 analysis found that the top 5% of users in a company often consume more tokens than the other 95% combined. Accenture's own internal analysis found roughly 19% of users accounted for about 80% of spend on a single AI tool. Usage of that tool rose 113-fold in ten weeks.

So when a leader opens a usage report, a handful of names dominate it. Those names are doing one of three different things. The report cannot tell them apart.

Three different people, one number. That is why usage alone can never answer the question in this post's title.

What actually signals someone is good at AI?

Four things. All four are visible from metadata alone.

Cost per unit of work, falling over time. This is the strongest single signal. Hold the output roughly constant and watch the spend per task. Someone learning the tool gets cheaper at the same job. Someone who is not learning stays flat or climbs.

Model choice matched to task difficulty. Reaching for the most expensive model for every task is a novice tell. The organisational version is well documented. Bain reported that AT&T, running 8 billion tokens a day, cut cost by 90% and tripled throughput by routing tasks to smaller domain models. The same instinct shows up in individuals. It is learnable.

Depth rather than novelty. Someone who returns to the same three workflows for months has found something that works. Someone who tried twenty tools once each has found nothing yet. Breadth of tools looks like enthusiasm in a report. It is usually the opposite of skill.

Use that survives past the novelty window. Most individual AI adoption spikes and decays. Sustained use at week twelve is a much better signal than heavy use in week one. Almost nobody measures it, because the pilot report is written in week three.

What can metadata not tell you about AI skill?

This is where honesty matters more than the pitch.

Metadata cannot see quality. It knows a developer generated code and how much it cost. It does not know whether the code was good, whether it shipped, or whether someone else spent a day fixing it.

Metadata cannot see judgement. The most valuable AI skill is often knowing when not to use it. That decision leaves no trace in any dashboard, because the thing that happened was nothing.

Metadata cannot see transfer. The person who quietly rewrote your team's prompt templates changed everyone else's numbers and not their own.

So the honest position is this. The data tells you where to look. It does not tell you what you will find when you look. A manager reading a distribution, spotting three unusual patterns and going to talk to those three people is measurement working correctly. A dashboard that ranks forty people is measurement being misused.

Should you use this in performance reviews?

No. Three reasons, in order of how much they will cost you.

It stops being true the moment you do. Meta ran the experiment for you. Grade the number and the number becomes a thing people produce rather than a thing you observe. You do not get a worse metric, you get a fictional one.

It punishes the behaviour you want. The efficient user has a low token count. Reward the high count and you have taught the whole company to be wasteful on purpose.

It changes what the tool is. Attribution and surveillance are not the same thing. Conflating them is how a measurement programme gets stopped by a works council, a union or a general counsel. It usually takes the useful parts down with it.

There is a version of this that works. Use the data to find people worth learning from, then ask them what they are doing. That makes strong performers a resource rather than a ranking. It is the only use of this data that gets better the more openly you run it.

Does any of this connect to actual returns?

The evidence that measurement pays is not ours, which is what makes it usable.

KPMG surveyed 2,145 senior leaders across 20 countries. Organisations with AI cost monitoring dashboards report established ROI five times as often as those without.

It is a correlation and we will not claim more. A survey cannot tell you which way the causation runs. Disciplined companies tend to do several things well at once. It is still a Big Four firm finding that the companies able to see their AI costs are the ones getting a return.

How do you find what good looks like and spread it?

Four questions, in the order worth asking them.

1. What is the spread? The ratio between your heaviest 5% and your median user. This tells you whether you have a concentration problem, an adoption problem or neither.

2. Who got cheaper? Anyone whose cost per unit of work fell while their output held. This is your shortlist. It is far shorter than the usage leaderboard.

3. What are they doing differently? This question cannot be answered by any dashboard. Go and ask them.

4. Can it be copied? Most of what a strong AI user does is a prompt pattern, a model choice or a workflow. All three travel.

Steps one and two are an instrument problem. Steps three and four are a management problem. Confusing the two is why so many AI enablement programmes produce a dashboard and no change.

FAQ

How do you tell if an employee is good at AI? Look at cost per unit of work over time rather than raw usage. Someone learning the tool produces the same output for less spend. Also look at model choice, depth on a few workflows rather than novelty across many, plus whether the use survives past the first few weeks.

Does high token usage mean an employee is productive with AI? No. A well-scoped prompt costs less than a badly scoped one, so skill often shows up as lower consumption. High usage can mean heavy legitimate workloads, a skill gap causing repeated retries, or someone optimising a metric.

What is tokenmaxxing? Inflating token consumption to look productive, usually in response to a metric or leaderboard that rewards usage. Meta stopped using token counts in engineer evaluations in September 2026 after exactly this happened.

Can you measure AI skill without reading employees' prompts? Yes. Cost per task, model selection, tool depth and sustained use are all visible from metadata. Reading prompt content adds risk without improving any of those signals.

Should AI usage data be used in performance reviews? No. Grading the metric corrupts it. It also punishes efficient users, who consume less. Use it to find people worth learning from, not to rank people.

What can usage data not tell you? Whether the output was any good, whether someone was right not to use AI at all, plus who quietly improved everyone else's results. Those need a manager, not a dashboard.