Skip to main content

Definition: AI Provider Coverage & Limitations

Written by Tom Williams

The AI Usage dataset is designed to normalise five sources onto one schema so they can be reported side by side. Only GitHub Copilot is connected today. The other four integrations are built and specified, and will start producing records as they are released.

Provider

Status

GitHub Copilot

Available

Anthropic Claude

Coming soon

Cursor

Coming soon

OpenAI Platform

Coming soon

OpenAI Codex

Coming soon

Every record you can query today therefore comes from GitHub Copilot, and any field Copilot does not publish is empty everywhere. The field articles say which providers populate each field, and mark the ones that are not connected yet.

What GitHub Copilot publishes today

Copilot's coverage is uneven in ways a reader of a dashboard will notice:

  • Tokens: command line and desktop app only. There are no token counts for IDE work.

  • Model attribution: chat only. Completions carry no model.

  • Repository: yes, as pull request metrics on REPOSITORY records.

  • Credits: yes, on day total records.

  • Code outcomes: yes - suggestions, lines of code and pull request activity.

Copilot publishes nothing at all for prompt cache tokens, explicitly rejected suggestions, commits authored by the assistant, or web searches. Those fields exist in the schema because other providers publish them, and they stay empty until those providers are connected.

What the other providers will add

This is what each integration will contribute once it is released. Nothing below is queryable yet.

Provider

Tokens

Model

Repository

Credits

Code outcomes

Anthropic Claude

yes, including the cache split

yes

no

no

yes

Cursor

yes

yes

no

no

yes

OpenAI Platform

yes

yes

no

no

no

OpenAI Codex

none

none

no

yes

code review only

An empty measure always means the provider does not report it. It never means zero, and averaging or summing across providers over a measure only some of them publish will understate the ones that do.

Known limitations

Each of the following is expected behaviour rather than a defect. They are listed here because each one looks like a bug the first time it is seen in a report.

Applying today, to GitHub Copilot:

  • No tokens for IDE work. GitHub publishes token counts for the Copilot command line and desktop app only, so per-developer token reporting is not possible for IDE usage. Treat it as not available rather than as zero.

  • Web chat and GitHub Mobile are excluded from usage metrics while still consuming credits, so Copilot usage will always understate Copilot billing.

  • Active user counts will be lower than GitHub's own Copilot dashboard. GitHub counts users it sees only through server-side telemetry, who by its own documentation may not appear in the per-user breakdowns these records come from.

  • Passive code review has no equivalent in GitHub's dashboard. GitHub counts a user with both active and passive review signals as active only. Keypup keeps both as separate records, so filtering to surface_name = code_review_active reproduces GitHub's definition.

  • Accepted suggestions can exceed offered suggestions. A suggestion generated one day and applied the next does exactly that, so no report should assume the ratio stays below 100% on a single day. Compute acceptance rates over a window rather than per day.

  • Organization records overlap across organizations. Copilot attributes organization-scoped usage by membership rather than by where the work happened, so a user belonging to three organizations is counted in all three. Organization totals must not be summed across organizations.

  • Suggested deletions are always zero. GitHub publishes the counter but documents it as not yet implemented.

Applying once the other providers are connected:

  • Codex will report no tokens at all, and OpenAI Platform no credits. The two meter differently, which is why they are two separate integrations rather than one.

  • Anthropic will see Claude API traffic only. A team running Claude Code against Amazon Bedrock, Google Vertex or Microsoft Foundry will report nothing here even though it is spending money.

  • Acceptance rates will not be comparable across providers. Anthropic reports accepted and rejected proposals but never how many were shown, so its rate must be computed as accepted over accepted plus rejected - a different denominator from Copilot's and Cursor's.

  • Actors will not be comparable across providers. GitHub identifies a person by their login; Anthropic, Cursor and Codex identify them by email address. The same person will therefore appear as two actors when two providers are connected, and a cross-provider count of distinct actors will be too high.

One limitation applies whatever is connected: currency amounts are not in this dataset. Only metered consumption units - AI credits - are carried, in credits_used. Spend belongs to billing, not usage.

Did this answer your question?