GitHub Copilot AI Credits Optimization — Module 3: Measurement & Optimization at Scale — Workshop Guide

Duration: ~180 minutes (including one 10-minute break)
Format: Presentation + Live Demo + Hands-On Exercises
Audience: FinOps practitioners, platform/DevEx engineers, engineering managers, Copilot program owners, advanced admins
Focus: Reporting and observability layers, token cost calculations, KPI baselines, outcome economics, budget tuning, and a sustainable operating model at scale

Module Overview

This is Module 3 of 3 in the GitHub Copilot AI Credits Optimization workshop series. Where Module 1 focuses on admin guardrail setup and Module 2 focuses on developer-level optimization habits, Module 3 is built for the people who own the numbers: FinOps, platform/DevEx, engineering managers, Copilot program owners, and advanced admins who need to measure consumption at scale, distinguish legitimate heavy use from waste, tune budgets across every layer, and run a repeatable monthly operating cadence.

Note: This module is standalone. You do not need to have attended Module 1 or Module 2 to complete it — the Welcome & Budget Refresher section below re-establishes the budget vocabulary and mechanics this module builds on. If your team has already run Module 1 (admin guardrails) and Module 2 (developer optimization), this module is the natural third session.

Important: Pricing and most dated figures in this module reflect GitHub documentation as of 2026-08-08; the standard included-credit amounts and historical promotional window were re-verified on 2026-09-01. These details change over time — always verify against the live docs linked in the Appendix before delivering this module.

Related Modules

Learning Objectives

• Locate and filter AI usage reporting by user, model, organization, and cost center, and know where individual users see their own consumption
• Select the least-complex visibility layer for a question: session commands, AI usage dashboard/CSV, or optional privacy-reviewed Copilot CLI OpenTelemetry
• Apply the detailed token cost formula to calculate USD and AI-credit cost for a Copilot interaction, using live, current pricing rather than memorized numbers
• Establish KPI baselines — burn rate, projected exhaustion date, credits per active user/task, blocked users, overage, feature/model mix, and adoption/quality — from a live export or a supplied sample dataset
• Distinguish legitimate heavy use from waste using session-level signals, not just total credits consumed
• Analyze a sample dataset correctly, including recognizing when a correlation between two metrics does not establish that one caused the other
• Define a measurable outcome, first-pass durable acceptance rule, and stabilization period before calculating economic value
• Calculate total assisted-delivery cost, full cost per durable accepted outcome, and net contribution using sampled human effort and explicit assumptions
• Distinguish realized cash savings from cost avoidance, capacity released, revenue contribution, and risk reduction
• Evaluate model and multi-stage workflow economics using durable acceptance, rolled yield, and amortized enablement cost
• Tune budgets across every layer — universal, cost-center, and individual user-level budgets, plus cost-center/organization/enterprise spending budgets — and apply included usage controls and power-user exceptions
• Explain the "lowest remaining headroom wins" behavior when multiple budgets apply to the same user
• Draft a 30-day optimization backlog and an operating model (RACI, review cadence, anomaly response, showback/chargeback) that a FinOps or platform team can run every cycle

Prerequisites

Important: No section of this module requires making a live paid AI request. Every calculation can be completed with the worked formulas and the supplied sample dataset. If you export live data during Section 1, treat it as read-only reporting — no budget changes are required to complete this module.

Session Agenda

Total: ~180 min (~3 hours)

Welcome & Budget Refresher (15 min)

Key Points

• Position this session: Module 3 of the GitHub Copilot AI Credits Optimization series, standalone-capable, aimed at the people who own budgets, reporting, and the operating model
• GitHub AI Credits are the billing unit for Copilot Business and Enterprise: 1 AI credit = $0.01 USD
• Every Copilot Business license includes 1,900 AI credits/user/month; every Enterprise license includes 3,900 AI credits/user/month — pooled at the billing entity, reset at 00:00:00 UTC on the first of each month, with no rollover
• Budget controls exist at four levels and work together, not as alternatives:

Budget Control Comparison

Historical allotment note: The temporary June 1–September 1, 2026 promotional allotment was 3,000 credits/user/month (Business) and 7,000 credits/user/month (Enterprise). The durable examples in this module use the standard 1,900 / 3,900 amounts. Historical exports from the promotional period may therefore show a larger included-credit pool. Re-verify live amounts before delivery.

• A $0 or exhausted ULB always blocks a user immediately — there is no fallback to a cheaper model
• An unconfigured cost-center/organization/enterprise budget is an alert, not a guardrail — "Stop usage when budget limit is reached" is off by default

Discussion Points

• If your organization already ran Module 1 or 2, what changed in your budget configuration since then?
• Who in this room owns the monthly review of these numbers today — is that ownership clear?
• What's the one measurement question you most want answered by the end of this session?
• Do your current baselines reflect the standard allotments, or are they still influenced by promotional-period data?

1. Reporting & Data Sources (20 min)

Key Points

• The AI usage page (Billing & Licensing → AI usage, at the organization or enterprise level) is the primary reporting surface for AI credit consumption
• Filter and group usage by user, model, organization, or cost center; use the "Group by" control to switch between per-model and per-SKU views, and the Timeframe control to change the reporting window
• Export the filtered view (CSV) for offline KPI calculation, trend analysis, or sharing with stakeholders who don't have dashboard access
• Metered usage search (under Usage → Metered usage) accepts query syntax such as product:copilot and cost_center:<name> to isolate Copilot spend for one team
• Individual visibility: every Business/Enterprise user sees their own consumption under Copilot settings → Usage — the card shows credits consumed against their budget if one applies (for example, "450 / 1,000 AI credits used"), or total credits consumed if no ULB applies to them; VS Code and other IDEs also surface personal usage and reset date
• Cost center home pages show "AI credit pool enabled" and current consumption against the included-usage cap when a cost center has an included usage control turned on

Three Visibility Layers

Optional Advanced Telemetry with OpenTelemetry

• Copilot CLI can export OTel traces and metrics when explicitly enabled; confirm current configuration with copilot help monitoring
• Use it when dashboard aggregation cannot answer a bounded engineering question such as "which tool stage drives retries or token growth?"
• Keep full prompt/response capture disabled by default. The content-capture setting can expose source, prompts, tool output, or sensitive data to the telemetry pipeline
• Define retention, access, redaction, data residency, and incident-response ownership before enabling an exporter
• Do not build chargeback from raw traces alone. Reconcile telemetry with the AI usage dashboard/CSV, which remains the billing-oriented source

Important: Start with /usage, /context, and the AI usage dashboard. Add OTel only when a specific observability question justifies its operational and privacy cost.

Live Prices, Not Memorized Numbers

Important: Pricing, included-credit amounts, and reporting UI details change over time. Treat every number in this module — including the pricing snapshot in Section 2 — as a dated example, not a permanent reference. Always confirm against the live Models and pricing page and the live AI usage dashboard before quoting numbers to stakeholders or making a budget decision.

🖥️ Demo: AI Usage Dashboard Walkthrough

1. Navigate to Billing & Licensing → AI usage (organization or enterprise settings)
2. Filter by cost center to isolate one team's consumption
3. Switch "Group by" between model and SKU to see which models or plan types drive the largest share of spend
4. Change the Timeframe to compare the current cycle against the prior one
5. Export the filtered view (CSV) for offline analysis
6. Open Copilot settings → Usage as an individual user to show the personal visibility view attendees' own developers will see

Discussion Points

• Who in your organization currently has access to the AI usage dashboard — is that the right set of people?
• Should individual contributors be told where to find their own usage view proactively, or only on request?
• What would you do if a cost center's actual usage didn't match what its assigned licenses should fund?
• How often should a CSV export be pulled for offline analysis versus relying on the live dashboard?

2. Token Cost Calculations (Hands-On) (25 min)

Key Points

• Every Copilot interaction consumes tokens, and each token category is priced independently per model:
  • Input tokens — what you send to the model (your prompt + context)
  • Cached input tokens — context the model reuses from a previous turn, typically billed at about 10% of that model's fresh-input price (roughly 10x cheaper). This is the common pattern, not a universal rule — verify the selected model's row before quoting or encoding a ratio
  • Cache write tokens — the cost of writing context into the cache so it can be reused later (not all models charge this)
  • Output tokens — what the model generates back to you, typically the most expensive category per token
• Cost formula:

USD = Σ (tokens_in_category / 1,000,000 × price_per_million_for_that_category)
AI credits = USD × 100
Auto model selection applies an additional 10% discount to the model-cost total.

• Pricing is per million tokens and changes over time — always check the live pricing page before quoting numbers to stakeholders (see Appendix)

Pricing Snapshot (2026-08-08 — verify before use)

All prices are USD per 1 million tokens. Source: Models and pricing for GitHub Copilot.

Worked Example — Longer Coding Agent Session (Claude Sonnet 5, no Auto)

• 150,000 input tokens, 40,000 cached input tokens, 20,000 cache-write tokens, 6,000 output tokens

Input:      150,000 / 1,000,000 × $2.00  = $0.300000
Cached:      40,000 / 1,000,000 × $0.20  = $0.008000
Cache write: 20,000 / 1,000,000 × $2.50  = $0.050000
Output:       6,000 / 1,000,000 × $10.00 = $0.060000
------------------------------------------------------
Total USD                                = $0.418000
AI credits (USD × 100)                   = 41.8 credits

With Auto model selection (×0.90 discount): $0.418 × 0.90 = $0.3762 → 37.62 credits (a savings of ~4.18 credits).

Why This Matters at Scale

• A single session's cost looks trivial. At a 500-user scale, a 10-credit-per-session difference across 20 sessions/user/month is 100,000 credits/month — larger than an entire 50-user cost center's included-usage cap
• Cost-per-session math is the building block for every KPI in Section 3 — you cannot calculate "credits per active user/task" without first understanding what drives an individual session's cost

Discussion Points

• Why does model choice, not just prompt length, dominate cost at scale?
• Cached input is typically about 10% of a model's fresh-input price, but not for every model. What does that imply about session-continuity policies your team might set, and how would you verify the ratio before encoding it in a policy or a cost model?
• How would you use the "monthly credit difference" calculation from the exercise to justify a model-selection policy to engineering leadership?
• Would you expect a quick Ask-mode question or a long cloud-agent session to consume more credits, and by roughly what order of magnitude?

3. KPI Baselines & Anomaly Analysis (Hands-On) (30 min)

Key Points

• Before tuning any budget, establish a baseline of current consumption patterns using live data (Section 1) or the supplied sample dataset below
• Core KPIs to track every cycle:

Legitimate Heavy Use vs. Waste

Total credits consumed alone does not tell you whether spend is healthy. Pair consumption with session-level signals:

Important: A single high-consumption user is not automatically a problem, and a single low-consumption user is not automatically efficient. Always read credit volume alongside the quality and session-behavior signals above before labeling anyone's usage as waste.

Correlation Is Not Causation

• Two metrics moving together in the same reporting window does not prove one caused the other — always look for a confounding factor before concluding a causal link
• Common trap: overage rises in the same week a quality metric declines. The reflexive conclusion — "overage is degrading quality" — is often wrong. Check for a shared root cause first, such as a release deadline, a large onboarding cohort, or an incident response effort that would independently drive both more usage and more rushed output
• Test for causation, don't assume it: if quality recovers while overage or credit volume stays elevated, overage was very unlikely to be the cause — the true driver already resolved (for example, the deadline passed) while spend was still working through the backlog it created

Sample Dataset — "Growth Platform" Cost Center

60 licensed Copilot Business users. Included-usage cap = 60 × 1,900 = 114,000 AI credits for the 30-day cycle.

Note: The included-usage cap (114,000 credits) is crossed during Day 9 — 17,800 credits remained after Day 8, but Day 9 consumed 21,300, pushing 3,500 credits into metered overage ($35.00 at $0.01/credit). Recheck this math yourself in the hands-on exercise; the table's displayed overage reflects the cumulative running total, not each day in isolation.

Read the Release Deadline column before drawing conclusions: the PR review pass rate drops sharply on Days 6–8, exactly when the "Release Deadline?" flag is Yes — and credits consumed also rises those same days as teams push more work through Copilot under time pressure. On Day 9, overage begins after the deadline has passed, yet the pass rate recovers to 79% and continues recovering to 85% on Day 10 even though overage is still accumulating and growing. If overage itself were degrading quality, the pass rate would not recover while overage was still rising — the deadline, not the overage, was the shared root cause of both effects.

Discussion Points

• What would you do differently in month one if you saw this dataset for your own cost center?
• Besides a release deadline, what other confounding factors could produce a false correlation between cost and quality in your organization?
• How often should these KPIs be reviewed — weekly, biweekly, monthly — and does the answer change for a cost center currently trending toward exhaustion?
• Who should receive an anomaly alert when blocked users or burn rate spike mid-cycle, and how quickly should they be expected to respond?

☕ Break (10 min)

Use the break to reset before the module's conceptually heavier outcome-economics and operating-model sections.

4. Economics of Accepted Outcomes (Hands-On) (45 min)

Key Points

• Token cost is an input, not the outcome. AI credits answer "what did the model cost?" but not "did the assisted work create durable value?"
• Freeze the outcome definition, acceptance rule, stabilization period, and reporting period before collecting or comparing data
• Measure teams and repeatable workflow categories — not individual developers as a performance ranking based on token consumption
• Use representative samples and ranges; do not create measurement overhead larger than the decision it supports

Freeze the Outcome Definition

Note: The example is not a universal standard. Documentation, production code, security remediation, and operational workflows can require different outcome units and stabilization periods.

Report outcomes by cohort or with a reporting lag: work still inside its stabilization period is pending, not accepted or rejected.

First-Pass Durable Acceptance

An attempted outcome counts as durably accepted only when it meets the defined acceptance rule without a material corrective cycle and remains valid through the stabilization period.

Durable acceptance rate =
durable accepted outcomes ÷ attempted outcomes

This is stricter than suggestion acceptance, initial test pass, or PR merge rate. Those remain useful leading indicators, but none proves the result stayed accepted.

Total Assisted-Delivery Cost

Total assisted-delivery cost =
AI-credit (model) cost
+ human prompting, review, correction, and escalation cost
+ relevant compute and CI/CD rerun cost
+ allocated license cost
+ amortized enablement and workflow-development cost

Full cost per durable accepted outcome =
total assisted-delivery cost ÷ durable accepted outcomes

Human effort can be estimated from a representative sample:

Sampled human cost =
sampled hours × approved loaded labor rate

Document the sample size, roles, rates, exclusions, and uncertainty. Do not rely on self-reported "hours saved" as the sole monetary input.

Economic Scorecard

Important: Never report unit-cost improvement alone. Pair it with accepted-output volume or net contribution so reduced delivery cannot masquerade as improved efficiency.

Benefit Classification

Important: If payroll and headcount did not change, multiplying "hours saved" by a labor rate does not automatically produce cash savings. It usually represents capacity released or cost avoidance until an actual expense or measurable output changes.

One-Minute Paired Benefit Check

Classify each outcome before assigning a dollar value:

1. A team delivers the same backlog with unchanged payroll and no headcount reduction.
2. The organization cancels an external code-review service after an accepted pilot.

Answer: Scenario 1 is capacity released; Scenario 2 is realized cash savings. Both can support an investment decision, but they belong in separate scorecard rows.

Advanced Economics: Multi-Stage Yield & Enablement

When every workflow stage must succeed:

End-to-end durable acceptance =
stage 1 acceptance × stage 2 acceptance × ... × stage n acceptance

Four required stages at 90% each:
0.90 × 0.90 × 0.90 × 0.90 = 65.61%

Shorten unnecessary chains, improve the weakest stage, or make failed stages recoverable rather than optimizing token cost alone.

Treat instructions, skills, agent definitions, evaluations, training, and workflow design as enablement investment:

Amortized enablement cost per outcome =
enablement investment ÷ expected durable outcomes over its useful period

$30,000 ÷ 3,000 durable outcomes = $10 per outcome

Expect a maturation J-curve: early enablement cost can increase before acceptance and throughput improve. Amortize the investment, but do not hide it.

Evidence & Uncertainty

Use evidence in this order:

1. Operational outcome and durable-failure data
2. Sampled human review, correction, and escalation effort
3. AI-credit, compute, license, and enablement cost
4. Surveys for friction, confidence, and explanatory context — not as the sole financial source of truth

Use low/expected/high assumptions for labor rate, review time, acceptance, useful life, and outcome value. Report a defensible range when the inputs do not support a precise point estimate.

Discussion Points

• Which outcome in your organization is valuable and measurable enough to pilot this scorecard?
• Who should approve the acceptance rule and stabilization period?
• Which human-effort component is most likely to reverse a token-only model decision?
• What evidence would Finance require before recognizing cash savings rather than capacity released?

5. Budget Tuning & Operating Model Capstone (Hands-On) (35 min)

Live Run of Show

Key Points

• Budget tuning spans two independent dimensions that must both be sized correctly:
  • User-level budgets (ULBs) — universal, cost-center, and individual — cap what each person can consume (pool + metered), and always hard-stop
  • Spending budgets — cost-center, organization, and enterprise — cap metered charges only, after the pool is exhausted, and only hard-stop if "Stop usage when budget limit is reached" is explicitly enabled
• Included usage controls cap a cost center's draw from the shared pool to the AI credits its own licenses fund (for example, 80 Business licenses → an automatically calculated 152,000-credit cap); when the cap is reached, you choose whether the team is blocked or continues into paid overage
• Power-user exceptions: use an individual ULB override for known heavy but legitimate users rather than raising the universal ULB for everyone — keep the exception path lightweight (a single request form or ticket type) so legitimate power users aren't blocked for days waiting on approval

"Lowest Remaining Headroom Wins"

• ULBs and spending budgets are independent controls — a user can be blocked by whichever one has the least capacity remaining, even if their own individual budget still has room
• Worked example: a user has $5 remaining on their individual ULB, but the enterprise spending budget only has $1 remaining. The enterprise budget blocks them first, even though their personal ULB is not exhausted
• The reverse also holds: raising a cost-center or enterprise budget does not unblock a user who has already hit their own ULB — only raising that user's individual (or universal) ULB does
• Practical rule: whenever you raise ULBs, recheck that your spending budgets can still cover the resulting gap between what your ULBs collectively allow and what the shared pool provides

Sizing Check (the calculation every tuning exercise should run)

1. Max ULB consumption = Σ (users at each ULB tier × that tier's ULB amount)
2. Included-credit pool value =
   [(Business seats × 1,900) + (Enterprise seats × 3,900)] × $0.01/credit
3. Gap to cover         = Max ULB consumption − Pool value
   → Your cost-center + organization + enterprise budgets must cover this gap,
     or users will be blocked by a spending budget before reaching their own ULB.

Use included-credit amounts—not license prices—to value the pool. At the standard monthly amounts, 1,900 Business credits equal $19 and 3,900 Enterprise credits equal $39 at the fixed $0.01-per-credit conversion. Those equivalents do not state the license price: GitHub lists Copilot Business at $19 per user per month and Copilot Enterprise pricing as variable. Use your contracted seat rates separately when analyzing total license economics.

Post-Session Reference — Operating Model Building Blocks

Use this table to support the capstone and later implementation planning. During the live session, orient learners to the building blocks rather than teaching every row.

Post-Session Reference — RACI: Who Owns What

Use this table to assign the capstone owners. Resolve organization-specific role changes after the workshop instead of debating every row live.

R = Responsible, A = Accountable, C = Consulted, I = Informed.

Discussion Points

• What's the single biggest blocker to executing your 30-day optimization backlog as written?
• Who owns anomaly response after today — is it the same person who owns monthly KPI review, or should it be different given how quickly anomalies need triage?
• How will you know, one cycle from now, whether your budget tuning changes worked?
• What material change in allotments, pricing, or usage would make you revisit your enterprise spending budget sooner than one cycle?

Recap

• Reporting lives in the AI usage dashboard — filter by user, model, organization, or cost center, and export for offline analysis; individual users have their own visibility too
• Cost is tokens × per-model, per-category pricing — always re-verify against live pricing, never memorize it
• Baseline before you tune: burn rate, exhaustion date, credits/user, blocked users, overage, feature/model mix, and quality together — not cost in isolation
• Total credits alone don't identify waste — pair volume with quality and session-behavior signals, and never assume correlation is causation without checking for a confounding factor
• Define durable accepted outcomes and calculate total assisted-delivery cost — token cost alone does not establish economic value
• Report full cost per durable accepted outcome with accepted volume or net contribution; classify benefits honestly and expose uncertainty
• Budgets are tuned across two independent dimensions — ULBs and spending budgets — and the lowest remaining headroom wins when they disagree
• A written 30-day optimization backlog, owned through a clear RACI, turns this session into a lasting operating cadence

Note — where this goes next: This module measures and governs consumption at the organization level. Teams that run Copilot as unattended automation need the same discipline one level down — per-lane telemetry, lane-level cost per durable accepted outcome, pre-spend gates, and per-run credit caps — because GitHub's billing views attribute usage to identities, models, organizations, and cost centers, not to your lane names. Optional Module 4: Agent Fleet Efficiency and Cost Governance covers that layer. It is optional and excluded from the core curriculum duration.

Q&A Topics to Prepare For

• "Can we see cost by repository or team, not just by user or cost center?"
• "How do we handle third-party coding agents in our KPI and budget math?"
• "How should we normalize baselines that include promotional-period data?"
• "If overage and a quality dip happen together again, how fast can we tell whether they're actually related?"
• "Can we count developer time saved as cash savings, or is it capacity released?"
• "How long should our stabilization period be before an outcome counts as durable?"

Appendix: Official Sources & Facilitator Reference

Key URLs

Note: The pricing snapshot reflects the docs as verified on 2026-08-08. Standard included-credit amounts and the historical promotional-window dates were re-verified on 2026-09-01. Re-verify against these live URLs before every delivery of this module.

Instructor Preflight Checklist

• Re-verify the pricing snapshot table (Section 2) against the live Models and pricing page
• Re-verify the promotional-window dates and standard credit amounts (Welcome section) against the live Usage-based billing page
• Confirm which attendees have AI usage dashboard access ahead of time so you can plan the Section 1 live demo versus a screenshot walkthrough
• Click through Billing & Licensing → AI usage and Usage → Metered usage in the delivery tenant; verify the current labels, CSV export flow, and product:copilot / cost_center:<name> query syntax
• If discussing OTel, run copilot help monitoring on the installed CLI version and review the organization's telemetry privacy, retention, and access requirements; do not enable prompt/response capture for the workshop
• Print or share the Sample Dataset and Answer Key (LAB guide) in case attendees lack access to a real AI usage export
• Confirm the outcome-economics exercise assumptions are presented as illustrative, not GitHub product benchmarks or universal accounting rules
• For remote delivery, pre-assign breakout groups of 3–4 and name the FinOps, Platform/DevEx, Engineering Manager, and Program Owner roles before Labs 4–6
• Bookmark all Key URLs above in a dedicated browser folder for quick reference during Q&A
• If your organization has already run Module 1 and/or Module 2, review those decisions (budget amounts, developer optimization habits already adopted) so this module's tuning exercise reflects real numbers where possible

Backup Plan

If the live AI usage dashboard is unavailable (no access, connectivity issue, or UI change):

1. Use the Sample Dataset (Section 3) for the entire KPI and anomaly-analysis exercise instead of a live export
2. Walk through the Reporting & Data Sources demo steps as a slide-only, screenshot-based discussion
3. Reference the official docs URLs above and screen-share the documentation pages as a substitute for the live product UI
4. Keep the hands-on worksheets (LAB guide) as the primary artifact attendees leave with, regardless of demo availability

Workshop guide for Module 3: Measurement & Optimization at Scale, part of the GitHub Copilot AI Credits Optimization workshop series (standalone-capable)