GitHub Copilot AI Credits Optimization — Optional Module 4: Agent Fleet Efficiency and Cost Governance Workshop

Duration: 170 minutes (including one 10-minute break); the optional runnable lab appendix adds 45 minutes and is delivered separately
Format: Optional advanced module — presentation + design exercises + optional runnable appendix
Audience: Platform and DevEx engineers, automation and SRE-adjacent engineers, Copilot program owners who operate unattended Copilot automation
Focus: Programmatic GitHub Copilot CLI and SDK automation — fixed session cost, lane profiles, isolation, pre-spend gates, layered budgets, failure handling, durable landing, and safe rollout of cost changes

Note: This is Optional Module 4 of the GitHub Copilot AI Credits Optimization curriculum. It is not part of the advertised 7-hour-10-minute core (Modules 1–3) and is not required for any core learning path. Deliver it only for teams that already run — or are about to run — Copilot programmatically in scripts, schedulers, workers, or CI/CD.

Important: Every product behavior, flag, default, and feature status in this module was verified against GitHub documentation on 2026-08-12, and the command surfaces were cross-checked against Copilot CLI v1.0.79 help output on the same date. Copilot CLI and SDK surfaces change quickly, and several controls used here are public preview or experimental. Re-verify every flag in the Appendix control reference — and re-run copilot help, copilot help environment, and copilot help limits for your installed version — before each delivery.


Workshop Overview

Modules 1–3 optimize what a person does with Copilot and what an organization measures. This optional module addresses a different failure mode: unattended automation that spends AI credits while nobody is watching.

A fleet is any set of programmatic Copilot invocations — a nightly batch, a queue worker, a scheduled audit, a CI step, or an SDK service. In a fleet, every configuration decision is multiplied by invocation volume, and every unbounded input, silent retry, duplicated worker, or discarded result becomes a recurring cost line instead of a one-time annoyance.

This module treats fleet cost as an engineering problem with four levers:

Lever Question it answers
Fixed cost per invocation What does every call pay for before it does any useful work?
Variable cost per invocation How much work, fan-out, and retry does the task actually require?
Avoided calls Which paid calls should never happen at all?
Realized value Did the paid work land, get consumed, and get accepted?

Note: This module does not reteach Modules 1–3. It assumes their vocabulary — AI credits, token categories, cached input, budgets, user-level budgets, durable accepted outcomes, and full cost per durable accepted outcome — and extends it to unattended work.

Learning Objectives

By the end of this module, participants will be able to:

  1. Measure the fixed and variable cost of a programmatic Copilot lane, and design a repeatable baseline experiment instead of estimating from intuition.
  2. Build a least-context, least-tool, least-permission lane profile using documented Copilot CLI controls, with feature status recorded for every preview or experimental control used.
  3. Design safe pre-spend gates and a complete cache key that cannot serve a stale or wrong verdict.
  4. Design layered budget, fan-out, retry, backoff, and circuit-breaker controls that fail closed without generating retry storms.
  5. Verify that paid output lands durably, is consumed downstream, and is attributed to a durable accepted outcome.
  6. Produce a canary-ready fleet optimization plan with an explicit quality-parity gate and rollback trigger.

Each objective is exercised in the paired lab: Objective 1 → Lab 1, Objective 2 → Lab 2 and Lab 3, Objective 3 → Lab 4, Objective 4 → Lab 5, Objectives 5 and 6 → Lab 6.

Prerequisites

Requirement Details
Curriculum prerequisite Module 2 (Developer Efficiency) and Module 3 (Measurement and Optimization at Scale), or equivalent working knowledge of AI credits, cached input, budgets, and cost per durable accepted outcome
Role prerequisite You own, build, or operate programmatic Copilot invocations (Copilot CLI in scripts/CI, or a Copilot SDK service)
Engineering prerequisite Comfort with shell scripting, queues or schedulers, exit codes, and caching concepts
Copilot license Business or Enterprise for the discussion content; no license and no paid call is required to complete any core exercise
Optional appendix only A personal or non-production sandbox repository, a Copilot license you are authorized to spend from, and Copilot CLI installed

Important: No core section of this module requires making a paid AI request. All six core exercises are design and review activities completed with the supplied worksheets and answer keys. The runnable appendix in the lab guide is optional, bounded, and clearly labeled, and it must never be run against a production repository, production identity, or customer data.

Important: Tool-permission flags discussed in this module are a security boundary, not only a cost control. Treat every change to them as a security-relevant change, reviewed accordingly.

Module Workshop Relationship to this module
Module 1 Administration and Budget Guardrails Supplies the organization-level budget layer this module operates inside
Module 2 Developer Efficiency Supplies the per-session habits this module automates and enforces in a harness
Module 3 Measurement and Optimization at Scale Supplies the outcome definition and cost-per-durable-accepted-outcome metric this module reports against
Curriculum overview GitHub Copilot AI Credits Optimization Role-based learning paths and duration treatment

Session Agenda

Section Topic Time
— Welcome, Prerequisites, and the Fleet Cost Model 10 min
1 Measure the Fixed Session Tax 20 min
2 Lane Profiles for Model, Context, Tools, MCP, and Permissions 30 min
3 Isolation, Worktrees, and Concurrent Ownership 20 min
☕ Break 10 min
4 Pre-Spend Gates, Cache Correctness, and Prompt Budgets 30 min
5 Runtime Governance for Budgets, Fan-Out, Retries, and Breakers 30 min
6 Durable Landing, Telemetry, Canary Rollout, and Capstone 20 min

Total: 170 min (2h 50min), including the 10-minute break

Optional runnable lab appendix: +45 min — delivered as a separate hands-on block, not included in the 170 minutes.


Welcome, Prerequisites, and the Fleet Cost Model (10 min)

Key Points

The Fleet Cost Identity

Use this as the mental model for every later section:

Fleet cost per window
  = Σ over lanes of [ invocations
                      × (session tax + task work + fan-out + retries)
                      × (1 − avoided-call rate) ]
  + human correction cost

Value per window
  = durable accepted outcomes × value per outcome

Decision metric (Module 3)
  = full cost per durable accepted outcome
Term Where it is controlled Covered in
Session tax Instructions, tools, MCP servers, memory, injected context Sections 1–3
Task work Model, reasoning effort, context tier, prompt budget Sections 2 and 4
Fan-out Subagents, votes, parallel workers Section 5
Retries Outcome taxonomy, backoff, breakers Section 5
Avoided calls Pre-spend gates and caches Section 4
Human correction Quality parity, verification, landing Sections 4–6

Lane Lifecycle

Every section of this module maps to one stage of this lifecycle:

                    ┌──────────────────── Governance envelope (§5) ────────────────────┐
                    │                                                                  │
  ┌──────────┐   ┌──┴───────────────┐   ┌──────────────────┐   ┌──────────────┐   ┌────┴──────────┐
  │  Queue   │──►│ Pre-spend gates  │──►│ Lane profile     │──►│  Paid call   │──►│ Verify + land │
  │ (work    │   │ (§4)             │   │ (§2) in isolation│   │ (bounded by  │   │ (§6)          │
  │  items)  │   │ probe · dedupe   │   │ (§3)             │   │  credit cap) │   │ atomic · ack  │
  └──────────┘   │ cache · ledger   │   └──────────────────┘   └──────────────┘   └────┬──────────┘
                 └──────────────────┘             ▲                                    │
                          │                       │                                    ▼
                          │ skip (free)           └────── measured session tax (§1) ── Telemetry
                          ▼                                                            ledger (§6)
                    Recorded outcome

Discussion Points


1. Measure the Fixed Session Tax (20 min)

Key Points

Session-Tax Contributors and Their Instruments

Contributor How it enters the session How to inspect it
System prompt and built-in tool definitions Always present /context — System Prompt, System Tools rows
Custom instructions (AGENTS.md, .github/copilot-instructions.md, and related) Discovered from the working directory and configured directories /context — Custom Instructions row; copilot plugins list --kind instruction
MCP server tool schemas Every enabled server contributes its tool definitions /context — MCP Tools row; copilot plugins list --kind mcp
Skills and plugins Discovered per working directory and scope copilot plugins list --kind skill --kind plugin
Memory Disabled by default in prompt mode; enabled with --enable-memory Lane profile review; transcript inspection via --share
Conversation and tool output Accumulates during the run /context — Messages row; /usage for per-model token totals

Note: Prompt mode (-p) is deliberately conservative by default: memory is off unless --enable-memory is passed, and tools do not run automatically without an explicit permission grant (--allow-all-tools, also settable through COPILOT_ALLOW_ALL). Additional prompt-mode trust toggles have existed under version-specific environment variables; CLI v1.0.79 does not list any of them in copilot help environment, so do not teach them as stable controls. Confirm what your harness sets, and re-read copilot help environment for your installed version before assuming a clean baseline.

Designing a Baseline Experiment

A defensible baseline changes one factor at a time and reports a distribution, not a single run.

Design element Requirement
Probe prompt Trivial, deterministic, and identical across arms (for example, "Reply with the single word OK.")
Pinned configuration Same model, same reasoning effort, same context tier, same working directory
Repetitions Enough runs per arm to see spread; report median and range, never one run
Recorded fields AI credits, input/cached-input/output tokens, latency, outcome, exit status
Isolation No unrelated work in the same session; fresh session per arm
Single variable Change exactly one profile element per arm (for example, MCP servers enabled vs. disabled)
Quality control arm Repeat the comparison with a representative real task, not only the trivial probe, before adopting a reduced profile

Important: A trivial probe measures the session tax. It does not tell you whether a reduced profile still solves real tasks. Never adopt a cheaper profile on probe evidence alone — Section 6's canary gate exists for exactly this reason.

What Is Measurable Today, and What Must Be Verified

Question Status on 2026-08-12
Can I see the context breakdown of a session? Yes — /context in an interactive session
Can I inventory discovered instructions, MCP servers, skills, plugins, and LSP servers without a model call? Yes — copilot plugins list [--kind] [--scope] [--json]
Can I see per-model token totals for a session? Yes — /usage
Can I capture a full transcript of a programmatic run? Yes — --share=PATH writes the session transcript to Markdown after a programmatic run; treat transcripts as potentially sensitive
Can I get machine-readable run output? Yes — --output-format=json emits JSONL, one JSON object per line
Does that JSONL include a per-call AI-credit cost field? Verify in your version. The published CLI reference documents the JSONL output format, not a guaranteed per-call cost field. Do not design a billing ledger around an unverified field
Can an SDK application read per-call usage? Yes, with caveats — the Copilot SDK emits an assistant.usage event per model API call with model, inputTokens, outputTokens, and cost (documented as the premium request multiplier applied to that call). Accumulated totals come from session.usage.getMetrics, which is marked experimental (Usage and billing metrics)

Exercise Connection

Lab 1 — Session-Tax Inventory and Baseline Design (10 min): participants inventory the contributors in a supplied lane description, mark each as required/optional/unknown, and write a one-factor-at-a-time baseline experiment. No paid calls.

Facilitation — no live demo required: this section teaches instruments, not a demonstration. If you choose to show copilot plugins list live and it fails (no CLI, no network, unexpected output), stop after one attempt and switch to the illustrative sample output in the lab's Backup Path section. Do not troubleshoot the CLI in front of the room — the teaching point is what the categories mean, and the sample carries it.

Discussion Points


2. Lane Profiles for Model, Context, Tools, MCP, and Permissions (30 min)

Key Points

Model Routing and Measured Escalation

Route before you tune. In a fleet, the model decision is made once per lane and applied thousands of times, so it deserves a written rule rather than a preference.

Lane work shape Routing choice Why
Architecture, multi-file reasoning, hard debugging Reasoning-capable model, higher effort The failure mode is a wrong plan, and rework costs more than the call
Plan already specified in the work item Mid-tier model, low or medium effort The hard thinking is already done; the lane executes it
Routine, well-scoped, mechanical change Lighter model, low effort Capability beyond the task is paid-for and unused
Mixed or unpredictable task shapes in one lane auto Documented 10% discount on model costs for paid plans, and model changes happen only at cache boundaries

An escalation rule is only a cost control when it is measured and recorded in code:

Element Requirement Anti-pattern it prevents
Trigger A deterministic signal — the named check still fails after attempt 1 "The operator felt the answer was weak"
Ceiling One escalation step per fingerprint, then stop Silent laddering to the most expensive model
Record Attempt number, model used, and outcome written to the ledger An escalation nobody can audit or price
Review Escalation rate is a reported metric per lane Escalation becoming the default path without a decision

Note: A lane whose escalation rate climbs over time is telling you the cheap arm no longer clears the quality bar. That is a routing decision to remake with data, not a rule to loosen quietly.

Lane Profile Fields

Every lane profile in your fleet should be a reviewed file with these fields:

Field Copilot control Notes
Purpose and stop condition Prompt text One task shape per lane; explicit definition of done
Model --model (or COPILOT_MODEL, or model in settings) Pin explicitly per lane; auto is a deliberate choice, not a fallback
Reasoning effort --effort / --reasoning-effort (none, minimal, low, medium, high, xhigh, max) Choices as listed by CLI v1.0.79 help; configurable reasoning is available for supported models only; max is documented as the highest-depth tier for Anthropic models
Context window tier --context TIER (default, long_context) For tiered-pricing models; treat long_context as a justified exception
Tool visibility --available-tools, --excluded-tools Smallest set the lane can succeed with; use the tool names your installed version reports, not a copied list
Tool permission --allow-tool, --deny-tool Deny wins; scope shell and write tools by pattern
Path and URL scope -C, --add-dir, --allow-url, --deny-url, --disallow-temp-dir Least privilege, not convenience
MCP servers --disable-builtin-mcps, --disable-mcp-server=NAME, --additional-mcp-config Built-in disable currently covers github-mcp-server
GitHub MCP tool scope --add-github-mcp-toolset, --add-github-mcp-tool, --enable-all-github-mcp-tools The CLI enables a default subset; enabling everything is a cost and risk decision
Custom instructions --no-custom-instructions, COPILOT_CUSTOM_INSTRUCTIONS_DIRS Disable only when measured as unnecessary for the lane
Memory --enable-memory Disabled by default in prompt mode — leave it off unless the lane needs it
Credit cap --max-ai-credits Soft limit; public preview; CLI v1.0.79 rejects values below 30
Output contract --output-format=json, --share, --secret-env-vars, -s Machine-readable output plus redaction of named secret variables; -s prints the agent response only and suppresses stats, so keep it off measurement runs
Interaction contract --no-ask-user Prevents an unattended run from stalling on a clarifying question
Isolation -C, COPILOT_HOME, --worktree, --sandbox Worktree and sandbox flags are experimental (Section 3)
Fan-out caps Harness scheduler limits Cap concurrent subagent-style work and depth in your harness; CLI v1.0.79 help lists no supported environment variable for this (claim ledger)
Wait timeout Harness watchdog Bound how long the harness waits on a run and kill it deterministically; do not rely on an undocumented CLI timeout variable

Three Reference Lanes

Use these as the starting shapes in the lab. The values are design defaults to argue about, not recommendations to copy blindly.

Profile element Read-only audit lane Bounded code-fix lane Structured synthesis lane
Task shape Inspect and report; no changes Small, specified fix with a deterministic check Text-to-structured-output transformation
Model class Lighter model Mid-tier, escalate on signal Mid-tier
Reasoning effort Low Low to medium; raise only on escalation Low
Tool visibility Read/search tools only (names verified in your version) Read, search, edit, and the specific shell commands needed Minimal or none
Tool permission Deny write; deny shell mutation Allow scoped shell and write; deny shell(git push) Deny write, deny shell
MCP servers None unless the audit needs one Only servers the fix path uses None
Custom instructions Repository standards only if the report depends on them Yes — the fix must follow repository conventions Usually none
Credit cap Low Moderate Low
Isolation Working directory scope Isolated worktree or clone Working directory scope
Primary risk if under-configured Silent scope creep into writes Unverified or unmergeable changes Oversized inputs

Read-Only Means Enforced, Not Requested

A read-only rule written in prompt text is a request. A read-only rule enforced by the harness is a guarantee. Enforce it with permission first, because deny rules use identifiers the CLI itself documents and deny always wins:

# Read-only audit lane: mutation denied, no clarifying questions,
# machine-readable output, bounded credits, transcript captured for review.
copilot -p "$PROMPT" \
  --no-ask-user \
  --model "$LANE_MODEL" --effort low \
  --deny-tool 'write' --deny-tool 'shell' \
  --disable-builtin-mcps \
  --max-ai-credits "$LANE_CREDIT_CAP" \
  --output-format=json \
  --share "$RUN_DIR/transcript.md"

Narrowing visibility is the second, token-saving step, and it is version-specific:

# Add visibility restriction only after verifying the tool names your
# installed CLI uses — substitute the names you observed, do not copy a list.
  --available-tools '<TOOL-NAME>,<TOOL-NAME>'
Layer Flag What it buys you What it does not do
Permission --deny-tool 'write', --deny-tool 'shell' Enforcement — a denied tool cannot run, and deny beats allow, --allow-all, and saved approvals It does not remove the tool's schema from the session, so the token cost may remain
Visibility --available-tools, --excluded-tools Fewer tool schemas in the session, and less scope creep It is not a substitute for denial; a wrong or unrecognized tool name may simply not restrict anything

Important: Verify option spellings and values for your installed CLI version with copilot help before shipping a profile. An unknown option is rejected: CLI v1.0.79 exits non-zero with error: unknown option '--deny-toool' and makes no model call, so a misspelled flag fails loudly. The real fail-open risk is a valid flag carrying a wrong or unverified value — a tool name, pattern, or MCP server name that matches nothing still parses, and the lane runs without the restriction you believed you applied. Verify restrictions by observing behavior, not by reading your own command line.

Disabling Context Is a Trade, Not a Win

Reduction What you might gain What you might lose
--no-custom-instructions Fixed tokens per call Repository conventions, safety rules, build/test commands the agent needs to be correct
--disable-builtin-mcps / --disable-mcp-server Tool-schema tokens per call Capabilities the lane silently needs; the failure often looks like a bad answer, not an error
Narrower --available-tools Tool-schema tokens and scope creep Legitimate task paths, causing retries that cost more than the tokens saved
Lighter model or lower effort Per-token and per-call cost First-pass quality, which shows up as human correction cost, not credit cost

The only honest way to decide is to compare cost per durable accepted outcome on representative tasks, with a quality-parity gate. That is Section 6.

Exercise Connection

Lab 2 — Design Three Lane Profiles (15 min): teams complete the lane-profile worksheet for the three reference lanes, then justify each reduction with the evidence they would need before adopting it. No paid calls.

Facilitation — compression cue (30-minute section): if you enter this section more than five minutes behind, present the lane profile field tables as a read-out instead of a walkthrough and reduce Model Routing and Measured Escalation to the auto discount sentence plus the escalation trigger. Protect Visibility vs. Permission, Read-Only Means Enforced, and the full 15 minutes of Lab 2 — the worksheet is the artifact participants keep.

Discussion Points


3. Isolation, Worktrees, and Concurrent Ownership (20 min)

Key Points

Control What it does Status on 2026-08-12
-C DIRECTORY Change working directory before doing anything else Documented CLI option
--add-dir=PATH Add a directory to the allowed-paths list Documented CLI option
--disallow-temp-dir Prevent automatic access to the system temporary directory Documented CLI option
COPILOT_HOME Override the configuration and state directory (default $HOME/.copilot) Documented environment variable
-w / --worktree[=NAME] Create or reuse an isolated Git worktree under <repo>.worktrees/ and start the session inside it; conflicts with --resume, --continue, --connect Experimental — only available in experimental mode; not listed in CLI v1.0.79 top-level help
--sandbox / --no-sandbox Enable or disable the OS-level shell sandbox for this session only Experimental — only available in experimental mode (copilot help sandbox)
--secret-env-vars=VAR ... Redact named environment variable values in output (GITHUB_TOKEN and COPILOT_GITHUB_TOKEN are redacted by default) Documented CLI option

Important: --worktree and --sandbox are documented as experimental. Do not build a fleet whose safety depends on an experimental flag without a non-experimental fallback and an explicit re-verification date. Sensible fallbacks are generic engineering practice, not Copilot features: a per-worker clone plus filesystem permissions in place of --worktree, and running the worker inside your existing container or VM isolation in place of --sandbox. Neither is a documented Copilot control, and neither inherits Copilot's tool-permission model — you still need --deny-tool rules.

Important — worktree conflicts: worktrees are shared Git state, not private sandboxes. Two workers that reuse the same worktree name collide; a worktree left behind by a killed run keeps its lock and its branch checkout, so the next run either reuses dirty state or fails to create the worktree; and --worktree conflicts with --resume, --continue, and --connect. Every lane that creates worktrees needs a unique name per run, a cleanup step, and a stray-worktree recovery procedure (git worktree list, git worktree remove, git worktree prune).

Concurrent Ownership

Two watchers whose scopes overlap will race to fix the same target, and you will pay twice for one outcome — sometimes with conflicting results.

Control Purpose Minimum implementation
Work fingerprint Identify the problem, not the queue message Stable hash of target identifier plus error signature
Idempotency key Make a retried dispatch harmless Fingerprint plus attempt-independent key stored with the result
Single-owner lease Prevent two workers spending on one target Time-bounded claim with owner ID and expiry; renew or lose it
Disjoint scope Prevent overlap by construction Exactly one owner per target set; overlap is a configuration error, not a runtime race to resolve
Attempt ledger Stop infinite spend on an unfixable target Per-fingerprint attempt counter with a hard cap and a parked state
Workspace isolation Prevent workers corrupting each other's files One worktree or clone per worker, with a unique name and guaranteed cleanup

Note: Isolation without cleanup becomes its own problem. Every worktree, clone, temporary directory, and transcript needs an owner and a deletion rule — decided when the lane is designed, not after disk fills. Include stray-worktree recovery in the runbook: list them (git worktree list), remove the abandoned one (git worktree remove <path>, adding --force only for a worker you know is dead), then git worktree prune.

Exercise Connection

Lab 3 — Two-Watcher Fleet Review (10 min): participants review a supplied two-watcher fleet description, identify the overlap and inheritance defects, and specify the fingerprint, lease, and cleanup rules that fix them. No paid calls.

Discussion Points


☕ Break (10 min)

Use the break to note which lane in your own environment is the best canary candidate for Section 6.


4. Pre-Spend Gates, Cache Correctness, and Prompt Budgets (30 min)

Facilitation — post-break reset (60 seconds, inside this section's 30 minutes; the agenda is unchanged): before the first slide, invite the room to stand if they are able, roll the shoulders twice, look at something roughly 20 feet away for 20 seconds, and sit back down. State that participation is optional. Then give the section's one-sentence promise — "the cheapest call is the one you never make" — and start. The section budget is roughly 1 minute reset, 14 minutes of content, and 15 minutes for Lab 4.

Key Points

The Pre-Spend Gate Ladder

Evaluate in this order; stop at the first gate that resolves the item.

Order Gate Question Outcome when it fires
1 Configuration validation Is this lane's profile valid and complete? invalid_config — fail fast, alert, do not spend
2 Ownership lease Does another worker already own this target? already_owned — release without spending
3 Skip-if-done Does the required artifact already exist? already_done — record and stop (with an explicit force override implemented in your harness, for example --force)
4 Baseline probe Is the problem already resolved? Run the test, linter, or scan first already_green — record success for free
5 Attempt ledger Have we already failed on this fingerprint N times? parked — stop retrying, escalate to a human
6 Result cache Have we produced a verdict for exactly these inputs? cache_hit — reuse with provenance
7 Budget check Does the lane and fleet envelope allow this spend before the call? budget_refused — park the worker, keep the item queued, no credits spent

Note: Gate 7 is a pre-spend decision and produces budget_refused. A cap that is reached during a run — most commonly the per-run --max-ai-credits soft limit — is a different outcome, run_cap_exhausted, because credits were already spent. Section 5 separates the two.

Important: Gates 3 and 4 are the highest-value gates in most fleets and the easiest to get wrong. "The file exists" is not the same as "the correct current artifact exists," and "the test passed" is not the same as "the test ran."

Complete Cache Keys

A result cache is safe only when the key covers everything that could change the verdict.

Key component Why it must be included
Fully rendered prompt Template, rubric, and instruction text changes must invalidate old verdicts — hashing raw inputs alone silently pins results from an old rubric
Model identifier A different model can produce a different verdict
Reasoning effort Effort changes answer depth and quality
Context tier default vs long_context changes both behavior and price
Tool visibility and permission set A verdict produced without write access is not the same as one produced with it
MCP server configuration Available capabilities change the achievable answer
Instruction and rubric version Version the content, not just the file path
Relevant repository state Commit SHA or content hash of the inputs actually read
Harness and CLI version Behavior changes across versions; pin and record both

Cache hygiene rules that belong in the same design:

Prompt Budgets and Bounded Fields

Practice Implementation Failure it prevents
Reference, don't inline Point to a path and let the agent read on demand when the lane grants read access Paying to send a file the task never needed
Cap every growing field Hard truncation with a visible ...[truncated] marker Silent, invisible input growth
Trim to budget in priority order Drop least-important sections first, deterministically Random loss of the most important content
Refuse rather than overflow Emit a distinct context_too_large outcome before calling Paying to assemble and send a request that is rejected
Keep stable text stable Put invariant instruction blocks in a fixed position and change them rarely Unnecessary cache invalidation
Bound tool output Keep the documented 20 KiB default behavior; enforce your own input caps in the harness One command's output consuming the context window

Note: Whether a specific prompt layout improves provider-side cache reuse is not something GitHub documents at the byte level. What is documented is that mid-session changes to model, reasoning, context size, or the enabled tool/MCP set invalidate the cache, and that cached input tokens are typically billed at a fraction of fresh input. Optimize for stability of configuration first; treat prompt-layout cache theories as hypotheses to measure.

Exercise Connection

Lab 4 — Unsafe Cache-Key Review (15 min): teams review an intentionally unsafe cache design, list the omissions, and describe the specific wrong result each omission can produce. No paid calls.

Facilitation — compression cue (30-minute section): if you are behind, deliver Complete Cache Keys as the two split slides without commentary, reduce Cache Hygiene to its three bold rules (miss on unknown, never cache transient failures, log every bypass), and cut the prompt-budget table to the two rows the room reacted to. Never shorten Lab 4 — it is the highest-value exercise in the module.

Discussion Points


5. Runtime Governance for Budgets, Fan-Out, Retries, and Breakers (30 min)

Key Points

Layered Budget Controls

Layer Control Owner Enforcement
Enterprise / organization / cost center Spending budgets and user-level budgets Admin and FinOps (Modules 1 and 3) GitHub-enforced; a user-level budget always hard-stops
Fleet envelope Credits per lane per window Platform team Harness-enforced check() before every paid call
Run --max-ai-credits (CLI) or sessionLimits.maxAiCredits (SDK) Lane profile Copilot-enforced soft limit, public preview; minimum 30 in CLI v1.0.79
Fan-out Concurrency and depth caps for subagent-style work Lane profile Harness-enforced — CLI v1.0.79 help lists no supported environment variable for this (claim ledger)
Worker concurrency Parallel workers per lane Harness Scheduler-enforced
Retries Attempts per fingerprint Harness Attempt ledger

Important: The identity a fleet runs as consumes credits like any other user, and a user-level budget applied to that identity is a hard stop. Decide deliberately whether a hard stop on your automation identity is the outcome you want, and coordinate it with the Module 1 owner.

Outcome Taxonomy

The single most common dispatcher defect is treating every non-success as "the model failed." The second is treating "we declined to spend" and "we spent and ran out" as the same event.

Outcome Meaning Retry? Counts toward breaker? Queue behavior
success Verified result landed No No Acknowledge and remove
already_done / already_green Work was unnecessary No No Acknowledge with reason
model_failure Model produced unusable or failing output Yes, bounded Yes Retain with attempt increment
context_too_large Input exceeded the prompt budget No — fix the input first No Retain and flag for redesign
policy_denied Permission, policy, or content control blocked the action No No Retain and alert an owner
service_unavailable Transient service or network failure Yes, with backoff and jitter No Retain
budget_refused Pre-spend: a budget or envelope check refused the call before it was made; no credits were spent Not immediately — park the worker No Retain; retry after the park window; do not increment the attempt counter
run_cap_exhausted Post-spend: the run started and hit a credit ceiling mid-flight (per-run --max-ai-credits soft cap or the harness envelope) and stopped before finishing; credits were spent Not automatically No Retain; record the spend, increment the attempt counter, and require task decomposition or an explicit, reviewed cap change before the next attempt
invalid_config Lane profile or harness misconfiguration No No Halt the lane and alert
landing_failed The model produced output, but it did not land or was not consumed Yes, after the landing path is fixed No Retain — never acknowledge

Why budget_refused and run_cap_exhausted must be different outcomes:

Question budget_refused run_cap_exhausted
Were credits spent? No Yes — record them against the lane
Attempt counter Unchanged — the work was never tried Incremented — the fingerprint consumed budget without landing
Correct next step Wait for the window to reset, then retry unchanged Decompose the task, or make an explicit reviewed decision to raise the cap; only then retry
What automatic retry would cause A slower queue Repeated partial spend on work that cannot finish inside the cap
Signal to the owner "We are at our limit" "This task does not fit the lane's shape"

Important: An automatic retry after run_cap_exhausted is the most expensive mistake in this taxonomy: each attempt pays close to the full cap and lands nothing. Retry must be gated on a change — smaller work item, different lane, or a reviewed cap decision — never on time alone.

Backoff, Breakers, and Parking

Where Runtime Cost Data Comes From

Source Scope Status
/usage in a session Per-session usage metrics and per-model token totals Documented CLI command
--output-format=json (JSONL) Machine-readable programmatic output Documented; verify which usage fields your version emits before depending on them
--share=PATH transcript Full run transcript for audit Documented; may contain sensitive content
SDK assistant.usage event Per model API call: model, inputTokens, outputTokens, cost (premium request multiplier) Documented SDK event
SDK session.usage.getMetrics Accumulated AI credit and token totals Documented as experimental
AI usage dashboard and CSV export Billing-grade totals by user, model, organization, and cost center Documented (Module 3)

Important: GitHub's billing views attribute usage to identities, models, organizations, and cost centers — not to your lane names. Lane-level attribution is your harness's responsibility: record the lane, profile hash, and run ID in your own ledger and reconcile totals against the dashboard. Do not promise lane-level chargeback from GitHub reporting alone without verifying the dimensions available to you.

Exercise Connection

Lab 5 — Dispatcher Outcome Taxonomy (15 min): teams map nine supplied run results — including a pre-spend budget refusal and a post-spend cap exhaustion — to outcomes, retry behavior, breaker counting, and queue action, then identify which mapping errors would cause a retry storm, silent work loss, or repeated partial spend. No paid calls.

Facilitation — compression cue (30-minute section): if you are behind, teach the layered budget table and the first half of the outcome taxonomy live, hand the second half to the lab worksheet, and cut the backoff bullets to three (backoff with jitter, breakers on model_failure only, refused work is delayed not lost). Keep the budget_refused versus run_cap_exhausted contrast — it is the new decision this section exists to teach — and keep Lab 5 at 15 minutes.

Discussion Points


6. Durable Landing, Telemetry, Canary Rollout, and Capstone (20 min)

Key Points

The Landing Contract

Step Requirement Evidence it worked
1. Validate Output matches the expected schema or check Validation result recorded
2. Persist Atomic write or verified merge Artifact exists at the resolved path
3. Attribute Provenance: lane, run ID, profile hash, model, cache status Ledger row
4. Signal Distinct exit status for success, no-op, and failure Caller behavior verified for each
5. Consume A downstream consumer actually reads it Consumer log or downstream artifact
6. Acknowledge Remove the queue item only after 1–5 Queue depth reflects real progress
7. Deploy Restart the worker and verify in live logs Post-restart log line showing new behavior

Fleet Telemetry: What Every Run Records

Field group Fields
Identity Lane, run ID, fingerprint, owner/lease ID, harness and CLI version
Configuration Profile hash, model, effort, context tier, tool and MCP set, instruction version
Spend AI credits, token counts by category, fan-out call count
Behavior Outcome, retries, backoff events, breaker state, cache result, gate that fired
Value Landed, consumed, human correction time, durable acceptance after the stabilization period

Those fields roll directly into the Module 3 metric: full cost per durable accepted outcome, per lane. Cost per invocation and tokens saved are diagnostics; they are not the decision metric.

Canary Rollout for Cost Changes

Treat a cost-optimization change exactly like a production code change.

Baseline ──► Canary one lane ──► Quality-parity gate ──► Adversarial review ──► Expand ──► Monitor
   │              │                      │                      │                 │         │
   │              │                      │                      │                 │         └─ rollback trigger stays armed
   │              │                      │                      │                 └─ one lane at a time
   │              │                      │                      └─ integration seams, cache keys, flag spelling and values, exit-code blast radius
   │              │                      └─ durable acceptance must not regress
   │              └─ same task mix, same window, single variable
   └─ measured, not remembered
Gate Pass condition If it fails
Baseline captured Median and range recorded for the canary lane before the change Do not proceed
Single variable Exactly one profile element changed Split the change
Quality parity Durable acceptance and human correction time do not regress beyond the agreed tolerance Roll back
Cost improvement Full cost per durable accepted outcome improves Roll back or keep the old profile
Adversarial review A reviewer explicitly hunts integration seams, cache-key completeness, flag spelling and flag values, and exit-code blast radius Fix and re-review
Deployed and verified Live logs confirm the new behavior in the running worker Not done

Important: Report savings as measured cost per durable accepted outcome, not as assumed token reductions. A profile that saves tokens and loses acceptance is a cost increase wearing a savings label.

Exercise Connection

Lab 6 — Capstone: Fleet Lane Optimization Checklist and Canary Scorecard (12 min): teams complete the capstone templates for one anonymized real lane, naming the canary lane, the single variable, the quality-parity gate, the rollback trigger, and the owner. No paid calls.

Facilitation — pacing (20-minute section): this section carries a 12-minute lab, so the live budget is about 8 minutes — roughly 2 minutes on the landing contract, 2 on telemetry, 3 on canary rollout, and 1 for the hand-off. Deliver the tables as pointers, not readings. The claim ledger and the "Translating Playbooks" table are reference material in the deck appendix; do not teach them live unless the room asks.

Discussion Points


Recap


Appendix

Package Artifacts

Artifact File
Canonical workshop guide (this document) github-copilot-token-platform-automation-workshop.md
Hands-on lab, including the optional runnable appendix and printable template copies github-copilot-token-platform-automation-LAB.md
Slidev deck github-copilot-token-platform-automation.slidev.md
Lane profile worksheet (source) templates/lane-profile-worksheet.md
Cache-key review worksheet (source) templates/cache-key-review-worksheet.md
Failure taxonomy and breaker worksheet (source) templates/failure-taxonomy-worksheet.md
Fleet lane optimization checklist and canary scorecard (source) templates/canary-scorecard.md

Important: The four templates are also inlined as printable copies in Appendix B of the lab guide, because the published site renders the lab page but does not publish the templates/ folder. Participants reading the published lab therefore need no repository access. The templates/ files remain the editable sources — change both copies in the same edit and keep them identical in content.

Key URLs

Topic URL
GitHub Copilot CLI command reference https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-command-reference
GitHub Copilot CLI programmatic reference https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-programmatic-reference
Running GitHub Copilot CLI programmatically https://docs.github.com/en/copilot/how-tos/copilot-cli/automate-copilot-cli/run-cli-programmatically
Allowing and denying tool use in GitHub Copilot CLI https://docs.github.com/en/copilot/how-tos/copilot-cli/use-copilot-cli/allowing-tools
Setting an AI credit session limit in GitHub Copilot CLI https://docs.github.com/en/copilot/how-tos/copilot-cli/use-copilot-cli/set-session-limit
Managing context in GitHub Copilot CLI https://docs.github.com/en/copilot/concepts/agents/copilot-cli/context-management
Optimizing your AI usage to maximize efficiency and reduce cost https://docs.github.com/en/copilot/tutorials/optimize-ai-usage
Copilot SDK usage and billing metrics https://docs.github.com/en/copilot/how-tos/copilot-sdk/features/usage-and-billing
Copilot SDK session limits https://docs.github.com/en/copilot/how-tos/copilot-sdk/features/session-limits
About cloud and local sandboxes for GitHub Copilot (coding-agent sandboxes — not the CLI --sandbox flag) https://docs.github.com/en/copilot/concepts/about-cloud-and-local-sandboxes
Models and pricing for GitHub Copilot https://docs.github.com/en/copilot/reference/copilot-billing/models-and-pricing
Monitoring your GitHub AI Credits usage https://docs.github.com/en/copilot/how-tos/manage-and-track-spending/monitor-ai-usage

Copilot CLI Control Reference (documentation verified 2026-08-12; help output cross-checked against CLI v1.0.79)

Concern Control Documented behavior and status
Programmatic run -p / --prompt, piped stdin Executes a prompt and exits; piped input is ignored if -p is also supplied
Version check copilot --version Prints the installed version; proves installation only, not authentication
Authentication copilot login (--device-code, --web-flow, --host), or COPILOT_GITHUB_TOKEN / GH_TOKEN / GITHUB_TOKEN in that precedence v1.0.79 exposes no non-interactive "auth status" subcommand; the first real run is the authoritative check
Clean output -s / --silent Outputs only the agent response and suppresses stats — do not use it on measurement runs
No stalling --no-ask-user Prevents the agent from pausing for clarification
Model --model, COPILOT_MODEL, settings model Precedence: custom agent → --model → COPILOT_MODEL → settings → CLI default; auto selects automatically and carries a documented 10% model-cost discount on paid plans
Reasoning effort --effort / --reasoning-effort v1.0.79 choices: none, minimal, low, medium, high, xhigh, max; available for supported models only; max documented as the highest-depth tier for Anthropic models
Context tier --context TIER default or long_context for tiered-pricing models; documented as overriding the persisted setting and honored in fresh interactive sessions — verify behavior for -p runs in your version
Tool visibility --available-tools, --excluded-tools Allowlist wins if both are supplied; invisible tools cannot be used even if allowed; tool-name values are version- and configuration-specific and are not enumerated in the published reference
Tool permission --allow-tool, --deny-tool, --allow-all-tools, --allow-all / --yolo Deny always wins, including over saved approvals; documented example identifiers include write, shell(git:*), shell(git push), and SERVER(tool); allow-all options are session-wide permission grants intended for isolated environments
Unknown options any misspelling Rejected: v1.0.79 prints error: unknown option '<flag>', suggests the closest match, and exits non-zero without a model call
Path and URL scope -C, --add-dir, --allow-url, --deny-url, --allow-all-paths, --disallow-temp-dir Deny URL takes precedence over allow URL
MCP scope --disable-builtin-mcps, --disable-mcp-server=NAME, --additional-mcp-config Built-in disable currently covers github-mcp-server
GitHub MCP tools --add-github-mcp-toolset, --add-github-mcp-tool, --enable-all-github-mcp-tools The CLI enables a default subset unless overridden
MCP instructions --allow-all-mcp-server-instructions By default only allowlisted server instructions are included up front; others are retrieved on demand
Custom instructions --no-custom-instructions, COPILOT_CUSTOM_INSTRUCTIONS_DIRS Disables loading of AGENTS.md and related files
Memory --enable-memory Memory is disabled by default in prompt mode
Broad permission via environment COPILOT_ALLOW_ALL Documented equivalent of --allow-all-tools; audit whether your harness sets it
Credit cap --max-ai-credits, /limits set max-ai-credits Soft limit; public preview; minimum accepted value is 30 in v1.0.79; documentation advises setting it above 30; in non-interactive mode the run ends when the limit is reached
Output --output-format=json, -s, --share=PATH, --share-gist, --secret-env-vars JSON format emits JSONL; gists are unavailable to EMU and *.ghe.com
Isolation COPILOT_HOME, -w / --worktree, --sandbox / --no-sandbox COPILOT_HOME is a documented environment variable; worktree and sandbox flags are experimental and are not listed in v1.0.79 top-level help (copilot help sandbox documents sandboxing as experimental)
Fan-out and run timeouts Harness scheduler and watchdog No supported CLI environment variable is listed in v1.0.79 copilot help environment — implement concurrency, depth, and timeout caps in your harness (claim ledger)
Tool output size Documented 20 KiB behavior; threshold variable Larger tool output is written to a file and previewed; the threshold environment variable named in the context-management documentation is not listed in v1.0.79 help — treat as version-sensitive (claim ledger)
Inventory copilot plugins list [--kind] [--scope] [--json] Inventories plugins, MCP servers, skills, instruction sources, and language servers; --kind supports plugin, mcp, skill, instruction, lsp; excludes custom agents and session-scoped hooks
Session inspection /context, /usage, /compact Interactive session commands

Claim Ledger: Verify Before Delivery

These items are not settled facts and must be presented as measurement questions, not product guarantees.

Item Why it is uncertain How to handle it in delivery
Token overhead per instruction file, tool schema, MCP server, or plugin Not published by GitHub; environment-specific Measure locally with /context; never quote a number from another environment
Per-call AI-credit detail in --output-format=json output The JSONL format is documented; a guaranteed per-call cost field is not Verify with your installed version before building a ledger on it
--context behavior in -p runs Documented as honored in fresh interactive sessions Verify empirically; do not promise programmatic behavior you have not observed
Valid --available-tools / --excluded-tools values Tool names vary by version, agent, and MCP configuration; not enumerated in the published reference Discover names in your environment, pin them per lane, and re-verify on upgrade — never copy a tool list from a slide
Model availability per environment Model catalogs change over time and differ by plan and host Pin a model per lane and treat an unknown model name as invalid_config. Do not claim the CLI silently falls back to another model — that behavior is unverified
Subagent concurrency/depth and run-wait environment variables Not listed in v1.0.79 copilot help environment and not found in the current CLI reference pages Do not teach them as controls; cap fan-out and wait time in the harness
Tool-output threshold environment variable Named in the context-management documentation but not listed in v1.0.79 help Teach the documented 20 KiB behavior; bound inputs in the harness; verify the variable before depending on it
Prompt-mode trust environment variables (extensions, repository hooks, workspace MCP) Not listed in v1.0.79 copilot help environment Teach only what your installed version documents; audit COPILOT_ALLOW_ALL
Cross-run prompt-cache reuse and provider-side cache benefit from prompt layout Not documented at that level of detail Teach configuration stability; treat layout and cross-run cache theories as hypotheses to measure
Per-lane cost attribution in GitHub billing views Reporting attributes to identities, models, organizations, and cost centers Attribute in your own ledger; reconcile with the dashboard
Per-lane COPILOT_HOME profiles as an isolation boundary Built on a documented variable, but not a documented fleet feature Teach as a harness pattern and verify in your environment
Experimental controls (--worktree, --sandbox) and preview features (session limits) Explicitly labeled experimental or public preview and subject to change State the status out loud every time; require a fallback

Translating Playbooks From Other Agent CLIs

Teams often arrive with a playbook written for a different agent CLI. The principles usually transfer; the commands usually do not.

Principle (portable) Do not assume Copilot equivalent to verify
Suppress memory and injected context for headless runs Another tool's environment variables or settings-file overrides Memory is off by default in prompt mode; --enable-memory opts in
Give a call only the tools it needs Another tool's allow/disallow flag names or its tool-name values --available-tools / --excluded-tools for visibility plus --allow-tool / --deny-tool for permission; discover valid tool names in your own environment
Restrict MCP servers per lane Another tool's strict-config flags --disable-builtin-mcps, --disable-mcp-server, --additional-mcp-config
Control inherited instruction files Another tool's instruction-file hierarchy semantics Working-directory scope, --no-custom-instructions, COPILOT_CUSTOM_INSTRUCTIONS_DIRS
Cap spend per invocation That the cap is hard --max-ai-credits is a soft limit in public preview
Hook into every tool call Another tool's hook matcher syntax Copilot hooks and SDK hook surfaces have their own documented model — verify before designing around them

Important: Never present a flag from another vendor's CLI as a GitHub Copilot flag. If a control does not exist in the current Copilot documentation, say so, and implement the principle in your harness instead.

Instructor Preflight Checklist

Backup Plan

If any live element is unavailable (no CLI, no license, restricted network, or a UI change):

  1. Deliver all six core exercises from the lab worksheets — none of them requires a live product.
  2. Replace any live inventory demonstration with the illustrative sample copilot plugins list output in the lab's Backup Path section (Lab 1 and Lab 3 both reference it). Say out loud that the sample is synthetic and that real output differs by machine.
  3. Screen-share the linked documentation pages instead of the product when a flag's behavior is questioned.
  4. Skip the optional appendix entirely; it is not required for any learning objective.
  5. If a documented flag has changed since verification, say so explicitly and record it as a follow-up rather than improvising a replacement.

Workshop guide for GitHub Copilot AI Credits Optimization — Optional Module 4: Agent Fleet Efficiency and Cost Governance (optional; excluded from the 7-hour-10-minute core curriculum of Modules 1–3)