GitHub Copilot AI Credits Optimization — Optional Module 4: Agent Fleet Efficiency and Cost Governance Workshop
Duration: 170 minutes (including one 10-minute break); the optional runnable lab appendix adds 45 minutes and is delivered separately
Format: Optional advanced module — presentation + design exercises + optional runnable appendix
Audience: Platform and DevEx engineers, automation and SRE-adjacent engineers, Copilot program owners who operate unattended Copilot automation
Focus: Programmatic GitHub Copilot CLI and SDK automation — fixed session cost, lane profiles, isolation, pre-spend gates, layered budgets, failure handling, durable landing, and safe rollout of cost changes
Note: This is Optional Module 4 of the GitHub Copilot AI Credits Optimization curriculum. It is not part of the advertised 7-hour-10-minute core (Modules 1–3) and is not required for any core learning path. Deliver it only for teams that already run — or are about to run — Copilot programmatically in scripts, schedulers, workers, or CI/CD.
Important: Every product behavior, flag, default, and feature status in this module was verified against GitHub documentation on 2026-08-12, and the command surfaces were cross-checked against Copilot CLI v1.0.79 help output on the same date. Copilot CLI and SDK surfaces change quickly, and several controls used here are public preview or experimental. Re-verify every flag in the Appendix control reference — and re-run
copilot help,copilot help environment, andcopilot help limitsfor your installed version — before each delivery.
Workshop Overview
Modules 1–3 optimize what a person does with Copilot and what an organization measures. This optional module addresses a different failure mode: unattended automation that spends AI credits while nobody is watching.
A fleet is any set of programmatic Copilot invocations — a nightly batch, a queue worker, a scheduled audit, a CI step, or an SDK service. In a fleet, every configuration decision is multiplied by invocation volume, and every unbounded input, silent retry, duplicated worker, or discarded result becomes a recurring cost line instead of a one-time annoyance.
This module treats fleet cost as an engineering problem with four levers:
| Lever | Question it answers |
|---|---|
| Fixed cost per invocation | What does every call pay for before it does any useful work? |
| Variable cost per invocation | How much work, fan-out, and retry does the task actually require? |
| Avoided calls | Which paid calls should never happen at all? |
| Realized value | Did the paid work land, get consumed, and get accepted? |
Note: This module does not reteach Modules 1–3. It assumes their vocabulary — AI credits, token categories, cached input, budgets, user-level budgets, durable accepted outcomes, and full cost per durable accepted outcome — and extends it to unattended work.
Learning Objectives
By the end of this module, participants will be able to:
- Measure the fixed and variable cost of a programmatic Copilot lane, and design a repeatable baseline experiment instead of estimating from intuition.
- Build a least-context, least-tool, least-permission lane profile using documented Copilot CLI controls, with feature status recorded for every preview or experimental control used.
- Design safe pre-spend gates and a complete cache key that cannot serve a stale or wrong verdict.
- Design layered budget, fan-out, retry, backoff, and circuit-breaker controls that fail closed without generating retry storms.
- Verify that paid output lands durably, is consumed downstream, and is attributed to a durable accepted outcome.
- Produce a canary-ready fleet optimization plan with an explicit quality-parity gate and rollback trigger.
Each objective is exercised in the paired lab: Objective 1 → Lab 1, Objective 2 → Lab 2 and Lab 3, Objective 3 → Lab 4, Objective 4 → Lab 5, Objectives 5 and 6 → Lab 6.
Prerequisites
| Requirement | Details |
|---|---|
| Curriculum prerequisite | Module 2 (Developer Efficiency) and Module 3 (Measurement and Optimization at Scale), or equivalent working knowledge of AI credits, cached input, budgets, and cost per durable accepted outcome |
| Role prerequisite | You own, build, or operate programmatic Copilot invocations (Copilot CLI in scripts/CI, or a Copilot SDK service) |
| Engineering prerequisite | Comfort with shell scripting, queues or schedulers, exit codes, and caching concepts |
| Copilot license | Business or Enterprise for the discussion content; no license and no paid call is required to complete any core exercise |
| Optional appendix only | A personal or non-production sandbox repository, a Copilot license you are authorized to spend from, and Copilot CLI installed |
Important: No core section of this module requires making a paid AI request. All six core exercises are design and review activities completed with the supplied worksheets and answer keys. The runnable appendix in the lab guide is optional, bounded, and clearly labeled, and it must never be run against a production repository, production identity, or customer data.
Important: Tool-permission flags discussed in this module are a security boundary, not only a cost control. Treat every change to them as a security-relevant change, reviewed accordingly.
Related Modules
| Module | Workshop | Relationship to this module |
|---|---|---|
| Module 1 | Administration and Budget Guardrails | Supplies the organization-level budget layer this module operates inside |
| Module 2 | Developer Efficiency | Supplies the per-session habits this module automates and enforces in a harness |
| Module 3 | Measurement and Optimization at Scale | Supplies the outcome definition and cost-per-durable-accepted-outcome metric this module reports against |
| Curriculum overview | GitHub Copilot AI Credits Optimization | Role-based learning paths and duration treatment |
Session Agenda
| Section | Topic | Time |
|---|---|---|
| — | Welcome, Prerequisites, and the Fleet Cost Model | 10 min |
| 1 | Measure the Fixed Session Tax | 20 min |
| 2 | Lane Profiles for Model, Context, Tools, MCP, and Permissions | 30 min |
| 3 | Isolation, Worktrees, and Concurrent Ownership | 20 min |
| ☕ | Break | 10 min |
| 4 | Pre-Spend Gates, Cache Correctness, and Prompt Budgets | 30 min |
| 5 | Runtime Governance for Budgets, Fan-Out, Retries, and Breakers | 30 min |
| 6 | Durable Landing, Telemetry, Canary Rollout, and Capstone | 20 min |
Total: 170 min (2h 50min), including the 10-minute break
Optional runnable lab appendix: +45 min — delivered as a separate hands-on block, not included in the 170 minutes.
Welcome, Prerequisites, and the Fleet Cost Model (10 min)
Key Points
- A fleet is any set of unattended, programmatic Copilot invocations. The defining property is not scale — it is that no human reviews the cost of each call before it happens.
- Interactive users self-correct: they notice a bad answer, stop, and rephrase. A scheduler does not. Every configuration mistake in a fleet is charged once per invocation, forever, until someone measures it.
- The unit of analysis for the whole module is the lane: a named class of work with one invocation profile. "Nightly dependency audit" is a lane. "Fix failing test in service X" is a lane. A fleet is a set of lanes.
- Cost optimization chooses among options that already clear the quality bar. It never lowers the bar. If a cheaper profile produces work humans must redo, it is more expensive — Module 3's full cost per durable accepted outcome is the arbiter.
- The largest savings in practice come from two places, and neither is "shorter prompts":
- Not paying for things the agent never uses — inherited instructions, tool schemas, MCP servers, and injected context that no task in the lane can act on.
- Not paying twice — duplicate workers, unnecessary retries, cache misses on identical inputs, and paid results that are thrown away.
The Fleet Cost Identity
Use this as the mental model for every later section:
Fleet cost per window
= Σ over lanes of [ invocations
× (session tax + task work + fan-out + retries)
× (1 − avoided-call rate) ]
+ human correction cost
Value per window
= durable accepted outcomes × value per outcome
Decision metric (Module 3)
= full cost per durable accepted outcome
| Term | Where it is controlled | Covered in |
|---|---|---|
| Session tax | Instructions, tools, MCP servers, memory, injected context | Sections 1–3 |
| Task work | Model, reasoning effort, context tier, prompt budget | Sections 2 and 4 |
| Fan-out | Subagents, votes, parallel workers | Section 5 |
| Retries | Outcome taxonomy, backoff, breakers | Section 5 |
| Avoided calls | Pre-spend gates and caches | Section 4 |
| Human correction | Quality parity, verification, landing | Sections 4–6 |
Lane Lifecycle
Every section of this module maps to one stage of this lifecycle:
┌──────────────────── Governance envelope (§5) ────────────────────┐
│ │
┌──────────┐ ┌──┴───────────────┐ ┌──────────────────┐ ┌──────────────┐ ┌────┴──────────┐
│ Queue │──►│ Pre-spend gates │──►│ Lane profile │──►│ Paid call │──►│ Verify + land │
│ (work │ │ (§4) │ │ (§2) in isolation│ │ (bounded by │ │ (§6) │
│ items) │ │ probe · dedupe │ │ (§3) │ │ credit cap) │ │ atomic · ack │
└──────────┘ │ cache · ledger │ └──────────────────┘ └──────────────┘ └────┬──────────┘
└──────────────────┘ ▲ │
│ │ ▼
│ skip (free) └────── measured session tax (§1) ── Telemetry
▼ ledger (§6)
Recorded outcome
Discussion Points
- Which of your automated Copilot invocations could run today without a human noticing the cost for a full week?
- If your fleet doubled tomorrow, which cost line would grow fastest — session tax, retries, or duplicated work?
- Who currently approves a change to an automated lane's model, tools, or permissions, and is that the same reviewer who approves production code?
1. Measure the Fixed Session Tax (20 min)
Key Points
- The session tax is the fixed cost every invocation pays before the task-specific work begins: system prompt, custom instructions, tool definitions, MCP tool schemas, and anything else injected at session start.
- One-shot runs cannot amortize conversation context across turns. An interactive user spreads the session tax over many turns in one session; a one-shot worker pays the full session-start cost for a single turn of work. Whether any provider-side cache reuse survives between separate runs is not something GitHub documents — do not assume either a cold start or a warm one; measure your own lane.
- Copilot CLI exposes the composition of a session's context window through the
/contextcommand, which reports token usage and context-window percentage for System Prompt, Custom Instructions, System Tools, MCP Tools, Messages, Free Space, and Buffer (Managing context in GitHub Copilot CLI). Those categories are your session-tax line items. copilot plugins listnon-interactively inventories every plugin, MCP server, skill, instruction source, and language server discovered for the current working directory, grouped by kind and configuration scope, with--kind,--scope, and--jsonfilters. It is an inventory command, not a model call. Custom agents and session-scoped hooks are not covered by it and require a live session (GitHub Copilot CLI command reference).- GitHub does not publish a per-instruction-file, per-tool, or per-MCP-server token overhead. Any specific number is environment-specific. Measure yours; never quote someone else's.
Session-Tax Contributors and Their Instruments
| Contributor | How it enters the session | How to inspect it |
|---|---|---|
| System prompt and built-in tool definitions | Always present | /context — System Prompt, System Tools rows |
Custom instructions (AGENTS.md, .github/copilot-instructions.md, and related) |
Discovered from the working directory and configured directories | /context — Custom Instructions row; copilot plugins list --kind instruction |
| MCP server tool schemas | Every enabled server contributes its tool definitions | /context — MCP Tools row; copilot plugins list --kind mcp |
| Skills and plugins | Discovered per working directory and scope | copilot plugins list --kind skill --kind plugin |
| Memory | Disabled by default in prompt mode; enabled with --enable-memory |
Lane profile review; transcript inspection via --share |
| Conversation and tool output | Accumulates during the run | /context — Messages row; /usage for per-model token totals |
Note: Prompt mode (
-p) is deliberately conservative by default: memory is off unless--enable-memoryis passed, and tools do not run automatically without an explicit permission grant (--allow-all-tools, also settable throughCOPILOT_ALLOW_ALL). Additional prompt-mode trust toggles have existed under version-specific environment variables; CLI v1.0.79 does not list any of them incopilot help environment, so do not teach them as stable controls. Confirm what your harness sets, and re-readcopilot help environmentfor your installed version before assuming a clean baseline.
Designing a Baseline Experiment
A defensible baseline changes one factor at a time and reports a distribution, not a single run.
| Design element | Requirement |
|---|---|
| Probe prompt | Trivial, deterministic, and identical across arms (for example, "Reply with the single word OK.") |
| Pinned configuration | Same model, same reasoning effort, same context tier, same working directory |
| Repetitions | Enough runs per arm to see spread; report median and range, never one run |
| Recorded fields | AI credits, input/cached-input/output tokens, latency, outcome, exit status |
| Isolation | No unrelated work in the same session; fresh session per arm |
| Single variable | Change exactly one profile element per arm (for example, MCP servers enabled vs. disabled) |
| Quality control arm | Repeat the comparison with a representative real task, not only the trivial probe, before adopting a reduced profile |
Important: A trivial probe measures the session tax. It does not tell you whether a reduced profile still solves real tasks. Never adopt a cheaper profile on probe evidence alone — Section 6's canary gate exists for exactly this reason.
What Is Measurable Today, and What Must Be Verified
| Question | Status on 2026-08-12 |
|---|---|
| Can I see the context breakdown of a session? | Yes — /context in an interactive session |
| Can I inventory discovered instructions, MCP servers, skills, plugins, and LSP servers without a model call? | Yes — copilot plugins list [--kind] [--scope] [--json] |
| Can I see per-model token totals for a session? | Yes — /usage |
| Can I capture a full transcript of a programmatic run? | Yes — --share=PATH writes the session transcript to Markdown after a programmatic run; treat transcripts as potentially sensitive |
| Can I get machine-readable run output? | Yes — --output-format=json emits JSONL, one JSON object per line |
| Does that JSONL include a per-call AI-credit cost field? | Verify in your version. The published CLI reference documents the JSONL output format, not a guaranteed per-call cost field. Do not design a billing ledger around an unverified field |
| Can an SDK application read per-call usage? | Yes, with caveats — the Copilot SDK emits an assistant.usage event per model API call with model, inputTokens, outputTokens, and cost (documented as the premium request multiplier applied to that call). Accumulated totals come from session.usage.getMetrics, which is marked experimental (Usage and billing metrics) |
Exercise Connection
Lab 1 — Session-Tax Inventory and Baseline Design (10 min): participants inventory the contributors in a supplied lane description, mark each as required/optional/unknown, and write a one-factor-at-a-time baseline experiment. No paid calls.
Facilitation — no live demo required: this section teaches instruments, not a demonstration. If you choose to show
copilot plugins listlive and it fails (no CLI, no network, unexpected output), stop after one attempt and switch to the illustrative sample output in the lab's Backup Path section. Do not troubleshoot the CLI in front of the room — the teaching point is what the categories mean, and the sample carries it.
Discussion Points
- Do you currently know what fraction of a typical automated invocation is session tax versus task work? If not, what would it cost you to find out?
- Which instruction files, MCP servers, or skills in your automation exist for humans rather than for the agent?
- What would convince your team that a reduced profile is safe — a token delta, a credit delta, or a quality comparison on real tasks?
2. Lane Profiles for Model, Context, Tools, MCP, and Permissions (30 min)
Key Points
- A lane profile is the explicit, version-controlled configuration for one class of programmatic work. Nothing in a fleet should inherit "whatever the CLI defaults to" — defaults change, and an inherited default is an unreviewed decision.
- Copilot CLI separates tool visibility from tool permission, and the distinction matters for both cost and security (Allowing and denying tool use in GitHub Copilot CLI):
--available-toolsand--excluded-toolscontrol which tools the model is aware of and can choose from. A tool that is not visible cannot be used at all, even if it is also allowed. These options take tool names as your installed CLI and agent configuration name them; GitHub's reference does not publish a stable enumeration, so discover the names in your own environment before pinning them in a profile.--allow-tooland--deny-toolcontrol permission — whether a tool runs without prompting. Deny rules always take precedence over allow rules, even when--allow-allis set or an approval was previously saved. The CLI's own documented examples use identifiers such aswrite,shell,shell(git:*),shell(git push), andSERVER(tool).- If
--available-toolsand--excluded-toolsare used together, the CLI applies the allowlist and ignores the denylist. - Visibility is a token and scope-creep control; permission is the security boundary. Enforce a read-only lane with denials first, then narrow visibility once you have verified the tool names your version uses.
- Command-line
--allow-tooland--deny-toolapply only to the current session and are not written to the persistedpermissions-config.json. Saved approvals from interactive use can otherwise leak into automation running as the same identity — an audit item for every fleet. - Model selection is a routing decision, not a habit. GitHub's guidance is to match capability to work: reasoning models for architecture and complex debugging, mid-tier models when the plan is already clear, lighter models for routine well-scoped changes; auto model selection is recommended as the default and carries a documented 10% discount on model costs for paid plans, and it changes models only at cache boundaries (Optimizing your AI usage).
- Escalate model capability on a measured signal recorded in code (for example, "escalate only when the deterministic check still fails after the first attempt"), never by default and never by operator habit.
- Configuration churn is expensive: changing reasoning effort, context size, or the set of enabled tools and MCP servers during a session invalidates the cache. Fleet lanes should decide everything before the session starts and change nothing during it.
Model Routing and Measured Escalation
Route before you tune. In a fleet, the model decision is made once per lane and applied thousands of times, so it deserves a written rule rather than a preference.
| Lane work shape | Routing choice | Why |
|---|---|---|
| Architecture, multi-file reasoning, hard debugging | Reasoning-capable model, higher effort | The failure mode is a wrong plan, and rework costs more than the call |
| Plan already specified in the work item | Mid-tier model, low or medium effort | The hard thinking is already done; the lane executes it |
| Routine, well-scoped, mechanical change | Lighter model, low effort | Capability beyond the task is paid-for and unused |
| Mixed or unpredictable task shapes in one lane | auto |
Documented 10% discount on model costs for paid plans, and model changes happen only at cache boundaries |
An escalation rule is only a cost control when it is measured and recorded in code:
| Element | Requirement | Anti-pattern it prevents |
|---|---|---|
| Trigger | A deterministic signal — the named check still fails after attempt 1 | "The operator felt the answer was weak" |
| Ceiling | One escalation step per fingerprint, then stop | Silent laddering to the most expensive model |
| Record | Attempt number, model used, and outcome written to the ledger | An escalation nobody can audit or price |
| Review | Escalation rate is a reported metric per lane | Escalation becoming the default path without a decision |
Note: A lane whose escalation rate climbs over time is telling you the cheap arm no longer clears the quality bar. That is a routing decision to remake with data, not a rule to loosen quietly.
Lane Profile Fields
Every lane profile in your fleet should be a reviewed file with these fields:
| Field | Copilot control | Notes |
|---|---|---|
| Purpose and stop condition | Prompt text | One task shape per lane; explicit definition of done |
| Model | --model (or COPILOT_MODEL, or model in settings) |
Pin explicitly per lane; auto is a deliberate choice, not a fallback |
| Reasoning effort | --effort / --reasoning-effort (none, minimal, low, medium, high, xhigh, max) |
Choices as listed by CLI v1.0.79 help; configurable reasoning is available for supported models only; max is documented as the highest-depth tier for Anthropic models |
| Context window tier | --context TIER (default, long_context) |
For tiered-pricing models; treat long_context as a justified exception |
| Tool visibility | --available-tools, --excluded-tools |
Smallest set the lane can succeed with; use the tool names your installed version reports, not a copied list |
| Tool permission | --allow-tool, --deny-tool |
Deny wins; scope shell and write tools by pattern |
| Path and URL scope | -C, --add-dir, --allow-url, --deny-url, --disallow-temp-dir |
Least privilege, not convenience |
| MCP servers | --disable-builtin-mcps, --disable-mcp-server=NAME, --additional-mcp-config |
Built-in disable currently covers github-mcp-server |
| GitHub MCP tool scope | --add-github-mcp-toolset, --add-github-mcp-tool, --enable-all-github-mcp-tools |
The CLI enables a default subset; enabling everything is a cost and risk decision |
| Custom instructions | --no-custom-instructions, COPILOT_CUSTOM_INSTRUCTIONS_DIRS |
Disable only when measured as unnecessary for the lane |
| Memory | --enable-memory |
Disabled by default in prompt mode — leave it off unless the lane needs it |
| Credit cap | --max-ai-credits |
Soft limit; public preview; CLI v1.0.79 rejects values below 30 |
| Output contract | --output-format=json, --share, --secret-env-vars, -s |
Machine-readable output plus redaction of named secret variables; -s prints the agent response only and suppresses stats, so keep it off measurement runs |
| Interaction contract | --no-ask-user |
Prevents an unattended run from stalling on a clarifying question |
| Isolation | -C, COPILOT_HOME, --worktree, --sandbox |
Worktree and sandbox flags are experimental (Section 3) |
| Fan-out caps | Harness scheduler limits | Cap concurrent subagent-style work and depth in your harness; CLI v1.0.79 help lists no supported environment variable for this (claim ledger) |
| Wait timeout | Harness watchdog | Bound how long the harness waits on a run and kill it deterministically; do not rely on an undocumented CLI timeout variable |
Three Reference Lanes
Use these as the starting shapes in the lab. The values are design defaults to argue about, not recommendations to copy blindly.
| Profile element | Read-only audit lane | Bounded code-fix lane | Structured synthesis lane |
|---|---|---|---|
| Task shape | Inspect and report; no changes | Small, specified fix with a deterministic check | Text-to-structured-output transformation |
| Model class | Lighter model | Mid-tier, escalate on signal | Mid-tier |
| Reasoning effort | Low | Low to medium; raise only on escalation | Low |
| Tool visibility | Read/search tools only (names verified in your version) | Read, search, edit, and the specific shell commands needed | Minimal or none |
| Tool permission | Deny write; deny shell mutation |
Allow scoped shell and write; deny shell(git push) |
Deny write, deny shell |
| MCP servers | None unless the audit needs one | Only servers the fix path uses | None |
| Custom instructions | Repository standards only if the report depends on them | Yes — the fix must follow repository conventions | Usually none |
| Credit cap | Low | Moderate | Low |
| Isolation | Working directory scope | Isolated worktree or clone | Working directory scope |
| Primary risk if under-configured | Silent scope creep into writes | Unverified or unmergeable changes | Oversized inputs |
Read-Only Means Enforced, Not Requested
A read-only rule written in prompt text is a request. A read-only rule enforced by the harness is a guarantee. Enforce it with permission first, because deny rules use identifiers the CLI itself documents and deny always wins:
# Read-only audit lane: mutation denied, no clarifying questions,
# machine-readable output, bounded credits, transcript captured for review.
copilot -p "$PROMPT" \
--no-ask-user \
--model "$LANE_MODEL" --effort low \
--deny-tool 'write' --deny-tool 'shell' \
--disable-builtin-mcps \
--max-ai-credits "$LANE_CREDIT_CAP" \
--output-format=json \
--share "$RUN_DIR/transcript.md"
Narrowing visibility is the second, token-saving step, and it is version-specific:
# Add visibility restriction only after verifying the tool names your
# installed CLI uses — substitute the names you observed, do not copy a list.
--available-tools '<TOOL-NAME>,<TOOL-NAME>'
| Layer | Flag | What it buys you | What it does not do |
|---|---|---|---|
| Permission | --deny-tool 'write', --deny-tool 'shell' |
Enforcement — a denied tool cannot run, and deny beats allow, --allow-all, and saved approvals |
It does not remove the tool's schema from the session, so the token cost may remain |
| Visibility | --available-tools, --excluded-tools |
Fewer tool schemas in the session, and less scope creep | It is not a substitute for denial; a wrong or unrecognized tool name may simply not restrict anything |
Important: Verify option spellings and values for your installed CLI version with
copilot helpbefore shipping a profile. An unknown option is rejected: CLI v1.0.79 exits non-zero witherror: unknown option '--deny-toool'and makes no model call, so a misspelled flag fails loudly. The real fail-open risk is a valid flag carrying a wrong or unverified value — a tool name, pattern, or MCP server name that matches nothing still parses, and the lane runs without the restriction you believed you applied. Verify restrictions by observing behavior, not by reading your own command line.
Disabling Context Is a Trade, Not a Win
| Reduction | What you might gain | What you might lose |
|---|---|---|
--no-custom-instructions |
Fixed tokens per call | Repository conventions, safety rules, build/test commands the agent needs to be correct |
--disable-builtin-mcps / --disable-mcp-server |
Tool-schema tokens per call | Capabilities the lane silently needs; the failure often looks like a bad answer, not an error |
Narrower --available-tools |
Tool-schema tokens and scope creep | Legitimate task paths, causing retries that cost more than the tokens saved |
| Lighter model or lower effort | Per-token and per-call cost | First-pass quality, which shows up as human correction cost, not credit cost |
The only honest way to decide is to compare cost per durable accepted outcome on representative tasks, with a quality-parity gate. That is Section 6.
Exercise Connection
Lab 2 — Design Three Lane Profiles (15 min): teams complete the lane-profile worksheet for the three reference lanes, then justify each reduction with the evidence they would need before adopting it. No paid calls.
Facilitation — compression cue (30-minute section): if you enter this section more than five minutes behind, present the lane profile field tables as a read-out instead of a walkthrough and reduce Model Routing and Measured Escalation to the
autodiscount sentence plus the escalation trigger. Protect Visibility vs. Permission, Read-Only Means Enforced, and the full 15 minutes of Lab 2 — the worksheet is the artifact participants keep.
Discussion Points
- Which of your automated invocations currently run with broader tool permissions than the task requires, and what would break if you narrowed them tomorrow?
- Where would an escalation rule ("try cheap, escalate on failure") beat always using a capable model — and where would it cost more?
- Who reviews lane profile changes today? Should a permission change require the same review as a production deployment?
3. Isolation, Worktrees, and Concurrent Ownership (20 min)
Key Points
- The working directory is a configuration input. Copilot CLI discovers instruction sources, MCP servers, skills, and language servers relative to the current working directory and configuration scope. Moving a worker changes what it inherits — in both directions.
- A worker started inside a directory tree that carries large human-oriented instruction files inherits them on every invocation. A worker moved out of that tree may silently lose the MCP servers or instructions it actually needed. Both are configuration bugs; both are found the same way — run
copilot plugins listfrom the exact working directory the harness uses, not from your shell's default directory. - Resolve configuration paths relative to the harness's own location rather than hardcoding absolute paths, and commit configuration files in the same change as the code that references them. A clone or worktree copy must keep working without edits.
- Copilot CLI provides documented isolation controls with distinct maturity levels:
| Control | What it does | Status on 2026-08-12 |
|---|---|---|
-C DIRECTORY |
Change working directory before doing anything else | Documented CLI option |
--add-dir=PATH |
Add a directory to the allowed-paths list | Documented CLI option |
--disallow-temp-dir |
Prevent automatic access to the system temporary directory | Documented CLI option |
COPILOT_HOME |
Override the configuration and state directory (default $HOME/.copilot) |
Documented environment variable |
-w / --worktree[=NAME] |
Create or reuse an isolated Git worktree under <repo>.worktrees/ and start the session inside it; conflicts with --resume, --continue, --connect |
Experimental — only available in experimental mode; not listed in CLI v1.0.79 top-level help |
--sandbox / --no-sandbox |
Enable or disable the OS-level shell sandbox for this session only | Experimental — only available in experimental mode (copilot help sandbox) |
--secret-env-vars=VAR ... |
Redact named environment variable values in output (GITHUB_TOKEN and COPILOT_GITHUB_TOKEN are redacted by default) |
Documented CLI option |
Important:
--worktreeand--sandboxare documented as experimental. Do not build a fleet whose safety depends on an experimental flag without a non-experimental fallback and an explicit re-verification date. Sensible fallbacks are generic engineering practice, not Copilot features: a per-worker clone plus filesystem permissions in place of--worktree, and running the worker inside your existing container or VM isolation in place of--sandbox. Neither is a documented Copilot control, and neither inherits Copilot's tool-permission model — you still need--deny-toolrules.
Important — worktree conflicts: worktrees are shared Git state, not private sandboxes. Two workers that reuse the same worktree name collide; a worktree left behind by a killed run keeps its lock and its branch checkout, so the next run either reuses dirty state or fails to create the worktree; and
--worktreeconflicts with--resume,--continue, and--connect. Every lane that creates worktrees needs a unique name per run, a cleanup step, and a stray-worktree recovery procedure (git worktree list,git worktree remove,git worktree prune).
- Per-lane configuration isolation — giving each lane its own
COPILOT_HOMEso that saved permissions, settings, and state do not leak between lanes or from interactive use — is a harness pattern built on a documented environment variable, not a documented fleet feature. Teach it as a pattern, verify it in your environment, and record the verification. - Session transcripts and gists can carry sensitive content.
--share-gistpublishes to GitHub.com and is not available to Enterprise Managed Users or GitHub Enterprise Cloud with data residency (*.ghe.com). Prefer--share=PATHinto a controlled location with a retention rule.
Concurrent Ownership
Two watchers whose scopes overlap will race to fix the same target, and you will pay twice for one outcome — sometimes with conflicting results.
| Control | Purpose | Minimum implementation |
|---|---|---|
| Work fingerprint | Identify the problem, not the queue message | Stable hash of target identifier plus error signature |
| Idempotency key | Make a retried dispatch harmless | Fingerprint plus attempt-independent key stored with the result |
| Single-owner lease | Prevent two workers spending on one target | Time-bounded claim with owner ID and expiry; renew or lose it |
| Disjoint scope | Prevent overlap by construction | Exactly one owner per target set; overlap is a configuration error, not a runtime race to resolve |
| Attempt ledger | Stop infinite spend on an unfixable target | Per-fingerprint attempt counter with a hard cap and a parked state |
| Workspace isolation | Prevent workers corrupting each other's files | One worktree or clone per worker, with a unique name and guaranteed cleanup |
Note: Isolation without cleanup becomes its own problem. Every worktree, clone, temporary directory, and transcript needs an owner and a deletion rule — decided when the lane is designed, not after disk fills. Include stray-worktree recovery in the runbook: list them (
git worktree list), remove the abandoned one (git worktree remove <path>, adding--forceonly for a worker you know is dead), thengit worktree prune.
Exercise Connection
Lab 3 — Two-Watcher Fleet Review (10 min): participants review a supplied two-watcher fleet description, identify the overlap and inheritance defects, and specify the fingerprint, lease, and cleanup rules that fix them. No paid calls.
Discussion Points
- Could two of your automations act on the same target today? How would you find out — and would the duplicate spend be visible in your reporting?
- What in your automation would break first if a worker ran from a different directory than expected?
- Which experimental controls would you be comfortable depending on, and what fallback would you require before doing so?
☕ Break (10 min)
Use the break to note which lane in your own environment is the best canary candidate for Section 6.
4. Pre-Spend Gates, Cache Correctness, and Prompt Budgets (30 min)
Facilitation — post-break reset (60 seconds, inside this section's 30 minutes; the agenda is unchanged): before the first slide, invite the room to stand if they are able, roll the shoulders twice, look at something roughly 20 feet away for 20 seconds, and sit back down. State that participation is optional. Then give the section's one-sentence promise — "the cheapest call is the one you never make" — and start. The section budget is roughly 1 minute reset, 14 minutes of content, and 15 minutes for Lab 4.
Key Points
- The cheapest call is the one you never make. Every paid invocation should sit behind a ladder of deterministic checks that can answer "this work is unnecessary" for free.
- A gate that is only usually right is worse than no gate: a wrong "already done" verdict silently drops real work. Gates must be conservative — when uncertain, spend.
- Cache correctness is a data-integrity problem, not an optimization. A cache key that omits an input that can change the answer will confidently serve a wrong verdict, and the failure is invisible in cost reporting because it looks like a saving.
- Bound every prompt field that can grow. Unbounded diffs, log tails, accumulated notes, and generated file contents are the most common cause of oversized, expensive, low-quality calls.
- Copilot CLI already bounds one growth source for you: tool output larger than 20 KiB is saved to a temporary file by default, and the model receives the file path and a preview instead of the full output (Managing context in GitHub Copilot CLI). The documentation names an environment variable for that threshold, but CLI v1.0.79 does not list it in
copilot help environment— treat the variable as version-sensitive (claim ledger) and bound large inputs in your harness, where the cap is yours to test.
The Pre-Spend Gate Ladder
Evaluate in this order; stop at the first gate that resolves the item.
| Order | Gate | Question | Outcome when it fires |
|---|---|---|---|
| 1 | Configuration validation | Is this lane's profile valid and complete? | invalid_config — fail fast, alert, do not spend |
| 2 | Ownership lease | Does another worker already own this target? | already_owned — release without spending |
| 3 | Skip-if-done | Does the required artifact already exist? | already_done — record and stop (with an explicit force override implemented in your harness, for example --force) |
| 4 | Baseline probe | Is the problem already resolved? Run the test, linter, or scan first | already_green — record success for free |
| 5 | Attempt ledger | Have we already failed on this fingerprint N times? | parked — stop retrying, escalate to a human |
| 6 | Result cache | Have we produced a verdict for exactly these inputs? | cache_hit — reuse with provenance |
| 7 | Budget check | Does the lane and fleet envelope allow this spend before the call? | budget_refused — park the worker, keep the item queued, no credits spent |
Note: Gate 7 is a pre-spend decision and produces
budget_refused. A cap that is reached during a run — most commonly the per-run--max-ai-creditssoft limit — is a different outcome,run_cap_exhausted, because credits were already spent. Section 5 separates the two.
Important: Gates 3 and 4 are the highest-value gates in most fleets and the easiest to get wrong. "The file exists" is not the same as "the correct current artifact exists," and "the test passed" is not the same as "the test ran."
Complete Cache Keys
A result cache is safe only when the key covers everything that could change the verdict.
| Key component | Why it must be included |
|---|---|
| Fully rendered prompt | Template, rubric, and instruction text changes must invalidate old verdicts — hashing raw inputs alone silently pins results from an old rubric |
| Model identifier | A different model can produce a different verdict |
| Reasoning effort | Effort changes answer depth and quality |
| Context tier | default vs long_context changes both behavior and price |
| Tool visibility and permission set | A verdict produced without write access is not the same as one produced with it |
| MCP server configuration | Available capabilities change the achievable answer |
| Instruction and rubric version | Version the content, not just the file path |
| Relevant repository state | Commit SHA or content hash of the inputs actually read |
| Harness and CLI version | Behavior changes across versions; pin and record both |
Cache hygiene rules that belong in the same design:
- Missing key component in an old entry = cache miss. Never guess.
- Record provenance with every stored result: key components, timestamp, run ID, and whether it was later accepted.
- Decide the negative-result policy explicitly. Caching failures can be correct (deterministic refusal) or dangerous (transient outage). Default to not caching transient failures.
- Provide a documented bypass implemented in your harness (for example, a
--no-cacheflag on your own runner — not a Copilot CLI option) and log every bypass so a permanent bypass cannot hide. - Set a TTL based on how quickly the underlying inputs go stale, and invalidate on profile change.
Prompt Budgets and Bounded Fields
| Practice | Implementation | Failure it prevents |
|---|---|---|
| Reference, don't inline | Point to a path and let the agent read on demand when the lane grants read access | Paying to send a file the task never needed |
| Cap every growing field | Hard truncation with a visible ...[truncated] marker |
Silent, invisible input growth |
| Trim to budget in priority order | Drop least-important sections first, deterministically | Random loss of the most important content |
| Refuse rather than overflow | Emit a distinct context_too_large outcome before calling |
Paying to assemble and send a request that is rejected |
| Keep stable text stable | Put invariant instruction blocks in a fixed position and change them rarely | Unnecessary cache invalidation |
| Bound tool output | Keep the documented 20 KiB default behavior; enforce your own input caps in the harness | One command's output consuming the context window |
Note: Whether a specific prompt layout improves provider-side cache reuse is not something GitHub documents at the byte level. What is documented is that mid-session changes to model, reasoning, context size, or the enabled tool/MCP set invalidate the cache, and that cached input tokens are typically billed at a fraction of fresh input. Optimize for stability of configuration first; treat prompt-layout cache theories as hypotheses to measure.
Exercise Connection
Lab 4 — Unsafe Cache-Key Review (15 min): teams review an intentionally unsafe cache design, list the omissions, and describe the specific wrong result each omission can produce. No paid calls.
Facilitation — compression cue (30-minute section): if you are behind, deliver Complete Cache Keys as the two split slides without commentary, reduce Cache Hygiene to its three bold rules (miss on unknown, never cache transient failures, log every bypass), and cut the prompt-budget table to the two rows the room reacted to. Never shorten Lab 4 — it is the highest-value exercise in the module.
Discussion Points
- Which paid call in your current automation could be replaced by a deterministic check tomorrow?
- If your rubric or prompt template changed today, would any cached verdict in your system still be served? How would you know?
- What is your policy for caching failures — and does it distinguish a deterministic refusal from a service outage?
5. Runtime Governance for Budgets, Fan-Out, Retries, and Breakers (30 min)
Key Points
- Autonomous lanes need governance because a single bad day can erase a quarter of savings. Governance is layered: organization budgets bound the blast radius, the harness bounds the fleet, and per-run caps bound one invocation.
- A lane that bypasses the harness budget module is invisible spend. Every paid call goes through one
check()before and onerecord()after — no exceptions, including ad-hoc scripts. - Record honestly. If one logical task fans out into three model calls, record three. Undercounting hides exactly the lanes you most need to see.
- Copilot's own per-run cap is a soft limit and is in public preview.
--max-ai-creditssets a soft maximum for AI credits; if a response is in progress when the limit is reached, that response completes, so actual usage may slightly exceed the configured number. In non-interactive mode, the run ends when the limit is reached. GitHub documents session limits as currently in public preview and subject to change, and notes they work best when set above 30 AI credits because most model calls cost more than 20 (Setting an AI credit session limit in GitHub Copilot CLI). CLI v1.0.79 also enforces a floor of 30:--max-ai-credits 10is rejected withUse at least 30 AI credits. Pick a value above 30 — this module uses 40 — and treat a cap that stops a run as a signal to decompose the task, not a number to raise. - Session limits do not replace user-level budgets or spending limits, which govern overall monthly consumption (Optimizing your AI usage). Module 1 owns that layer.
Layered Budget Controls
| Layer | Control | Owner | Enforcement |
|---|---|---|---|
| Enterprise / organization / cost center | Spending budgets and user-level budgets | Admin and FinOps (Modules 1 and 3) | GitHub-enforced; a user-level budget always hard-stops |
| Fleet envelope | Credits per lane per window | Platform team | Harness-enforced check() before every paid call |
| Run | --max-ai-credits (CLI) or sessionLimits.maxAiCredits (SDK) |
Lane profile | Copilot-enforced soft limit, public preview; minimum 30 in CLI v1.0.79 |
| Fan-out | Concurrency and depth caps for subagent-style work | Lane profile | Harness-enforced — CLI v1.0.79 help lists no supported environment variable for this (claim ledger) |
| Worker concurrency | Parallel workers per lane | Harness | Scheduler-enforced |
| Retries | Attempts per fingerprint | Harness | Attempt ledger |
Important: The identity a fleet runs as consumes credits like any other user, and a user-level budget applied to that identity is a hard stop. Decide deliberately whether a hard stop on your automation identity is the outcome you want, and coordinate it with the Module 1 owner.
Outcome Taxonomy
The single most common dispatcher defect is treating every non-success as "the model failed." The second is treating "we declined to spend" and "we spent and ran out" as the same event.
| Outcome | Meaning | Retry? | Counts toward breaker? | Queue behavior |
|---|---|---|---|---|
success |
Verified result landed | No | No | Acknowledge and remove |
already_done / already_green |
Work was unnecessary | No | No | Acknowledge with reason |
model_failure |
Model produced unusable or failing output | Yes, bounded | Yes | Retain with attempt increment |
context_too_large |
Input exceeded the prompt budget | No — fix the input first | No | Retain and flag for redesign |
policy_denied |
Permission, policy, or content control blocked the action | No | No | Retain and alert an owner |
service_unavailable |
Transient service or network failure | Yes, with backoff and jitter | No | Retain |
budget_refused |
Pre-spend: a budget or envelope check refused the call before it was made; no credits were spent | Not immediately — park the worker | No | Retain; retry after the park window; do not increment the attempt counter |
run_cap_exhausted |
Post-spend: the run started and hit a credit ceiling mid-flight (per-run --max-ai-credits soft cap or the harness envelope) and stopped before finishing; credits were spent |
Not automatically | No | Retain; record the spend, increment the attempt counter, and require task decomposition or an explicit, reviewed cap change before the next attempt |
invalid_config |
Lane profile or harness misconfiguration | No | No | Halt the lane and alert |
landing_failed |
The model produced output, but it did not land or was not consumed | Yes, after the landing path is fixed | No | Retain — never acknowledge |
Why budget_refused and run_cap_exhausted must be different outcomes:
| Question | budget_refused |
run_cap_exhausted |
|---|---|---|
| Were credits spent? | No | Yes — record them against the lane |
| Attempt counter | Unchanged — the work was never tried | Incremented — the fingerprint consumed budget without landing |
| Correct next step | Wait for the window to reset, then retry unchanged | Decompose the task, or make an explicit reviewed decision to raise the cap; only then retry |
| What automatic retry would cause | A slower queue | Repeated partial spend on work that cannot finish inside the cap |
| Signal to the owner | "We are at our limit" | "This task does not fit the lane's shape" |
Important: An automatic retry after
run_cap_exhaustedis the most expensive mistake in this taxonomy: each attempt pays close to the full cap and lands nothing. Retry must be gated on a change — smaller work item, different lane, or a reviewed cap decision — never on time alone.
Backoff, Breakers, and Parking
- Exponential backoff with jitter (for example, 10m → 20m → 40m, capped) recovers quickly when conditions change. A long fixed timeout is a throughput tax that outlives the incident that caused it.
- Park on refusal, do not poll. When a pre-spend budget check refuses, park the whole worker for a fixed window and log one summary line. Per-item refusal checks at poll cadence generate thousands of log events per day and hide real signal.
- Escalate, do not retry, on
run_cap_exhausted. Record the spend, increment the attempt, and route the item to a human or a decomposition step. The item stays queued but blocked until someone changes the work or the cap. - Trip breakers on consecutive
model_failureonly. Counting routing, policy, budget, or "already done" outcomes punishes the model for the dispatcher's mistakes and takes down healthy lanes. - Half-open carefully. After the open window (a defensible default is 20 minutes), allow exactly one probe item; close on success, reopen for the next window on failure.
- Dead-letter with context. An item that exhausts its attempts must land in a dead-letter queue with the last outcome, the profile hash, and the transcript pointer — never be deleted.
- Refused work is delayed, not lost. The queue must still contain it after the park window.
Where Runtime Cost Data Comes From
| Source | Scope | Status |
|---|---|---|
/usage in a session |
Per-session usage metrics and per-model token totals | Documented CLI command |
--output-format=json (JSONL) |
Machine-readable programmatic output | Documented; verify which usage fields your version emits before depending on them |
--share=PATH transcript |
Full run transcript for audit | Documented; may contain sensitive content |
SDK assistant.usage event |
Per model API call: model, inputTokens, outputTokens, cost (premium request multiplier) |
Documented SDK event |
SDK session.usage.getMetrics |
Accumulated AI credit and token totals | Documented as experimental |
| AI usage dashboard and CSV export | Billing-grade totals by user, model, organization, and cost center | Documented (Module 3) |
Important: GitHub's billing views attribute usage to identities, models, organizations, and cost centers — not to your lane names. Lane-level attribution is your harness's responsibility: record the lane, profile hash, and run ID in your own ledger and reconcile totals against the dashboard. Do not promise lane-level chargeback from GitHub reporting alone without verifying the dimensions available to you.
Exercise Connection
Lab 5 — Dispatcher Outcome Taxonomy (15 min): teams map nine supplied run results — including a pre-spend budget refusal and a post-spend cap exhaustion — to outcomes, retry behavior, breaker counting, and queue action, then identify which mapping errors would cause a retry storm, silent work loss, or repeated partial spend. No paid calls.
Facilitation — compression cue (30-minute section): if you are behind, teach the layered budget table and the first half of the outcome taxonomy live, hand the second half to the lab worksheet, and cut the backoff bullets to three (backoff with jitter, breakers on
model_failureonly, refused work is delayed not lost). Keep thebudget_refusedversusrun_cap_exhaustedcontrast — it is the new decision this section exists to teach — and keep Lab 5 at 15 minutes.
Discussion Points
- Does your automation today distinguish "the model failed" from "the service was unavailable" and "the work was already done"?
- What would your fleet do if a budget refused every call for two hours — and how many log lines would that generate?
- Which layer would catch a runaway lane first in your environment, and how long would it take?
6. Durable Landing, Telemetry, Canary Rollout, and Capstone (20 min)
Key Points
- The most expensive token is one spent on a result that gets thrown away. A fleet is only as efficient as its landing path.
- No silent success. A merge, write, or publish step that finds nothing to do must say so distinctly — a dedicated exit signal or a prominent warning — not exit successfully while the caller acknowledges and deletes the work item.
- Exit codes are a contract. Adding a new failure signal changes the behavior of every caller. Trace all callers before changing one, because a signal that is correct for one lane can quarantine every item in another.
- Never acknowledge on ambiguous success. Validate the output, write atomically (or verify the merge), record provenance, and only then acknowledge. If validation is inconclusive, retain the item.
- Fixes do not count until deployed. Long-running workers load code at start; committing a change without restarting the service means the old code is still spending. Commit, deploy, restart, then confirm the new behavior in live logs.
- Resolve shared paths through one helper. Two components with different ideas of "the current round" means paid work written where nobody reads it.
The Landing Contract
| Step | Requirement | Evidence it worked |
|---|---|---|
| 1. Validate | Output matches the expected schema or check | Validation result recorded |
| 2. Persist | Atomic write or verified merge | Artifact exists at the resolved path |
| 3. Attribute | Provenance: lane, run ID, profile hash, model, cache status | Ledger row |
| 4. Signal | Distinct exit status for success, no-op, and failure | Caller behavior verified for each |
| 5. Consume | A downstream consumer actually reads it | Consumer log or downstream artifact |
| 6. Acknowledge | Remove the queue item only after 1–5 | Queue depth reflects real progress |
| 7. Deploy | Restart the worker and verify in live logs | Post-restart log line showing new behavior |
Fleet Telemetry: What Every Run Records
| Field group | Fields |
|---|---|
| Identity | Lane, run ID, fingerprint, owner/lease ID, harness and CLI version |
| Configuration | Profile hash, model, effort, context tier, tool and MCP set, instruction version |
| Spend | AI credits, token counts by category, fan-out call count |
| Behavior | Outcome, retries, backoff events, breaker state, cache result, gate that fired |
| Value | Landed, consumed, human correction time, durable acceptance after the stabilization period |
Those fields roll directly into the Module 3 metric: full cost per durable accepted outcome, per lane. Cost per invocation and tokens saved are diagnostics; they are not the decision metric.
Canary Rollout for Cost Changes
Treat a cost-optimization change exactly like a production code change.
Baseline ──► Canary one lane ──► Quality-parity gate ──► Adversarial review ──► Expand ──► Monitor
│ │ │ │ │ │
│ │ │ │ │ └─ rollback trigger stays armed
│ │ │ │ └─ one lane at a time
│ │ │ └─ integration seams, cache keys, flag spelling and values, exit-code blast radius
│ │ └─ durable acceptance must not regress
│ └─ same task mix, same window, single variable
└─ measured, not remembered
| Gate | Pass condition | If it fails |
|---|---|---|
| Baseline captured | Median and range recorded for the canary lane before the change | Do not proceed |
| Single variable | Exactly one profile element changed | Split the change |
| Quality parity | Durable acceptance and human correction time do not regress beyond the agreed tolerance | Roll back |
| Cost improvement | Full cost per durable accepted outcome improves | Roll back or keep the old profile |
| Adversarial review | A reviewer explicitly hunts integration seams, cache-key completeness, flag spelling and flag values, and exit-code blast radius | Fix and re-review |
| Deployed and verified | Live logs confirm the new behavior in the running worker | Not done |
Important: Report savings as measured cost per durable accepted outcome, not as assumed token reductions. A profile that saves tokens and loses acceptance is a cost increase wearing a savings label.
Exercise Connection
Lab 6 — Capstone: Fleet Lane Optimization Checklist and Canary Scorecard (12 min): teams complete the capstone templates for one anonymized real lane, naming the canary lane, the single variable, the quality-parity gate, the rollback trigger, and the owner. No paid calls.
Facilitation — pacing (20-minute section): this section carries a 12-minute lab, so the live budget is about 8 minutes — roughly 2 minutes on the landing contract, 2 on telemetry, 3 on canary rollout, and 1 for the hand-off. Deliver the tables as pointers, not readings. The claim ledger and the "Translating Playbooks" table are reference material in the deck appendix; do not teach them live unless the room asks.
Discussion Points
- Where in your pipeline could paid work land somewhere nobody reads, and how long would it take you to notice?
- Which lane is your best canary — highest volume, lowest blast radius, clearest acceptance signal?
- What is your rollback trigger, and who is allowed to pull it without a meeting?
Recap
- A fleet's cost is fixed session tax plus variable task work, multiplied by invocations and inflated by retries, fan-out, duplication, and discarded results.
- Measure the session tax before optimizing it:
/context,/usage,copilot plugins list, and a one-factor-at-a-time baseline — never someone else's token numbers. - A lane profile pins model, effort, context tier, tool visibility, tool permission, MCP set, instructions, memory, credit cap, isolation, and fan-out. Nothing inherits a default.
- Tool visibility and tool permission are different layers; deny always wins; permission changes are security changes.
- Isolation controls exist at several maturity levels —
-C,--add-dir, andCOPILOT_HOMEare documented options;--worktreeand--sandboxare experimental and need a fallback. - Deterministic gates and complete cache keys avoid paid calls safely; an incomplete cache key is a correctness bug that looks like a saving.
- Governance is layered: organization budgets, fleet envelope, per-run soft credit cap (public preview, minimum 30 in CLI v1.0.79), harness fan-out caps, worker concurrency, and retry caps — with honest accounting at every level.
- A failure taxonomy prevents retry storms and breaker misfires; a pre-spend refusal parks the work, a post-spend
run_cap_exhaustedrecords the spend and blocks retry until the task or the cap changes; dead-lettered work keeps its context. - Paid work counts only when it lands, is consumed, and is accepted — verified in live logs after deployment.
- Roll out cost changes like production changes: baseline, canary, quality parity, adversarial review, expand, and keep the rollback trigger armed.
Appendix
Package Artifacts
| Artifact | File |
|---|---|
| Canonical workshop guide (this document) | github-copilot-token-platform-automation-workshop.md |
| Hands-on lab, including the optional runnable appendix and printable template copies | github-copilot-token-platform-automation-LAB.md |
| Slidev deck | github-copilot-token-platform-automation.slidev.md |
| Lane profile worksheet (source) | templates/lane-profile-worksheet.md |
| Cache-key review worksheet (source) | templates/cache-key-review-worksheet.md |
| Failure taxonomy and breaker worksheet (source) | templates/failure-taxonomy-worksheet.md |
| Fleet lane optimization checklist and canary scorecard (source) | templates/canary-scorecard.md |
Important: The four templates are also inlined as printable copies in Appendix B of the lab guide, because the published site renders the lab page but does not publish the
templates/folder. Participants reading the published lab therefore need no repository access. Thetemplates/files remain the editable sources — change both copies in the same edit and keep them identical in content.
Key URLs
Copilot CLI Control Reference (documentation verified 2026-08-12; help output cross-checked against CLI v1.0.79)
| Concern | Control | Documented behavior and status |
|---|---|---|
| Programmatic run | -p / --prompt, piped stdin |
Executes a prompt and exits; piped input is ignored if -p is also supplied |
| Version check | copilot --version |
Prints the installed version; proves installation only, not authentication |
| Authentication | copilot login (--device-code, --web-flow, --host), or COPILOT_GITHUB_TOKEN / GH_TOKEN / GITHUB_TOKEN in that precedence |
v1.0.79 exposes no non-interactive "auth status" subcommand; the first real run is the authoritative check |
| Clean output | -s / --silent |
Outputs only the agent response and suppresses stats — do not use it on measurement runs |
| No stalling | --no-ask-user |
Prevents the agent from pausing for clarification |
| Model | --model, COPILOT_MODEL, settings model |
Precedence: custom agent → --model → COPILOT_MODEL → settings → CLI default; auto selects automatically and carries a documented 10% model-cost discount on paid plans |
| Reasoning effort | --effort / --reasoning-effort |
v1.0.79 choices: none, minimal, low, medium, high, xhigh, max; available for supported models only; max documented as the highest-depth tier for Anthropic models |
| Context tier | --context TIER |
default or long_context for tiered-pricing models; documented as overriding the persisted setting and honored in fresh interactive sessions — verify behavior for -p runs in your version |
| Tool visibility | --available-tools, --excluded-tools |
Allowlist wins if both are supplied; invisible tools cannot be used even if allowed; tool-name values are version- and configuration-specific and are not enumerated in the published reference |
| Tool permission | --allow-tool, --deny-tool, --allow-all-tools, --allow-all / --yolo |
Deny always wins, including over saved approvals; documented example identifiers include write, shell(git:*), shell(git push), and SERVER(tool); allow-all options are session-wide permission grants intended for isolated environments |
| Unknown options | any misspelling | Rejected: v1.0.79 prints error: unknown option '<flag>', suggests the closest match, and exits non-zero without a model call |
| Path and URL scope | -C, --add-dir, --allow-url, --deny-url, --allow-all-paths, --disallow-temp-dir |
Deny URL takes precedence over allow URL |
| MCP scope | --disable-builtin-mcps, --disable-mcp-server=NAME, --additional-mcp-config |
Built-in disable currently covers github-mcp-server |
| GitHub MCP tools | --add-github-mcp-toolset, --add-github-mcp-tool, --enable-all-github-mcp-tools |
The CLI enables a default subset unless overridden |
| MCP instructions | --allow-all-mcp-server-instructions |
By default only allowlisted server instructions are included up front; others are retrieved on demand |
| Custom instructions | --no-custom-instructions, COPILOT_CUSTOM_INSTRUCTIONS_DIRS |
Disables loading of AGENTS.md and related files |
| Memory | --enable-memory |
Memory is disabled by default in prompt mode |
| Broad permission via environment | COPILOT_ALLOW_ALL |
Documented equivalent of --allow-all-tools; audit whether your harness sets it |
| Credit cap | --max-ai-credits, /limits set max-ai-credits |
Soft limit; public preview; minimum accepted value is 30 in v1.0.79; documentation advises setting it above 30; in non-interactive mode the run ends when the limit is reached |
| Output | --output-format=json, -s, --share=PATH, --share-gist, --secret-env-vars |
JSON format emits JSONL; gists are unavailable to EMU and *.ghe.com |
| Isolation | COPILOT_HOME, -w / --worktree, --sandbox / --no-sandbox |
COPILOT_HOME is a documented environment variable; worktree and sandbox flags are experimental and are not listed in v1.0.79 top-level help (copilot help sandbox documents sandboxing as experimental) |
| Fan-out and run timeouts | Harness scheduler and watchdog | No supported CLI environment variable is listed in v1.0.79 copilot help environment — implement concurrency, depth, and timeout caps in your harness (claim ledger) |
| Tool output size | Documented 20 KiB behavior; threshold variable | Larger tool output is written to a file and previewed; the threshold environment variable named in the context-management documentation is not listed in v1.0.79 help — treat as version-sensitive (claim ledger) |
| Inventory | copilot plugins list [--kind] [--scope] [--json] |
Inventories plugins, MCP servers, skills, instruction sources, and language servers; --kind supports plugin, mcp, skill, instruction, lsp; excludes custom agents and session-scoped hooks |
| Session inspection | /context, /usage, /compact |
Interactive session commands |
Claim Ledger: Verify Before Delivery
These items are not settled facts and must be presented as measurement questions, not product guarantees.
| Item | Why it is uncertain | How to handle it in delivery |
|---|---|---|
| Token overhead per instruction file, tool schema, MCP server, or plugin | Not published by GitHub; environment-specific | Measure locally with /context; never quote a number from another environment |
Per-call AI-credit detail in --output-format=json output |
The JSONL format is documented; a guaranteed per-call cost field is not | Verify with your installed version before building a ledger on it |
--context behavior in -p runs |
Documented as honored in fresh interactive sessions | Verify empirically; do not promise programmatic behavior you have not observed |
Valid --available-tools / --excluded-tools values |
Tool names vary by version, agent, and MCP configuration; not enumerated in the published reference | Discover names in your environment, pin them per lane, and re-verify on upgrade — never copy a tool list from a slide |
| Model availability per environment | Model catalogs change over time and differ by plan and host | Pin a model per lane and treat an unknown model name as invalid_config. Do not claim the CLI silently falls back to another model — that behavior is unverified |
| Subagent concurrency/depth and run-wait environment variables | Not listed in v1.0.79 copilot help environment and not found in the current CLI reference pages |
Do not teach them as controls; cap fan-out and wait time in the harness |
| Tool-output threshold environment variable | Named in the context-management documentation but not listed in v1.0.79 help | Teach the documented 20 KiB behavior; bound inputs in the harness; verify the variable before depending on it |
| Prompt-mode trust environment variables (extensions, repository hooks, workspace MCP) | Not listed in v1.0.79 copilot help environment |
Teach only what your installed version documents; audit COPILOT_ALLOW_ALL |
| Cross-run prompt-cache reuse and provider-side cache benefit from prompt layout | Not documented at that level of detail | Teach configuration stability; treat layout and cross-run cache theories as hypotheses to measure |
| Per-lane cost attribution in GitHub billing views | Reporting attributes to identities, models, organizations, and cost centers | Attribute in your own ledger; reconcile with the dashboard |
Per-lane COPILOT_HOME profiles as an isolation boundary |
Built on a documented variable, but not a documented fleet feature | Teach as a harness pattern and verify in your environment |
Experimental controls (--worktree, --sandbox) and preview features (session limits) |
Explicitly labeled experimental or public preview and subject to change | State the status out loud every time; require a fallback |
Translating Playbooks From Other Agent CLIs
Teams often arrive with a playbook written for a different agent CLI. The principles usually transfer; the commands usually do not.
| Principle (portable) | Do not assume | Copilot equivalent to verify |
|---|---|---|
| Suppress memory and injected context for headless runs | Another tool's environment variables or settings-file overrides | Memory is off by default in prompt mode; --enable-memory opts in |
| Give a call only the tools it needs | Another tool's allow/disallow flag names or its tool-name values | --available-tools / --excluded-tools for visibility plus --allow-tool / --deny-tool for permission; discover valid tool names in your own environment |
| Restrict MCP servers per lane | Another tool's strict-config flags | --disable-builtin-mcps, --disable-mcp-server, --additional-mcp-config |
| Control inherited instruction files | Another tool's instruction-file hierarchy semantics | Working-directory scope, --no-custom-instructions, COPILOT_CUSTOM_INSTRUCTIONS_DIRS |
| Cap spend per invocation | That the cap is hard | --max-ai-credits is a soft limit in public preview |
| Hook into every tool call | Another tool's hook matcher syntax | Copilot hooks and SDK hook surfaces have their own documented model — verify before designing around them |
Important: Never present a flag from another vendor's CLI as a GitHub Copilot flag. If a control does not exist in the current Copilot documentation, say so, and implement the principle in your harness instead.
Instructor Preflight Checklist
- Re-verify every row of the Copilot CLI Control Reference against the live command reference; update the verification date in the guide, lab, and deck together
- Confirm the current status of
--max-ai-credits(public preview),--worktree, and--sandbox(experimental) and say the status out loud when the slide appears - Confirm the audience actually operates programmatic Copilot automation; if not, deliver Module 2 or Module 3 instead
- Decide before the session whether the optional runnable appendix will be delivered, and confirm participants have a non-production sandbox and authorization to spend
- Print or share the four templates for participants without laptops — printable copies are inlined in Appendix B of the lab guide, and the editable sources are in
templates/ - Prepare one anonymized real lane from your own environment as a worked example; remove customer names, repository names, and internal figures
- Confirm the Module 1 budget owner is identified for the layered-governance discussion
- Re-run
copilot help,copilot help environment, andcopilot help limitson the delivery machine and note any drift from the control reference
Backup Plan
If any live element is unavailable (no CLI, no license, restricted network, or a UI change):
- Deliver all six core exercises from the lab worksheets — none of them requires a live product.
- Replace any live inventory demonstration with the illustrative sample
copilot plugins listoutput in the lab's Backup Path section (Lab 1 and Lab 3 both reference it). Say out loud that the sample is synthetic and that real output differs by machine. - Screen-share the linked documentation pages instead of the product when a flag's behavior is questioned.
- Skip the optional appendix entirely; it is not required for any learning objective.
- If a documented flag has changed since verification, say so explicitly and record it as a follow-up rather than improvising a replacement.
Workshop guide for GitHub Copilot AI Credits Optimization — Optional Module 4: Agent Fleet Efficiency and Cost Governance (optional; excluded from the 7-hour-10-minute core curriculum of Modules 1–3)