CustomLabs
Adoption

The tools work. Getting 180 engineers to use them is the other half of the job.

Six places a rollout stalls. 24 practices, split by who owns each move. Five ways a programme fakes momentum. The numbers that separate real adoption from seat activation. Most of what kills an agent rollout has nothing to do with the model.

Updated First published

33 min read

Markdown

A working pilot proves the harness can work somewhere. It does not show whether the other engineers will pick it up. It does not show whether the review queue can absorb five times the diffs. It does not show whether anyone can tell a real improvement from a number measured against no baseline. Most rollouts stall in the gap between a proven pilot and an organisation that runs on it.

The Agentic Delivery Playbook covers the machine: task definition, isolation, stage gates and spend. This page covers what must be true in the organisation around that machine. Without it, the machine produces nothing at scale.

The six surfaces

Where a rollout stalls.#

Each surface fails a different way. None of them substitute for each other.

Diagram in six lanes, one per adoption surface, in order: the mandate; the first team; and the paved road. The other three are absorbing the output; measurement; and sustain. Each lane holds one node that names what that surface covers. The surfaces connect in order, with a return edge from measurement back to the first team labeled expand or hold. THE MANDATE THE FIRST TEAM THE PAVED ROAD ABSORBING THE OUTPUT MEASUREMENT SUSTAIN Named sponsor Exit criteria Sanctioned harness Review capacity Pre-rollout baseline Plateau past launch expand or hold
An agent rollout moves through six stages, from the first mandate to a sustained program. Measurement results decide whether the pilot expands or holds.
01

The mandate

Covers: The problem the rollout must solve, one named accountable sponsor, and a written definition of success. Agree all three before the first licence is bought.

Breaks when: The mandate says "roll out AI" and names no sponsor. When the pilot numbers turn ambiguous, nobody has the authority or the duty to make the call.

Watch: Time between the first licence purchased and a written definition of success

02

The first team

Covers: Choose a pilot team whose workload, codebase health and appetite generalise. Name the exit-from-pilot criteria before the pilot starts.

Breaks when: The pilot team is the platform group's best engineer in a clean, greenfield service. The result then predicts nothing about a legacy team that carries real technical debt.

Watch: Share of exit-from-pilot criteria written down before the pilot's first week

03

The paved road

Covers: The sanctioned harness and defaults, sandboxing and repo hygiene. Agent instructions files, tests that gate a merge, and the boundary of what an agent may touch. Enablement built as pairing in the team's own repo.

Breaks when: The rollout ships a licence and a slide deck with no sanctioned default. Every team improvises its own guardrailsGuardrails are the checks that keep an LLM or agent inside acceptable bounds in production. at a different level of care. People then blame the tool for the friction.

Watch: Share of teams that run on the sanctioned harness instead of a bespoke one

04

Absorbing the output

Covers: Review capacity, CI throughput and spec quality. Who owns the merge. How the senior engineer's role changes once writing code stops being the bottleneck.

Breaks when: Agents multiply the diffs a team produces and review capacity stays the same. The queue that said "waiting on code" now says "waiting on review", at the same total cycle time.

Watch: Review latency on agent-authored changes against the team's own pre-rollout baseline

05

Measurement

Covers: The baseline captured before rollout. Every adoption number paired with a quality number. The split between the median engineer and the power user.

Breaks when: Nobody captured a pre-rollout baseline. A real improvement and a number that merely looks better than a guess are then impossible to tell apart.

Watch: Whether a dated pre-rollout baseline exists for every metric on the scoreboard

06

Sustain

Covers: Plateau and regression past the launch spike, and spend per team. Unsanctioned use read as a signal. Retiring pilot scaffolding on schedule. A named platform owner with a refresh cadence.

Breaks when: Adoption plateaus at the enthusiasts within the first month. The rollout reports the plateau as steady-state success, because nobody asked why everyone else stopped coming back.

Watch: Active weekly usage among engineers with tool access, tracked for months past the launch spike instead of only the first one

The practice bank

24 practices, six surfaces.#

Filter by how far a practice has to reach (pilot, team, org), then copy the visible list as a Markdown checklist.

01 The mandate

Every agent rollout starts with an announcement. Most skip the part that gives the announcement meaning. That part is one named owner and a written definition of success. Agree both before the first licence is bought. Without these, the rollout runs on borrowed authority. A budget gets approved because a competitor shipped something or a demo landed well. Nobody answers in writing what problem the rollout solves or how anyone will know it worked. The gap does not show at first. It shows months in, when the pilot numbers are ambiguous. By then no sponsor is left to make the call, and no written definition of success exists to judge it against.

What it covers
A single named executive who is accountable for whether the rollout succeeds. This person is separate from whoever runs the pilot day to day.
How to build it
Put a name and a title on the mandate document itself. Sign it before procurement starts. Do not assign a sponsor after the pilot already has momentum.
Produces
A single person to ask when the rollout stalls, instead of a committee nobody can point to.
Tradeoff
A sponsor who changes teams or leaves takes the mandate's authority with them. Re-confirm ownership at every leadership change. Do not assume it survives one.
Prove it
The current sponsor can state the rollout's definition of success from memory. They do not need to re-read the deck.
What it covers
A short, specific statement of what the rollout must change. Agree it in writing before the first licence or seat is purchased.
How to build it
Draft it as a testable claim with a stated target and a date. Do not write a mission statement. Get the sponsor to sign it.
Produces
A rollout that people can judge against what it said it would do. Nobody can reinterpret it after the fact to match what happened.
Tradeoff
A specific target invites scrutiny that a vague one avoids. That scrutiny is the point.
Prove it
The definition of success predates the first purchase order. Check it against the dates.
What it covers
A stated business problem that the rollout must move, such as slow delivery, a backlog nobody can clear or an unmet SLA. State it ahead of any product decision.
How to build it
Write the problem statement before the vendor conversation. Route every later tool choice back through one question: does it move that problem?
Produces
A rollout that can say no to a feature or a vendor that does not serve the problem. It does not collect capability for its own sake.
Tradeoff
A narrow problem statement rules out adjacent wins that the tool could also deliver. Note them and do not chase them yet.
Prove it
Someone outside the rollout team can restate the problem in one sentence without reading the deck.
What it covers
A short, written list of outcomes that the rollout is not judged on this cycle. Examples are headcount reduction, replacing a specific role or a fixed seat-count target.
How to build it
Add a non-goals section to the mandate document and have the sponsor agree it. Repeat it in the first team-facing announcement.
Produces
Fewer engineers who treat the rollout as a threat and route around it. The thing they feared was already ruled out in writing.
Tradeoff
A non-goal stated today can look naive in a year if the plan changes. Revisit it on the same cadence as the mandate. Say so out loud when it changes.
Prove it
A team lead can quote the non-goals list without checking, and it matches what leadership says in private.

02 The first team

The team that pilots an agent rollout decides more about the result than the tool does. Teams get picked for an enthusiastic manager, a clean codebase or unusually well-scoped tickets. Such a pilot looks great and predicts almost nothing about the teams that wait behind it. The best team is the wrong target. Find a representative team. Then name what "worked" means before the pilot starts. This stops the decision to expand from going to whoever tells the best story afterward.

What it covers
A pilot team selected because its workload shape resembles the organisation's median. Look at ticket size, codebase age and test coverage. Its manager volunteering first is not a reason.
How to build it
Score candidate teams against the organisation's workload distribution. Do this before you ask anyone if they want to go first.
Produces
A pilot result that predicts what the rest of the organisation will see. It is not a best case nobody else can reach.
Tradeoff
A representative team is harder to recruit than an enthusiastic outlier. Recruit on the strength of the mandate. Do not recruit whoever happens to be excited.
Prove it
The pilot team's ticket-size and test-coverage numbers sit within a stated range of the org median. Check them against real data.
What it covers
A numeric bar that the pilot must clear before the rollout expands past it. Examples are an acceptance rate, a review latency and a rollback-rate ceiling.
How to build it
Agree the bar with the sponsor before the pilot's first week. Publish it to the pilot team so they know what they are measured against.
Produces
An expansion decision made against a rule agreed in advance. It is not a judgment call made once the pilot has run a while.
Tradeoff
A bar set too high can kill a pilot that was working, only slowly. Calibrate it against the first team's baseline. Do not use an aspirational number.
Prove it
The exit-criteria document predates the pilot's first week. The expand-or-hold decision cites it directly.
What it covers
A quick read on the pilot repo's test coverage, CI speed and documentation state. An agent inherits whatever review and test discipline already exists there.
How to build it
Run the health check the team would apply before any other significant investment. If a repo has no tests or a broken CI pipeline, fix that first or pick a different pilot.
Produces
A pilot result that comes from the tooling. A codebase that happened to make everything look easy does not distort it.
Tradeoff
Waiting for a healthier repo costs calendar time that a sponsor may not want to spend. Weigh that against a result nobody trusts once it fails to repeat elsewhere.
Prove it
The pilot repo's coverage and CI numbers were checked and recorded before the pilot's first change merged.
What it covers
A direct check with the candidate pilot team on their appetite for the change. Their manager's enthusiasm is a different thing.
How to build it
Talk to the engineers who will do the work before the decision is final. Treat a reluctant team as a signal to pick a different one. Do not push it through as a change-management problem.
Produces
A pilot that a team who never wanted it does not sandbag.
Tradeoff
Deferring to appetite can mean picking a less representative team. Weigh that against a pilot result that nobody believes because the team's heart was not in it.
Prove it
The pilot team can name, unprompted, something about the rollout that they are curious about. They do not only repeat what they were told to expect.

03 The paved road

A licence and a link to the vendor's documentation is not a paved road. It is the absence of one, dressed as adoption. An agent works well in a codebase only when four things exist. It needs a sandboxed environment and a repo-local instructions file. It needs tests that gate a merge. It needs a stated boundary of what it may touch. They must exist before a second team is asked to use any of it. Enablement should be a worked example in the team's own repo, done beside a person who knows the harness. It beats a webinar every time, because it is the only version that survives contact with a real ticket.

What it covers
A single supported configuration that every team starts from. It includes sandboxing, a default instructions file and credential scoping. A licence and a link to the vendor's documentation is not a substitute.
How to build it
Build and pilot the default harness on the first team. Then package it as the thing a new team installs on day one. Do not let teams converge on it after a month of improvising.
Produces
A second team's onboarding measured in days against a working baseline. Without it, teams spend weeks to reinvent the first team's decisions.
Tradeoff
A single sanctioned default will not fit every workload exactly. Let teams diverge on purpose once they can name why it does not fit. Do not let divergence be the default.
Prove it
A new team's first week of agent-assisted changes runs on the sanctioned harness. Check their configuration. The onboarding checklist's claim is not proof.
What it covers
A file in the repository itself. It states the tests that gate a merge and the boundary of what an agent may touch. It also holds the house conventions that a human reviewer would otherwise explain in every pull request.
How to build it
Write the first version from the pilot team's review comments. Then require it when you onboard any new repo onto the paved road.
Produces
An agent and a new human hire that read the file the team works from. A wiki page that drifted out of date months ago does not replace it.
Tradeoff
A repo-local file needs its own upkeep, so it is one more thing that can go stale. Tie updates to the same review that catches other doc drift. Do not start a separate process that nobody owns.
Prove it
A repo picked at random has an instructions file that matches its current test-gating and merge rules. Last quarter's rules do not count.
What it covers
An explicit, written scope. It lists which files, systems and actions an agent may change unsupervised, and which always route to a human.
How to build it
Start narrow. The agent reads and proposes, and never merges to a protected branch unattended. Widen the boundary only after the review data supports it. Do not widen it on day one.
Produces
A rollout that can say precisely what went wrong when something fails. Nobody debates afterward whether the agent should have been allowed to do it.
Tradeoff
A narrow boundary limits the benefit teams see early. That is real friction against the mandate's own definition of success. Widen the boundary on a measured schedule. Do not widen it to quiet a complaint.
Prove it
The current boundary is written down where a new team member reads it. The last incident review checked the action against it.
What it covers
A worked example built against the team's own codebase and backlog. A person walks the team through it in the room. A generic training deck does not do this job.
How to build it
Pair a platform engineer with two or three engineers from the new team for their first real ticket. Then let them run the second one alone.
Produces
A team that has seen the harness work on a ticket they recognise. A slide deck they nodded through does not do this.
Tradeoff
Pairing does not scale to a large organisation by headcount alone. Train a small number of team-level champions this way. Then let peer diffusion carry it past the platform team's own capacity.
Prove it
A newly onboarded engineer can name the specific ticket they paired on. Knowing that a training session happened is not enough.

04 Absorbing the output

An agent that writes code faster changes nothing on its own if the team downstream cannot absorb the output. Review capacity, CI throughput and spec quality were sized for the old rate of change. A rollout that does not resize them only moves the bottleneck from writing code to reviewing it, at the same overall speed. The senior engineer's job changes here too, whether anyone planned for it or not. Once code is no longer scarce, judgment about what to build and what to approve becomes the main work. If a team never says so, its senior engineers end up competing with agents at the one task that stopped being the bottleneck.

What it covers
A concrete plan for who reviews the extra pull requests that an agent-assisted team produces. Agree it before rollout. Do not discover it later as a growing queue.
How to build it
Model the expected diff volume from the pilot's own numbers. Check whether current reviewers can absorb it inside their existing hours. If they cannot, the review step itself needs a design change.
Produces
A rollout that does not convert a code-writing bottleneck into a review-queue bottleneck of the same size.
Tradeoff
Adding review capacity has a real cost, either headcount or a deliberately slower ramp. Both are cheaper than a queue nobody planned for.
Prove it
The team's review latency after rollout sits at or below its pre-rollout baseline. Check it directly. Do not assume it from the plan.
What it covers
A stated rule for who is accountable for a merged change's correctness. The rule holds whether an agent or a human wrote the diff.
How to build it
Keep the existing human-approver requirement on every merge. Say explicitly that approving an agent-authored diff carries the same accountability as approving a human-authored one.
Produces
A merge queue where "the agent wrote it" is never a legitimate answer to "who approved this?"
Tradeoff
This adds no new process, which is the point. It does mean reviewers cannot treat an agent's diff as lower stakes to wave through. Say that out loud. Do not leave it implied.
Prove it
A reviewer asked why they approved a specific agent-authored change gives the same kind of answer they would give for a human-authored one.
What it covers
A minimum bar for what a ticket contains before it goes to an agent. It needs acceptance criteria, the files in scope and a picture of what "done" looks like. A competent engineer would want the same before starting the ticket.
How to build it
Reject or return a ticket that fails the bar at triage. A team applies the same discipline before it assigns a ticket to a person.
Produces
Fewer plausible-looking diffs against a vague ticket. These diffs are the biggest driver of review time on agent-assisted work.
Tradeoff
A stricter triage bar slows down ticket creation. That reads as friction against the speed the rollout should deliver. The friction pays for itself in review time saved.
Prove it
A sample of tickets from the last sprint each clear the stated bar. Check the ticket text itself. The triage process's own account is not proof.
What it covers
An explicit statement that review, spec-writing and architectural judgment are now the senior engineer's main output. It replaces an implicit expectation that their value is measured in lines shipped.
How to build it
Update how you evaluate and staff senior engineers. Credit review quality and spec clarity alongside their own commit volume. Do this before the incentives punish the people who do the most valuable work.
Produces
Senior engineers who spend their time where the bottleneck moved. They do not compete with agents at the one thing agents are now faster at.
Tradeoff
Changing how a role is evaluated is a management change. It is not a tooling rollout, and it lands slower than the harness. Start the conversation with the pilot team early, before the mismatch shows up in someone's review.
Prove it
A senior engineer on the pilot team can describe their own role change in their own words. The rollout announcement's words do not count.

05 Measurement

A number reported after rollout with no baseline before it is an anecdote wearing a dashboard. Most programmes never capture what cycle time, review latency or change-failure rate looked like before the harness arrived. Every improvement claim afterward is then measured against a guess. The fix is procedural. Take the baseline in week one, on every team the rollout plans to compare later. Then pair every adoption number with the quality number that would catch it flattering the programme.

What it covers
A recorded measurement of every scoreboard metric for the pilot team, taken before the harness is installed. The metrics include cycle time, review latency and change-failure rate.
How to build it
Run the measurement pass as the pilot's first step, before the first agent-assisted change merges. Store the result where the post-rollout comparison can reach it.
Produces
A true before-and-after comparison. Without it, a post-rollout number has nothing honest to compare against.
Tradeoff
Measuring a baseline delays the pilot's visible start by a sprint or two. That delay is what makes the later comparison mean anything.
Prove it
A dated baseline snapshot exists for the pilot team, and it predates the first agent-assisted merge. Check it against the timeline.
What it covers
A rule that no adoption metric gets reported without a paired quality metric. Adoption metrics include seats activated, weekly active users and diffs merged. The quality metric would catch the number flattering the programme.
How to build it
Publish the two numbers together on the same dashboard and in the same report, every time. Never report the adoption number alone.
Produces
A scoreboard that nobody can read as "adoption is working" when the real story is usage up and quality flat or worse.
Tradeoff
A paired number is more work to compute. It can undercut a headline the sponsor wanted to lead with. Report it anyway. Otherwise the number is one nobody trusts six months later.
Prove it
The last quarterly rollout report shows every adoption figure with its paired quality figure on the same page.
What it covers
A reporting split between the top decile of usage and the median engineer. A rollout's average is usually carried by a small number of enthusiasts.
How to build it
Segment every adoption metric by usage decile before you report an org-wide average. State both the median and the top decile.
Produces
An honest read on whether the rollout reached the organisation or only its most eager early adopters.
Tradeoff
A segmented report is a less flattering headline than a single averaged number. It is also the only version that predicts what happens when the enthusiasts change teams.
Prove it
The current adoption report states the median engineer's usage figure. It does not report only the mean.
What it covers
A cost figure counted in what shipped: a merged, reviewed change that nobody reverted. A per-seat licence cost says nothing about output.
How to build it
Divide a team's total rollout spend by its count of merged agent-assisted changes over the same period. Track it on the same cadence as any other delivery metric.
Produces
A spend number that moves with delivery. Cost per outcome is then visible before the finance review finds it first.
Tradeoff
Cost per merged change needs the same ledger discipline that a fleet's per-task spend tracking already requires. It does not fall out of a seat count for free.
Prove it
The last quarter's cost-per-merged-change figure for the pilot team is checked against billing. Do not estimate it from seat count.

06 Sustain

Adoption that plateaus at the enthusiasts looks like success if the only number anyone tracks is whether the licence got used at all. The harder questions come later. Why did the other engineers stop after week one? What do the unsanctioned workarounds say about the paved road? Who owns the harness once the pilot team that built it has moved on? A rollout that never revisits its scaffolding ages into the rigidity it was meant to replace. The same happens if it never sets a cadence to refresh the harness as the underlying tooling changes.

What it covers
A standing check for shadow use. Engineers route around the sanctioned harness with a personal account or an unapproved tool. Treat this first as evidence that the paved road is missing something.
How to build it
Ask the engineers who do it what the sanctioned harness does not cover. Do this before you write a policy that blocks the workaround.
Produces
A paved road that closes the gap that drives the workaround. A ban would only push the same behaviour further out of sight.
Tradeoff
Treating shadow use as a discovery process is slower than blocking it. It also means you tolerate some ungoverned use while the gap gets fixed. The security review sets the limit.
Prove it
The last confirmed instance of unsanctioned use led to a named change to the sanctioned harness. A warning alone does not count.
What it covers
A team or role accountable for the sanctioned harness after the pilot ends. It is separate from the pilot team, which will move to other work.
How to build it
Make the ownership handover a named step in the exit-from-pilot criteria. Do not leave it as an afterthought once the pilot team has moved on.
Produces
A paved road that survives the move of the people who built it to a different project.
Tradeoff
A dedicated platform owner is a real, ongoing cost. It is not a one-time project expense. Budget it from the mandate stage, so it is not a surprise once the pilot ends.
Prove it
The current platform owner is not the person who ran the original pilot. If they are the same person, an explicit reason is on record.
What it covers
A stated date to remove the temporary shortcuts, manual approvals and extra logging that a pilot runs. A scaled rollout should not carry them indefinitely.
How to build it
Write the retirement date into the exit-from-pilot criteria beside the metrics bar. Track scaffolding that is still in place past that date as a debt item. It is a cost that someone can see.
Produces
A rollout that does not carry pilot-era overhead into a process meant to run at many times the pilot's scale.
Tradeoff
Some scaffolding turns out to be load-bearing once you try to remove it. Make it a permanent, supported control. Do not leave it as an unowned pilot leftover.
Prove it
The scaffolding list from the pilot's exit criteria shows each item as retired or explicitly promoted to a supported control.
What it covers
A recurring, scheduled review of the sanctioned default. Cover the harness, the instructions-file template and the tool boundary. Compare each against how the underlying agent tooling has changed since launch.
How to build it
Put the review on the same calendar discipline as any other platform component's support lifecycle. Give it a named owner and a fixed interval. Do not trigger it only by a complaint.
Produces
A paved road that keeps pace with what the tooling can now do. It does not ossify at whatever the pilot shipped with.
Tradeoff
A fixed refresh cadence spends platform time on a review that sometimes concludes nothing needs to change. That outcome is cheaper than a harness nobody revisits until it is visibly behind.
Prove it
The last scheduled harness review happened on the calendar date it was due. It produced a change or a stated reason that none was needed.
Five ways a rollout lies to itself

A busy dashboard is a claim, not proof.#

Every one of these looks like a working rollout right up until the number it was supposed to produce turns out to be hollow.

Seat activation counted as adoption

Looks like: A dashboard shows every engineer with a licence assigned and at least one login. The rollout reports that as adoption.

Costs you: An assigned licence and one login say nothing about whether anyone landed a merged change. The rollout can report full activation while real usage sits with a handful of engineers.

Fix: Report the paired quality number beside every adoption figure. Segment by usage decile so the median engineer's number cannot hide behind the top decile's.

See the practice: Pair every adoption number with a quality number →

The power user's numbers generalised to the org

Looks like: One enthusiastic team lead reports a large personal speedup. The rollout report quotes that number as the expected result org-wide.

Costs you: That number is true for one person who was already motivated to make the tool work. It says nothing about the engineers who have not tried it yet. Expectations set against it make every ordinary result look like underperformance.

Fix: Segment every metric by usage decile and state the median. Show it beside the number that made the best slide.

See the practice: Measure the power user and the median engineer separately →

Time saved with no throughput or quality change

Looks like: Engineers report that the tool feels faster. The rollout counts that as a win.

Costs you: Review may have absorbed the saved time through a bigger queue and more back-and-forth on agent-authored diffs. The change-failure rate may also have crept up. Either way, the time saved never reached a shipped, working outcome. The bottleneck moved downstream where nobody was watching.

Fix: Size review capacity to the new diff volume before rollout. Track cycle time and change-failure rate, because they would catch the trade as it happens.

See the practice: Size review capacity to the new diff volume before it arrives →

A mandate with no paved road

Looks like: Leadership announces the rollout and a licence budget. It expects teams to work out the rest themselves.

Costs you: Every team invents its own guardrails at a different level of care. Engineers blame the tool for the friction when the harness is what is missing. The mandate's credibility erodes before the paved road ships.

Fix: Ship one sanctioned default harness, piloted and packaged, before the mandate reaches teams beyond the pilot.

See the practice: Ship one sanctioned default harness before asking teams to adopt anything →

A pilot that depended on someone babysitting it

Looks like: The pilot clears every exit criterion, and the team credits the harness.

Costs you: The real reason it worked may be one platform engineer who fixed broken configs and coached people through failures all quarter. The result then does not survive that person moving on. The second team's onboarding fails in ways the pilot never surfaced.

Fix: Run enablement as pairing and worked examples in the team's own repo. Add a named platform owner who is separate from whoever ran the original pilot.

See the practice: Name a platform owner before the pilot team disperses →
The scoreboard

Six numbers, and how each one lies.#

Every one of these is measurable today. None of them is trustworthy read alone. Pair it with the number next to it.

Share of engineers landing a merged agent-assisted change weekly

Why it matters
The most direct read on whether the rollout reached the organisation or stopped at its early adopters.
How it misleads
A rising share can come entirely from the same enthusiasts merging more often, with no new engineer ever starting.
Pair with
Count of distinct engineers who landed their first agent-assisted merge this month

Acceptance rate of agent-authored changes

Why it matters
Whether the diffs an agent proposes are good enough to ship. Volume alone does not show that.
How it misleads
A high rate on a narrow, well-scoped ticket type says nothing about the harder tickets that nobody has routed to an agent yet.
Pair with
Share of ticket types routed to an agent, beyond the ones the rate is measured on

Review latency and review-hours per merged change

Why it matters
Catches a code-writing bottleneck that turns into a review-queue bottleneck of the same size.
How it misleads
Latency can look fine if reviewers work unpaid overtime to keep the queue moving. Hours per change catches that.
Pair with
Reviewer-reported time spent per agent-authored change, tracked separately from latency

Change-failure and rollback rate on agent-authored changes

Why it matters
Whether shipping faster came at the cost of shipping worse.
How it misleads
A flat rollback rate can hide a rise in near-misses. Review caught them before they reached production, so the rate never counts them.
Pair with
Defects caught in review per agent-authored change, tracked as its own number

Cost per merged change

Why it matters
The spend number that moves with real output. It does not move with how many licences were assigned.
How it misleads
A falling cost per change can hide a rising rollback rate if nobody checks the change against quality.
Pair with
Change-failure rate on the same set of changes, read on the same dashboard

A new joiner's time to first merged agent-assisted change

Why it matters
The clearest read on whether the paved road onboards someone who was not there for the pilot.
How it misleads
A fast time for a joiner on the original pilot team says nothing about a joiner on a team that gets little platform attention.
Pair with
The same figure measured on a team outside the original pilot
What this is built from

Verifiable, not claimed.#

No invented survey statistics. This section lists what the site already documents, and how the pieces connect.

Sources

  1. NIST - AI Risk Management Framework (AI RMF 1.0)

    The risk-management functions our governance and adoption controls map onto. Retrieved 2026-08-24.

  2. Stanford HAI - AI Index Report

    The annual report our adoption page's industry-wide statistics are drawn from. Retrieved 2026-08-24.

Not sure the pilot would generalise?

A Ship Audit checks the paved road and the absorption math against your specific team before the rollout reaches the next 179 engineers.

Questions

Before you announce a rollout.#

What teams ask us before they take an agent programme past the first team.

01 Isn't this just change management with new labels?

Change management is the general discipline. This page is the version for an agent rollout. It adds a paved road, review-capacity math and a scoreboard. A generic change programme does not ship with these.

Link to this answer: Isn't this just change management with new labels?
02 We already ran a successful pilot. Why isn't that enough?

A pilot proves the harness can work somewhere. It does not prove the result generalises to a team with a different workload. It also does not prove the organisation can absorb several times the review volume once the rollout scales past the pilot team.

Link to this answer: We already ran a successful pilot. Why isn't that enough?
03 What's the fastest way to kill a rollout without meaning to?

Skip the paved road and ship a licence with no sanctioned default. Every team improvises its own guardrails at a different level of care. People read the friction as the tool being bad. The mandate loses credibility before anyone builds the thing that would have made it work.

Link to this answer: What's the fastest way to kill a rollout without meaning to?
04 How do we know adoption is real, rather than seat activation with nothing behind it?

Pair every adoption figure with its quality counterpart. Check the median engineer beside the top decile. Activation with no merged output and no movement in review latency is the seat-activation trap. On a dashboard that reports only the average, it looks identical to real adoption.

Link to this answer: How do we know adoption is real, rather than seat activation with nothing behind it?
05 Who should own this, the platform team or engineering leadership?

Both, at different points. Leadership owns the mandate and the definition of success. The platform team owns the paved road and its upkeep once the pilot ends. A rollout that lacks either one stalls in a different place, depending on which is missing.

Link to this answer: Who should own this, the platform team or engineering leadership?
06 Does this still apply once we're past the pilot and scaling org-wide?

Yes. Most of what breaks a rollout shows up after the pilot succeeds. This includes absorption capacity, the median-versus-power-user gap and unsanctioned use. Revisit the surfaces above at the scaling stage, even if the mandate and first-team work already happened.

Link to this answer: Does this still apply once we're past the pilot and scaling org-wide?
Find out before the org finds out

A mandate with no paved road is one of five ways a rollout stalls, and one power user rarely represents the org. A Technical Diligence review checks whether your own rollout is actually built on that road.

Source: https://customlabs.io/adoption/

↑↓ navigate ↵ select esc close