How mature is your DevOps organization's AI adoption?
Prepared for ACME Corp. — a B2B SaaS company with ~280 engineers — following a 30-minute structured evaluation with its platform engineering leadership. Company details in this sample are illustrative.
Executive summary
One composite score, five maturity levels, and the three findings that matter most.
ACME Corp. has moved decisively past experimentation: coding assistants are standard issue across engineering, and two background agents — built on shared LangChain-based libraries and a common skills repository — are running in production. That places the organization ahead of the median for its peer group. However, AI adoption remains confined to single-user and fire-and-forget patterns. The collaborative, multi-user workflows where DevOps work actually happens — releases, environment provisioning, incident response — remain untouched by AI, and the organization has none of the shared infrastructure (access control, audit, shared context, cost governance) that those workflows require.
| L1 | Ad hoc | Coding assistants at most; individual experimentation |
| L2 | Emerging | Assistants standardized; background agents running; no multiplayer capability |
| L3 | Structured | First multiplayer agents, built piecemeal; shared libraries emerging |
| L4 | Managed | An Agentic Platform provides most categories; centrally governed |
| L5 | AI-native | Agentic Platform provides all categories; agents hold only use-case logic |
Ahead of the median for its peer group — and one multiplayer use case away from Structured.
Assistant adoption is genuinely broad
~85% of engineers use a coding assistant weekly. IaC and pipeline authoring are routinely AI-accelerated — real leverage, not shelfware.
No multiplayer AI capability
Every AI session is single-user. No shared sessions, shared context, or team-visible agent activity exists — each engineer's AI works in isolation.
Ungoverned agent access is accumulating
Both production agents hold static, broadly-scoped credentials with no audit trail of agent-initiated actions and no human-in-the-loop policy.
Scorecard
A flat, additive scorecard out of 5.0. The first three milestones measure what runs today; the last two measure what is shared. No platform is assumed — the first 3.0 points are earnable without one.
| Milestone | ACME Corp. today | Points |
|---|---|---|
| 1 · Using coding assistantsSingle-user AI in daily engineering use | Claude Code and Cursor deployed org-wide; ~85% of engineers use one weekly for IaC, pipeline and script authoring. | 1.0 / 1.0 |
| 2 · Building background agentsPre-coded, trigger-driven, unattended | Two in production: an incident-triage summarizer on PagerDuty alerts and a nightly cost anomaly report. | +0.5 / 0.5 |
| 3 · Building multiplayer agents, piecemealMulti-user agents, each a bespoke build | None. Release management, environment provisioning, troubleshooting and PR reviews remain manual, ticket-driven workflows. | 0 / 1.0 |
| 4 · Shared libraries & reusable codeCommon code agents reuse — each running its own copy | Shared skills repositories are in active use, and both background agents are built on common libraries on top of LangChain — reusable code, though each agent runs its own copy. | +0.5 / 0.5 |
| 5 · Agentic PlatformShared components run as a service all agents plug into — not copied | Scored across the seven categories below, ~0.29 each. Partial credit for the agent-accessible cloud automation estate and early observability tooling. | +0.35 / 2.0 |
| Σ Composite | Level 2 · Emerging — the single-user, background-agent and shared-code points are fully banked; the multiplayer and Agentic Platform points are almost entirely unclaimed. | 2.35 / 5.0 |
Agentic Platform — Milestone 5 breakdown
Seven categories, ~0.29 points each.
SSO exists for humans; agents use shared static credentials. No workspaces or per-user credential mapping.
No shared sessions, shared context or agent memory, skill distribution, or model routing service.
Application observability is strong; agent audit trails, prompt-injection posture and approvals are absent.
Mature Terraform, EKS and pipelines — a real shared automation surface agents could act through.
Both agents run on hand-rolled infrastructure; no shared deployment, update or fault-handling service.
All AI interaction is chat or Slack text. No shared forms, dashboards, approvals, or API/CLI invocation.
No way to describe a use case in plain English and have the platform build the agent — every agent is hand-coded by engineers.
Reading the scorecard: ACME Corp. has banked everything below the sharing line — assistants, background agents, and a genuine shared-code layer — but none of the multiplayer points and little of the Agentic Platform. At 2.35, it sits near the top of Emerging: the first multiplayer use case (milestone 3) alone would cross it into Structured, and every fraction of milestone 5 earned makes each subsequent agent cheaper to build.
AI cost efficiency
Where DevOps-related tokens go today across the three agent categories — and the four spend patterns driving avoidable cost.
Coding assistants
Single-user sessions on individual laptops. High duplication: the same organizational context is re-assembled, in tokens, engineer by engineer.
Background agents
Two production agents. Inflated by LLM-in-the-loop design — the triage agent spends tokens on deterministic steps that need no model at all.
Collaborative (multiplayer) agents
No spend — because no capability exists. 100% of tokens serve single-user or unattended patterns; none serve the multi-user workflows where DevOps value concentrates.
The distribution is the diagnosis: a maturing organization's token mix shifts right — from individual assistants toward shared, collaborative workflows where one governed session serves many users. ACME's mix is entirely on the left, which is why AI spend grows linearly with headcount rather than with outcomes.
Siloed individual sessions
Every engineer runs isolated AI sessions. Context assembled by one engineer — architecture, conventions, runbooks — is re-purchased in tokens by the next.
Frontier model for everything
A single frontier model serves all workloads, including routine summarization and formatting tasks that cheaper models handle at a fraction of the cost.
LLM in the loop for routine actions
The incident-triage agent invokes the LLM for every alert — including high-frequency, fully deterministic steps that need no model judgment at all.
No token accounting
Spend is visible only as a monthly invoice total. No breakdown by team, use case, or workflow — so waste has nowhere to be seen, let alone managed.
35–50% of ACME Corp's current AI spend is structurally addressable through shared context, per-workflow model right-sizing, and removing the LLM from deterministic high-frequency actions — before any negotiation on unit price.
Methodology & definitions
How this evaluation works: the five additive milestones behind the scorecard, how they map to maturity levels, and the reference architecture that defines the upper levels.
30-minute structured interview
Covers adoption patterns, agent inventory, platform capabilities and AI spend posture. No access to code, cloud accounts or internal systems is required.
Five milestones, additive to 5.0
A DevOps platform specialist scores each milestone from the interview responses. Points accumulate — from coding assistants through the Agentic Platform — and map to five maturity levels.
Positioned against peers
The composite is placed against evaluated organizations of comparable engineering headcount and cloud footprint, identifying where the separation from the top quartile sits.
The scorecard is flat and additive: five milestones, each worth a fixed number of points, summing to a 5.0 maximum. The first three milestones score what an organization runs — no platform is assumed or required. The last two score what is shared, and distinguish code that is merely reused from components that are hosted once for every agent to plug into.
| 1.0 PT | 1 · Using coding assistants | Single-user AI in daily engineering use — one engineer, one session, generating IaC, pipelines and scripts consumed the traditional way. |
| +0.5 PT | 2 · Building background agents | Pre-coded, trigger-driven agents running unattended — alert triage, scheduled remediation, reports. No human in the loop during execution. |
| +1.0 PT | 3 · Building multiplayer agents, piecemeal | Multi-user, interactive agents where humans and agents act together on shared state — releases, environments, troubleshooting — each built as its own bespoke stack. |
| +0.5 PT | 4 · Shared libraries & reusable code | Common code multiple agents reuse — SDKs, prompt and skill libraries, model proxies — but where each agent still runs its own copy. |
| Up to +2.0 | 5 · Agentic Platform | Shared components provided as a running service every agent plugs into — SSO, workspaces, RBAC & policy model, governance, shared context, model routing, hosting, UX, and a no-code agent builder. Scored across seven categories (~0.29 each) by how much the platform provides versus what each agent must still bring itself. |
| 0.0–1.4 | L1 · Ad hoc | Coding assistants at most — individual experimentation, no organizational posture. |
| 1.5–2.4 | L2 · Emerging | Assistants standardized and background agents in production; no multiplayer capability. |
| 2.5–3.4 | L3 · Structured | First multiplayer agents, built piecemeal; shared libraries emerging. |
| 3.5–4.4 | L4 · Managed | The Agentic Platform provides the majority of the seven categories; access, audit and cost governed centrally. |
| 4.5–5.0 | L5 · AI-native | The Agentic Platform provides substantially all seven categories — agents contain only use-case logic. Multiplayer sessions and model routing by default. |
Get your organization's evaluation
A 30-minute call with a DevOps platform specialist. Your scored report is delivered within two business days.