Scoped agent credentials, pinned regression replay, and the Agent ROI guide

Three agent-facing updates shipped recently on Devotel Orbit: scoped credentials on every downstream agent call, pre-promotion regression replay for prompt changes, and a documentation guide for the Agent ROI dashboard. This is the worked version of that announcement — actual request shapes, reading the delegation chain, a regression-pin walkthrough, and ROI numbers you can sanity-check.

Scoped, auditable credentials on every agent call

AI agents act downstream — LLM tool calls, MCP server integrations, paid API calls — and those actions too often run on whatever ambient session token the agent happens to hold. Orbit's agent OAuth surface speaks OAuth 2.0 Token Exchange (RFC 8693): an agent presents the token it holds, along with its own actor token, and receives back a shorter-lived credential narrowed to exactly the tools the task at hand requires — never wider than what it already held, and bound to only the resource it names.

A concrete exchange asks for one narrow scope:

POST /api/v1/agents/:id/token-exchange
Content-Type: application/x-www-form-urlencoded

grant_type=urn:ietf:params:oauth:grant-type:token-exchange
&subject_token=at_appY4J1
&subject_token_type=urn:ietf:params:oauth:token-type:access_token
&actor_token=at_agentH0stQ9
&actor_token_type=urn:ietf:params:oauth:token-type:access_token
&requested_actor=user-erol
&scope=messages:read
&resource=https://orbit.devotel.io/mcp

The response is the narrowed credential:

HTTP/1.1 200 OK
Content-Type: application/json

{
  "access_token": "at_narrowedKv83",
  "issued_token_type": "urn:ietf:params:oauth:token-type:access_token",
  "token_type": "Bearer",
  "expires_in": 900,
  "scope": "messages:read"
}

Every element of the request is load-bearing. subject_token is what the agent currently holds and gives up. actor_token authenticates the agent itself, so the exchange cannot run unattributable. requested_actor names the sponsoring principal. scope is the narrowing request. resource (RFC 8707) audience-binds the returned token to one protected resource — a token minted for the hosted MCP server is useless anywhere else, and the server rejects unrecognised resource values outright rather than issuing an unbound token. Omitting the three required fields returns invalid_request; requesting a scope the installed app was never granted returns invalid_scope; a bad resource returns invalid_target.

Reading the delegation chain

The exchanged token carries a signed delegation chain, so a security review reconstructs provenance straight from the credential. The chain reads in this order:

1. Actor — the agent (for example, agent-support-bot).
2. Sponsor — the requested_actor, the human or principal who authorised the delegation (user-erol in the exchange above).
3. Permissions — the granted scopes after narrowing (messages:read).

Read that as "this agent did it, on behalf of this person, with these permissions." Because delegation may only narrow the parent app's granted scopes, the chain can never claim more power than the sponsor consented to, and a request without all three of subject token, actor token, and requested actor is refused. You can inspect the same attribution in the dashboard under Agents → Authorization mandates, and the exchange contract is in the API reference next to the agent OAuth endpoints.

Prove a prompt before you promote it

Agent regression tests — the saved conversations under an agent's testing panel — can be pinned to a prompt version, and a run can override that pin with a candidate version id. The walkthrough, end to end:

1. Pin a version. Under the agent's versions page, take the prompt version currently in production — say pver-014. Pin the regression suite to that id so every replay scores a frozen prompt, not whatever config is live.
2. Run the saved corpus. Replaying scores every saved conversation against pver-014 and reports a pooled pass rate plus per-set regression versus the prior run.
3. Override with a candidate. Before promoting a rewritten prompt — say pver-015 — rerun the suite with the candidate version id overriding the pin. The live agent stays on pver-014; the replay runs in the same sandbox mode as the studio test panel.
4. Read pass/fail. The run reports the pooled pass rate and flags any golden set whose score or latency regressed past its threshold. A healthy look: 100 of 100 passed, worst regression 0.00 — promote. An unhealthy one: a set regressed 14% on judge score — hold the promotion.
5. Promote on evidence. Because the promotion gate and the saved test runs draw on the same corpus, green means promote and red means iterate — either way the gate and the suite agree.

Where a regression used to announce itself only after the change shipped, promotion is now gated on evidence from the exact version you are about to flip live.

Reading Agent ROI with real assumptions

The Agent ROI surface compares per-agent AI cost against the attributed revenue of resolved outcomes, with a daily trend and a pricing config you own. A new Guides → Agent ROI attribution docs page explains how to set the assumptions; here is a worked setup:

The config behaves as read-merge-write: set defaults, then override only the rubrics you need to. Margin per agent follows from two numbers: in June an agent resolved 2,140 conversations at a blended AI cost of $0.11 per conversation — $235.40 — while attributing 2,140 × $4.00 = $8,560 in outcome value. Margin: $8,324.60, roughly 97% of attributed value.

Three summary endpoints feed the dashboard: the overall summary (cost, attributed value, margin), the daily trend, and the config itself. Access is role-gated so the numbers stay visible only to those who should see spend and revenue:

Attribution numbers are only as meaningful as the assumptions behind them; the guide shows how to set a defensible per-outcome value so the per-agent margin is worth acting on.

What this adds up to

AI agents in production are judged on three axes: can you audit what they did, can you prove a change is safe before it ships, and can you tell whether the agent pays for itself. Scoped token exchange handles attribution, pinned regression replay gates promotion, and the ROI guide makes the cost-versus-outcome numbers readable. Each lives in the Devotel Orbit dashboard and is documented in the guides section of the docs.