Agent Trace adds request-level observability to supported agent traffic sent through R9S. Open Agent observability in the Console to inspect request counts, sessions, token activity, costs, and the expandable Trace list.
This is operational model-request telemetry, not a recording of an agent’s entire execution. It does not automatically instrument local tools, shell commands, file changes, or every span in your application.
What is recognized
R9S recognizes these agent identifiers from supported User-Agent patterns or an explicit supported metadata.agent value:
| Agent | Telemetry identifier |
|---|---|
| Claude Code / Claude CLI | claude_code |
| Codex | codex |
| Pi | pi |
| OpenCode | opencode |
| ZCode | zcode |
Agent recognition and model compatibility are different checks. Recognition does not certify a particular client version or protocol. See Agent Integrations for client setup, and confirm the key, model, and endpoint independently.
Preserve the real client User-Agent when using a wrapper or proxy. An unrecognized browser, generic SDK, or other client does not enter agent telemetry merely because it sends a trace ID. If an actual supported agent’s wrapper cannot preserve its User-Agent, use that agent’s exact supported metadata identifier; do not relabel unrelated traffic as a different agent.
Choose a compatible key
Agent telemetry follows supported requests, not a separate Trace API key. It can accompany standard, Plan, BYOK, and Router traffic under the applicable authorization and billing context.
- For standard keys, use the public model and endpoint available to the key.
- For Plan Keys, follow the Token Plan model aliases and protocol settings.
- For BYOK, match the connection protocol and authorize the provider.
- For Router keys, use Chat Completions only. An agent configured for Responses or Messages cannot use a Router key by changing only the model name.
Correlation fields
Send opaque identifiers without customer content. Keep a session ID stable across an agent conversation, a trace ID stable across a logical operation, and request/span IDs distinct for each model request or retry.
| Request header | Metadata equivalent | Purpose |
|---|---|---|
X-R9S-Agent-Session-Id |
session_id |
Group requests into an agent session |
X-R9S-Request-Id |
request_id |
Correlate one model request with application logs |
X-R9S-Trace-Id |
trace_id |
Link model requests belonging to the same operation |
X-R9S-Span-Id |
span_id |
Identify the individual model-call span |
X-R9S-Parent-Span-Id |
parent_span_id |
Associate the model call with its parent span |
Headers take precedence over the corresponding metadata fields. IDs are trimmed, limited to 256 characters, and accept only ASCII letters, digits, and ._:@/+~=-. Empty, invalid, overlong, or control-character values are omitted. When using headers, usage-tag validation also applies, including its prohibition on = in values. Use UUIDs or similarly opaque values, not email addresses, prompts, file paths, or secrets.
If a request ID is absent, the gateway uses an available gateway call ID or response ID. If a span ID is absent, it uses the request ID. Session, trace, and parent-span IDs are not automatically reconstructed when the client does not provide them. Do not assume standard W3C traceparent alone populates these R9S fields.
Minimal header example
This illustrates a model call from an OpenCode integration; the sample User-Agent version is illustrative, not a version-verification claim. Real integrations should preserve the client’s actual User-Agent. Use a standard key with access to glm-5.3:
export R9S_API_KEY="<R9S_STANDARD_API_KEY>"
curl https://gw.r9s.ai/v1/chat/completions \
-H "Authorization: Bearer $R9S_API_KEY" \
-H "Content-Type: application/json" \
-H "User-Agent: opencode/1.0.0" \
-H "X-R9S-Agent-Session-Id: session-demo-001" \
-H "X-R9S-Trace-Id: trace-demo-001" \
-H "X-R9S-Request-Id: request-demo-001" \
-d '{
"model": "glm-5.3",
"messages": [{"role": "user", "content": "Say hello in one sentence."}],
"max_tokens": 64,
"stream": false
}'
Generate fresh IDs for real work; do not reuse the demo request ID for multiple calls. The response remains a normal Chat Completions response. Inspect the Console after usage delivery rather than expecting a new trace payload in the inference response.
Custom X-R9S-* request headers also pass through current usage-tag validation, which permits at most three distinct name/value pairs. Do not send every header in the table together or add more custom X-R9S-* headers to this example. For richer correlation, prefer supported request metadata instead of placing all IDs in custom headers. See API Reference.
Metadata alternative
For a client and endpoint that accept request metadata, the following Chat Completions payload carries the complete correlation set without five custom headers. Use it instead of the header example’s correlation fields:
{
"model": "glm-5.3",
"messages": [{"role": "user", "content": "Say hello in one sentence."}],
"max_tokens": 64,
"metadata": {
"agent": "opencode",
"session_id": "session-demo-001",
"request_id": "request-demo-001",
"trace_id": "trace-demo-001",
"span_id": "span-demo-001",
"parent_span_id": "span-parent-001"
}
}
The exact agent value is required when neither the User-Agent nor other supported client metadata identifies the actual agent. Do not assume that a client’s arbitrary local configuration fields become gateway request metadata: inspect its outgoing request.
Optional telemetry fields include agent_version, speed, speed_mode, service_tier, reasoning_effort, and request_source. Most non-ID scalar fields are limited to 128 characters, and agent_version to 64. These fields describe observed request context; setting an observability header is not a guarantee that an upstream model enables that speed or reasoning feature.
Read the Console
Select the correct organization, billing scope, and time range before investigating a missing or unexpected record. Agent reporting combines the applicable platform balance, Plan, and BYOK usage contexts; the same agent can appear across them without making their balances interchangeable.
| View | What it shows |
|---|---|
| Summary and usage trend | Requests, observed sessions, token volume, cost, and daily activity |
| Usage by agent | The same operational measures grouped by supported agent |
| Agent sessions | Requests grouped by agent and session ID, with observed model/provider context |
| Trace list | Request-level timeline, duration, tokens, cost, status, and agent |
| Expanded trace | Available model/provider, HTTP status, request/trace/span/parent IDs, session ID, and safe failure metadata |
Copy the request ID to correlate a row with your own logs or a support report. Duration comes from measured request timing; it is not time to first token or the total wall-clock duration of the entire agent task. The timeline is an operational view, not a complete distributed parent/child execution tree.
Usage is delivered asynchronously and queries are bounded. The current summary returns up to 100 session rows and 200 trace rows, not a complete export. Narrow the time range or request filters for investigation. A missing record is not proof that no request ran or no provider charge occurred, especially for failures before agent identification or telemetry capture.
Tokens, costs, and failures
- Token reporting depends on what the provider returns. Agent metrics can include input, output, cache-read, cache-creation, and thinking categories; Codex telemetry does not emit a separate cache-creation metric.
- Cache and reasoning categories can overlap other usage counts. Do not sum every displayed or raw category as if all categories were independent billable tokens.
- Cost follows the applicable billing context. Plan allowance consumption is not wallet cash; BYOK sale/cost values come from configured prices and do not replace the provider invoice.
- Captured failures can include status, HTTP status code, safe error code, and failure reason. An absent duration, token category, provider, or status means that field was unavailable, not necessarily zero or success.
- Session and trace IDs are client-supplied correlation labels, not authorization credentials or an idempotency guarantee. A retry is another model request and may incur additional usage.
Privacy and troubleshooting
The agent telemetry path retains an allowlisted set of operational scalar fields and usage metrics. It does not include prompt or response bodies, messages, source files, or tool arguments. This is a boundary of agent telemetry, not an end-to-end zero-data-retention promise for providers, gateway logs, or third-party tools. Follow your organization’s data-handling policy and the response storage guidance.
| Symptom | Check |
|---|---|
| No agent rows, but ordinary usage exists | Verify the actual User-Agent or supported metadata.agent; trace IDs alone do not identify an agent |
| Requests appear without sessions | Send a valid stable session ID; R9S does not invent conversation membership |
| Trace or parent-span ID is missing | Confirm the client transmitted it, check precedence and character/length rules, and avoid relying on traceparent alone |
invalid_billing_usage_tag after adding trace headers |
Keep custom X-R9S-* tags within the three-pair limit or move correlation into supported metadata |
| Fewer rows than expected | Check organization, scope, date range, filters, asynchronous delivery, and row limits |
| Router calls fail before useful trace data appears | Verify Chat Completions, the profile mapping, and selected-target admission; inspect the request error and Router audit through support |
For an incident, share the organization, time range, model, request ID, trace ID, and safe error code. Never attach API keys or sensitive prompts. Continue with Monitoring and Operations for incident and cost workflows.
