Error handling

Use the HTTP status and response body to decide whether an operation succeeded. Budget headers describe admission, not the result of the endpoint: an allowed request can still fail validation or encounter a server error.

The coworld and softmax CLIs apply this page for you on Observatory API calls: a failed request prints the status, the server’s detail, a next step, the request id, and the documentation link instead of a traceback. The browser sign-in exchange (softmax exchange-code) reports its own shorter failure message.

Diagnose the responseCopy Anchor Link

The base URL is https://softmax.com/api/observatory, including /observatory. An HTML 404 often means the request reached the website instead of the API. Inspect Content-Type before trying to parse an error as JSON.

See Authentication and identities for user and player credentials.

The SQL console cancels queries after 20 seconds. Its 408 response uses type: "request_timeout" and a text detail with the next step. Do not automatically retry an unchanged query that exceeded this execution limit.

Distinguish throttling from other failuresCopy Anchor Link

Shared API budget rejections use detail.type: "api_rate_limit_exceeded" when enforcement is active. These requests do not run the endpoint handler. Follow Rate limits for the response headers and retry example.

Other protections can also return 429. An operation-specific write limit uses detail.type: "write_rate_limit_exceeded". Concurrent-request limits may return a text detail instead. Do not assume every error body has the same shape, or that every 429 came from the shared minute/hour buckets. Branch on a documented stable type, not the human-readable message.

Retry safelyCopy Anchor Link

Honor Retry-After when supplied. It is a minimum delay, not a reservation: another worker sharing your budget may spend the available credits first. Add jitter, bound attempts and total elapsed time, and avoid retries at several nested layers.

Reads are normally safe to repeat. For writes, use the endpoint’s documented idempotency key or deduplication behavior; reuse the same key and payload on a retry. Do not invent an Idempotency-Key header for an endpoint that does not support one. After an ambiguous timeout, inspect the operation’s status before submitting a new operation.

Do not retry a successful response because X-RateLimit-Outcome says would_reject or unavailable. Those outcomes allow the handler to run. Repeating a successful write can duplicate work.

Collect a useful debugging reportCopy Anchor Link

Observatory returns X-Request-Id on responses and includes the same request_id in API error bodies. Share that ID so the team can find the matching server logs. Errors generated before reaching Observatory may lack it.

On an Observatory error page, choose Copy error details. Client-side failures include the request ID, status, route, and timestamp. Server-rendered failures retain the request ID. The copied details exclude query strings and response bodies.

Capture the request method, route, UTC timestamp, HTTP status, Content-Type, response body, X-Request-ID, and any Retry-After or X-RateLimit-* headers. Include whether the caller was a user, player, or service, and whether several workers shared the identity.

Keep credentials, cookies, private request payloads, and personal data out of shared reports. Never include the Authorization header. Contact Softmax through the contact page with the request ID and a redacted reproduction.

Statistics workloadCopy Anchor Link

statistics_workload_exceeded means the requested statistics exceed a synchronous query’s work budget. An episode reference links a round to a requested or recorded game. Pending requests consume work even before a game records a score. A reward row records one policy version’s reward in one episode. Matchup work also depends on the number of policy pairs.

The error identifies each exceeded dimension, its maximum, and an observed lower bound. The server stops counting at a guard where possible. observed_at_least is not an exact workload estimate. These are per-query limits. Reducing the output limit or following an output cursor does not reduce input history.

Use the structured detail.recovery action to retain the requested history:

1. Authenticate as a user or player that can read every selected round, then send its body to the supplied POST path under the same Observatory API base URL. Anonymous and commissioner credentials cannot create reports; use an eligible credential or request a smaller synchronous scope. If you lack access, contact the league owner.
2. Retain the returned report ID and idempotency key. Poll the report’s status, respecting Retry-After.
3. Read results when status becomes completed, following every next_cursor.
4. For a matrix, read the report’s matrix resource with the desired max_versions.

Reuse the same idempotency key after a lost response. A changed request requires a new key. A report in failed or cancelled state has no complete result. Inspect its structured error before changing the request. An expired report returns statistics_report_expired; create a new report with a new key.

A statistics query that exceeds its execution budget returns request_timeout with the same recovery action when available. Retrying that synchronous query unchanged does not resolve the budget failure.

Report selection fixes the source IDs. Its as_of bounds source creation, while rewards are observed during processing. Completed results remain stable. The coverage fields distinguish unfinished sources, requests without recorded episodes, and recorded episodes without rewards. Request a fresh report to observe later results.

Use policy-version, policy, or player groupings for performance. Player source counts deduplicate across versions. League rewards default to raw policy rewards. Experience comparisons may request per_agent normalization. That normalization changes score averages, while wins and matchup comparisons continue to use raw rewards. Standard error uses sample variance and is undefined for a single observation.

Report creation and reads use ordinary API visibility. A report ID does not grant access to private sources. A visibility change can make an existing report inaccessible. Staff ownership lookup and internal owner counts require team elevation.

Pagination scopeCopy Anchor Link

cursor_scope_mismatch and statistics_cursor_mismatch identify a cursor from another resource, reader, or filter set. Restart pagination without the cursor after changing filters. Do not treat these errors as workload limits or shorten history.

For experience-request dashboards, a report’s experience-requests collection pages its fixed source set, including requests without scores. Its total is exact. Clients can collect those bounded pages for custom sorting. The existing experience summary’s cohort counts parents created within its selected interval. Its current gauges retain their meaning across all creation dates.