Uptime and incidents

Where to watch availability and how failures behave.

Updated 2026-10-25 · MainAI Compute docs

The public status page is mainai.dk/status. It carries current incidents and maintenance. It is the source of truth during a degraded window; individual support threads may lag behind it.

Failure behaviorLink to this section

Condition	Customer sees	What to do
Inference route unreachable	`503 upstream_unavailable`	Backoff and retry.
Engine overloaded	`429` or `503`	Backoff; reduce concurrency.
Stream idle too long	Connection closes	Reconnect if partial output is acceptable.
Wallet at zero	Upstream budget error	Top up; traffic resumes after the webhook.
Organization suspended	`403 org_suspended`	Resolve with the owner or support.
Cache layer down	Routing degradation	Retry; portal stays available.
Usage sync lag	Dashboard shows an `asOf` time	Billing uses the ledger, not live usage.

Inference route unreachable

Customer sees: 503 upstream_unavailable
What to do: Backoff and retry.

Engine overloaded

Customer sees: 429 or 503
What to do: Backoff; reduce concurrency.

Stream idle too long

Customer sees: Connection closes
What to do: Reconnect if partial output is acceptable.

Wallet at zero

Customer sees: Upstream budget error
What to do: Top up; traffic resumes after the webhook.

Organization suspended

Customer sees: 403 org_suspended
What to do: Resolve with the owner or support.

Cache layer down

Customer sees: Routing degradation
What to do: Retry; portal stays available.

Usage sync lag

Customer sees: Dashboard shows an asOf time
What to do: Billing uses the ledger, not live usage.

The API returns OpenAI-shaped errors when it can. Ingress timeouts can still produce a plain proxy response; always check the HTTP status and x-request-id.

Incident statesLink to this section

State	Meaning
`investigating`	Impact confirmed; cause unknown.
`identified`	Cause found; mitigation underway.
`monitoring`	Mitigation applied; watching recovery.
`resolved`	Customer impact over.

investigating

Meaning: Impact confirmed; cause unknown.

identified

Meaning: Cause found; mitigation underway.

monitoring

Meaning: Mitigation applied; watching recovery.

resolved

Meaning: Customer impact over.

Updates are written for customers, not for a postmortem audience: what is affected, what changed, and the next update time. A resolved incident is not deleted.

MaintenanceLink to this section

Planned work that can interrupt inference is announced on the status page.
Schema and key rotations aim for no downtime, but are still announced when a restart is required.
Maintenance windows are sized to the rollback, not just the change.

What to send supportLink to this section

01Timestamp with timezone.
02Endpoint and model ID.
03HTTP status and error.code.
04x-request-id from the response.
05Whether the failure is steady or intermittent.

Never send a live API key. The last four characters and x-request-id are enough to locate a key record.

PostmortemsLink to this section

Incidents with customer impact get an internal review: timeline, contributing conditions, detection gap, and owned actions with dates. Enterprise customers can request the customer-safe summary.

Monitoring from your sideLink to this section

Cheap liveness check with a real key

bash

curl -fsS -o /dev/null -w "%{http_code} %{time_total}\n" \
  https://cloud.mainai.dk/v1/models \
  -H "Authorization: Bearer $MAINAI_API_KEY"

Run from your own region, not only from a single host.
Treat only 200 as healthy; 401 means your probe key expired.
Alert on three consecutive failures, not one.
Do not use a large completion as a probe; use a 1-token request.