Open Status
Run monitor workers

Run monitor workers

Deploy workers and configure check execution and result delivery.

Monitor workers are separately deployed processes that schedule and execute checks and deliver results to the Status application. A worker keeps running from its last known configuration during application outages and queues results locally until delivery succeeds.

The setup wizard generates a shared worker token, which an owner can replace from Settings → Instance → Monitoring workers. Set the same token (at least 32 characters) on every trusted worker:

HTTP
Authorization: Bearer <MONITOR_WORKER_TOKEN>

Without a valid token configured, the worker endpoints return 503; an incorrect token returns 401.

The shared credential serves every organization on the instance. For checks inside a private network, register a per-organization private agent instead—see Monitoring agents.

Configuration sync

Workers poll the control plane for their configuration:

HTTP
GET /api/worker/config
Authorization: Bearer <token>
If-Config-Revision: <last revision>
X-Worker-ID: worker-ord-1
X-Worker-Region: us-central
X-Worker-Version: 1.0.0

A changed configuration returns 200 with the full active monitor configuration and the new revision in X-Config-Revision. An unchanged configuration returns 304.

The three worker identity headers are required. Their values register or refresh the worker on the Agents page, where operators can inspect the reported version, region, revision, and last configuration/result activity. An agent is shown as stale after two minutes without either kind of activity.

A worker should persist each 200 response before activating it, and retain the previous configuration on timeouts, 5xx responses, or invalid payloads. A paused or deleted monitor disappears from the next snapshot. Monitoring continues during an application outage; configuration changes wait for recovery.

Each active or future maintenance occurrence that applies to a monitor appears in the monitor's suppressionWindows. Workers must not start a check while the current time is within a window (startsAt inclusive, endsAt exclusive). Because end times are part of the durable snapshot, a cached worker resumes checks after maintenance even during an application outage. Late or in-flight results received during maintenance are kept for history but cannot change current state or the automatic incident lifecycle. Daily and weekly monitor schedules are expanded in their configured timezone for the next 90 days; their local wall-clock times remain stable across daylight saving changes.

Result delivery

Workers submit results in batches of up to 100:

HTTP
POST /api/worker/results
Authorization: Bearer <token>
Content-Type: application/json
JSON
{
  "workerId": "worker-ord-1",
  "region": "us-central",
  "version": "1.0.0",
  "results": [
    {
      "id": "afbdd83d-dbf8-4bb2-8681-1e636a4e33f0",
      "monitorId": "3d53ed3f-8027-4f46-9377-a91ca63e29da",
      "status": "up",
      "checkedAt": "2025-12-05T18:00:00.000Z",
      "responseTimeMs": 42,
      "statusCode": 204,
      "message": null
    }
  ]
}

The endpoint returns 202 with accepted and ignored counts. Each result id is an idempotency key: generate it once and preserve it across retries. Duplicate IDs and results for monitors deleted while the worker was offline are ignored. An older buffered result is retained in history but cannot replace a monitor's newer current state.

Workers should durably enqueue results before attempting delivery, retry network errors and 5xx responses with exponential backoff and jitter, and remove only accepted or intentionally ignored results after a 202 response.

Regional routing and aggregation

Network monitors may list up to 20 worker region IDs. A worker receives a monitor when the list is empty or contains that worker's region. Results from a region outside an explicit list remain in history but cannot change current state or incidents.

For private agents, the scheduling mode also applies. Active agents receive the normal filtered configuration. Draining agents receive an empty configuration while result ingestion stays available so in-flight checks can finish. Disabled agents fail authentication immediately. The region a worker reports controls execution; a declared region that differs from the report is surfaced on the Agents page as placement drift.

The application combines the newest fresh observation from each region before applying confirmation and recovery periods:

  • all reports up only when every participating region is up.
  • any reports up when at least one participating region is up.
  • majority reports the strict majority and waits when the vote is tied.

An explicitly selected region set waits until every selected region has a fresh observation. Observations are fresh for the greater of two minutes or three monitor intervals. With no explicit set, all fresh reporting regions participate. Heartbeat monitors use their deadline state directly rather than regional voting.

Check semantics

  • http: supports GET, HEAD, POST, PUT, PATCH, DELETE, and OPTIONS, repeated query parameters, custom headers, JSON/text/form request bodies, bounded redirects, inclusive status ranges, and contains, not-contains, or regular-expression body assertions. Responses are capped at 1 MiB.
  • HTTPS checks can require a trusted certificate, fail within a configured certificate-expiration window, require an exact, substring, or regular-expression match against the peer issuer distinguished name, and check domain registration expiration. An empty issuer rule and zero-day thresholds disable their corresponding checks.
  • tcp: establish a TCP connection to host:port.
  • ping: perform an ICMP reachability check for host.
  • dns: resolve the configured record type for host and optionally require an exact, substring, or regular-expression answer match.
  • tls: complete a TLS handshake to host:port, optionally override SNI, optionally require a trusted certificate, and fail when the certificate expires within the warning window or its issuer fails the configured rule.
  • udp: send a bounded UTF-8 datagram and require the first response to satisfy an exact, substring, or regular-expression assertion.
  • smtp: require a 220 greeting and successful EHLO response over plain SMTP, STARTTLS, or implicit TLS. Encrypted modes can require a trusted hostname-matching certificate.
  • ssh: require a valid SSH protocol identification line and optionally require configured banner text such as an OpenSSH version.
  • database: verify a PostgreSQL SSL-negotiation response or a MySQL/MariaDB protocol greeting. This is a credential-free wire-protocol availability check, not an application query.
  • websocket: perform an HTTP upgrade over WS or WSS, send configured handshake headers, and verify the status, upgrade tokens, and Sec-WebSocket-Accept response.
  • heartbeat: compare the last authenticated push timestamp with the expected cadence plus grace period. New heartbeat monitors stay pending until their first deadline; after a push, workers can detect a missed deadline. The application reconciles stale worker snapshots against the current timestamp before applying state or incident transitions.

Every check must honor timeoutMs. intervalSeconds is the target cadence; workers should add small stable jitter to avoid synchronized bursts. ipVersion can leave address selection automatic or force IPv4/IPv6 for network checks.

The application applies the confirmation period before changing a healthy monitor to down and opening an automatic incident. It applies the recovery period before returning a down monitor to up and resolving that automatic incident. A healthy result interrupts a pending failure transition, and a failed result interrupts a pending recovery transition. Manual incidents are never automatically resolved.

Configured monitor dependencies are evaluated after a result is stored. If an upstream dependency is down—or is itself suppressed by an unhealthy dependency—the downstream failure remains in history but cannot change current state or open an automatic incident. Dependency graphs are ownership-scoped and cycle-free. A later downstream result is evaluated again, so normal failure handling resumes after its dependencies recover.