Troubleshooting
Diagnose failed operations, unreachable nodes, storage issues, and connectivity failures.
Troubleshooting starts with the affected resource ID, its last observation, and the operation responsible for the most recent change. Preserve those IDs before restarting services or retrying requests.
Investigate a failed operation
- Open Operations → Queue for active work or Operations → History for a finished request. Select the operation and record its type, phase, error code, and request ID.
- Read its completed steps, child results, and retained resource IDs.
- Open each target from those results and compare its observed state with the requested configuration revision.
- In the operation journal, find the failed stage and its dependency error.
- Correct that condition. For uncertain execution, use the operation's Recovery tab as described in operation recovery. For a retryable failed request, follow the retry action in its result.
A failed parent can contain successful children. A network timeout can leave work running. Reuse the original idempotency key when retrying an uncertain API request, and avoid repeating destructive actions against a resource whose state is still reconciling.
A droplet cannot be scheduled
Open Placement → Explain. Check region access, quota, pool selection, required labels, taints, CPU profile, placement groups, volume access, and allocatable capacity. The first failed requirement on each candidate explains its exclusion.
Raising quota helps only when quota is the blocker. Changing a preferred zone does not override a required zone-spread group. Add eligible capacity or update the actual constraint through a reviewed change.
An agent or node is unreachable
Check host power, management-controller status, management routing, DNS, time, TLS trust, and local agent service logs. Compare other nodes in the same zone to identify a shared network failure.
From a terminal on the affected node, inspect the agent service:
sudo systemctl status infra-agentRead its recent journal entries:
sudo journalctl -u infra-agent --since '30 minutes ago' --no-pagerRun its local connectivity and capability diagnostics:
sudo infra-agent doctorCorrect the service, trust, or transport error reported by these checks. Confirm the node reports a fresh heartbeat in Fleet → Nodes before resuming management changes.
Do not delete agent state to force a new identity. If the host is lost, cordon its inventory record and establish fencing before recovering guests elsewhere. The original hypervisor might still execute workloads during a network partition.
A migration does not converge
Inspect dirty-memory rate, transfer rate, mesh health, destination capacity, and storage latency. A write-heavy guest can repeatedly change pages while transfer continues. Consider reducing workload pressure or scheduling a cold move.
To stop the migration, open its operation and check cancelable. If it is true,
select Cancel and wait for the terminal result. Once commit begins, cancellation
can be unavailable. Follow the ownership result in the operation's Recovery
tab before starting either copy; see migration recovery.
Private network traffic fails
Check both interfaces, assigned addresses, guest routes, effective firewall rules, application listeners, and node mesh peers. Test same-node and cross-node traffic separately. A cross-node-only failure points toward the transport path.
For small requests that succeed while larger transfers stall, inspect MTU and encapsulation overhead. For IPv6-only failures, check IPv6 routes, address-family rules, and required ICMPv6 behavior.
A volume is attached but unavailable
Compare the attachment ID with guest device discovery. Check volume lifecycle, replica health, storage ownership, and the guest filesystem. A block device can exist without a mounted filesystem. Do not format a volume until its identity and intended contents are verified.
If another writer is recorded, complete node fencing before attempting replacement writes. If replicas do not meet the write requirement, open Volume → Replicas and follow replica repair. When no authoritative set can be established, use the reviewed forced-recovery procedure or restore a verified backup. Forced source selection can discard unreplicated changes.
A backup fails
Inspect source consistency hooks, destination credentials, network reachability, quota, free staging space, and residency policy. Check the age of the latest completed recovery point rather than the last attempted run.
A partial upload is not a complete application backup. Correct the failed stage and run a verification or restore exercise after the next successful copy.
The API returns a conflict
Use the error code to choose the recovery action:
| Code | Next action |
|---|---|
revision_conflict | Read the resource again, compare its current configuration with the intended change, and send the revised request with the returned ETag |
operation_conflict | Open the blocking operation ID under Operations → Queue; wait for it or follow cancellation |
idempotency_conflict | Compare the request with the original method, path, and body; reuse the key only for the identical request |
Retrying the same conflicting request without addressing its cause will not resolve any of these errors.
Collect a diagnostic bundle
Include resource IDs, operation IDs, request IDs, release versions, affected time range, health conditions, and relevant logs. Exclude private keys, tokens, secret files, and unnecessary guest data. Share the bundle through an approved support channel permitted by data residency policy.