Node fencing and writer ownership
Establish that a failed source cannot write before promoting replacement execution.
Fencing prevents a previous host or guest from continuing to write after Infra assigns execution or storage ownership to a replacement. Loss of an agent heartbeat does not establish that the workload stopped.
A partitioned host
Open Fleet → Nodes → Node → Recovery to inspect ownership and the configured fencing integration. Use Fence host only after reviewing the affected guests and storage roles shown there.
A host can lose its management network while its hypervisor and storage path remain active. Starting the same droplet elsewhere at that point can create two application writers even if only one appears in the current console view.
Infra leaves recovery blocked while source ownership is unresolved. The droplet
operation reports fencing_required with the source node and affected attachment
IDs. The storage service can additionally reject a new writer while a prior
lease remains valid.
Supported evidence
Use an approved out-of-band power controller, a completed hypervisor stop with verified execution state, or the storage system's supported lease-revocation procedure. Evidence must identify the host and ownership generation affected. A failed ping, an expired console session, or an operator assumption is not accepted as a stopped-source observation.
Fencing integrations run outside the failed host's normal agent path. Their credentials should belong to a dedicated management scope and remain available when the production network is unavailable.
Recover a droplet after host loss
- Cordon the source inventory record and open its recovery operation.
- Inspect the last observed execution placement and volume writer generations.
- Select the configured fencing method and review every affected workload.
- Execute fencing and wait for its verified result.
- Allow Infra to reconcile storage ownership and reserve a destination.
- Start the replacement and validate the application before restoring traffic.
Fencing a combined compute/storage node can remove a live storage replica. Check surviving write requirements before choosing a host-wide power action. Recovering compute cannot restore data that existed only on a failed local disk.
Uncertain or failed fencing
If a controller times out after accepting a power command, the operation remains unresolved until power state is read back. Repeating the recovery with another operation ID does not remove this requirement. Inspect the original fencing attempt and the independent controller record.
Manual acknowledgement requires the separate Recovery Override permission and records the evidence, operator, reason, and affected ownership generation. It cannot make an unsafe source safe; the operator must complete the physical or storage action before acknowledging it.
Return the source
A fenced node reconnects cordoned. Reconcile its local hypervisor inventory against authoritative placements before allowing guest starts. Stale guest definitions must remain stopped. A former owner cannot reuse its old storage lease to resume writes.
Use operation recovery for ambiguous execution, replica repair for storage redundancy, and node maintenance for a responsive source.