When PureLB allocates a local address, only one node should announce it – otherwise, multiple nodes would respond to ARP requests for the same IP, causing packet loss. PureLB’s election system determines the winner.
Remote addresses skip the election entirely: all nodes announce them, and the upstream router handles distribution via ECMP.
How It Works#
Each LBNodeAgent creates a Kubernetes Lease object in the purelb-system namespace. The Lease contains:
- Holder identity: The node name.
- Subnets annotation (
purelb.io/subnets): The list of subnets reachable from this node’s interfaces. - Renewal timestamp: Updated periodically to prove the node is alive.
sequenceDiagram
participant N as LBNodeAgent
participant K as Kubernetes API
participant L as Lease (purelb-node-X)
N->>K: Create Lease with subnets
loop Every retryPeriod
N->>K: Renew Lease timestamp
K->>L: Update renewTime
end
Note over N,L: Other LBNodeAgents watch all Leases<br/>to build a list of healthy nodes + subnets
Winner Selection#
When a LoadBalancer address needs to be announced, each LBNodeAgent independently determines the winner using the same deterministic algorithm:
- Collect healthy nodes: Find all nodes with valid (non-expired) Leases.
- Filter by subnet: Keep only nodes whose Lease subnets contain the address.
- Deterministic hash: Compute SHA256 of each node name combined with the service key. Sort by hash value.
- First in list wins: The node with the lowest hash announces the address.
Because every LBNodeAgent uses the same algorithm and the same Lease data, they all reach the same conclusion without coordination.
Failover#
When a node becomes unhealthy:
- Its Lease expires (not renewed within
leaseDuration). - Other nodes detect the expiry via their Lease informer.
- The remaining nodes re-evaluate the election – a new winner is chosen.
- The new winner adds the address to its interface and sends Gratuitous ARP to update network switches.
Graceful Shutdown#
When a node shuts down cleanly:
- The LBNodeAgent marks itself as unhealthy (its
Winner()function returns empty for all addresses). - A
ForceSyncwithdraws all addresses from the node’s interfaces. - The Lease is deleted, so other nodes see the change immediately.
- Remaining nodes elect new winners.
This is faster than waiting for Lease expiry and avoids connectivity gaps during planned maintenance.
Configuration#
Election timing is controlled via environment variables (set through Helm’s leaseConfig):
| Variable | Default | Description |
|---|---|---|
PURELB_LEASE_DURATION | 10s | How long a Lease is valid without renewal |
PURELB_RENEW_DEADLINE | 7s | How long to retry renewals before giving up |
PURELB_RETRY_PERIOD | 2s | Interval between renewal attempts |
The code default for
PURELB_LEASE_DURATIONis5s, but the Helm chart overrides it to10s. If installing via manifest, the deployed value is10s.
Subnet-Aware Election#
The election is subnet-aware: only nodes that have the address’s subnet in their Lease annotations are candidates. This means:
- On a multi-subnet cluster, an address from subnet A will only be announced by a node on subnet A.
- The
interfacesfield in the LBNodeAgent configuration can add additional interfaces for subnet detection.
Monitoring#
Election health is exposed via Prometheus metrics. Key metrics: purelb_election_lease_healthy, purelb_election_member_count, and purelb_election_winner_changes_total.