In BGP mode without leader election, every kube-vip instance announces
the same VIP. ECMP distributes traffic evenly, but if a node's
kube-apiserver goes down, kube-vip (a static pod tied to kubelet) keeps
advertising the route — sending a fraction of all API traffic into a
black hole.
An optional health check loop now polls the local kube-apiserver and
withdraws the BGP route after a configurable number of consecutive
failures, removing the dead node from the ECMP set. The route is
re-announced once the API server recovers.
Signed-off-by: Guillermo Gaston <ggastonloren@nvidia.com>
* improve logging in election.go and set defaultBGPPort constant
Signed-off-by: Matthew Carpenter <mattcarp88@gmail.com>
* update log messages per code review
Signed-off-by: Matthew Carpenter <mattcarp88@gmail.com>
---------
Signed-off-by: Matthew Carpenter <mattcarp88@gmail.com>
* This fixes the rules for egress allowed-networks
Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>
* lint fix
Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>
* fixes a potential issue with out of bounds array access
Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>
---------
Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>
* Check BGP config node annotations before creating server
Signed-off-by: Martynas Deveikis <martynasdev@gmail.com>
* Add E2E test for BGP config via node annotations
Signed-off-by: Martynas Deveikis <martynasdev@gmail.com>
* Escape prefix meta chars when parsing BGP config annotations
Signed-off-by: Martynas Deveikis <martynasdev@gmail.com>
* Fail when no peer IP in BGP config annotations
Signed-off-by: Martynas Deveikis <martynasdev@gmail.com>
* Stop allowing trailing characters in BGP config annotations
Signed-off-by: Martynas Deveikis <martynasdev@gmail.com>
* Compile bgp config annotation regexes outside of loops
Signed-off-by: Martynas Deveikis <martynasdev@gmail.com>
* Allow setting multi hop via BGP config annotations
Signed-off-by: Martynas Deveikis <martynasdev@gmail.com>
---------
Signed-off-by: Martynas Deveikis <martynasdev@gmail.com>
* services: retry on 403/401 in ServicesWatcher using exponential backoff
On Kubernetes 1.34+, admin.conf uses O=kubeadm:cluster-admins instead
of O=system:masters. On joining control plane nodes, kube-vip starts
before etcd is promoted from learner to full member. RBAC data is not
yet available through the learner etcd, causing a transient 403
Forbidden or 401 Unauthorized on the first Watch call.
Replace the bare Watch call in ServicesWatcher's WatchFunc with
wait.ExponentialBackoffWithContext (from k8s.io/apimachinery/pkg/util/wait,
already in go.mod). Parameters: 2s initial, 2x factor, 0.1 jitter,
30s cap, 10 steps. Non-auth errors are returned immediately. Context
cancellation stops the retry loop cleanly.
Change Processor.rwClientSet from *kubernetes.Clientset to
kubernetes.Interface to enable fake client injection in tests.
watchEndpoint() uses p.clientSet (*kubernetes.Clientset) directly for
CreateRetryWatcher, avoiding an unnecessary type assertion.
Add unit tests covering: 403/401 retry succeeds, context cancellation
during retry. Update CHANGELOG.md.
Fixes: https://github.com/kube-vip/kube-vip/issues/1464
Signed-off-by: Soto Sugita <sotosugi@amazon.co.jp>
* fix(services): refactor watchWithAuthRetry as standalone function
Extract the retry logic into a standalone watchWithAuthRetry function
to avoid changing rwClientSet from *kubernetes.Clientset to
kubernetes.Interface. Tests now directly test watchWithAuthRetry via
closures without fake clientset injection or struct type changes.
Revert Processor.rwClientSet back to *kubernetes.Clientset and remove
the type assertion in watchEndpoint().
Signed-off-by: Soto Sugita <sotosugi@amazon.co.jp>
* fix(services): simplify watchWithAuthRetry tests to table-driven style
Signed-off-by: Soto Sugita <sotosugi@amazon.co.jp>
---------
Signed-off-by: Soto Sugita <sotosugi@amazon.co.jp>
* do not masquerade for local endpoints
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
* fix: do not add VIP to lo in wg mode
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
* fix: setup policy routing for wg interface
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
* refactor: use k8s API types for protocol
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
* conservatively apply packet mark
only apply the ct mark as packet mark if it matches our calculated
fwmark
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
* use new nftable setup
the nftable setup now uses only one set of chains per tunnel and
utilizes named maps and sets to match NAT the connections properly
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
* ensure proper cleanup
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
* watch kubernetes endpoints
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
* refactor wireguard nftables implementation
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
* use helper for if name determination
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
---------
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
PrometheusCollector() was registering bgpServer.BGPSessionInfoGauge while the worker was writing to Manager.bgpSessionInfoGauge — two separate objects representing the same metric. This caused kube_vip_manager_bgp_session_info to always appear empty in Prometheus.
Signed-off-by: Sebastian Gaiser <sebastiangaiser@users.noreply.github.com>
* Now will ensure egress rules are cleaned for nft-internal
Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>
* Fixed missing IPv6 rule
Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>
---------
Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>
Prevents log spam in control-plane-only deployments where
ServiceInstances is always empty. The UPNP refresh goroutine
still runs but no longer logs every 5 minutes when there's
nothing to refresh.
Fixes noise in SIEM systems and log aggregators.
Signed-off-by: Olli Hauer <ohauer@gmx.de>
PR #1252 (fix for #1243) removed the guard that prevented adding the
service IP to the interface in routing table mode. This made AddIP()
unconditional in StartLoadBalancerService(), causing the LoadBalancer IP
to be added to the interface on ALL nodes.
In no-election routing table mode (vip_leaderelection=false,
svc_election=false), this leads to traffic blackholing: every node has
the VIP bound to its interface, but only the node with local endpoints
can actually serve traffic when externalTrafficPolicy is Local.
The original code correctly guarded AddIP() behind `!c.EnableRoutingTable`
because in L3 modes (routing table / BGP), traffic is attracted via
routing protocols, not by having the IP present on the interface. The
endpoint watchers (endpoints_routing_table.go, endpoints_bgp.go) manage
routes/advertisements based on local endpoint presence.
Restore the guard and extend it to also cover BGP mode:
- Routing table mode: only routes are managed (by endpoint watcher or
leader), no IP on interface
- BGP mode: only BGP advertisements are managed, no IP on interface
- ARP/L2 mode: IP is added to interface (unchanged behavior)
Control plane (cp_enable) is unaffected as vipService() has its own
independent AddIP() call.
Existing stale IPs from the buggy version are cleaned up automatically
on restart, as StartLoadBalancerService() unconditionally calls
DeleteIP() before the now-guarded AddIP().
Signed-off-by: Christopher Dziomba <christopher.dziomba@telekom.de>
