Compare commits

..

154 Commits

Author SHA1 Message Date
Dan Finneran
adfc1d20fc This adds the capability to filter on ports individually per rule
Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>
2026-03-29 15:16:45 +00:00
Soto Sugita
f1065a4a8c services: retry on 403 Forbidden in ServicesWatcher WatchFunc kubeadm 1.34.x onwards (#1465)
* services: retry on 403/401 in ServicesWatcher using exponential backoff

On Kubernetes 1.34+, admin.conf uses O=kubeadm:cluster-admins instead
of O=system:masters. On joining control plane nodes, kube-vip starts
before etcd is promoted from learner to full member. RBAC data is not
yet available through the learner etcd, causing a transient 403
Forbidden or 401 Unauthorized on the first Watch call.

Replace the bare Watch call in ServicesWatcher's WatchFunc with
wait.ExponentialBackoffWithContext (from k8s.io/apimachinery/pkg/util/wait,
already in go.mod). Parameters: 2s initial, 2x factor, 0.1 jitter,
30s cap, 10 steps. Non-auth errors are returned immediately. Context
cancellation stops the retry loop cleanly.

Change Processor.rwClientSet from *kubernetes.Clientset to
kubernetes.Interface to enable fake client injection in tests.
watchEndpoint() uses p.clientSet (*kubernetes.Clientset) directly for
CreateRetryWatcher, avoiding an unnecessary type assertion.

Add unit tests covering: 403/401 retry succeeds, context cancellation
during retry. Update CHANGELOG.md.

Fixes: https://github.com/kube-vip/kube-vip/issues/1464
Signed-off-by: Soto Sugita <sotosugi@amazon.co.jp>

* fix(services): refactor watchWithAuthRetry as standalone function

Extract the retry logic into a standalone watchWithAuthRetry function
to avoid changing rwClientSet from *kubernetes.Clientset to
kubernetes.Interface. Tests now directly test watchWithAuthRetry via
closures without fake clientset injection or struct type changes.

Revert Processor.rwClientSet back to *kubernetes.Clientset and remove
the type assertion in watchEndpoint().

Signed-off-by: Soto Sugita <sotosugi@amazon.co.jp>

* fix(services): simplify watchWithAuthRetry tests to table-driven style

Signed-off-by: Soto Sugita <sotosugi@amazon.co.jp>

---------

Signed-off-by: Soto Sugita <sotosugi@amazon.co.jp>
2026-03-26 13:21:15 +01:00
Daniel
50993b63f1 Enhance WireGuard nftables and endpoint handling (#1469)
* do not masquerade for local endpoints

Signed-off-by: Daniel Nägele <daniel@naegele.dev>

* fix: do not add VIP to lo in wg mode

Signed-off-by: Daniel Nägele <daniel@naegele.dev>

* fix: setup policy routing for wg interface

Signed-off-by: Daniel Nägele <daniel@naegele.dev>

* refactor: use k8s API types for protocol

Signed-off-by: Daniel Nägele <daniel@naegele.dev>

* conservatively apply packet mark

only apply the ct mark as packet mark if it matches our calculated
fwmark

Signed-off-by: Daniel Nägele <daniel@naegele.dev>

* use new nftable setup

the nftable setup now uses only one set of chains per tunnel and
utilizes named maps and sets to match NAT the connections properly

Signed-off-by: Daniel Nägele <daniel@naegele.dev>

* ensure proper cleanup

Signed-off-by: Daniel Nägele <daniel@naegele.dev>

* watch kubernetes endpoints

Signed-off-by: Daniel Nägele <daniel@naegele.dev>

* refactor wireguard nftables implementation

Signed-off-by: Daniel Nägele <daniel@naegele.dev>

* use helper for if name determination

Signed-off-by: Daniel Nägele <daniel@naegele.dev>

---------

Signed-off-by: Daniel Nägele <daniel@naegele.dev>
2026-03-24 14:25:05 +01:00
Patryk Strusiewicz-Surmacki
4108a8b32a Updated services error handling (#1481)
* Updated services error handling

Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>

* Fix service instance not found error

Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>

---------

Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
Co-authored-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2026-03-23 20:26:46 +01:00
dependabot[bot]
e6658ff32f Bump go.etcd.io/etcd/api/v3 from 3.6.8 to 3.6.9 (#1482)
Bumps [go.etcd.io/etcd/api/v3](https://github.com/etcd-io/etcd) from 3.6.8 to 3.6.9.
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.6.8...v3.6.9)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/api/v3
  dependency-version: 3.6.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-23 13:33:54 +01:00
dependabot[bot]
4bb5103f47 Bump go.etcd.io/etcd/client/v3 from 3.6.8 to 3.6.9 (#1484)
Bumps [go.etcd.io/etcd/client/v3](https://github.com/etcd-io/etcd) from 3.6.8 to 3.6.9.
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.6.8...v3.6.9)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/client/v3
  dependency-version: 3.6.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-23 13:33:41 +01:00
dependabot[bot]
e3961d7404 Bump k8s.io/api from 0.35.2 to 0.35.3 (#1486)
Bumps [k8s.io/api](https://github.com/kubernetes/api) from 0.35.2 to 0.35.3.
- [Commits](https://github.com/kubernetes/api/compare/v0.35.2...v0.35.3)

---
updated-dependencies:
- dependency-name: k8s.io/api
  dependency-version: 0.35.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-23 12:35:34 +01:00
dependabot[bot]
b10375824f Bump anchore/sbom-action from 0.23.1 to 0.24.0 (#1487)
Bumps [anchore/sbom-action](https://github.com/anchore/sbom-action) from 0.23.1 to 0.24.0.
- [Release notes](https://github.com/anchore/sbom-action/releases)
- [Changelog](https://github.com/anchore/sbom-action/blob/main/RELEASE.md)
- [Commits](https://github.com/anchore/sbom-action/compare/v0.23.1...v0.24.0)

---
updated-dependencies:
- dependency-name: anchore/sbom-action
  dependency-version: 0.24.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-23 12:16:23 +01:00
dependabot[bot]
1981efc95b Bump go.etcd.io/etcd/client/pkg/v3 from 3.6.8 to 3.6.9 (#1483)
Bumps [go.etcd.io/etcd/client/pkg/v3](https://github.com/etcd-io/etcd) from 3.6.8 to 3.6.9.
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.6.8...v3.6.9)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/client/pkg/v3
  dependency-version: 3.6.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-23 12:11:04 +01:00
dependabot[bot]
7cd2b00cee Bump github.com/google/go-containerregistry from 0.21.2 to 0.21.3 (#1485)
Bumps [github.com/google/go-containerregistry](https://github.com/google/go-containerregistry) from 0.21.2 to 0.21.3.
- [Release notes](https://github.com/google/go-containerregistry/releases)
- [Commits](https://github.com/google/go-containerregistry/compare/v0.21.2...v0.21.3)

---
updated-dependencies:
- dependency-name: github.com/google/go-containerregistry
  dependency-version: 0.21.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-23 12:10:50 +01:00
Matej Kern
ba334acf7a fix(egress): prevent unnecessary SNAT updates and fix missed updates (#1433)
* fix(egress): prevent unnecessary SNAT updates and fix missed updates #1428

Signed-off-by: RnkeZ <matej.kern@gmail.com>

* Remove ActionUpdate from services.go

Signed-off-by: RnkeZ <matej.kern@gmail.com>

* refactor(egress): remove redundant service DeepCopy

Signed-off-by: RnkeZ <matej.kern@gmail.com>

* refactor(endpoints): pass clientSet via function parameter instead of
  storing in struct

Signed-off-by: RnkeZ <matej.kern@gmail.com>

* refactor(endpoints): pass egressUpdateFunc via parameter instead of storing in struct

Signed-off-by: RnkeZ <matej.kern@gmail.com>

* refactor(endpoints): update AddOrModify parameters

Signed-off-by: RnkeZ <matej.kern@gmail.com>

* fix formatting issues

Signed-off-by: RnkeZ <matej.kern@gmail.com>

---------

Signed-off-by: RnkeZ <matej.kern@gmail.com>
2026-03-20 15:01:34 +01:00
Daniel Finneran
3a387b87c5 Update version to v1.1.1
Bumping for a new release
2026-03-20 11:05:12 +00:00
Patryk Strusiewicz-Surmacki
ca47abfc3a Fixed services error handling (#1478)
* Fixed services error handling

Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>

* Removed redundant code

Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>

---------

Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
Co-authored-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2026-03-20 10:44:32 +01:00
Patryk Strusiewicz-Surmacki
dcd8fe0392 Fixed leader election restart issue (#1477)
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
Co-authored-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2026-03-19 11:29:26 +01:00
dependabot[bot]
4e0de5277d Bump google.golang.org/grpc from 1.79.2 to 1.79.3 (#1476)
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.79.2 to 1.79.3.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.79.2...v1.79.3)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.79.3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-19 11:25:46 +01:00
dependabot[bot]
897a1fe6d8 Bump github.com/florianl/go-conntrack from 0.6.0 to 0.7.0 (#1470)
Bumps [github.com/florianl/go-conntrack](https://github.com/florianl/go-conntrack) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/florianl/go-conntrack/releases)
- [Commits](https://github.com/florianl/go-conntrack/compare/v0.6.0...v0.7.0)

---
updated-dependencies:
- dependency-name: github.com/florianl/go-conntrack
  dependency-version: 0.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-17 14:57:29 +01:00
dependabot[bot]
25f215e38d Bump anchore/sbom-action from 0.23.0 to 0.23.1 (#1471)
Bumps [anchore/sbom-action](https://github.com/anchore/sbom-action) from 0.23.0 to 0.23.1.
- [Release notes](https://github.com/anchore/sbom-action/releases)
- [Changelog](https://github.com/anchore/sbom-action/blob/main/RELEASE.md)
- [Commits](https://github.com/anchore/sbom-action/compare/v0.23.0...v0.23.1)

---
updated-dependencies:
- dependency-name: anchore/sbom-action
  dependency-version: 0.23.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-17 14:57:20 +01:00
Patryk Strusiewicz-Surmacki
52d7c23db7 Fixed BGP route advertisement in control-plane only mode (#1468)
* Fixed BGP shutdown in control-plane only mode

Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>

* Added E2E tests for control-plane only mode

Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>

---------

Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
Co-authored-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2026-03-13 16:18:07 +01:00
Marcel Fest
64880b62ab Update to v1.1.0
Signed-off-by: Marcel Fest <marcel.fest@telekom.de>
2026-03-09 14:12:37 +01:00
Sebastian Gaiser
494c48bf21 fix(metrics): register correct gauge for BGP session info (#1460)
PrometheusCollector() was registering bgpServer.BGPSessionInfoGauge while the worker was writing to Manager.bgpSessionInfoGauge — two separate objects representing the same metric. This caused kube_vip_manager_bgp_session_info to always appear empty in Prometheus.

Signed-off-by: Sebastian Gaiser <sebastiangaiser@users.noreply.github.com>
2026-03-09 12:44:12 +01:00
dependabot[bot]
ff93e58cfd Bump golang.org/x/sys from 0.41.0 to 0.42.0 (#1452)
Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.41.0 to 0.42.0.
- [Commits](https://github.com/golang/sys/compare/v0.41.0...v0.42.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-version: 0.42.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-09 11:24:00 +01:00
dependabot[bot]
857891f695 Bump docker/build-push-action from 6 to 7 (#1454)
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 6 to 7.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v6...v7)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-09 08:06:08 +01:00
dependabot[bot]
377153bc52 Bump docker/setup-qemu-action from 3 to 4 (#1456)
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 3 to 4.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](https://github.com/docker/setup-qemu-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-09 08:05:48 +01:00
dependabot[bot]
a46f0e76c5 Bump docker/setup-buildx-action from 3 to 4 (#1457)
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3 to 4.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-09 08:05:37 +01:00
dependabot[bot]
51725463d6 Bump docker/metadata-action from 5.10.0 to 6.0.0 (#1455)
Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 5.10.0 to 6.0.0.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Commits](https://github.com/docker/metadata-action/compare/v5.10.0...v6.0.0)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-version: 6.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-09 08:05:22 +01:00
dependabot[bot]
001c467e36 Bump golang.org/x/sync from 0.19.0 to 0.20.0 (#1451)
Bumps [golang.org/x/sync](https://github.com/golang/sync) from 0.19.0 to 0.20.0.
- [Commits](https://github.com/golang/sync/compare/v0.19.0...v0.20.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sync
  dependency-version: 0.20.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-09 08:05:01 +01:00
dependabot[bot]
ff33ebe6ab Bump github.com/google/go-containerregistry from 0.21.1 to 0.21.2 (#1450)
Bumps [github.com/google/go-containerregistry](https://github.com/google/go-containerregistry) from 0.21.1 to 0.21.2.
- [Release notes](https://github.com/google/go-containerregistry/releases)
- [Commits](https://github.com/google/go-containerregistry/compare/v0.21.1...v0.21.2)

---
updated-dependencies:
- dependency-name: github.com/google/go-containerregistry
  dependency-version: 0.21.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-09 08:04:43 +01:00
dependabot[bot]
97fb2bd711 Bump k8s.io/klog/v2 from 2.130.1 to 2.140.0 (#1453)
Bumps [k8s.io/klog/v2](https://github.com/kubernetes/klog) from 2.130.1 to 2.140.0.
- [Release notes](https://github.com/kubernetes/klog/releases)
- [Changelog](https://github.com/kubernetes/klog/blob/main/RELEASE.md)
- [Commits](https://github.com/kubernetes/klog/compare/v2.130.1...2.140.0)

---
updated-dependencies:
- dependency-name: k8s.io/klog/v2
  dependency-version: 2.140.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-09 08:04:27 +01:00
dependabot[bot]
e36ed5a189 Bump google.golang.org/grpc from 1.79.1 to 1.79.2 (#1449)
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.79.1 to 1.79.2.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.79.1...v1.79.2)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.79.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-09 08:04:10 +01:00
dependabot[bot]
342709a63e Bump golang from 1.26.0-alpine3.23 to 1.26.1-alpine3.23 (#1458)
Bumps golang from 1.26.0-alpine3.23 to 1.26.1-alpine3.23.

---
updated-dependencies:
- dependency-name: golang
  dependency-version: 1.26.1-alpine3.23
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-09 07:20:14 +01:00
dependabot[bot]
e61453ace2 Bump docker/login-action from 3 to 4 (#1459)
Bumps [docker/login-action](https://github.com/docker/login-action) from 3 to 4.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-09 07:19:52 +01:00
Daniel Finneran
d62dd9313a Now will ensure egress rules are cleaned for nft-internal (#1447)
* Now will ensure egress rules are cleaned for nft-internal

Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>

* Fixed missing IPv6 rule

Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>

---------

Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>
2026-03-06 17:44:20 +01:00
Patryk Strusiewicz-Surmacki
eb18c59519 Cleanup of channels (#1444)
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
Co-authored-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2026-03-06 16:06:02 +01:00
Olli Hauer
5f11053080 fix: skip UPNP refresh logging when no service instances (#1446)
Prevents log spam in control-plane-only deployments where
ServiceInstances is always empty. The UPNP refresh goroutine
still runs but no longer logs every 5 minutes when there's
nothing to refresh.

Fixes noise in SIEM systems and log aggregators.

Signed-off-by: Olli Hauer <ohauer@gmx.de>
2026-03-05 23:56:42 +01:00
k-jun
7caf5f656a fix: align CLI lease flag defaults with Kubernetes client-go defaults (15/10/2) (#1429)
Signed-off-by: k-jun <keijun091221@gmail.com>
2026-03-04 16:42:26 +01:00
Patryk Strusiewicz-Surmacki
88efcc7bc2 Added waitgroups (#1434)
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
Co-authored-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2026-03-03 18:06:15 +01:00
Chris
abb4741d1f fix: don't add VIP to interface in routing table and BGP service modes (#1442)
PR #1252 (fix for #1243) removed the guard that prevented adding the
service IP to the interface in routing table mode. This made AddIP()
unconditional in StartLoadBalancerService(), causing the LoadBalancer IP
to be added to the interface on ALL nodes.

In no-election routing table mode (vip_leaderelection=false,
svc_election=false), this leads to traffic blackholing: every node has
the VIP bound to its interface, but only the node with local endpoints
can actually serve traffic when externalTrafficPolicy is Local.

The original code correctly guarded AddIP() behind `!c.EnableRoutingTable`
because in L3 modes (routing table / BGP), traffic is attracted via
routing protocols, not by having the IP present on the interface. The
endpoint watchers (endpoints_routing_table.go, endpoints_bgp.go) manage
routes/advertisements based on local endpoint presence.

Restore the guard and extend it to also cover BGP mode:
- Routing table mode: only routes are managed (by endpoint watcher or
  leader), no IP on interface
- BGP mode: only BGP advertisements are managed, no IP on interface
- ARP/L2 mode: IP is added to interface (unchanged behavior)

Control plane (cp_enable) is unaffected as vipService() has its own
independent AddIP() call.

Existing stale IPs from the buggy version are cleaned up automatically
on restart, as StartLoadBalancerService() unconditionally calls
DeleteIP() before the now-guarded AddIP().

Signed-off-by: Christopher Dziomba <christopher.dziomba@telekom.de>
2026-03-03 17:33:09 +01:00
Daniel
81cc332fd8 WireGuard for services (#1414)
* add tunnel manager

Signed-off-by: Daniel Nägele <daniel@naegele.dev>

* add wireguard endpoints and services

Signed-off-by: Daniel Nägele <daniel@naegele.dev>

* add udp support

Signed-off-by: Daniel Nägele <daniel@naegele.dev>

* fix endpoint watching

Signed-off-by: Daniel Nägele <daniel@naegele.dev>

* refactor code

Signed-off-by: Daniel Nägele <daniel@naegele.dev>

* fix after rebase

Signed-off-by: Daniel Nägele <daniel@naegele.dev>

---------

Signed-off-by: Daniel Nägele <daniel@naegele.dev>
Co-authored-by: Marcel Fest <marcel.fest@live.de>
2026-03-02 21:56:23 +01:00
dependabot[bot]
19b63cdbf6 Bump anchore/sbom-action from 0.22.2 to 0.23.0 (#1439)
Bumps [anchore/sbom-action](https://github.com/anchore/sbom-action) from 0.22.2 to 0.23.0.
- [Release notes](https://github.com/anchore/sbom-action/releases)
- [Changelog](https://github.com/anchore/sbom-action/blob/main/RELEASE.md)
- [Commits](https://github.com/anchore/sbom-action/compare/v0.22.2...v0.23.0)

---
updated-dependencies:
- dependency-name: anchore/sbom-action
  dependency-version: 0.23.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 21:10:22 +01:00
dependabot[bot]
61f5dc9cf9 Bump k8s.io/api from 0.35.1 to 0.35.2 (#1437)
Bumps [k8s.io/api](https://github.com/kubernetes/api) from 0.35.1 to 0.35.2.
- [Commits](https://github.com/kubernetes/api/compare/v0.35.1...v0.35.2)

---
updated-dependencies:
- dependency-name: k8s.io/api
  dependency-version: 0.35.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 21:09:51 +01:00
dependabot[bot]
bed8c4ccec Bump k8s.io/client-go from 0.35.1 to 0.35.2 (#1438)
Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.35.1 to 0.35.2.
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kubernetes/client-go/compare/v0.35.1...v0.35.2)

---
updated-dependencies:
- dependency-name: k8s.io/client-go
  dependency-version: 0.35.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 21:09:33 +01:00
dependabot[bot]
c74b4e4de6 Bump actions/upload-artifact from 6 to 7 (#1440)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 6 to 7.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 18:59:47 +01:00
dependabot[bot]
ae36ce1f04 Bump github.com/google/go-containerregistry from 0.21.0 to 0.21.1 (#1436)
Bumps [github.com/google/go-containerregistry](https://github.com/google/go-containerregistry) from 0.21.0 to 0.21.1.
- [Release notes](https://github.com/google/go-containerregistry/releases)
- [Commits](https://github.com/google/go-containerregistry/compare/v0.21.0...v0.21.1)

---
updated-dependencies:
- dependency-name: github.com/google/go-containerregistry
  dependency-version: 0.21.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 18:58:38 +01:00
dependabot[bot]
490163171a Bump k8s.io/apimachinery from 0.35.1 to 0.35.2 (#1435)
Bumps [k8s.io/apimachinery](https://github.com/kubernetes/apimachinery) from 0.35.1 to 0.35.2.
- [Commits](https://github.com/kubernetes/apimachinery/compare/v0.35.1...v0.35.2)

---
updated-dependencies:
- dependency-name: k8s.io/apimachinery
  dependency-version: 0.35.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 18:58:10 +01:00
Patryk Strusiewicz-Surmacki
afc35f335b Added context inheritance for services (#1432)
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
Co-authored-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2026-02-25 10:51:52 +01:00
dependabot[bot]
239bd3b047 Bump github.com/florianl/go-conntrack from 0.5.0 to 0.6.0 (#1431)
Bumps [github.com/florianl/go-conntrack](https://github.com/florianl/go-conntrack) from 0.5.0 to 0.6.0.
- [Release notes](https://github.com/florianl/go-conntrack/releases)
- [Commits](https://github.com/florianl/go-conntrack/compare/v0.5.0...v0.6.0)

---
updated-dependencies:
- dependency-name: github.com/florianl/go-conntrack
  dependency-version: 0.6.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-23 08:48:15 +01:00
dependabot[bot]
df9c190248 Bump github.com/google/go-containerregistry from 0.20.7 to 0.21.0 (#1430)
Bumps [github.com/google/go-containerregistry](https://github.com/google/go-containerregistry) from 0.20.7 to 0.21.0.
- [Release notes](https://github.com/google/go-containerregistry/releases)
- [Commits](https://github.com/google/go-containerregistry/compare/v0.20.7...v0.21.0)

---
updated-dependencies:
- dependency-name: github.com/google/go-containerregistry
  dependency-version: 0.21.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-23 08:47:56 +01:00
Patryk Strusiewicz-Surmacki
208c55fbfa Improved global lease (#1426)
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
Co-authored-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2026-02-20 23:08:50 +01:00
Daniel Finneran
0f32c712c3 Merge pull request #1427 from kube-vip/skaffold
Adds skaffold for development
2026-02-19 15:15:59 +00:00
Dan Finneran
09947db639 Fix paths to seperate manifest
Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>
2026-02-18 17:29:16 +00:00
Dan Finneran
12085ab747 Adds skaffold for development
Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>
2026-02-18 17:10:50 +00:00
dependabot[bot]
089bc2e217 Bump google.golang.org/protobuf from 1.36.10 to 1.36.11 (#1420)
Bumps google.golang.org/protobuf from 1.36.10 to 1.36.11.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-version: 1.36.11
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-16 11:33:16 +01:00
dependabot[bot]
edca162f8f Bump k8s.io/client-go from 0.35.0 to 0.35.1 (#1423)
Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.35.0 to 0.35.1.
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kubernetes/client-go/compare/v0.35.0...v0.35.1)

---
updated-dependencies:
- dependency-name: k8s.io/client-go
  dependency-version: 0.35.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-16 11:33:00 +01:00
dependabot[bot]
1240cff958 Bump golang from 1.25.7-alpine3.23 to 1.26.0-alpine3.23 (#1424)
Bumps golang from 1.25.7-alpine3.23 to 1.26.0-alpine3.23.

---
updated-dependencies:
- dependency-name: golang
  dependency-version: 1.26.0-alpine3.23
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-16 09:20:10 +01:00
dependabot[bot]
e2ac746260 Bump google.golang.org/grpc from 1.78.0 to 1.79.1 (#1422)
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.78.0 to 1.79.1.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.78.0...v1.79.1)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.79.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-16 09:19:42 +01:00
dependabot[bot]
463e4408f6 Bump go.etcd.io/etcd/client/v3 from 3.6.7 to 3.6.8 (#1421)
Bumps [go.etcd.io/etcd/client/v3](https://github.com/etcd-io/etcd) from 3.6.7 to 3.6.8.
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.6.7...v3.6.8)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/client/v3
  dependency-version: 3.6.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-16 09:19:27 +01:00
dependabot[bot]
3b6c40cc4b Bump k8s.io/api from 0.35.0 to 0.35.1 (#1419)
Bumps [k8s.io/api](https://github.com/kubernetes/api) from 0.35.0 to 0.35.1.
- [Commits](https://github.com/kubernetes/api/compare/v0.35.0...v0.35.1)

---
updated-dependencies:
- dependency-name: k8s.io/api
  dependency-version: 0.35.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-16 09:18:55 +01:00
Patryk Strusiewicz-Surmacki
e679ba206d Refactored manager's code for mode selection (#1395)
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
Co-authored-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2026-02-15 21:58:24 +01:00
Patryk Strusiewicz-Surmacki
9cea0e8b8d Fixed actions failure when whoami image cannot be loaded
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2026-02-12 22:18:46 +01:00
Patryk Strusiewicz-Surmacki
6fc134258d Reverted missing change from #623
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2026-02-12 17:59:23 +01:00
dependabot[bot]
9be058291f Bump go.etcd.io/etcd/client/v3 from 3.6.6 to 3.6.7
Bumps [go.etcd.io/etcd/client/v3](https://github.com/etcd-io/etcd) from 3.6.6 to 3.6.7.
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.6.6...v3.6.7)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/client/v3
  dependency-version: 3.6.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-11 20:14:23 +01:00
dependabot[bot]
0addea441e Bump go.etcd.io/etcd/client/pkg/v3 from 3.6.6 to 3.6.7
Bumps [go.etcd.io/etcd/client/pkg/v3](https://github.com/etcd-io/etcd) from 3.6.6 to 3.6.7.
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.6.6...v3.6.7)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/client/pkg/v3
  dependency-version: 3.6.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-10 16:19:20 +01:00
dependabot[bot]
cc741d523f Bump anchore/sbom-action from 0.22.1 to 0.22.2
Bumps [anchore/sbom-action](https://github.com/anchore/sbom-action) from 0.22.1 to 0.22.2.
- [Release notes](https://github.com/anchore/sbom-action/releases)
- [Changelog](https://github.com/anchore/sbom-action/blob/main/RELEASE.md)
- [Commits](https://github.com/anchore/sbom-action/compare/v0.22.1...v0.22.2)

---
updated-dependencies:
- dependency-name: anchore/sbom-action
  dependency-version: 0.22.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-10 16:18:38 +01:00
dependabot[bot]
9917b46396 Bump golang.org/x/sys from 0.40.0 to 0.41.0
Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.40.0 to 0.41.0.
- [Commits](https://github.com/golang/sys/compare/v0.40.0...v0.41.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-version: 0.41.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-10 16:18:21 +01:00
dependabot[bot]
bc973361e9 Bump google.golang.org/grpc from 1.77.0 to 1.78.0
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.77.0 to 1.78.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.77.0...v1.78.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.78.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-10 16:18:11 +01:00
dependabot[bot]
f0db910b8d Bump github.com/onsi/ginkgo/v2 from 2.28.0 to 2.28.1
Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.28.0 to 2.28.1.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/ginkgo/compare/v2.28.0...v2.28.1)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-version: 2.28.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-10 16:17:59 +01:00
Patryk Strusiewicz-Surmacki
dab1bb6201 Fixed E2E ARP tests
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2026-02-10 14:19:09 +01:00
dependabot[bot]
f9f5d75183 Bump golang from 1.25.6-alpine3.23 to 1.25.7-alpine3.23 (#1409)
Bumps golang from 1.25.6-alpine3.23 to 1.25.7-alpine3.23.

---
updated-dependencies:
- dependency-name: golang
  dependency-version: 1.25.7-alpine3.23
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-10 14:05:38 +01:00
dependabot[bot]
686431af94 Bump github.com/sirupsen/logrus from 1.9.3 to 1.9.4 (#1399)
Bumps [github.com/sirupsen/logrus](https://github.com/sirupsen/logrus) from 1.9.3 to 1.9.4.
- [Release notes](https://github.com/sirupsen/logrus/releases)
- [Changelog](https://github.com/sirupsen/logrus/blob/master/CHANGELOG.md)
- [Commits](https://github.com/sirupsen/logrus/compare/v1.9.3...v1.9.4)

---
updated-dependencies:
- dependency-name: github.com/sirupsen/logrus
  dependency-version: 1.9.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-03 12:48:00 +01:00
dependabot[bot]
e0520d6864 Bump github.com/onsi/gomega from 1.39.0 to 1.39.1 (#1398)
Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.39.0 to 1.39.1.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/gomega/compare/v1.39.0...v1.39.1)

---
updated-dependencies:
- dependency-name: github.com/onsi/gomega
  dependency-version: 1.39.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-03 07:55:39 +01:00
dependabot[bot]
374f858ead Bump sigs.k8s.io/kind from 0.30.0 to 0.31.0 (#1402)
Bumps [sigs.k8s.io/kind](https://github.com/kubernetes-sigs/kind) from 0.30.0 to 0.31.0.
- [Release notes](https://github.com/kubernetes-sigs/kind/releases)
- [Commits](https://github.com/kubernetes-sigs/kind/compare/v0.30.0...v0.31.0)

---
updated-dependencies:
- dependency-name: sigs.k8s.io/kind
  dependency-version: 0.31.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-03 07:55:17 +01:00
dependabot[bot]
e07c86b9a0 Bump anchore/sbom-action from 0.22.0 to 0.22.1 (#1403)
Bumps [anchore/sbom-action](https://github.com/anchore/sbom-action) from 0.22.0 to 0.22.1.
- [Release notes](https://github.com/anchore/sbom-action/releases)
- [Changelog](https://github.com/anchore/sbom-action/blob/main/RELEASE.md)
- [Commits](https://github.com/anchore/sbom-action/compare/v0.22.0...v0.22.1)

---
updated-dependencies:
- dependency-name: anchore/sbom-action
  dependency-version: 0.22.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-03 07:55:03 +01:00
dependabot[bot]
4efa98c676 Bump alpine from 3.23.2 to 3.23.3 (#1404)
Bumps alpine from 3.23.2 to 3.23.3.

---
updated-dependencies:
- dependency-name: alpine
  dependency-version: 3.23.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-03 07:54:53 +01:00
Daniel Finneran
ff40ff06a3 Update Makefile
Bumping version for a new release
2026-01-29 13:55:15 +00:00
Daniel Finneran
8b90925a69 Merge pull request #1384 from daniel-naegele/improve-wireguard
Improve wireguard
2026-01-28 11:13:50 +00:00
Daniel Nägele
d9e1fcd288 fix linting issues
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
2026-01-27 22:55:23 +01:00
Daniel Nägele
0c30654282 add sysctl
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
2026-01-27 22:55:23 +01:00
Daniel Nägele
265094df3b cleanup code
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
2026-01-27 22:55:23 +01:00
Daniel Nägele
b85f411502 make nftables impl work ?!?
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
2026-01-27 22:55:23 +01:00
Daniel Nägele
671046751e add IPVS
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
2026-01-27 22:55:23 +01:00
Daniel Nägele
8e539558f7 fixed up wireguard
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
2026-01-27 22:55:23 +01:00
Daniel Nägele
f81e42eab7 fix wg
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
2026-01-27 22:55:23 +01:00
Daniel Nägele
6eec9451bf add mtu and remove routes
goal is to get close to the wg-quick config format and behaviour

Signed-off-by: Daniel Nägele <daniel@naegele.dev>
2026-01-27 22:55:23 +01:00
Daniel Nägele
5b109de522 add sloppy nftmanager DNAT impl for wireguard
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
2026-01-27 22:55:23 +01:00
Daniel Nägele
92a7987f31 improve impl
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
2026-01-27 22:55:23 +01:00
Daniel Nägele
058beca51b fix lint issues
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
2026-01-27 22:55:23 +01:00
Daniel Nägele
ec899d1723 add minimalistic changelog
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
2026-01-27 22:55:23 +01:00
Daniel Nägele
9365d21e59 impl wg manager
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
2026-01-27 22:55:23 +01:00
Daniel Nägele
e148794d66 "handcraft" wg controller
Signed-off-by: Daniel Nägele <daniel@naegele.dev>
2026-01-27 22:55:22 +01:00
Patryk Strusiewicz-Surmacki
50da0c19d1 Fix IP refresh when using FQDN for VIP (#1390)
* Fix IP refresh when using FQDN for VIP

Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>

* Made AddIP a variadic function

Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>

---------

Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
Co-authored-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2026-01-27 13:26:32 +01:00
Patryk Strusiewicz-Surmacki
16b369575b Fixed cpCluster closing for BGP and RT modes
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2026-01-26 12:47:17 +01:00
Patryk Strusiewicz-Surmacki
dd8feefb96 Fixed cluster close
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2026-01-26 12:47:17 +01:00
Patryk Strusiewicz-Surmacki
5818a6c661 Do not use panic()
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2026-01-26 12:47:17 +01:00
Patryk Strusiewicz-Surmacki
33c8bc08ac Fixed context propagation
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
Co-authored-by: Ricardo Pchevuzinske Katz <ricardo.katz@gmail.com>
2026-01-26 12:47:17 +01:00
dependabot[bot]
4b802feb88 Bump github.com/onsi/gomega from 1.38.2 to 1.38.3 (#1362)
Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.38.2 to 1.38.3.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/gomega/compare/v1.38.2...v1.38.3)

---
updated-dependencies:
- dependency-name: github.com/onsi/gomega
  dependency-version: 1.38.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-26 09:02:05 +01:00
dependabot[bot]
4e5b12bf9f Bump alpine from 3.23.0 to 3.23.2 (#1366)
Bumps alpine from 3.23.0 to 3.23.2.

---
updated-dependencies:
- dependency-name: alpine
  dependency-version: 3.23.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-26 08:17:12 +01:00
dependabot[bot]
fda288134c Bump anchore/sbom-action from 0.21.0 to 0.22.0 (#1394)
Bumps [anchore/sbom-action](https://github.com/anchore/sbom-action) from 0.21.0 to 0.22.0.
- [Release notes](https://github.com/anchore/sbom-action/releases)
- [Changelog](https://github.com/anchore/sbom-action/blob/main/RELEASE.md)
- [Commits](https://github.com/anchore/sbom-action/compare/v0.21.0...v0.22.0)

---
updated-dependencies:
- dependency-name: anchore/sbom-action
  dependency-version: 0.22.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-26 08:16:32 +01:00
dependabot[bot]
9ddb91386a Bump go.etcd.io/etcd/api/v3 from 3.6.6 to 3.6.7 (#1363)
Bumps [go.etcd.io/etcd/api/v3](https://github.com/etcd-io/etcd) from 3.6.6 to 3.6.7.
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.6.6...v3.6.7)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/api/v3
  dependency-version: 3.6.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-26 08:15:42 +01:00
dependabot[bot]
3519e638db Bump github.com/onsi/ginkgo/v2 from 2.27.2 to 2.27.3 (#1364)
Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.27.2 to 2.27.3.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/ginkgo/compare/v2.27.2...v2.27.3)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-version: 2.27.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-26 08:12:00 +01:00
dependabot[bot]
60afc05a15 Bump golang.org/x/sync from 0.18.0 to 0.19.0 (#1361)
Bumps [golang.org/x/sync](https://github.com/golang/sync) from 0.18.0 to 0.19.0.
- [Commits](https://github.com/golang/sync/compare/v0.18.0...v0.19.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sync
  dependency-version: 0.19.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-26 08:11:43 +01:00
dependabot[bot]
309917507a Bump k8s.io/client-go from 0.34.3 to 0.35.0 (#1365)
Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.34.3 to 0.35.0.
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kubernetes/client-go/compare/v0.34.3...v0.35.0)

---
updated-dependencies:
- dependency-name: k8s.io/client-go
  dependency-version: 0.35.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-26 08:11:23 +01:00
dependabot[bot]
caa46fdcbc Bump golang from 1.25.5-alpine3.23 to 1.25.6-alpine3.23 (#1393)
Bumps golang from 1.25.5-alpine3.23 to 1.25.6-alpine3.23.

---
updated-dependencies:
- dependency-name: golang
  dependency-version: 1.25.6-alpine3.23
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-26 08:10:56 +01:00
Patryk Strusiewicz-Surmacki
545199246d Fixed leaderelection retry when endpoint changes node (#1386)
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
Co-authored-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2026-01-21 07:19:07 +01:00
Patryk Strusiewicz-Surmacki
531e3fd6c6 Bumped e2e test kubernetes version to 1.35.0 (#1387)
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
Co-authored-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2026-01-19 15:04:09 +01:00
Brian Davis
93a2fbec34 fix common lease fix from 1.0.1 (#1383)
* fix common lease fix from 1.0.1

Test coverage specific to this issue
- TestManager_LeaderElectionRestartScenario - Simulates the bug from the issue

Additional tests for leases
- TestManager_Add_NewLease - Verifies new lease creation
- TestManager_Add_ExistingLease - Verifies returning existing lease
- TestManager_Delete_DecrementCounter - Verifies reference counting
- TestManager_Delete_CancelsContext - Verifies context cancellation on delete
- TestManager_Add_AfterDelete_CreatesNewLease - Critical test for the bug fix - verifies a new lease is created after delete
- TestManager_Add_DifferentServices - Verifies different services get different leases
- TestManager_Add_SameNameDifferentNamespace - Verifies namespace isolation
- TestManager_ConcurrentAccess - Verifies thread safety
- TestLease_StartedChannel - Verifies Started channel behavior
- TestGetName_WithoutAnnotation - Verifies default lease naming
- TestGetName_WithAnnotation - Verifies custom lease naming
- TestUsesCommon - Verifies common lease detection
- TestManager_CommonLeaseScenario - Verifies the common lease feature still works

  Affects issue #1377

Signed-off-by: Brian Davis <dbrian@vmware.com>

* handle 0 LB addresses

Signed-off-by: Brian Davis <slimm609@gmail.com>
Signed-off-by: Brian Davis <dbrian@vmware.com>

* update tests for hanging connections

Signed-off-by: Brian Davis <slimm609@gmail.com>

* fix lint and remaining tests

Signed-off-by: Brian Davis <slimm609@gmail.com>

* fix failing tests for waiting for lease

Signed-off-by: Brian Davis <slimm609@gmail.com>

* handle race in intergration test

Signed-off-by: Brian Davis <slimm609@gmail.com>

---------

Signed-off-by: Brian Davis <dbrian@vmware.com>
Signed-off-by: Brian Davis <slimm609@gmail.com>
2026-01-16 07:33:52 +01:00
hindung
a2042d46e1 Fixed an issue with default gateway interface retrieval in multi-path… (#1373)
* Fixed an issue with default gateway interface retrieval in multi-path routing scenarios

Signed-off-by: huangxd <huangxd@winhong.com>

* RouteListFiltered filtering function enhances query efficiency,
separately handles IPv4 and IPv6 route lookups to avoid unnecessary route merging operations.

Signed-off-by: huangxd <huangxd@winhong.com>

* Extract the embedded `findDefault` function from the `GetDefaultGatewayInterface` function into a separate `getDefaultRoute` function.

Signed-off-by: huangxd <huangxd@winhong.com>

* Update pkg/vip/util.go

Co-authored-by: Marcel Fest <marcel@fest4.eu>
Signed-off-by: huangxd <huangxd@winhong.com>

* Update pkg/vip/util.go

Co-authored-by: Marcel Fest <marcel@fest4.eu>
Signed-off-by: huangxd <huangxd@winhong.com>

* Update pkg/vip/util.go

Co-authored-by: Marcel Fest <marcel@fest4.eu>
Signed-off-by: huangxd <huangxd@winhong.com>

* Update pkg/vip/util.go

Co-authored-by: Marcel Fest <marcel@fest4.eu>
Signed-off-by: huangxd <huangxd@winhong.com>

* Optimize error handling in the getDefaultRoute function

Signed-off-by: huangxd <huangxd@winhong.com>

---------

Signed-off-by: huangxd <huangxd@winhong.com>
Co-authored-by: huangxd <huangxd@winhong.com>
Co-authored-by: Marcel Fest <marcel@fest4.eu>
2026-01-14 09:29:17 +01:00
Patryk Strusiewicz-Surmacki
3ce54c4f97 Added pre-pull for testing images + some minor e2e tweaks (#1380)
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
Co-authored-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2026-01-10 22:52:20 +01:00
Patryk Strusiewicz-Surmacki
42393bf5fc Fixed endpointslices handling in dualstack clusters (#1379)
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
Co-authored-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2026-01-10 22:02:32 +01:00
Patryk Strusiewicz-Surmacki
7e671624b0 Configurable DHCP retry (#1374)
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
Co-authored-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2026-01-05 12:20:13 +01:00
Patryk Strusiewicz-Surmacki
9953699239 Fixed preserveVipOnLeadershipLoss setting in manifest generation (#1376)
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
Co-authored-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2025-12-31 18:41:49 +01:00
dependabot[bot]
3a5dad57ba Bump anchore/sbom-action from 0.20.11 to 0.21.0 (#1372)
Bumps [anchore/sbom-action](https://github.com/anchore/sbom-action) from 0.20.11 to 0.21.0.
- [Release notes](https://github.com/anchore/sbom-action/releases)
- [Changelog](https://github.com/anchore/sbom-action/blob/main/RELEASE.md)
- [Commits](https://github.com/anchore/sbom-action/compare/v0.20.11...v0.21.0)

---
updated-dependencies:
- dependency-name: anchore/sbom-action
  dependency-version: 0.21.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-29 04:06:22 +01:00
Daniel Finneran
ba7a71bc5f Merge pull request #1368 from p-strusiewiczsurmacki-mobica/fix/svc-leaderelection-restart
Improved leaderelection context and ingress comparison
2025-12-24 17:00:16 +00:00
Patryk Strusiewicz-Surmacki
40d15fed9c Fixed ingress comparison
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2025-12-23 17:47:57 +01:00
Patryk Strusiewicz-Surmacki
fad5176f6a Improved leaderelection context
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2025-12-23 17:47:46 +01:00
Daniel Finneran
28bec598f5 Merge pull request #1360 from p-strusiewiczsurmacki-mobica/fix/redundant-delete-svcelection
Fixed service deletion when service leader election is enabled
2025-12-19 09:13:37 +00:00
Patryk Strusiewicz-Surmacki
200d0d960c Fixed service deletion when service leader election is enabled
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2025-12-18 17:16:39 +01:00
Daniel Finneran
6e0f2132ca Merge pull request #1359 from kube-vip/repeat-of-prev-PR-to-move-error-to-warn 2025-12-17 19:55:31 +00:00
Daniel Finneran
19152ccd15 Merge pull request #1358 from kube-vip/bump-Makefile 2025-12-17 19:55:15 +00:00
Daniel Finneran
741d4a63e0 Move an Error to a warning 2025-12-17 18:04:56 +00:00
Daniel Finneran
badd66c4e8 Update Makefile 2025-12-17 18:02:56 +00:00
Daniel Finneran
fbb0717cb4 Merge pull request #1357 from p-strusiewiczsurmacki-mobica/fix/service-port-security
Fixed service port security rules for iptables
2025-12-17 18:02:29 +00:00
Marcel Fest
414ba0c6a6 Merge branch 'main' into fix/service-port-security 2025-12-17 18:09:11 +01:00
Daniel Finneran
c4054fa86d Merge pull request #1356 from p-strusiewiczsurmacki-mobica/fix/service-dns
Fixed service DNS resolve
2025-12-17 16:06:39 +00:00
Patryk Strusiewicz-Surmacki
5bfec6e426 Fixed service port security rules
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2025-12-17 12:49:06 +01:00
Daniel Finneran
075639996e Merge pull request #1354 from kube-vip/dependabot/github_actions/anchore/sbom-action-0.20.11
Bump anchore/sbom-action from 0.20.10 to 0.20.11
2025-12-16 14:16:50 +00:00
Daniel Finneran
d90b7ae20e Merge pull request #1353 from kube-vip/dependabot/github_actions/actions/upload-artifact-6
Bump actions/upload-artifact from 5 to 6
2025-12-16 14:16:32 +00:00
Daniel Finneran
b200e9a3f7 Merge pull request #1352 from kube-vip/dependabot/go_modules/k8s.io/client-go-0.34.3
Bump k8s.io/client-go from 0.34.1 to 0.34.3
2025-12-16 14:16:17 +00:00
dependabot[bot]
e6c48839a7 Bump anchore/sbom-action from 0.20.10 to 0.20.11
Bumps [anchore/sbom-action](https://github.com/anchore/sbom-action) from 0.20.10 to 0.20.11.
- [Release notes](https://github.com/anchore/sbom-action/releases)
- [Changelog](https://github.com/anchore/sbom-action/blob/main/RELEASE.md)
- [Commits](https://github.com/anchore/sbom-action/compare/v0.20.10...v0.20.11)

---
updated-dependencies:
- dependency-name: anchore/sbom-action
  dependency-version: 0.20.11
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-16 12:58:25 +00:00
dependabot[bot]
53a86b6164 Bump actions/upload-artifact from 5 to 6
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 5 to 6.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-16 12:58:13 +00:00
dependabot[bot]
1988a69d1b Bump k8s.io/client-go from 0.34.1 to 0.34.3
Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.34.1 to 0.34.3.
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kubernetes/client-go/compare/v0.34.1...v0.34.3)

---
updated-dependencies:
- dependency-name: k8s.io/client-go
  dependency-version: 0.34.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-16 12:58:06 +00:00
Daniel Finneran
3b97bb7360 Merge pull request #1351 from kube-vip/dependabot/go_modules/k8s.io/api-0.34.3
Bump k8s.io/api from 0.34.2 to 0.34.3
2025-12-16 12:54:44 +00:00
Daniel Finneran
09800c9be6 Merge pull request #1346 from kube-vip/dependabot/go_modules/github.com/florianl/go-conntrack-0.5.0
Bump github.com/florianl/go-conntrack from 0.4.0 to 0.5.0
2025-12-16 12:54:32 +00:00
Patryk Strusiewicz-Surmacki
c8e0a72be6 Fixed service DNS resolve
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2025-12-15 17:48:03 +01:00
Daniel Finneran
002a83fa65 Merge pull request #1347 from kube-vip/dependabot/docker/alpine-3.23.0
Bump alpine from 3.22.2 to 3.23.0
2025-12-15 16:42:42 +00:00
Daniel Finneran
9ad84e3ae6 Merge pull request #1345 from kube-vip/dependabot/go_modules/github.com/spf13/cobra-1.10.2
Bump github.com/spf13/cobra from 1.10.1 to 1.10.2
2025-12-15 16:10:36 +00:00
Daniel Finneran
ad50b9c3ef Merge pull request #1344 from kube-vip/dependabot/go_modules/testing/e2e/e2e/github.com/sirupsen/logrus-1.9.1
Bump github.com/sirupsen/logrus from 1.9.0 to 1.9.1 in /testing/e2e/e2e
2025-12-15 16:10:10 +00:00
dependabot[bot]
d98a6ef660 Bump k8s.io/api from 0.34.2 to 0.34.3
Bumps [k8s.io/api](https://github.com/kubernetes/api) from 0.34.2 to 0.34.3.
- [Commits](https://github.com/kubernetes/api/compare/v0.34.2...v0.34.3)

---
updated-dependencies:
- dependency-name: k8s.io/api
  dependency-version: 0.34.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-15 15:33:29 +00:00
dependabot[bot]
c83a8cea2f Bump github.com/florianl/go-conntrack from 0.4.0 to 0.5.0
Bumps [github.com/florianl/go-conntrack](https://github.com/florianl/go-conntrack) from 0.4.0 to 0.5.0.
- [Release notes](https://github.com/florianl/go-conntrack/releases)
- [Commits](https://github.com/florianl/go-conntrack/compare/v0.4.0...v0.5.0)

---
updated-dependencies:
- dependency-name: github.com/florianl/go-conntrack
  dependency-version: 0.5.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-15 15:32:50 +00:00
dependabot[bot]
08966e9bed Bump github.com/spf13/cobra from 1.10.1 to 1.10.2
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 1.10.1 to 1.10.2.
- [Release notes](https://github.com/spf13/cobra/releases)
- [Commits](https://github.com/spf13/cobra/compare/v1.10.1...v1.10.2)

---
updated-dependencies:
- dependency-name: github.com/spf13/cobra
  dependency-version: 1.10.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-15 15:32:32 +00:00
dependabot[bot]
6bf0e37282 Bump github.com/sirupsen/logrus from 1.9.0 to 1.9.1 in /testing/e2e/e2e
Bumps [github.com/sirupsen/logrus](https://github.com/sirupsen/logrus) from 1.9.0 to 1.9.1.
- [Release notes](https://github.com/sirupsen/logrus/releases)
- [Changelog](https://github.com/sirupsen/logrus/blob/master/CHANGELOG.md)
- [Commits](https://github.com/sirupsen/logrus/compare/v1.9.0...v1.9.1)

---
updated-dependencies:
- dependency-name: github.com/sirupsen/logrus
  dependency-version: 1.9.1
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-15 15:32:00 +00:00
dependabot[bot]
ea1d6136be Bump alpine from 3.22.2 to 3.23.0
Bumps alpine from 3.22.2 to 3.23.0.

---
updated-dependencies:
- dependency-name: alpine
  dependency-version: 3.23.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-15 15:31:50 +00:00
Daniel Finneran
ff3e85c6a6 Merge pull request #1355 from kube-vip/golang_bump
Bump to golang to fix stdlib vuln
2025-12-15 15:30:49 +00:00
Dan Finneran
38a6fdb457 bump alpine release for build
Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>
2025-12-15 14:57:21 +00:00
Dan Finneran
dfd1b1da19 path up other ci complaints.
Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>
2025-12-15 14:53:21 +00:00
Dan Finneran
aecc0264b0 upgrade lint config
Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>
2025-12-15 12:02:41 +00:00
Dan Finneran
a35849d4f8 bump ci workflow
Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>
2025-12-15 11:57:46 +00:00
Dan Finneran
4c3a2d57ee Bump to golang to fix stdlib vuln
Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>
2025-12-15 11:53:23 +00:00
Daniel Finneran
48c99ca320 Merge pull request #1342 from kube-vip/dependabot/github_actions/docker/metadata-action-5.10.0
Bump docker/metadata-action from 5.9.0 to 5.10.0
2025-12-15 11:50:31 +00:00
Daniel Finneran
83d1a9e455 Merge pull request #1341 from kube-vip/dependabot/go_modules/github.com/google/go-containerregistry-0.20.7
Bump github.com/google/go-containerregistry from 0.20.6 to 0.20.7
2025-12-15 11:50:18 +00:00
Daniel Finneran
11649c9e79 Merge pull request #1350 from ivucica/ivucica-upnp-lease-duration
Support specifying UPNP lease duration as annotation on a service.
2025-12-15 11:50:06 +00:00
Ivan Vučica
b31de5ca61 Support specifying UPNP lease duration as annotation on a service.
The annotation `kube-vip.io/upnp-lease-duration` takes in a duration where the string is parseable by `time.ParseDuration`.

There is no new configuration option that would allow setting this globally, since this new feature is most helpful in limited cases (such as incorrect implementations that have trouble with certain values).

There is a minimal test added for the value returned by the new private helper `upnpLeaseDurationForService`. Due to a lack of dependency injection in `service.Processor`'s `upnpMap`, it is infeasible to add a more complete test at this time, to check what would be passed, exactly, into `AddPinholeCtx` and `AddPortMapping`.

Signed-off-by: Ivan Vučica <ivan@vucica.net>
Signed-off-by: Ivan Vučica <ivucica@google.com>
Fixes: #1349
2025-12-12 12:22:37 +00:00
Daniel Finneran
18544a5d54 Merge pull request #1343 from kube-vip/error_warn
Move this error to a warning
2025-12-01 15:56:14 +00:00
dependabot[bot]
099d8f759b Bump docker/metadata-action from 5.9.0 to 5.10.0
Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 5.9.0 to 5.10.0.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Commits](https://github.com/docker/metadata-action/compare/v5.9.0...v5.10.0)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-version: 5.10.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-01 02:14:54 +00:00
dependabot[bot]
e99fa71ed6 Bump github.com/google/go-containerregistry from 0.20.6 to 0.20.7
Bumps [github.com/google/go-containerregistry](https://github.com/google/go-containerregistry) from 0.20.6 to 0.20.7.
- [Release notes](https://github.com/google/go-containerregistry/releases)
- [Commits](https://github.com/google/go-containerregistry/compare/v0.20.6...v0.20.7)

---
updated-dependencies:
- dependency-name: github.com/google/go-containerregistry
  dependency-version: 0.20.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-01 02:03:39 +00:00
118 changed files with 9878 additions and 3183 deletions

View File

@@ -26,6 +26,6 @@ jobs:
with:
ref: ${{ github.ref_name }}
- name: Anchore SBOM Action
uses: anchore/sbom-action@v0.20.10
uses: anchore/sbom-action@v0.24.0
with:
format: cyclonedx-json

View File

@@ -25,23 +25,23 @@ jobs:
run: make dockerx86Local
- name: Run Manifest generation tests
run: make manifest-test
- name: Run ARP mode tests v1.29.0 onwards
run: E2E_KEEP_LOGS=true make e2e-tests129-arp
- name: Run ARP mode tests
run: E2E_KEEP_LOGS=true make e2e-tests-arp
if: matrix.mode== 'arp'
- name: Run RT mode tests v1.29.0 onwards
run: E2E_KEEP_LOGS=true make e2e-tests129-rt
- name: Run RT mode tests
run: E2E_KEEP_LOGS=true make e2e-tests-rt
if: matrix.mode== 'rt'
- name: Get GoBGP binaries
run: make get-gobgp
if: matrix.mode== 'bgp'
- name: Run BGP mode tests v1.29.0 onwards
run: sudo -E PATH=$PATH DOCKER_API_VERSION=1.48 E2E_KEEP_LOGS=true make e2e-tests129-bgp
- name: Run BGP mode tests
run: sudo -E PATH=$PATH DOCKER_API_VERSION=1.48 E2E_KEEP_LOGS=true make e2e-tests-bgp
if: matrix.mode== 'bgp'
- name: Change log directory permissions
run: sudo chmod -R 755 /tmp/kube-vip-test-${{ matrix.mode }}*
if: matrix.mode== 'bgp' && always()
- name: Save logs
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@v7
with:
name: e2e-test-logs-${{ matrix.mode }}-${{ steps.date.outputs.date }}
path: /tmp/kube-vip-test-${{ matrix.mode }}*
@@ -64,7 +64,7 @@ jobs:
- name: Run tests
run: DOCKERTAG=action E2E_KEEP_LOGS=true make service-tests
- name: Save logs
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@v7
with:
name: services-test-logs-${{ steps.date.outputs.date }}
path: /tmp/kube-vip-service-tests*

View File

@@ -10,7 +10,7 @@ jobs:
- name: Init
run: sudo apt-get update && sudo apt-get install -y build-essential && sudo sysctl fs.inotify.max_user_instances=8192 && sudo sysctl fs.inotify.max_user_watches=524288
- name: Install golangci-lint
run: curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.64.8
run: curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v2.7.2
- name: Checkout code
uses: actions/checkout@v6
- name: Install Go

View File

@@ -13,23 +13,23 @@ jobs:
- name: Checkout
uses: actions/checkout@v6
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
uses: docker/setup-qemu-action@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@v4
- name: Login to DockerHub
uses: docker/login-action@v3
uses: docker/login-action@v4
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to Github Packages
uses: docker/login-action@v3
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build standard version
id: docker_build
uses: docker/build-push-action@v6
uses: docker/build-push-action@v7
with:
context: .
platforms: linux/amd64,linux/arm/v7,linux/arm64,linux/ppc64le,linux/s390x
@@ -39,7 +39,7 @@ jobs:
ghcr.io/kube-vip/kube-vip:${{ github.ref_name }}
- name: Build iptables version
id: docker_build_iptables
uses: docker/build-push-action@v6
uses: docker/build-push-action@v7
with:
context: .
file: Dockerfile_iptables

View File

@@ -13,29 +13,29 @@ jobs:
- name: Checkout
uses: actions/checkout@v6
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
uses: docker/setup-qemu-action@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@v4
- name: Login to DockerHub
uses: docker/login-action@v3
uses: docker/login-action@v4
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to Github Packages
uses: docker/login-action@v3
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Generate Metadata
uses: docker/metadata-action@v5.9.0
uses: docker/metadata-action@v6.0.0
id: metadata
with:
labels: |
org.opencontainers.image.documentation=https://kube-vip.io/docs/
- name: Build and push main branch
id: docker_build
uses: docker/build-push-action@v6
uses: docker/build-push-action@v7
with:
context: .
platforms: linux/amd64,linux/arm/v7,linux/arm64,linux/ppc64le,linux/s390x
@@ -48,7 +48,7 @@ jobs:
ghcr.io/kube-vip/kube-vip:latest
- name: Build iptables version and push main branch
id: docker_build_iptables
uses: docker/build-push-action@v6
uses: docker/build-push-action@v7
with:
context: .
file: Dockerfile_iptables

View File

@@ -1,13 +1,38 @@
run:
timeout: 10m
version: "2"
linters:
enable:
- bodyclose
- gofmt
- goimports
- revive
- gosec
- misspell
- unconvert
- unparam
- bodyclose
- gosec
- misspell
- unconvert
- unparam
settings:
misspell:
ignore-rules:
- creater
staticcheck:
checks:
- all
# Disable QF1008 to retain embedded fields for better readability.
- "-QF1008"
exclusions:
generated: lax
presets:
- comments
- common-false-positives
- legacy
- std-error-handling
paths:
- third_party$
- builtin$
- examples$
formatters:
enable:
- gofmt
- goimports
exclusions:
generated: lax
paths:
- third_party$
- builtin$
- examples$

View File

@@ -7,6 +7,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Fixed
- Retry on 403 Forbidden and 401 Unauthorized in `ServicesWatcher` at startup with exponential backoff. Fixes #1464.
### Added
- SIGUSR1 signal handler for runtime configuration dumps (#1301)
- Send SIGUSR1 to kube-vip process to dump current configuration to stdout
@@ -26,6 +29,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Changed
- Updated signal handlers in manager_arp.go, manager_bgp.go, manager_wireguard.go, and manager_table.go to use switch statement pattern for handling multiple signals (SIGUSR1, SIGINT, SIGTERM)
- wireguard.go now manages a complete wireguard interface on the current network namespace
- manager_wireguard.go uses the new wireguard.go implementation
## [v1.0.1] - Previous Release

View File

@@ -1,6 +1,6 @@
# syntax=docker/dockerfile:experimental
FROM golang:1.25.4-alpine3.22 as dev
FROM golang:1.26.1-alpine3.23 as dev
RUN apk add --no-cache git ca-certificates make
RUN adduser -D appuser
COPY . /src/

View File

@@ -1,6 +1,6 @@
# syntax=docker/dockerfile:experimental
FROM golang:1.25.4-alpine3.22 as dev
FROM golang:1.26.1-alpine3.23 as dev
RUN apk add --no-cache git make
RUN adduser -D appuser
COPY . /src/
@@ -11,7 +11,7 @@ RUN --mount=type=cache,sharing=locked,id=gomod,target=/go/pkg/mod/cache \
--mount=type=cache,sharing=locked,id=goroot,target=/root/.cache/go-build \
CGO_ENABLED=0 GOOS=linux make build
FROM alpine:3.22.2
FROM alpine:3.23.3
# Update pkgs and add iptables
RUN apk upgrade && \
apk add --no-cache iptables iptables-legacy

View File

@@ -5,7 +5,7 @@ TARGET := kube-vip
.DEFAULT_GOAL := $(TARGET)
# These will be provided to the target
VERSION := v1.0.2
VERSION := v1.1.1
BUILD := `git rev-parse HEAD`
@@ -16,8 +16,10 @@ TARGETOS=linux
LDFLAGS=-ldflags "-s -w -X=main.Version=$(VERSION) -X=main.Build=$(BUILD) -extldflags -static"
DOCKERTAG ?= $(VERSION)
REPOSITORY ?= docker.io/plndr
GO_VERSION := 1.25.5
K8S_VERSION ?= v1.35.0
.PHONY: all build clean install uninstall simplify check run e2e-tests
.PHONY: all build clean install uninstall simplify check run e2e-tests unit-tests integration-tests unit-tests-docker integration-tests-docker
all: check install
@@ -127,26 +129,27 @@ manifest-test:
docker run $(REPOSITORY)/$(TARGET):$(DOCKERTAG) manifest daemonset --interface eth0 --vip 192.168.0.1 --image "$(REPOSITORY)/$(TARGET):$(DOCKERTAG)" --bgp --leaderElection --controlplane --services --inCluster
unit-tests:
go test ./...
go test -race ./...
unit-tests-docker:
docker run --rm -w /kube-vip -v $$(pwd):/kube-vip golang:$(GO_VERSION) make unit-tests
integration-tests:
go test -tags=integration,e2e -v ./pkg/etcd
e2e-tests:
GOMAXPROCS=4 E2E_IMAGE_PATH=$(REPOSITORY)/$(TARGET):$(DOCKERTAG) go run github.com/onsi/ginkgo/v2/ginkgo --tags=e2e -v -p ./testing/e2e ./testing/e2e/etcd
e2e-tests-arp: get-whoami
GOMAXPROCS=4 TEST_MODE=arp K8S_IMAGE_PATH=kindest/node:$(K8S_VERSION) E2E_IMAGE_PATH=$(REPOSITORY)/$(TARGET):$(DOCKERTAG) go run github.com/onsi/ginkgo/v2/ginkgo --tags=e2e -v -p ./testing/e2e
e2e-tests129-arp:
GOMAXPROCS=4 TEST_MODE=arp V129=true K8S_IMAGE_PATH=kindest/node:v1.29.0 E2E_IMAGE_PATH=$(REPOSITORY)/$(TARGET):$(DOCKERTAG) go run github.com/onsi/ginkgo/v2/ginkgo --tags=e2e -v -p ./testing/e2e
e2e-tests-rt: get-whoami
GOMAXPROCS=4 TEST_MODE=rt K8S_IMAGE_PATH=kindest/node:$(K8S_VERSION) E2E_IMAGE_PATH=$(REPOSITORY)/$(TARGET):$(DOCKERTAG) go run github.com/onsi/ginkgo/v2/ginkgo --tags=e2e -v -p ./testing/e2e
e2e-tests129-rt:
GOMAXPROCS=4 TEST_MODE=rt V129=true K8S_IMAGE_PATH=kindest/node:v1.29.0 E2E_IMAGE_PATH=$(REPOSITORY)/$(TARGET):$(DOCKERTAG) go run github.com/onsi/ginkgo/v2/ginkgo --tags=e2e -v -p ./testing/e2e
e2e-tests-bgp: get-whoami
GOMAXPROCS=4 TEST_MODE=bgp K8S_IMAGE_PATH=kindest/node:$(K8S_VERSION) E2E_IMAGE_PATH=$(REPOSITORY)/$(TARGET):$(DOCKERTAG) go run github.com/onsi/ginkgo/v2/ginkgo --tags=e2e -v -p ./testing/e2e
e2e-tests129-bgp:
GOMAXPROCS=4 TEST_MODE=bgp V129=true K8S_IMAGE_PATH=kindest/node:v1.29.0 E2E_IMAGE_PATH=$(REPOSITORY)/$(TARGET):$(DOCKERTAG) go run github.com/onsi/ginkgo/v2/ginkgo --tags=e2e -v -p ./testing/e2e
e2e-tests129: e2e-tests129-arp e2e-tests129-rt e2e-tests129-bgp
e2e-tests: e2e-tests-arp e2e-tests-rt e2e-tests-bgp
service-tests:
$(MAKE) -C testing/e2e/e2e dockerLocal
E2E_IMAGE_PATH=$(REPOSITORY)/$(TARGET):$(DOCKERTAG) go run ./testing/services -Services -simple -deployments -leaderActive -leaderFailover -localDeploy -egress -egressIPv6 -dualStack
trivy: dockerx86ActionIPTables
@@ -177,3 +180,5 @@ get-gobgp:
wget -nc --directory-prefix=bin https://github.com/osrg/gobgp/releases/download/v3.37.0/gobgp_3.37.0_linux_amd64.tar.gz
tar -xvzf bin/gobgp_3.37.0_linux_amd64.tar.gz -C bin
get-whoami:
docker pull ghcr.io/traefik/whoami:v1.11

View File

@@ -66,6 +66,17 @@ Thanks for taking the time to join our community and start contributing! We welc
:warning: This project has issue compiling on MacOS, please compile it on linux distribution
Additionally it is now relatively easy and quick to develop with [skaffold](https://skaffold.dev/), and the `skaffold.yaml` exists within the root folder of the gir repository.
### Set up a kind development environment
1. `kind create cluster --config ./testing/kind.yaml`
2. `kubectl apply -f https://kube-vip.io/manifests/rbac.yaml`
3. Create a load balancer range `configMap` from the kind cluster
4. Apply the CCM manifest
5. Start skaffold with `skaffold dev`
6. Start developing and see changes applied in real-time.
## Star History
[![Star History Chart](https://api.star-history.com/svg?repos=kube-vip/kube-vip&type=Date)](https://star-history.com/#kube-vip/kube-vip&Date)

View File

@@ -65,7 +65,11 @@ var kubeKubeadmInit = &cobra.Command{
}
}
cfg := kubevip.GeneratePodManifestFromConfig(&initConfig, image, Release.Version, inCluster)
cfg, err := kubevip.GeneratePodManifestFromConfig(&initConfig, image, Release.Version, inCluster)
if err != nil {
log.Error("unable to create manifest", "err", err)
return
}
fmt.Println(cfg) // output manifest to stdout
},
}
@@ -110,7 +114,11 @@ var kubeKubeadmJoin = &cobra.Command{
}
}
cfg := kubevip.GeneratePodManifestFromConfig(&initConfig, image, Release.Version, inCluster)
cfg, err := kubevip.GeneratePodManifestFromConfig(&initConfig, image, Release.Version, inCluster)
if err != nil {
log.Error("unable to create manifest", "err", err)
return
}
fmt.Println(cfg) // output manifest to stdout
},
}

View File

@@ -54,7 +54,7 @@ var kubeManifestPod = &cobra.Command{
// The control plane has a requirement for a VIP being specified
if initConfig.EnableControlPlane && (initConfig.VIP == "" && initConfig.Address == "" && !initConfig.DDNS) {
_ = cmd.Help()
log.Error("No address is specified for kube-vip to expose services on")
log.Error("no address is specified for kube-vip to expose services on")
return
}
@@ -67,7 +67,11 @@ var kubeManifestPod = &cobra.Command{
}
}
cfg := kubevip.GeneratePodManifestFromConfig(&initConfig, image, Release.Version, inCluster)
cfg, err := kubevip.GeneratePodManifestFromConfig(&initConfig, image, Release.Version, inCluster)
if err != nil {
log.Error("unable to create manifest", "err", err)
return
}
fmt.Println(cfg) // output manifest to stdout
},
}
@@ -87,7 +91,7 @@ var kubeManifestDaemon = &cobra.Command{
// The control plane has a requirement for a VIP being specified
if initConfig.EnableControlPlane && (initConfig.VIP == "" && initConfig.Address == "" && !initConfig.DDNS) {
_ = cmd.Help()
log.Error("No address is specified for kube-vip to expose services on")
log.Error("no address is specified for kube-vip to expose services on")
return
}
@@ -100,7 +104,11 @@ var kubeManifestDaemon = &cobra.Command{
}
}
cfg := kubevip.GenerateDaemonsetManifestFromConfig(&initConfig, image, Release.Version, inCluster, taint)
cfg, err := kubevip.GenerateDaemonsetManifestFromConfig(&initConfig, image, Release.Version, inCluster, taint)
if err != nil {
log.Error("unable to create manifest", "err", err)
return
}
fmt.Println(cfg) // output manifest to stdout
},
}
@@ -121,7 +129,7 @@ var kubeManifestRbac = &cobra.Command{
// The control plane has a requirement for a VIP being specified
if initConfig.EnableControlPlane && (initConfig.VIP == "" && initConfig.Address == "" && !initConfig.DDNS) {
_ = cmd.Help()
log.Error("No address is specified for kube-vip to expose services on")
log.Error("no address is specified for kube-vip to expose services on")
return
}

View File

@@ -7,7 +7,9 @@ import (
"net/http"
"os"
"slices"
"strconv"
"strings"
"sync"
"time"
log "log/slog"
@@ -80,9 +82,9 @@ func init() {
kubeVipCmd.PersistentFlags().BoolVar(&initConfig.EnableLeaderElection, "leaderElection", false, "Use the Kubernetes leader election mechanism for clustering")
kubeVipCmd.PersistentFlags().StringVar(&initConfig.LeaderElectionType, "leaderElectionType", "kubernetes", "Defines the backend to run the leader election: kubernetes or etcd. Defaults to kubernetes.")
kubeVipCmd.PersistentFlags().StringVar(&initConfig.LeaseName, "leaseName", "plndr-cp-lock", "Name of the lease that is used for leader election")
kubeVipCmd.PersistentFlags().IntVar(&initConfig.LeaseDuration, "leaseDuration", 5, "Length of time (in seconds) a Kubernetes leader lease can be held for")
kubeVipCmd.PersistentFlags().IntVar(&initConfig.RenewDeadline, "leaseRenewDuration", 3, "Length of time (in seconds) a Kubernetes leader can attempt to renew its lease")
kubeVipCmd.PersistentFlags().IntVar(&initConfig.RetryPeriod, "leaseRetry", 1, "Length of time (in seconds) the LeaderElector clients should wait between tries of actions")
kubeVipCmd.PersistentFlags().IntVar(&initConfig.LeaseDuration, "leaseDuration", 15, "Length of time (in seconds) a Kubernetes leader lease can be held for")
kubeVipCmd.PersistentFlags().IntVar(&initConfig.RenewDeadline, "leaseRenewDuration", 10, "Length of time (in seconds) a Kubernetes leader can attempt to renew its lease")
kubeVipCmd.PersistentFlags().IntVar(&initConfig.RetryPeriod, "leaseRetry", 2, "Length of time (in seconds) the LeaderElector clients should wait between tries of actions")
// BGP flags
kubeVipCmd.PersistentFlags().BoolVar(&initConfig.EnableBGP, "bgp", false, "This will enable BGP support within kube-vip")
@@ -133,6 +135,9 @@ func init() {
kubeVipCmd.PersistentFlags().BoolVar(&initConfig.EnableNodeLabeling, "enableNodeLabeling", false, fmt.Sprintf("Enable leader node labeling with %q, defaults to false", kubevip.HasIP))
kubeVipCmd.PersistentFlags().StringVar(&initConfig.ServicesLeaseName, "servicesLeaseName", "plndr-svcs-lock", "Name of the lease that is used for leader election for services (in arp mode)")
kubeVipCmd.PersistentFlags().StringVar(&initConfig.DNSMode, "dnsMode", "first", "Name of the mode that DNS lookup will be performed (first, ipv4, ipv6, dual)")
kubeVipCmd.PersistentFlags().StringVar(&initConfig.DHCPMode, "dhcpMode", "", "Mode DHCP resolving will use to obtain IP addresses (ipv4, ipv6, dual)")
kubeVipCmd.PersistentFlags().UintVar(&initConfig.DHCPBackoffAttempts, "dhcpBackoffAttempts", kubevip.DefaultDHCPBackoffAttempts,
fmt.Sprintf("number of times DHCP client will try to obtain an IP address (defaults to: %d, 0 for unlimited retries)", kubevip.DefaultDHCPBackoffAttempts))
kubeVipCmd.PersistentFlags().BoolVar(&initConfig.DisableServiceUpdates, "disableServiceUpdates", false, "If true, kube-vip will process services as usual, but will not update service's Status.LoadBalancer.Ingress slice")
kubeVipCmd.PersistentFlags().BoolVar(&initConfig.EnableEndpoints, "enableEndpoints", false, "If enabled, kube-vip will only advertise services, but will use the (deprecated since v1.33) endpoints for IP addresses")
kubeVipCmd.PersistentFlags().BoolVar(&initConfig.LoInterfaceGlobalScope, "loInterfaceGlobalScope", false, "If true, kube-vip will set global scope when using the lo interface, otherwise a host scope will be used by default")
@@ -246,8 +251,11 @@ var kubeVipService = &cobra.Command{
return
}
ctx, cancel := context.WithCancel(cmd.Context())
defer cancel()
// Start the service manager, this will watch the config Map and construct kube-vip services for it
err = mgr.Start()
err = mgr.Start(ctx)
if err != nil {
log.Error("manager start", "err", err)
return
@@ -295,10 +303,19 @@ var kubeVipManager = &cobra.Command{
// Welome messages
log.Info("kube-vip.io", "version", Release.Version, "build", Release.Build)
wg := sync.WaitGroup{}
defer wg.Wait()
// create main manager context
ctx, cancel := context.WithCancel(cmd.Context())
defer cancel()
// start prometheus server
if initConfig.PrometheusHTTPServer != "" {
go servePrometheusHTTPServer(cmd.Context(), PrometheusHTTPServerConfig{
Addr: initConfig.PrometheusHTTPServer,
wg.Go(func() {
servePrometheusHTTPServer(ctx, PrometheusHTTPServerConfig{
Addr: initConfig.PrometheusHTTPServer,
})
})
}
@@ -382,13 +399,13 @@ var kubeVipManager = &cobra.Command{
initConfig.Interface = defaultIF.Name
log.Info("kube-vip bind", "interface", initConfig.Interface)
go func() {
if err := vip.MonitorDefaultInterface(context.TODO(), defaultIF); err != nil {
wg.Go(func() {
if err := vip.MonitorDefaultInterface(ctx, defaultIF); err != nil {
log.Error("interface monitor", "err", err)
return
}
}()
})
}
}
// Perform a check on the state of the interface
@@ -413,7 +430,7 @@ var kubeVipManager = &cobra.Command{
prometheus.MustRegister(mgr.PrometheusCollector()...)
// Start the service manager, this will watch the config Map and construct kube-vip services for it
err = mgr.Start()
err = mgr.Start(ctx)
if err != nil {
log.Error("start manager", "err", err)
return
@@ -447,19 +464,20 @@ func servePrometheusHTTPServer(ctx context.Context, config PrometheusHTTPServerC
ReadHeaderTimeout: 2 * time.Second,
}
go func() {
wg := sync.WaitGroup{}
wg.Go(func() {
if err = srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
log.Error("prometheus HTTP server", "err", err)
return
}
}()
})
log.Info("prometheus HTTP server started")
<-ctx.Done()
log.Info("prometheus HTTP server stopped")
// create prometheus shutdown context (independent of other contexts)
ctxShutDown, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer func() {
cancel()
@@ -473,6 +491,10 @@ func servePrometheusHTTPServer(ctx context.Context, config PrometheusHTTPServerC
if err == http.ErrServerClosed {
err = nil
}
log.Info("prometheus HTTP server stopped")
wg.Wait()
}
func GenerateCidrRange(address string, dnsMode string) (string, error) {
@@ -483,23 +505,23 @@ func GenerateCidrRange(address string, dnsMode string) (string, error) {
ip := net.ParseIP(a)
if ip == nil {
// we probably are a DNS name
ips, err := utils.LookupHost(a, dnsMode)
ips, err := utils.LookupHost(a, dnsMode, true)
if len(ips) == 0 || err != nil {
return "", fmt.Errorf("invalid IP address: %s from [%s], %v", a, address, err)
}
for _, addr := range ips {
ip = net.ParseIP(addr)
if ip.To4() != nil {
cidrs = append(cidrs, "32")
cidrs = append(cidrs, strconv.Itoa(vip.DefaultMaskIPv4))
} else {
cidrs = append(cidrs, "128")
cidrs = append(cidrs, strconv.Itoa(vip.DefaultMaskIPv6))
}
}
} else {
if ip.To4() != nil {
cidrs = append(cidrs, "32")
cidrs = append(cidrs, strconv.Itoa(vip.DefaultMaskIPv4))
} else {
cidrs = append(cidrs, "128")
cidrs = append(cidrs, strconv.Itoa(vip.DefaultMaskIPv6))
}
}
}

View File

@@ -19,7 +19,7 @@ func main() {
var errorOccurred bool
for {
p := make([]byte, 2048)
conn, err := net.Dial("udp", fmt.Sprintf("%s:%d", *address, *port))
conn, err := net.Dial("udp", net.JoinHostPort(*address, fmt.Sprint(port)))
if err != nil {
if !errorOccurred {
errorTime = time.Now()

84
go.mod
View File

@@ -1,44 +1,45 @@
module github.com/kube-vip/kube-vip
go 1.24.4
go 1.25.7
require (
github.com/cloudflare/ipvs v0.11.0
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc
github.com/docker/docker v28.5.2+incompatible
github.com/florianl/go-conntrack v0.4.0
github.com/florianl/go-conntrack v0.7.0
github.com/google/go-cmp v0.7.0
github.com/google/go-containerregistry v0.20.6
github.com/google/go-containerregistry v0.21.3
github.com/google/nftables v0.3.0
github.com/gookit/slog v0.6.0
github.com/huin/goupnp v1.3.0
github.com/insomniacslk/dhcp v0.0.0-20241224095048-b56fa0d5f25d
github.com/jpillora/backoff v1.0.0
github.com/mdlayher/ndp v1.1.0
github.com/onsi/ginkgo/v2 v2.27.2
github.com/onsi/gomega v1.38.2
github.com/onsi/ginkgo/v2 v2.28.1
github.com/onsi/gomega v1.39.1
github.com/osrg/gobgp/v3 v3.37.0
github.com/pkg/errors v0.9.1
github.com/prometheus/client_golang v1.23.2
github.com/sirupsen/logrus v1.9.3
github.com/spf13/cobra v1.10.1
github.com/sirupsen/logrus v1.9.4
github.com/spf13/cobra v1.10.2
github.com/stretchr/testify v1.11.1
github.com/vishvananda/netlink v1.3.1
go.etcd.io/etcd/api/v3 v3.6.6
go.etcd.io/etcd/client/pkg/v3 v3.6.6
go.etcd.io/etcd/client/v3 v3.6.6
go.etcd.io/etcd/api/v3 v3.6.9
go.etcd.io/etcd/client/pkg/v3 v3.6.9
go.etcd.io/etcd/client/v3 v3.6.9
go.uber.org/zap v1.27.1
golang.org/x/exp v0.0.0-20250103183323-7d7fa50e5329
golang.org/x/sync v0.18.0
golang.org/x/sys v0.38.0
golang.org/x/sync v0.20.0
golang.org/x/sys v0.42.0
golang.zx2c4.com/wireguard/wgctrl v0.0.0-20241231184526-a9ab2273dd10
google.golang.org/grpc v1.77.0
google.golang.org/protobuf v1.36.10
k8s.io/api v0.34.2
k8s.io/apimachinery v0.34.2
k8s.io/client-go v0.34.1
k8s.io/klog/v2 v2.130.1
sigs.k8s.io/kind v0.30.0
google.golang.org/grpc v1.79.3
google.golang.org/protobuf v1.36.11
gopkg.in/yaml.v3 v3.0.1
k8s.io/api v0.35.3
k8s.io/apimachinery v0.35.3
k8s.io/client-go v0.35.2
k8s.io/klog/v2 v2.140.0
sigs.k8s.io/kind v0.31.0
sigs.k8s.io/yaml v1.6.0
)
@@ -51,12 +52,13 @@ require (
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/containerd/errdefs v1.0.0 // indirect
github.com/containerd/errdefs/pkg v0.3.0 // indirect
github.com/containerd/log v0.1.0 // indirect
github.com/coreos/go-semver v0.3.1 // indirect
github.com/coreos/go-systemd/v22 v22.5.0 // indirect
github.com/dgryski/go-farm v0.0.0-20240924180020-3414d57e47da // indirect
github.com/distribution/reference v0.6.0 // indirect
github.com/dlclark/regexp2 v1.11.4 // indirect
github.com/docker/go-connections v0.5.0 // indirect
github.com/docker/go-connections v0.6.0 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/eapache/channels v1.1.0 // indirect
github.com/eapache/queue v1.1.0 // indirect
@@ -74,7 +76,7 @@ require (
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/gnostic-models v0.7.0 // indirect
github.com/google/pprof v0.0.0-20250403155104-27863c87afa6 // indirect
github.com/google/pprof v0.0.0-20260115054156-294ebfa9ad83 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/gookit/color v1.6.0 // indirect
github.com/gookit/goutil v0.7.1 // indirect
@@ -95,10 +97,11 @@ require (
github.com/mdlayher/socket v0.5.1 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/sys/sequential v0.6.0 // indirect
github.com/moby/sys/atomicwriter v0.1.0 // indirect
github.com/moby/term v0.5.2 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee // indirect
github.com/morikuni/aec v1.1.0 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.1 // indirect
@@ -115,7 +118,7 @@ require (
github.com/sourcegraph/conc v0.3.0 // indirect
github.com/spf13/afero v1.11.0 // indirect
github.com/spf13/cast v1.7.1 // indirect
github.com/spf13/pflag v1.0.9 // indirect
github.com/spf13/pflag v1.0.10 // indirect
github.com/spf13/viper v1.19.0 // indirect
github.com/subosito/gotenv v1.6.0 // indirect
github.com/tj/go-spin v1.1.0 // indirect
@@ -128,38 +131,37 @@ require (
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 // indirect
go.opentelemetry.io/otel v1.38.0 // indirect
go.opentelemetry.io/otel v1.39.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.37.0 // indirect
go.opentelemetry.io/otel/metric v1.38.0 // indirect
go.opentelemetry.io/otel/trace v1.38.0 // indirect
go.opentelemetry.io/otel/metric v1.39.0 // indirect
go.opentelemetry.io/otel/trace v1.39.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.yaml.in/yaml/v2 v2.4.2 // indirect
go.yaml.in/yaml/v2 v2.4.3 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/crypto v0.45.0 // indirect
golang.org/x/mod v0.29.0 // indirect
golang.org/x/net v0.47.0 // indirect
golang.org/x/oauth2 v0.32.0 // indirect
golang.org/x/term v0.37.0 // indirect
golang.org/x/text v0.31.0 // indirect
golang.org/x/crypto v0.49.0 // indirect
golang.org/x/mod v0.34.0 // indirect
golang.org/x/net v0.52.0 // indirect
golang.org/x/oauth2 v0.36.0 // indirect
golang.org/x/term v0.41.0 // indirect
golang.org/x/text v0.35.0 // indirect
golang.org/x/time v0.9.0 // indirect
golang.org/x/tools v0.38.0 // indirect
golang.org/x/tools v0.43.0 // indirect
golang.zx2c4.com/wireguard v0.0.0-20231211153847-12269c276173 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20251022142026-3a174f9686a8 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20251022142026-3a174f9686a8 // indirect
gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20251202230838-ff82c1b0f217 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217 // indirect
gopkg.in/evanphx/json-patch.v4 v4.13.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/kube-openapi v0.0.0-20250710124328-f3f2b991d03b // indirect
k8s.io/utils v0.0.0-20250604170112-4c0f3b243397 // indirect
k8s.io/kube-openapi v0.0.0-20250910181357-589584f1c912 // indirect
k8s.io/utils v0.0.0-20251002143259-bc988d571ff4 // indirect
modernc.org/cc/v4 v4.24.4 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/opt v0.1.4 // indirect
modernc.org/sortutil v1.2.1 // indirect
modernc.org/strutil v1.2.1 // indirect
modernc.org/token v1.1.0 // indirect
sigs.k8s.io/json v0.0.0-20241014173422-cfa47c3a1cc8 // indirect
sigs.k8s.io/json v0.0.0-20250730193827-2d320260d730 // indirect
sigs.k8s.io/randfill v1.0.0 // indirect
sigs.k8s.io/structured-merge-diff/v6 v6.3.0 // indirect
)

273
go.sum
View File

@@ -2,8 +2,6 @@ al.essio.dev/pkg/shellescape v1.5.1 h1:86HrALUujYS/h+GtqoB26SBEdkWfmMI6FubjXlsXy
al.essio.dev/pkg/shellescape v1.5.1/go.mod h1:6sIqp7X2P6mThCQ7twERpZTuigpr6KbZWtls1U8I890=
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c h1:udKWzYgxTojEKWjV8V+WSxDXJ4NFATAsZjh8iIbsQIg=
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/toml v0.4.1/go.mod h1:CxXYINrC8qIiEnFrOxCa7Jy5BFHlXnUU2pbicEuybxQ=
github.com/BurntSushi/toml v1.4.0 h1:kuoIxZQy2WRRk1pttg9asf+WVv6tWQuBNVmK8+nqPr0=
github.com/BurntSushi/toml v1.4.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
github.com/Masterminds/semver/v3 v3.4.0 h1:Zog+i5UMtVoCU8oKka5P7i9q9HgrJeGzI9SA1Xbatp0=
@@ -16,8 +14,6 @@ github.com/cenkalti/backoff/v5 v5.0.2 h1:rIfFVxEf1QsI7E1ZHfp/B4DF/6QBAUhmgkxc0H7
github.com/cenkalti/backoff/v5 v5.0.2/go.mod h1:rkhZdG3JZukswDf7f0cwqPNk4K0sa+F97BxZthm/crw=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/cilium/ebpf v0.5.0/go.mod h1:4tRaxcgiL706VnOzHOdBlY8IEAIdxINsQBcU4xJJXRs=
github.com/cilium/ebpf v0.7.0/go.mod h1:/oI2+1shJiTGAMgl6/RgJr36Eo1jzrRcAWbcXO2usCA=
github.com/cloudflare/ipvs v0.11.0 h1:niLcbqfv8+RSYk+yI+jhiCIHobEdEhO++mcwCX0Znsw=
github.com/cloudflare/ipvs v0.11.0/go.mod h1:XsbuKcQpqb3rpjYPtsFsM8BNZlgLw+Z0iC6RhocqDd8=
github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI=
@@ -43,8 +39,8 @@ github.com/dlclark/regexp2 v1.11.4 h1:rPYF9/LECdNymJufQKmri9gV604RvvABwgOA8un7yA
github.com/dlclark/regexp2 v1.11.4/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/docker/docker v28.5.2+incompatible h1:DBX0Y0zAjZbSrm1uzOkdr1onVghKaftjlSWt4AFexzM=
github.com/docker/docker v28.5.2+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/go-connections v0.5.0 h1:USnMq7hx7gwdVZq1L49hLXaFtUdTADjXGp+uj1Br63c=
github.com/docker/go-connections v0.5.0/go.mod h1:ov60Kzw0kKElRwhNs9UlUHAE/F9Fe6GLaXnqyDdmEXc=
github.com/docker/go-connections v0.6.0 h1:LlMG9azAe1TqfR7sO+NJttz1gy6KO7VJBh+pMmjSD94=
github.com/docker/go-connections v0.6.0/go.mod h1:AahvXYshr6JgfUJGdDCs2b5EZG/vmaMAntpSFH5BFKE=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
@@ -59,9 +55,8 @@ github.com/evanphx/json-patch/v5 v5.9.0 h1:kcBlZQbplgElYIlo/n1hJbls2z/1awpXxpRi0
github.com/evanphx/json-patch/v5 v5.9.0/go.mod h1:VNkHZ/282BpEyt/tObQO8s5CMPmYYq14uClGH4abBuQ=
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
github.com/florianl/go-conntrack v0.4.0 h1:TlYkxytdwgVayfU0cKwkHurQA0Rd1ZSEBRckRYDUu18=
github.com/florianl/go-conntrack v0.4.0/go.mod h1:iPDx4oIats2T7X7Jm3PFyRCJM1GfZhJaSHOWROYOrE8=
github.com/frankban/quicktest v1.11.3/go.mod h1:wRf/ReqHper53s+kmmSZizM8NamnL3IM0I9ntUbOk+k=
github.com/florianl/go-conntrack v0.7.0 h1:kWbRhLRUqBlmi2ncASmygN5yI281wlpnTUJs69tcs7g=
github.com/florianl/go-conntrack v0.7.0/go.mod h1:7kzfTjtQCRsAVkuornpI8KiwFfa79QmcZXZQex1IXhU=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/fsnotify/fsnotify v1.8.0 h1:dAwr6QBTBZIkG8roQaJjGof0pp0EeF+tNV7YBP3F/8M=
@@ -98,22 +93,17 @@ github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/gnostic-models v0.7.0 h1:qwTtogB15McXDaNqTZdzPJRHvaVJlAl+HVQnLmJEJxo=
github.com/google/gnostic-models v0.7.0/go.mod h1:whL5G0m6dmc5cPxKc5bdKdEN3UjI7OUGxBlw57miDrQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.7/go.mod h1:n+brtR0CgQNWTVd5ZUFpTBC8YFBDLK/h/bpaJ8/DtOE=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/go-containerregistry v0.20.6 h1:cvWX87UxxLgaH76b4hIvya6Dzz9qHB31qAwjAohdSTU=
github.com/google/go-containerregistry v0.20.6/go.mod h1:T0x8MuoAoKX/873bkeSfLD2FAkwCDf9/HZgsFJ02E2Y=
github.com/google/go-containerregistry v0.21.3 h1:Xr+yt3VvwOOn/5nJzd7UoOhwPGiPkYW0zWDLLUXqAi4=
github.com/google/go-containerregistry v0.21.3/go.mod h1:D5ZrJF1e6dMzvInpBPuMCX0FxURz7GLq2rV3Us9aPkc=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/nftables v0.3.0 h1:bkyZ0cbpVeMHXOrtlFc8ISmfVqq5gPJukoYieyVmITg=
github.com/google/nftables v0.3.0/go.mod h1:BCp9FsrbF1Fn/Yu6CLUc9GGZFw/+hsxfluNXXmxBfRM=
github.com/google/pprof v0.0.0-20250403155104-27863c87afa6 h1:BHT72Gu3keYf3ZEu2J0b1vyeLSOYI8bm5wbJM/8yDe8=
github.com/google/pprof v0.0.0-20250403155104-27863c87afa6/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=
github.com/google/pprof v0.0.0-20260115054156-294ebfa9ad83 h1:z2ogiKUYzX5Is6zr/vP9vJGqPwcdqsWjOt+V8J7+bTc=
github.com/google/pprof v0.0.0-20260115054156-294ebfa9ad83/go.mod h1:MxpfABSjhmINe3F1It9d+8exIHFvUqtLIRCdOGNXqiI=
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 h1:El6M4kTTCOh6aBiKaUGG7oYTSPP8MxqL4YI3kZKwcP4=
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510/go.mod h1:pupxD2MaaD3pAXIBCelhxNneeOaAeabZDe5s4K6zSpQ=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
@@ -142,35 +132,23 @@ github.com/insomniacslk/dhcp v0.0.0-20241224095048-b56fa0d5f25d h1:VkCNWh6tuQLgD
github.com/insomniacslk/dhcp v0.0.0-20241224095048-b56fa0d5f25d/go.mod h1:VvGYjkZoJyKqlmT1yzakUs4mfKMNB0XdODP0+rdml6k=
github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY=
github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=
github.com/josharian/native v0.0.0-20200817173448-b6b71def0850/go.mod h1:7X/raswPFr05uY3HiLlYeyQntB6OO7E/d2Cu7qoaN2w=
github.com/josharian/native v1.0.0/go.mod h1:7X/raswPFr05uY3HiLlYeyQntB6OO7E/d2Cu7qoaN2w=
github.com/josharian/native v1.1.0 h1:uuaP0hAbW7Y4l0ZRQ6C9zfb7Mg1mbFKry/xzDAfmtLA=
github.com/josharian/native v1.1.0/go.mod h1:7X/raswPFr05uY3HiLlYeyQntB6OO7E/d2Cu7qoaN2w=
github.com/joshdk/go-junit v1.0.0 h1:S86cUKIdwBHWwA6xCmFlf3RTLfVXYQfvanM5Uh+K6GE=
github.com/joshdk/go-junit v1.0.0/go.mod h1:TiiV0PqkaNfFXjEiyjWM3XXrhVyCa1K4Zfga6W52ung=
github.com/jpillora/backoff v1.0.0 h1:uvFg412JmmHBHw7iwprIxkPMI+sGQ4kzOWsMeHnm2EA=
github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4=
github.com/jsimonetti/rtnetlink v0.0.0-20190606172950-9527aa82566a/go.mod h1:Oz+70psSo5OFh8DBl0Zv2ACw7Esh6pPUphlvZG9x7uw=
github.com/jsimonetti/rtnetlink v0.0.0-20200117123717-f846d4f6c1f4/go.mod h1:WGuG/smIU4J/54PblvSbh+xvCZmpJnFgr3ds6Z55XMQ=
github.com/jsimonetti/rtnetlink v0.0.0-20201009170750-9c6f07d100c1/go.mod h1:hqoO/u39cqLeBLebZ8fWdE96O7FxrAsRYhnVOdgHxok=
github.com/jsimonetti/rtnetlink v0.0.0-20201216134343-bde56ed16391/go.mod h1:cR77jAZG3Y3bsb8hF6fHJbFoyFukLFOkQ98S0pQz3xw=
github.com/jsimonetti/rtnetlink v0.0.0-20201220180245-69540ac93943/go.mod h1:z4c53zj6Eex712ROyh8WI0ihysb5j2ROyV42iNogmAs=
github.com/jsimonetti/rtnetlink v0.0.0-20210122163228-8d122574c736/go.mod h1:ZXpIyOK59ZnN7J0BV99cZUPmsqDRZ3eq5X+st7u/oSA=
github.com/jsimonetti/rtnetlink v0.0.0-20210212075122-66c871082f2b/go.mod h1:8w9Rh8m+aHZIG69YPGGem1i5VzoyRC8nw2kA8B+ik5U=
github.com/jsimonetti/rtnetlink v0.0.0-20210525051524-4cc836578190/go.mod h1:NmKSdU4VGSiv1bMsdqNALI4RSvvjtz65tTMCnD05qLo=
github.com/jsimonetti/rtnetlink v0.0.0-20211022192332-93da33804786/go.mod h1:v4hqbTdfQngbVSZJVWUhGE/lbTFf9jb+ygmNUDQMuOs=
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/k-sone/critbitgo v1.4.0 h1:l71cTyBGeh6X5ATh6Fibgw3+rtNT80BA0uNNWgkPrbE=
github.com/k-sone/critbitgo v1.4.0/go.mod h1:7E6pyoyADnFxlUBEKcnfS49b7SUAQGMK+OAp/UQvo0s=
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/klauspost/compress v1.18.4 h1:RPhnKRAQ4Fh8zU2FY/6ZFDwTVTxgJ/EMydqSTzE9a2c=
github.com/klauspost/compress v1.18.4/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
@@ -183,31 +161,16 @@ github.com/maruel/natural v1.1.1 h1:Hja7XhhmvEFhcByqDoHz9QZbkWey+COd9xWfCfn1ioo=
github.com/maruel/natural v1.1.1/go.mod h1:v+Rfd79xlw1AgVBjbO0BEQmptqb5HvL/k9GRHB7ZKEg=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mdlayher/ethtool v0.0.0-20210210192532-2b88debcdd43/go.mod h1:+t7E0lkKfbBsebllff1xdTmyJt8lH37niI6kwFk9OTo=
github.com/mdlayher/ethtool v0.0.0-20211028163843-288d040e9d60/go.mod h1:aYbhishWc4Ai3I2U4Gaa2n3kHWSwzme6EsG/46HRQbE=
github.com/mdlayher/genetlink v1.0.0/go.mod h1:0rJ0h4itni50A86M2kHcgS85ttZazNt7a8H2a2cw0Gc=
github.com/mdlayher/genetlink v1.3.2 h1:KdrNKe+CTu+IbZnm/GVUMXSqBBLqcGpRDa0xkQy56gw=
github.com/mdlayher/genetlink v1.3.2/go.mod h1:tcC3pkCrPUGIKKsCsp0B3AdaaKuHtaxoJRz3cc+528o=
github.com/mdlayher/ndp v1.1.0 h1:QylGKGVtH60sKZUE88+IW5ila1Z/M9/OXhWdsVKuscs=
github.com/mdlayher/ndp v1.1.0/go.mod h1:FmgESgemgjl38vuOIyAHWUUL6vQKA/pQNkvXdWsdQFM=
github.com/mdlayher/netlink v0.0.0-20190409211403-11939a169225/go.mod h1:eQB3mZE4aiYnlUsyGGCOpPETfdQq4Jhsgf1fk3cwQaA=
github.com/mdlayher/netlink v1.0.0/go.mod h1:KxeJAFOFLG6AjpyDkQ/iIhxygIUKD+vcwqcnu43w/+M=
github.com/mdlayher/netlink v1.1.0/go.mod h1:H4WCitaheIsdF9yOYu8CFmCgQthAPIWZmcKp9uZHgmY=
github.com/mdlayher/netlink v1.1.1/go.mod h1:WTYpFb/WTvlRJAyKhZL5/uy69TDDpHHu2VZmb2XgV7o=
github.com/mdlayher/netlink v1.2.0/go.mod h1:kwVW1io0AZy9A1E2YYgaD4Cj+C+GPkU6klXCMzIJ9p8=
github.com/mdlayher/netlink v1.2.1/go.mod h1:bacnNlfhqHqqLo4WsYeXSqfyXkInQ9JneWI68v1KwSU=
github.com/mdlayher/netlink v1.2.2-0.20210123213345-5cc92139ae3e/go.mod h1:bacnNlfhqHqqLo4WsYeXSqfyXkInQ9JneWI68v1KwSU=
github.com/mdlayher/netlink v1.3.0/go.mod h1:xK/BssKuwcRXHrtN04UBkwQ6dY9VviGGuriDdoPSWys=
github.com/mdlayher/netlink v1.4.0/go.mod h1:dRJi5IABcZpBD2A3D0Mv/AiX8I9uDEu5oGkAVrekmf8=
github.com/mdlayher/netlink v1.4.1/go.mod h1:e4/KuJ+s8UhfUpO9z00/fDZZmhSrs+oxyqAS9cNgn6Q=
github.com/mdlayher/netlink v1.5.0/go.mod h1:1Kr8BBFxGyUyNmztC9WLOayqYVAd2wsgOZm18nqGuzQ=
github.com/mdlayher/netlink v1.6.2/go.mod h1:O1HXX2sIWSMJ3Qn1BYZk1yZM+7iMki/uYGGiwGyq/iU=
github.com/mdlayher/netlink v1.7.3-0.20250113171957-fbb4dce95f42 h1:A1Cq6Ysb0GM0tpKMbdCXCIfBclan4oHk1Jb+Hrejirg=
github.com/mdlayher/netlink v1.7.3-0.20250113171957-fbb4dce95f42/go.mod h1:BB4YCPDOzfy7FniQ/lxuYQ3dgmM2cZumHbK8RpTjN2o=
github.com/mdlayher/packet v1.1.2 h1:3Up1NG6LZrsgDVn6X4L9Ge/iyRyxFEFD9o6Pr3Q1nQY=
github.com/mdlayher/packet v1.1.2/go.mod h1:GEu1+n9sG5VtiRE4SydOmX5GTwyyYlteZiFU+x0kew4=
github.com/mdlayher/socket v0.0.0-20210307095302-262dc9984e00/go.mod h1:GAFlyu4/XV68LkQKYzKhIo/WW7j3Zi0YRAz/BOoanUc=
github.com/mdlayher/socket v0.0.0-20211007213009-516dcbdf0267/go.mod h1:nFZ1EtZYK8Gi/k6QNu7z7CgO20i/4ExeQswwWuPmG/g=
github.com/mdlayher/socket v0.1.0/go.mod h1:mYV5YIZAfHh4dzDVzI8x8tWLWCliuX8Mon5Awbj+qDs=
github.com/mdlayher/socket v0.2.3/go.mod h1:bz12/FozYNH/VbvC3q7TRIK/Y6dH1kCKsXaUeXi/FmY=
github.com/mdlayher/socket v0.5.1 h1:VZaqt6RkGkt2OE9l3GcC6nZkqD3xKeQLyfleW/uBcos=
github.com/mdlayher/socket v0.5.1/go.mod h1:TjPLHI1UgwEv5J1B5q0zTZq12A/6H7nKmtTanQE37IQ=
github.com/mfridman/tparse v0.18.0 h1:wh6dzOKaIwkUGyKgOntDW4liXSo37qg5AXbIhkMV3vE=
@@ -230,14 +193,14 @@ github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJ
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee h1:W5t00kpgFdJifH4BDsTlE89Zl93FEloxaWZfGcifgq8=
github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
github.com/morikuni/aec v1.1.0 h1:vBBl0pUnvi/Je71dsRrhMBtreIqNMYErSAbEeb8jrXQ=
github.com/morikuni/aec v1.1.0/go.mod h1:xDRgiq/iw5l+zkao76YTKzKttOp2cwPEne25HDkJnBw=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/onsi/ginkgo/v2 v2.27.2 h1:LzwLj0b89qtIy6SSASkzlNvX6WktqurSHwkk2ipF/Ns=
github.com/onsi/ginkgo/v2 v2.27.2/go.mod h1:ArE1D/XhNXBXCBkKOLkbsb2c81dQHCRcF5zwn/ykDRo=
github.com/onsi/gomega v1.38.2 h1:eZCjf2xjZAqe+LeWvKb5weQ+NcPwX84kqJ0cZNxok2A=
github.com/onsi/gomega v1.38.2/go.mod h1:W2MJcYxRGV63b418Ai34Ud0hEdTVXq9NW9+Sx6uXf3k=
github.com/onsi/ginkgo/v2 v2.28.1 h1:S4hj+HbZp40fNKuLUQOYLDgZLwNUVn19N3Atb98NCyI=
github.com/onsi/ginkgo/v2 v2.28.1/go.mod h1:CLtbVInNckU3/+gC8LzkGUb9oF+e8W8TdUsxPwvdOgE=
github.com/onsi/gomega v1.39.1 h1:1IJLAad4zjPn2PsnhH70V4DKRFlrCzGBNrNaru+Vf28=
github.com/onsi/gomega v1.39.1/go.mod h1:hL6yVALoTOxeWudERyfppUcZXjMwIMLnuSfruD2lcfg=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040=
@@ -274,25 +237,25 @@ github.com/sagikazarmark/locafero v0.6.0 h1:ON7AQg37yzcRPU69mt7gwhFEBwxI6P9T4Qu3
github.com/sagikazarmark/locafero v0.6.0/go.mod h1:77OmuIc6VTraTXKXIs/uvUxKGUXjE1GbemJYHqdNjX0=
github.com/sagikazarmark/slog-shim v0.1.0 h1:diDBnUNK9N/354PgrxMywXnAwEr1QZcOr6gto+ugjYE=
github.com/sagikazarmark/slog-shim v0.1.0/go.mod h1:SrcSrq8aKtyuqEI1uvTDTK1arOWRIczQRv+GVI1AkeQ=
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/sirupsen/logrus v1.9.4 h1:TsZE7l11zFCLZnZ+teH4Umoq5BhEIfIzfRDZ1Uzql2w=
github.com/sirupsen/logrus v1.9.4/go.mod h1:ftWc9WdOfJ0a92nsE2jF5u5ZwH8Bv2zdeOC42RjbV2g=
github.com/sourcegraph/conc v0.3.0 h1:OQTbbt6P72L20UqAkXXuLOj79LfEanQ+YQFNpLA9ySo=
github.com/sourcegraph/conc v0.3.0/go.mod h1:Sdozi7LEKbFPqYX2/J+iBAM6HpqSLTASQIKqDmF7Mt0=
github.com/spf13/afero v1.11.0 h1:WJQKhtpdm3v2IzqG8VMqrr6Rf3UYpEF239Jy9wNepM8=
github.com/spf13/afero v1.11.0/go.mod h1:GH9Y3pIexgf1MTIWtNGyogA5MwRIDXGUr+hbWNoBjkY=
github.com/spf13/cast v1.7.1 h1:cuNEagBQEHWN1FnbGEjCXL2szYEXqfJPbP2HNUaca9Y=
github.com/spf13/cast v1.7.1/go.mod h1:ancEpBxwJDODSW/UG4rDrAqiKolqNNh2DX3mk86cAdo=
github.com/spf13/cobra v1.10.1 h1:lJeBwCfmrnXthfAupyUTzJ/J4Nc1RsHC/mSRU2dll/s=
github.com/spf13/cobra v1.10.1/go.mod h1:7SmJGaTHFVBY0jW4NXGluQoLvhqFQM+6XSKD+P4XaB0=
github.com/spf13/pflag v1.0.9 h1:9exaQaMOCwffKiiiYk6/BndUBv+iRViNW+4lEMi0PvY=
github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU=
github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4=
github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/viper v1.19.0 h1:RWq5SEjt8o25SROyN3z2OrDB9l7RPd3lwTWU8EcEdcI=
github.com/spf13/viper v1.19.0/go.mod h1:GQUN9bilAbhU/jgc1bKs99f/suXKeUMct8Adx5+Ntkg=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
@@ -325,31 +288,30 @@ github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavM
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.4.0/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
go.etcd.io/etcd/api/v3 v3.6.6 h1:mcaMp3+7JawWv69p6QShYWS8cIWUOl32bFLb6qf8pOQ=
go.etcd.io/etcd/api/v3 v3.6.6/go.mod h1:f/om26iXl2wSkcTA1zGQv8reJRSLVdoEBsi4JdfMrx4=
go.etcd.io/etcd/client/pkg/v3 v3.6.6 h1:uoqgzSOv2H9KlIF5O1Lsd8sW+eMLuV6wzE3q5GJGQNs=
go.etcd.io/etcd/client/pkg/v3 v3.6.6/go.mod h1:YngfUVmvsvOJ2rRgStIyHsKtOt9SZI2aBJrZiWJhCbI=
go.etcd.io/etcd/client/v3 v3.6.6 h1:G5z1wMf5B9SNexoxOHUGBaULurOZPIgGPsW6CN492ec=
go.etcd.io/etcd/client/v3 v3.6.6/go.mod h1:36Qv6baQ07znPR3+n7t+Rk5VHEzVYPvFfGmfF4wBHV8=
go.etcd.io/etcd/api/v3 v3.6.9 h1:UA7iKfEW1AzgihcBSGXci2kDGQiokSq41F9HMCI/RTI=
go.etcd.io/etcd/api/v3 v3.6.9/go.mod h1:csEk/qTfxKL36NqJdU15Tgtl65A8dyEY2BYo7PRsIwk=
go.etcd.io/etcd/client/pkg/v3 v3.6.9 h1:T8nuk8Lz64C+Hzb0coBFLMSlVSQZBpAtFk46swdM1DA=
go.etcd.io/etcd/client/pkg/v3 v3.6.9/go.mod h1:WEy3PpwbbEBVRdh1NVJYsuUe/8eyI21PNJRazeD8z/Y=
go.etcd.io/etcd/client/v3 v3.6.9 h1:3X555hQXmhRr27O37wls53g68CpUiPOiHXrZfz2Al+o=
go.etcd.io/etcd/client/v3 v3.6.9/go.mod h1:KO7H1HLYh1qaljuVZJQwBFk1lRce6pJzt+C81GEnrlM=
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 h1:F7Jx+6hwnZ41NSFTO5q4LYDtJRXBf2PD0rNBkeB/lus=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0/go.mod h1:UHB22Z8QsdRDrnAtX4PntOl36ajSxcdUMt1sF7Y6E7Q=
go.opentelemetry.io/otel v1.38.0 h1:RkfdswUDRimDg0m2Az18RKOsnI8UDzppJAtj01/Ymk8=
go.opentelemetry.io/otel v1.38.0/go.mod h1:zcmtmQ1+YmQM9wrNsTGV/q/uyusom3P8RxwExxkZhjM=
go.opentelemetry.io/otel v1.39.0 h1:8yPrr/S0ND9QEfTfdP9V+SiwT4E0G7Y5MO7p85nis48=
go.opentelemetry.io/otel v1.39.0/go.mod h1:kLlFTywNWrFyEdH0oj2xK0bFYZtHRYUdv1NklR/tgc8=
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.37.0 h1:Ahq7pZmv87yiyn3jeFz/LekZmPLLdKejuO3NcK9MssM=
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.37.0/go.mod h1:MJTqhM0im3mRLw1i8uGHnCvUEeS7VwRyxlLC78PA18M=
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.37.0 h1:bDMKF3RUSxshZ5OjOTi8rsHGaPKsAt76FaqgvIUySLc=
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.37.0/go.mod h1:dDT67G/IkA46Mr2l9Uj7HsQVwsjASyV9SjGofsiUZDA=
go.opentelemetry.io/otel/metric v1.38.0 h1:Kl6lzIYGAh5M159u9NgiRkmoMKjvbsKtYRwgfrA6WpA=
go.opentelemetry.io/otel/metric v1.38.0/go.mod h1:kB5n/QoRM8YwmUahxvI3bO34eVtQf2i4utNVLr9gEmI=
go.opentelemetry.io/otel/sdk v1.38.0 h1:l48sr5YbNf2hpCUj/FoGhW9yDkl+Ma+LrVl8qaM5b+E=
go.opentelemetry.io/otel/sdk v1.38.0/go.mod h1:ghmNdGlVemJI3+ZB5iDEuk4bWA3GkTpW+DOoZMYBVVg=
go.opentelemetry.io/otel/sdk/metric v1.38.0 h1:aSH66iL0aZqo//xXzQLYozmWrXxyFkBJ6qT5wthqPoM=
go.opentelemetry.io/otel/sdk/metric v1.38.0/go.mod h1:dg9PBnW9XdQ1Hd6ZnRz689CbtrUp0wMMs9iPcgT9EZA=
go.opentelemetry.io/otel/trace v1.38.0 h1:Fxk5bKrDZJUH+AMyyIXGcFAPah0oRcT+LuNtJrmcNLE=
go.opentelemetry.io/otel/trace v1.38.0/go.mod h1:j1P9ivuFsTceSWe1oY+EeW3sc+Pp42sO++GHkg4wwhs=
go.opentelemetry.io/otel/metric v1.39.0 h1:d1UzonvEZriVfpNKEVmHXbdf909uGTOQjA0HF0Ls5Q0=
go.opentelemetry.io/otel/metric v1.39.0/go.mod h1:jrZSWL33sD7bBxg1xjrqyDjnuzTUB0x1nBERXd7Ftcs=
go.opentelemetry.io/otel/sdk v1.39.0 h1:nMLYcjVsvdui1B/4FRkwjzoRVsMK8uL/cj0OyhKzt18=
go.opentelemetry.io/otel/sdk v1.39.0/go.mod h1:vDojkC4/jsTJsE+kh+LXYQlbL8CgrEcwmt1ENZszdJE=
go.opentelemetry.io/otel/sdk/metric v1.39.0 h1:cXMVVFVgsIf2YL6QkRF4Urbr/aMInf+2WKg+sEJTtB8=
go.opentelemetry.io/otel/sdk/metric v1.39.0/go.mod h1:xq9HEVH7qeX69/JnwEfp6fVq5wosJsY1mt4lLfYdVew=
go.opentelemetry.io/otel/trace v1.39.0 h1:2d2vfpEDmCJ5zVYz7ijaJdOF59xLomrvj7bjt6/qCJI=
go.opentelemetry.io/otel/trace v1.39.0/go.mod h1:88w4/PnZSazkGzz/w84VHpQafiU4EtqqlVdxWy+rNOA=
go.opentelemetry.io/proto/otlp v1.7.0 h1:jX1VolD6nHuFzOYso2E73H85i92Mv8JQYk0K9vz09os=
go.opentelemetry.io/proto/otlp v1.7.0/go.mod h1:fSKjH6YJ7HDlwzltzyMj036AJ3ejJLCgCSHGj4efDDo=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
@@ -358,103 +320,65 @@ go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/zap v1.27.1 h1:08RqriUEv8+ArZRYSTXy1LeBScaMpVSTBhCeaZYfMYc=
go.uber.org/zap v1.27.1/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI=
go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU=
go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0=
go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8=
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q=
golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4=
golang.org/x/crypto v0.49.0 h1:+Ng2ULVvLHnJ/ZFEq4KdcDd/cfjrrjjNSXNzxg0Y4U4=
golang.org/x/crypto v0.49.0/go.mod h1:ErX4dUh2UM+CFYiXZRTcMpEcN8b/1gxEuv3nODoYtCA=
golang.org/x/exp v0.0.0-20250103183323-7d7fa50e5329 h1:9kj3STMvgqy3YA4VQXBrN7925ICMxD5wzMRcgA30588=
golang.org/x/exp v0.0.0-20250103183323-7d7fa50e5329/go.mod h1:qj5a5QZpwLU2NLQudwIN5koi3beDhSAlJwa67PuM98c=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.5.1/go.mod h1:5OXOZSfqPIIbmVBIIKWRFfZjPR0E5r58TLhUjH0a2Ro=
golang.org/x/mod v0.29.0 h1:HV8lRxZC4l2cr3Zq1LvtOsi/ThTgWnUk/y64QSs8GwA=
golang.org/x/mod v0.29.0/go.mod h1:NyhrlYXJ2H4eJiRy/WDBO6HMqZQ6q9nk4JzS3NuCK+w=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/mod v0.34.0 h1:xIHgNUUnW6sYkcM5Jleh05DvLOtwc6RitGHbDk4akRI=
golang.org/x/mod v0.34.0/go.mod h1:ykgH52iCZe79kzLLMhyCUzhMci+nQj+0XkbXpNYtVjY=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190827160401-ba9fcec4b297/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20191007182048-72f939374954/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20201010224723-4f7140c49acb/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201216054612-986b41b23924/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20201224014010-6772e930b67b/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20210805182204-aaa1db679c0d/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20210928044308-7d9f5e0b762b/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20211020060615-d418f374d309/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20211209124913-491a49abca63/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20220107192237-5cfca573fb4d/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.47.0 h1:Mx+4dIFzqraBXUugkia1OOvlD6LemFo1ALMHjrXDOhY=
golang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU=
golang.org/x/oauth2 v0.32.0 h1:jsCblLleRMDrxMN29H3z/k1KliIvpLgCkE6R8FXXNgY=
golang.org/x/oauth2 v0.32.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA=
golang.org/x/net v0.0.0-20220923203811-8be639271d50/go.mod h1:YDH+HFinaLZZlnHAfSS6ZXJJ9M9t4Dl22yv3iI2vPwk=
golang.org/x/net v0.52.0 h1:He/TN1l0e4mmR3QqHMT2Xab3Aj3L9qjbhRm78/6jrW0=
golang.org/x/net v0.52.0/go.mod h1:R1MAz7uMZxVMualyPXb+VaqGSa3LIaUqk0eEt3w36Sw=
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.18.0 h1:kr88TuHDroi+UVf+0hZnirlk8o8T+4MrK6mr60WkH/I=
golang.org/x/sync v0.18.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/sync v0.0.0-20220923202941-7f9b1623fab7/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190411185658-b44545bcd369/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190826190057-c7b8b68b1456/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191008105621-543471e840be/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201009025420-dfb3f7c4e634/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201118182958-a01c418693c7/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201218084310-7d0127a74742/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210110051926-789bb1bd4061/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210123111255-9b0068b26619/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210216163648-f7da38b97c65/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210305230114-8fe3ee5dd75b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210525143221-35b2ab0089ea/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210906170528-6f6e22806c34/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211025201205-69cdffdb9359/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211210111614-af8b64212486/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220319134239-a9b59b0215f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220728004956-3c1f35247d10/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.10.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc=
golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.37.0 h1:8EGAD0qCmHYZg6J17DvsMy9/wJ7/D/4pV/wfnld5lTU=
golang.org/x/term v0.37.0/go.mod h1:5pB4lxRNYYVZuTLmy8oR2BH8dflOR+IbTYFD8fi3254=
golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo=
golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.41.0 h1:QCgPso/Q3RTJx2Th4bDLqML4W6iJiaXFq2/ftQF13YU=
golang.org/x/term v0.41.0/go.mod h1:3pfBgksrReYfZ5lvYM0kSO0LIkAl4Yl2bXOkKP7Ec2A=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM=
golang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.35.0 h1:JOVx6vVDFokkpaq1AEptVzLTpDe9KGpj5tR4/X+ybL8=
golang.org/x/text v0.35.0/go.mod h1:khi/HExzZJ2pGnjenulevKNX1W67CUy0AsXcNubPGCA=
golang.org/x/time v0.9.0 h1:EsRrnYcQiGH+5FfbgvV4AP7qEZstoyrHB0DzarOQ4ZY=
golang.org/x/time v0.9.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=
golang.org/x/tools v0.1.7/go.mod h1:LGqMHiF4EqQNHR1JncWGqT5BVaXmza+X+BDGol+dOxo=
golang.org/x/tools v0.38.0 h1:Hx2Xv8hISq8Lm16jvBZ2VQf+RLmbd7wVUsALibYI/IQ=
golang.org/x/tools v0.38.0/go.mod h1:yEsQ/d/YK8cjh0L6rZlY8tgtlKiBNTL14pGDJPJpYQs=
golang.org/x/tools v0.43.0 h1:12BdW9CeB3Z+J/I/wj34VMl8X+fEXBxVR90JeMX5E7s=
golang.org/x/tools v0.43.0/go.mod h1:uHkMso649BX2cZK6+RpuIPXS3ho2hZo4FVwfoy1vIk0=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@@ -465,44 +389,41 @@ golang.zx2c4.com/wireguard/wgctrl v0.0.0-20241231184526-a9ab2273dd10 h1:3GDAcqdI
golang.zx2c4.com/wireguard/wgctrl v0.0.0-20241231184526-a9ab2273dd10/go.mod h1:T97yPqesLiNrOYxkwmhMI0ZIlJDm+p0PMR8eRVeR5tQ=
gonum.org/v1/gonum v0.16.0 h1:5+ul4Swaf3ESvrOnidPp4GZbzf0mxVQpDCYUQE7OJfk=
gonum.org/v1/gonum v0.16.0/go.mod h1:fef3am4MQ93R2HHpKnLk4/Tbh/s0+wqD5nfa6Pnwy4E=
google.golang.org/genproto/googleapis/api v0.0.0-20251022142026-3a174f9686a8 h1:mepRgnBZa07I4TRuomDE4sTIYieg/osKmzIf4USdWS4=
google.golang.org/genproto/googleapis/api v0.0.0-20251022142026-3a174f9686a8/go.mod h1:fDMmzKV90WSg1NbozdqrE64fkuTv6mlq2zxo9ad+3yo=
google.golang.org/genproto/googleapis/rpc v0.0.0-20251022142026-3a174f9686a8 h1:M1rk8KBnUsBDg1oPGHNCxG4vc1f49epmTO7xscSajMk=
google.golang.org/genproto/googleapis/rpc v0.0.0-20251022142026-3a174f9686a8/go.mod h1:7i2o+ce6H/6BluujYR+kqX3GKH+dChPTQU19wjRPiGk=
google.golang.org/grpc v1.77.0 h1:wVVY6/8cGA6vvffn+wWK5ToddbgdU3d8MNENr4evgXM=
google.golang.org/grpc v1.77.0/go.mod h1:z0BY1iVj0q8E1uSQCjL9cppRj+gnZjzDnzV0dHhrNig=
google.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE=
google.golang.org/protobuf v1.36.10/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
google.golang.org/genproto/googleapis/api v0.0.0-20251202230838-ff82c1b0f217 h1:fCvbg86sFXwdrl5LgVcTEvNC+2txB5mgROGmRL5mrls=
google.golang.org/genproto/googleapis/api v0.0.0-20251202230838-ff82c1b0f217/go.mod h1:+rXWjjaukWZun3mLfjmVnQi18E1AsFbDN9QdJ5YXLto=
google.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217 h1:gRkg/vSppuSQoDjxyiGfN4Upv/h/DQmIR10ZU8dh4Ww=
google.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217/go.mod h1:7i2o+ce6H/6BluujYR+kqX3GKH+dChPTQU19wjRPiGk=
google.golang.org/grpc v1.79.3 h1:sybAEdRIEtvcD68Gx7dmnwjZKlyfuc61Dyo9pGXXkKE=
google.golang.org/grpc v1.79.3/go.mod h1:KmT0Kjez+0dde/v2j9vzwoAScgEPx/Bw1CYChhHLrHQ=
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/evanphx/json-patch.v4 v4.12.0 h1:n6jtcsulIzXPJaxegRbvFNNrZDjbij7ny3gmSPG+6V4=
gopkg.in/evanphx/json-patch.v4 v4.12.0/go.mod h1:p8EYWUEYMpynmqDbY58zCKCFZw8pRWMG4EsWvDvM72M=
gopkg.in/evanphx/json-patch.v4 v4.13.0 h1:czT3CmqEaQ1aanPc5SdlgQrrEIb8w/wwCvWWnfEbYzo=
gopkg.in/evanphx/json-patch.v4 v4.13.0/go.mod h1:p8EYWUEYMpynmqDbY58zCKCFZw8pRWMG4EsWvDvM72M=
gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc=
gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw=
gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=
gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gotest.tools/v3 v3.4.0 h1:ZazjZUfuVeZGLAmlKKuyv3IKP5orXcwtOwDQH6YVr6o=
gotest.tools/v3 v3.4.0/go.mod h1:CtbdzLSsqVhDgMtKsx03ird5YTGB3ar27v0u/yKBW5g=
honnef.co/go/tools v0.2.1/go.mod h1:lPVVZ2BS5TfnjLyizF7o7hv7j9/L+8cZY2hLyjP9cGY=
honnef.co/go/tools v0.2.2/go.mod h1:lPVVZ2BS5TfnjLyizF7o7hv7j9/L+8cZY2hLyjP9cGY=
k8s.io/api v0.34.2 h1:fsSUNZhV+bnL6Aqrp6O7lMTy6o5x2C4XLjnh//8SLYY=
k8s.io/api v0.34.2/go.mod h1:MMBPaWlED2a8w4RSeanD76f7opUoypY8TFYkSM+3XHw=
k8s.io/apimachinery v0.34.2 h1:zQ12Uk3eMHPxrsbUJgNF8bTauTVR2WgqJsTmwTE/NW4=
k8s.io/apimachinery v0.34.2/go.mod h1:/GwIlEcWuTX9zKIg2mbw0LRFIsXwrfoVxn+ef0X13lw=
k8s.io/client-go v0.34.1 h1:ZUPJKgXsnKwVwmKKdPfw4tB58+7/Ik3CrjOEhsiZ7mY=
k8s.io/client-go v0.34.1/go.mod h1:kA8v0FP+tk6sZA0yKLRG67LWjqufAoSHA2xVGKw9Of8=
k8s.io/klog/v2 v2.130.1 h1:n9Xl7H1Xvksem4KFG4PYbdQCQxqc/tTUyrgXaOhHSzk=
k8s.io/klog/v2 v2.130.1/go.mod h1:3Jpz1GvMt720eyJH1ckRHK1EDfpxISzJ7I9OYgaDtPE=
k8s.io/kube-openapi v0.0.0-20250710124328-f3f2b991d03b h1:MloQ9/bdJyIu9lb1PzujOPolHyvO06MXG5TUIj2mNAA=
k8s.io/kube-openapi v0.0.0-20250710124328-f3f2b991d03b/go.mod h1:UZ2yyWbFTpuhSbFhv24aGNOdoRdJZgsIObGBUaYVsts=
k8s.io/utils v0.0.0-20250604170112-4c0f3b243397 h1:hwvWFiBzdWw1FhfY1FooPn3kzWuJ8tmbZBHi4zVsl1Y=
k8s.io/utils v0.0.0-20250604170112-4c0f3b243397/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0=
k8s.io/api v0.35.3 h1:pA2fiBc6+N9PDf7SAiluKGEBuScsTzd2uYBkA5RzNWQ=
k8s.io/api v0.35.3/go.mod h1:9Y9tkBcFwKNq2sxwZTQh1Njh9qHl81D0As56tu42GA4=
k8s.io/apimachinery v0.35.3 h1:MeaUwQCV3tjKP4bcwWGgZ/cp/vpsRnQzqO6J6tJyoF8=
k8s.io/apimachinery v0.35.3/go.mod h1:jQCgFZFR1F4Ik7hvr2g84RTJSZegBc8yHgFWKn//hns=
k8s.io/client-go v0.35.2 h1:YUfPefdGJA4aljDdayAXkc98DnPkIetMl4PrKX97W9o=
k8s.io/client-go v0.35.2/go.mod h1:4QqEwh4oQpeK8AaefZ0jwTFJw/9kIjdQi0jpKeYvz7g=
k8s.io/klog/v2 v2.140.0 h1:Tf+J3AH7xnUzZyVVXhTgGhEKnFqye14aadWv7bzXdzc=
k8s.io/klog/v2 v2.140.0/go.mod h1:o+/RWfJ6PwpnFn7OyAG3QnO47BFsymfEfrz6XyYSSp0=
k8s.io/kube-openapi v0.0.0-20250910181357-589584f1c912 h1:Y3gxNAuB0OBLImH611+UDZcmKS3g6CthxToOb37KgwE=
k8s.io/kube-openapi v0.0.0-20250910181357-589584f1c912/go.mod h1:kdmbQkyfwUagLfXIad1y2TdrjPFWp2Q89B3qkRwf/pQ=
k8s.io/utils v0.0.0-20251002143259-bc988d571ff4 h1:SjGebBtkBqHFOli+05xYbK8YF1Dzkbzn+gDM4X9T4Ck=
k8s.io/utils v0.0.0-20251002143259-bc988d571ff4/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0=
modernc.org/cc/v4 v4.24.4 h1:TFkx1s6dCkQpd6dKurBNmpo+G8Zl4Sq/ztJ+2+DEsh0=
modernc.org/cc/v4 v4.24.4/go.mod h1:uVtb5OGqUKpoLWhqwNQo/8LwvoiEBLvZXIQ/SmO6mL0=
modernc.org/ccorpus2 v1.5.2 h1:Ui+4tc58mf/W+2arcYCJR903y3zl3ecsI7Fpaaqozyw=
@@ -519,10 +440,10 @@ modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
pgregory.net/rapid v1.1.0 h1:CMa0sjHSru3puNx+J0MIAuiiEV4N0qj8/cMWGBBCsjw=
pgregory.net/rapid v1.1.0/go.mod h1:PY5XlDGj0+V1FCq0o192FdRhpKHGTRIWBgqjDBTrq04=
sigs.k8s.io/json v0.0.0-20241014173422-cfa47c3a1cc8 h1:gBQPwqORJ8d8/YNZWEjoZs7npUVDpVXUUOFfW6CgAqE=
sigs.k8s.io/json v0.0.0-20241014173422-cfa47c3a1cc8/go.mod h1:mdzfpAEoE6DHQEN0uh9ZbOCuHbLK5wOm7dK4ctXE9Tg=
sigs.k8s.io/kind v0.30.0 h1:2Xi1KFEfSMm0XDcvKnUt15ZfgRPCT0OnCBbpgh8DztY=
sigs.k8s.io/kind v0.30.0/go.mod h1:FSqriGaoTPruiXWfRnUXNykF8r2t+fHtK0P0m1AbGF8=
sigs.k8s.io/json v0.0.0-20250730193827-2d320260d730 h1:IpInykpT6ceI+QxKBbEflcR5EXP7sU1kvOlxwZh5txg=
sigs.k8s.io/json v0.0.0-20250730193827-2d320260d730/go.mod h1:mdzfpAEoE6DHQEN0uh9ZbOCuHbLK5wOm7dK4ctXE9Tg=
sigs.k8s.io/kind v0.31.0 h1:UcT4nzm+YM7YEbqiAKECk+b6dsvc/HRZZu9U0FolL1g=
sigs.k8s.io/kind v0.31.0/go.mod h1:FSqriGaoTPruiXWfRnUXNykF8r2t+fHtK0P0m1AbGF8=
sigs.k8s.io/randfill v1.0.0 h1:JfjMILfT8A6RbawdsK2JXGBR5AQVfd+9TbzrlneTyrU=
sigs.k8s.io/randfill v1.0.0/go.mod h1:XeLlZ/jmk4i1HRopwe7/aU3H5n1zNUcX6TM94b3QxOY=
sigs.k8s.io/structured-merge-diff/v6 v6.3.0 h1:jTijUJbW353oVOd9oTlifJqOGEkUw2jB/fXCbTiQEco=

View File

@@ -1,6 +1,7 @@
package backend
import (
"context"
"fmt"
"time"
@@ -65,7 +66,7 @@ func (e *Entry) Check() bool {
return true
}
func Watch(tickAction func(), interval int, stop chan struct{}) {
func Watch(ctx context.Context, tickAction func(), interval int) {
if interval <= 0 {
interval = 5
}
@@ -75,7 +76,7 @@ func Watch(tickAction func(), interval int, stop chan struct{}) {
for {
select {
case <-stop:
case <-ctx.Done():
ticker.Stop()
return
case <-ticker.C:

View File

@@ -9,7 +9,7 @@ import (
)
// AddHost will update peers of a host
func (b *Server) AddHost(addr string) (err error) {
func (b *Server) AddHost(ctx context.Context, addr string) (err error) {
ip, _, err := net.ParseCIDR(addr)
if err != nil {
return err
@@ -20,7 +20,7 @@ func (b *Server) AddHost(addr string) (err error) {
return fmt.Errorf("failed to get path for %v", ip)
}
_, err = b.s.AddPath(context.Background(), &api.AddPathRequest{
_, err = b.s.AddPath(ctx, &api.AddPathRequest{
Path: p,
})
@@ -32,7 +32,7 @@ func (b *Server) AddHost(addr string) (err error) {
}
// DelHost will inform peers to remove a host
func (b *Server) DelHost(addr string) (err error) {
func (b *Server) DelHost(ctx context.Context, addr string) (err error) {
ip, _, err := net.ParseCIDR(addr)
if err != nil {
return err
@@ -42,7 +42,7 @@ func (b *Server) DelHost(addr string) (err error) {
return
}
return b.s.DeletePath(context.Background(), &api.DeletePathRequest{
return b.s.DeletePath(ctx, &api.DeletePathRequest{
Path: p,
})
}

View File

@@ -4,10 +4,12 @@ import (
"context"
"fmt"
"net"
"strconv"
//nolint
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/vip"
api "github.com/osrg/gobgp/v3/api"
"github.com/kube-vip/kube-vip/pkg/utils"
@@ -16,7 +18,7 @@ import (
)
// AddPeer will add peers to the BGP configuration
func (b *Server) AddPeer(peer kubevip.BGPPeer) (err error) {
func (b *Server) AddPeer(ctx context.Context, peer kubevip.BGPPeer) (err error) {
p := &api.Peer{
Conf: &api.PeerConf{
NeighborAddress: peer.Address,
@@ -74,16 +76,16 @@ func (b *Server) AddPeer(peer kubevip.BGPPeer) (err error) {
return fmt.Errorf("failed to get MP-BGP addresses: %w", err)
}
mask := "128"
mask := strconv.Itoa(vip.DefaultMaskIPv6)
address := ipv4Address
family := api.Family_AFI_IP
if utils.IsIPv4(p.Conf.NeighborAddress) {
mask = "32"
mask = strconv.Itoa(vip.DefaultMaskIPv4)
address = ipv6Address
family = api.Family_AFI_IP6
}
err = b.s.AddDefinedSet(context.Background(), &api.AddDefinedSetRequest{
err = b.s.AddDefinedSet(ctx, &api.AddDefinedSetRequest{
DefinedSet: &api.DefinedSet{
DefinedType: api.DefinedType_NEIGHBOR,
Name: fmt.Sprintf("peer-%s", p.Conf.NeighborAddress),
@@ -95,7 +97,7 @@ func (b *Server) AddPeer(peer kubevip.BGPPeer) (err error) {
}
if address != "" {
if err := insertPolicy(b.s, address, p, family); err != nil {
if err := insertPolicy(ctx, b.s, address, p, family); err != nil {
return fmt.Errorf("failed to add policy: %w", err)
}
}
@@ -109,7 +111,7 @@ func (b *Server) AddPeer(peer kubevip.BGPPeer) (err error) {
}
}
if err := b.s.AddPeer(context.Background(), &api.AddPeerRequest{Peer: p}); err != nil {
if err := b.s.AddPeer(ctx, &api.AddPeerRequest{Peer: p}); err != nil {
return fmt.Errorf("failed to add peer: %v", err)
}
@@ -128,7 +130,7 @@ func (b *Server) getPath(ip net.IP) (path *api.Path) {
//nolint
nlri, _ := anypb.New(&api.IPAddressPrefix{
Prefix: ip.String(),
PrefixLen: 32,
PrefixLen: vip.DefaultMaskIPv4,
})
//nolint
@@ -148,7 +150,7 @@ func (b *Server) getPath(ip net.IP) (path *api.Path) {
//nolint
nlri, _ := anypb.New(&api.IPAddressPrefix{
Prefix: ip.String(),
PrefixLen: 128,
PrefixLen: vip.DefaultMaskIPv6,
})
v6Family := &api.Family{
@@ -172,7 +174,7 @@ func (b *Server) getPath(ip net.IP) (path *api.Path) {
return
}
func insertPolicy(s *server.BgpServer, address string, p *api.Peer, family api.Family_Afi) error {
func insertPolicy(ctx context.Context, s *server.BgpServer, address string, p *api.Peer, family api.Family_Afi) error {
familyType := "v4"
if family == api.Family_AFI_IP6 {
familyType = "v6"
@@ -218,14 +220,14 @@ func insertPolicy(s *server.BgpServer, address string, p *api.Peer, family api.F
},
}
err := s.AddPolicy(context.Background(), &api.AddPolicyRequest{
err := s.AddPolicy(ctx, &api.AddPolicyRequest{
Policy: policy,
})
if err != nil {
return fmt.Errorf("failed to add policy: %w", err)
}
err = s.AddPolicyAssignment(context.Background(), &api.AddPolicyAssignmentRequest{
err = s.AddPolicyAssignment(ctx, &api.AddPolicyAssignmentRequest{
Assignment: &api.PolicyAssignment{
Name: "global",
Direction: api.PolicyDirection_EXPORT,

View File

@@ -52,10 +52,10 @@ func NewBGPServer(c kubevip.BGPConfig) (b *Server, err error) {
}
// Start starts the BGP server
func (b *Server) Start(peerStateChangeCallback func(*api.WatchEventResponse_PeerEvent)) (err error) {
func (b *Server) Start(ctx context.Context, peerStateChangeCallback func(*api.WatchEventResponse_PeerEvent)) (err error) {
go b.s.Serve()
if err = b.s.StartBgp(context.Background(), &api.StartBgpRequest{
if err = b.s.StartBgp(ctx, &api.StartBgpRequest{
Global: &api.Global{
Asn: b.c.AS,
RouterId: b.c.RouterID,
@@ -65,7 +65,7 @@ func (b *Server) Start(peerStateChangeCallback func(*api.WatchEventResponse_Peer
return
}
if err = b.s.WatchEvent(context.Background(), &api.WatchEventRequest{Peer: &api.WatchEventRequest_Peer{}}, func(r *api.WatchEventResponse) {
if err = b.s.WatchEvent(ctx, &api.WatchEventRequest{Peer: &api.WatchEventRequest_Peer{}}, func(r *api.WatchEventResponse) {
if p := r.GetPeer(); p != nil && p.Type == api.WatchEventResponse_PeerEvent_STATE {
log.Info("[BGP]", "peer", p.String())
if peerStateChangeCallback != nil {
@@ -77,13 +77,13 @@ func (b *Server) Start(peerStateChangeCallback func(*api.WatchEventResponse_Peer
}
for _, p := range b.c.Peers {
if err = b.AddPeer(p); err != nil {
if err = b.AddPeer(ctx, p); err != nil {
return
}
}
if b.c.Zebra.Enabled {
if err = b.s.EnableZebra(context.Background(), &api.EnableZebraRequest{
if err = b.s.EnableZebra(ctx, &api.EnableZebraRequest{
Url: b.c.Zebra.URL,
Version: b.c.Zebra.Version,
SoftwareName: b.c.Zebra.SoftwareName,
@@ -98,7 +98,8 @@ func (b *Server) Start(peerStateChangeCallback func(*api.WatchEventResponse_Peer
// Close will stop a running BGP Server
func (b *Server) Close() error {
ctx, cf := context.WithTimeout(context.Background(), 5*time.Second)
defer cf()
// create new BGP stop context (independent)
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
return b.s.StopBgp(ctx, &api.StopBgpRequest{})
}

View File

@@ -13,11 +13,10 @@ import (
// Cluster - The Cluster object manages the state of the cluster for a particular node
type Cluster struct {
stop chan bool
completed chan bool
once sync.Once
Network []vip.Network
arpMgr *arp.Manager
stop chan bool
once sync.Once
Network []vip.Network
arpMgr *arp.Manager
}
// InitCluster - Will attempt to initialise all of the required settings for the cluster
@@ -36,6 +35,7 @@ func InitCluster(c *kubevip.Config, disableVIP bool, intfMgr *networkinterface.M
newCluster := &Cluster{
Network: networks,
arpMgr: arpMgr,
stop: make(chan bool),
}
log.Debug("service security", "enabled", c.EnableServiceSecurity)
@@ -54,9 +54,9 @@ func startNetworking(c *kubevip.Config, intfMgr *networkinterface.Manager) ([]vi
networks := []vip.Network{}
for _, addr := range addresses {
network, err := vip.NewConfig(addr, c.Interface, c.LoInterfaceGlobalScope, c.VIPSubnet, c.DDNS, c.RoutingTableID,
c.RoutingTableType, c.RoutingProtocol, c.DNSMode, c.LoadBalancerForwardingMethod, c.IptablesBackend,
c.EnableLoadBalancer, c.EnableServiceSecurity, intfMgr)
network, err := vip.NewConfig(addr, c.Interface, c.LoInterfaceGlobalScope, c.VIPSubnet, c.DDNS, c.DHCPMode,
c.RequireDualStack, c.IsDualStack, c.RoutingTableID, c.RoutingTableType, c.RoutingProtocol, c.DNSMode,
c.LoadBalancerForwardingMethod, c.IptablesBackend, c.EnableLoadBalancer, c.EnableServiceSecurity, intfMgr)
if err != nil {
return nil, err
}
@@ -74,7 +74,4 @@ func (cluster *Cluster) Stop() {
close(cluster.stop)
})
}
// Wait until the completed channel is closed, signallign all shutdown tasks completed
<-cluster.completed
}

View File

@@ -2,6 +2,7 @@ package cluster
import (
"context"
"sync"
"github.com/kube-vip/kube-vip/pkg/vip"
)
@@ -12,9 +13,9 @@ import (
// during runtime if IP changes, startDDNS don't have to do reconfigure because
// dnsUpdater already have the functionality to keep trying resolve the IP
// and update the VIP configuration if it changes
func (cluster *Cluster) StartDDNS(ctx context.Context, network vip.Network) error {
ddnsMgr := vip.NewDDNSManager(ctx, network)
ip, err := ddnsMgr.Start()
func (cluster *Cluster) StartDDNS(ctx context.Context, network vip.Network, backoffAttempts uint, wg *sync.WaitGroup) error {
ddnsMgr := vip.NewDDNSManager(network, backoffAttempts)
ip, err := ddnsMgr.Start(ctx, wg)
if err != nil {
return err
}

View File

@@ -3,132 +3,59 @@ package cluster
import (
"context"
"fmt"
"os"
"os/signal"
"path/filepath"
"syscall"
"time"
"sync"
"github.com/davecgh/go-spew/spew"
"github.com/kube-vip/kube-vip/pkg/bgp"
"github.com/kube-vip/kube-vip/pkg/etcd"
"github.com/kube-vip/kube-vip/pkg/k8s"
"github.com/kube-vip/kube-vip/pkg/election"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/loadbalancer"
"github.com/kube-vip/kube-vip/pkg/lease"
"github.com/kube-vip/kube-vip/pkg/utils"
log "log/slog"
clientv3 "go.etcd.io/etcd/client/v3"
v1 "k8s.io/api/core/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/watch"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/cache"
"k8s.io/client-go/tools/leaderelection"
"k8s.io/client-go/tools/leaderelection/resourcelock"
watchtools "k8s.io/client-go/tools/watch"
)
// Manager degines the manager of the load-balancing services
type Manager struct {
KubernetesClient *kubernetes.Clientset
RetryWatcherClient *kubernetes.Clientset
// This channel is used to signal a shutdown
SignalChan chan os.Signal
// StartCluster - Begins a running instance of the Leader Election cluster
func (cluster *Cluster) StartCluster(ctx context.Context, c *kubevip.Config,
em *election.Manager, bgpServer *bgp.Server, leaseMgr *lease.Manager, killFunc func()) error {
EtcdClient *clientv3.Client
}
ns, leaseName := lease.NamespaceName(c.LeaseName, c)
// NewManager will create a new managing object
func NewManager(path string, inCluster bool, port int) (*Manager, error) {
var hostname string
leaseID := lease.NewID(c.LeaderElectionType, ns, leaseName)
// If inCluster is set then it will likely have started as a static pod or won't have the
// VIP up before trying to connect to the API server, we set the API endpoint to this machine to
// ensure connectivity. Else if the path passed is empty and not running in the cluster,
// attempt to look for a kubeconfig in the default HOME dir.
log.Info("cluster membership", "namespace", leaseID.Namespace(), "lock", leaseID.Name(), "id", c.NodeName)
hostname = fmt.Sprintf("kubernetes:%v", port)
objectName := lease.ObjectName(leaseID, "cp")
objLease := leaseMgr.Add(ctx, leaseID)
isNew := objLease.Add(objectName)
if len(path) == 0 && !inCluster {
path = filepath.Join(os.Getenv("HOME"), ".kube", "config")
wg := sync.WaitGroup{}
defer wg.Wait()
// We modify the config so that we can always speak to the correct host
id, err := os.Hostname()
if err != nil {
return nil, err
// Start a goroutine that will delete the lease when the service context is cancelled.
// This is important for proper cleanup when a service is deleted - it ensures that
// the lease context (svcLease.Ctx) gets cancelled, which causes RunOrDie to return.
// Without this, RunOrDie would continue running until leadership is naturally lost.
wg.Go(func() {
<-objLease.Ctx.Done()
leaseMgr.Delete(leaseID, objectName)
})
if !isNew {
log.Debug("this election was already done, waiting for it to finish", "lease", leaseName)
<-objLease.Ctx.Done()
return nil
}
wg.Go(func() {
select {
case <-cluster.stop:
case <-ctx.Done():
}
hostname = fmt.Sprintf("%s:%v", id, port)
}
config, err := k8s.NewRestConfig(path, inCluster, hostname)
if err != nil {
return nil, fmt.Errorf("failed to create k8s REST config: %w", err)
}
clientset, err := k8s.NewClientset(config)
if err != nil {
return nil, fmt.Errorf("error creating a new k8s clientset: %v", err)
}
rwConfig, err := k8s.NewRestConfig(path, inCluster, hostname)
if err != nil {
return nil, fmt.Errorf("failed to create k8s REST config for retryClientSet: %w", err)
}
rwConfig.Timeout = 0 // empty value to disable the timeout
rwClientSet, err := k8s.NewClientset(rwConfig)
if err != nil {
return nil, fmt.Errorf("failed to create k8s client for retry watcher: %w", err)
}
return &Manager{
KubernetesClient: clientset,
RetryWatcherClient: rwClientSet,
}, nil
}
// StartCluster - Begins a running instance of the Leader Election cluster
func (cluster *Cluster) StartCluster(c *kubevip.Config, sm *Manager, bgpServer *bgp.Server) error {
var err error
log.Info("cluster membership", "namespace", c.Namespace, "lock", c.LeaseName, "id", c.NodeName)
// use a Go context so we can tell the leaderelection code when we
// want to step down
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
// use a Go context so we can tell the arp loop code when we
// want to step down
ctxArp, cancelArp := context.WithCancel(context.Background())
defer cancelArp()
// use a Go context so we can tell the dns loop code when we
// want to step down
ctxDNS, cancelDNS := context.WithCancel(context.Background())
defer cancelDNS()
// listen for interrupts or the Linux SIGTERM signal and cancel
// our context, which the leader election code will observe and
// step down
signalChan := make(chan os.Signal, 1)
// Add Notification for Userland interrupt
signal.Notify(signalChan, syscall.SIGINT)
// Add Notification for SIGTERM (sent from Kubernetes)
signal.Notify(signalChan, syscall.SIGTERM)
go func() {
<-signalChan
log.Info("Received termination, signaling cluster shutdown")
// Cancel the context, which will in turn cancel the leadership
cancel()
// Cancel the arp context, which will in turn stop any broadcasts
}()
// Cancel the leader context, which will in turn cancel the leadership
objLease.Cancel()
})
// (attempt to) Remove the virtual IP, in case it already exists
@@ -142,310 +69,158 @@ func (cluster *Cluster) StartCluster(c *kubevip.Config, sm *Manager, bgpServer *
}
}
// Defer a function to check if the bgpServer has been created and if so attempt to close it
objLease.Lock()
defer func() {
if bgpServer != nil {
bgpServer.Close()
}
objLease.Unlock()
}()
if c.EnableBGP && bgpServer == nil {
// Lets start BGP
log.Info("Starting the BGP server to advertise VIP routes to VGP peers")
bgpServer, err = bgp.NewBGPServer(c.BGPConfig)
if err != nil {
log.Error("new BGP server", "err", err)
// this object is sharing lease with another object
if objLease.Elected.Load() {
log.Debug("this election was already done, shared lease", "lease", leaseName)
// wait for leader election to start or context to be done
select {
case <-objLease.Started:
case <-objLease.Ctx.Done():
// Lease was cancelled (e.g., leader election ended), return immediately
// This allows the restart loop to create a fresh lease
log.Debug("lease context cancelled before leader election started", "lease", leaseName)
return fmt.Errorf("lease %q context cancelled before leader election started", leaseName)
}
if err := bgpServer.Start(nil); err != nil {
log.Error("starting BGP server", "err", err)
cluster.OnStartedLeading(c, objLease, em, bgpServer, killFunc, true)
log.Debug("cluster waiting for leader context done", "lease", leaseName)
// wait for leaderelection to be finished
<-objLease.Ctx.Done()
cluster.OnStoppedLeading(c, objLease, bgpServer)
return nil
}
run := &election.RunConfig{
Config: c,
LeaseID: leaseID,
LeaseAnnotations: c.LeaseAnnotations,
Mgr: em,
OnStartedLeading: func(context.Context) { //nolint TODO: potential clean code
cluster.OnStartedLeading(c, objLease, em, bgpServer, killFunc, false)
},
OnStoppedLeading: func() {
objLease.Elected.Store(false)
cluster.OnStoppedLeading(c, objLease, bgpServer)
},
OnNewLeader: func(identity string) {
cluster.OnNewLeader(identity, c)
},
}
if err := election.RunOrDie(objLease.Ctx, run, c); err != nil {
objLease.Cancel()
return fmt.Errorf("leaderelection failed: %w", err)
}
return nil
}
func (cluster *Cluster) OnStartedLeading(c *kubevip.Config, objLease *lease.Lease,
em *election.Manager, bgpServer *bgp.Server, killFunc func(), isShared bool) {
objLease.Elected.Store(true)
objLease.Unlock()
// When we become leader, ensure we can take over VIPs even if they're preserved on other nodes
if !isShared {
close(objLease.Started)
}
if c.PreserveVIPOnLeadershipLoss {
log.Info("Becoming leader with VIP preservation enabled - ensuring VIP takeover")
// Force add the VIPs (this will work even if they exist due to the precheck logic)
for i := range cluster.Network {
added, err := cluster.Network[i].AddIP(true, false)
if err != nil {
log.Error("failed to ensure VIP on leader takeover", "vip", cluster.Network[i].IP(), "err", err)
} else if added {
log.Info("took over VIP as new leader", "IP", cluster.Network[i].IP(), "interface", cluster.Network[i].Interface())
} else {
log.Info("VIP already configured on interface", "IP", cluster.Network[i].IP(), "interface", cluster.Network[i].Interface())
}
}
}
run := &runConfig{
config: c,
leaseID: c.NodeName,
sm: sm,
onStartedLeading: func(ctx context.Context) { //nolint TODO: potential clean code
// When we become leader, ensure we can take over VIPs even if they're preserved on other nodes
if c.PreserveVIPOnLeadershipLoss {
log.Info("Becoming leader with VIP preservation enabled - ensuring VIP takeover")
// Force add the VIPs (this will work even if they exist due to the precheck logic)
for i := range cluster.Network {
added, err := cluster.Network[i].AddIP(true, false)
if err != nil {
log.Error("failed to ensure VIP on leader takeover", "vip", cluster.Network[i].IP(), "err", err)
} else if added {
log.Info("took over VIP as new leader", "IP", cluster.Network[i].IP(), "interface", cluster.Network[i].Interface())
} else {
log.Info("VIP already configured on interface", "IP", cluster.Network[i].IP(), "interface", cluster.Network[i].Interface())
}
}
}
// As we're leading lets start the vip service
err := cluster.StartVipService(objLease.Ctx, c, em, bgpServer, killFunc)
if err != nil {
log.Error("starting VIP service on leader", "err", err)
killFunc()
}
}
// Start ARP advertisements now that we have leadership
log.Info("Start ARP/NDP advertisement")
go cluster.arpMgr.StartAdvertisement(ctxArp)
func (cluster *Cluster) OnStoppedLeading(c *kubevip.Config, objLease *lease.Lease,
bgpServer *bgp.Server) {
// we can do cleanup here
log.Info("This node is becoming a follower within the cluster")
// As we're leading lets start the vip service
err := cluster.vipService(ctxArp, ctxDNS, c, sm, bgpServer, cancel)
if err != nil {
log.Error("starting VIP service on leader", "err", err)
}
},
onStoppedLeading: func() {
// we can do cleanup here
log.Info("This node is becoming a follower within the cluster")
// Stop the cluster context if it is running
objLease.Cancel()
// Stop the dns context
cancelDNS()
// Stop the Arp context if it is running
cancelArp()
// Stop the BGP server
if bgpServer != nil {
err := bgpServer.Close()
// Handle VIP cleanup based on configuration
if c.PreserveVIPOnLeadershipLoss {
// For IPv6, we must remove VIPs immediately to avoid DAD failures on the new leader
// IPv6 Duplicate Address Detection will fail if the new leader tries to add an IP that is still present on this node's interface
// We need to check each VIP individually and only remove IPv6 VIPs
for i := range cluster.Network {
if utils.IsIPv6(cluster.Network[i].IP()) {
log.Info("Removing IPv6 VIP immediately (required to prevent DAD failures on new leader)", "ip", cluster.Network[i].IP())
deleted, err := cluster.Network[i].DeleteIP()
if err != nil {
log.Warn("close BGP server", "err", err)
log.Warn("delete VIP", "err", err)
}
}
// Handle VIP cleanup based on configuration
if c.PreserveVIPOnLeadershipLoss {
// For IPv6, we must remove VIPs immediately to avoid DAD failures on the new leader
// IPv6 Duplicate Address Detection will fail if the new leader tries to add an IP that is still present on this node's interface
// We need to check each VIP individually and only remove IPv6 VIPs
for i := range cluster.Network {
if utils.IsIPv6(cluster.Network[i].IP()) {
log.Info("Removing IPv6 VIP immediately (required to prevent DAD failures on new leader)", "ip", cluster.Network[i].IP())
deleted, err := cluster.Network[i].DeleteIP()
if err != nil {
log.Warn("delete VIP", "err", err)
}
if deleted {
log.Info("deleted address", "IP", cluster.Network[i].IP(), "interface", cluster.Network[i].Interface())
}
} else {
log.Info("Preserving IPv4 VIP address on interface, only stopped ARP broadcasting", "ip", cluster.Network[i].IP())
}
if deleted {
log.Info("deleted address", "IP", cluster.Network[i].IP(), "interface", cluster.Network[i].Interface())
}
} else {
// Legacy behavior: delete VIP addresses on leadership loss
log.Info("Deleting VIP addresses on leadership loss (legacy behavior)")
for i := range cluster.Network {
deleted, err := cluster.Network[i].DeleteIP()
if err != nil {
log.Warn("delete VIP", "err", err)
}
if deleted {
log.Info("deleted address", "IP", cluster.Network[i].IP(), "interface", cluster.Network[i].Interface())
}
}
log.Info("Preserving IPv4 VIP address on interface, only stopped ARP broadcasting", "ip", cluster.Network[i].IP())
}
log.Error("lost leadership, restarting kube-vip")
panic("") // TODO - we could also return here
},
onNewLeader: func(identity string) {
// we're notified when new leader elected
log.Info("New leader", "leader", identity)
// If we're not the new leader and we have VIPs preserved from previous leadership,
// we need to clean them up to avoid conflicts.
if identity != c.NodeName && c.PreserveVIPOnLeadershipLoss {
log.Info("Cleaning up preserved VIPs as another node became leader", "new_leader", identity)
for i := range cluster.Network {
deleted, err := cluster.Network[i].DeleteIP()
if err != nil {
log.Warn("failed to cleanup preserved VIP", "vip", cluster.Network[i].IP(), "err", err)
}
if deleted {
log.Info("cleaned up preserved VIP to avoid conflict", "IP", cluster.Network[i].IP(), "interface", cluster.Network[i].Interface(), "new_leader", identity)
} else {
log.Debug("VIP was not present on this node", "IP", cluster.Network[i].IP(), "interface", cluster.Network[i].Interface())
}
}
}
} else {
// Legacy behavior: delete VIP addresses on leadership loss
log.Info("Deleting VIP addresses on leadership loss (legacy behavior)")
for i := range cluster.Network {
deleted, err := cluster.Network[i].DeleteIP()
if err != nil {
log.Warn("delete VIP", "err", err)
}
},
}
switch c.LeaderElectionType {
case "kubernetes", "":
cluster.runKubernetesLeaderElectionOrDie(ctx, run)
case "etcd":
cluster.runEtcdLeaderElectionOrDie(ctx, run)
default:
log.Info(fmt.Sprintf("LeaderElectionMode %s not supported, exiting", c.LeaderElectionType))
}
return nil
}
type runConfig struct {
config *kubevip.Config
leaseID string
sm *Manager
// onStartedLeading is called when this member starts leading.
onStartedLeading func(context.Context)
// onStoppedLeading is called when this member stops leading.
onStoppedLeading func()
// onNewLeader is called when the client observes a leader that is
// not the previously observed leader. This includes the first observed
// leader when the client starts.
onNewLeader func(identity string)
}
func (cluster *Cluster) runKubernetesLeaderElectionOrDie(ctx context.Context, run *runConfig) {
// we use the Lease lock type since edits to Leases are less common
// and fewer objects in the cluster watch "all Leases".
lock := &resourcelock.LeaseLock{
LeaseMeta: metav1.ObjectMeta{
Name: run.config.LeaseName,
Namespace: run.config.Namespace,
Annotations: run.config.LeaseAnnotations,
},
Client: run.sm.KubernetesClient.CoordinationV1(),
LockConfig: resourcelock.ResourceLockConfig{
Identity: run.leaseID,
},
}
// start the leader election code loop
leaderelection.RunOrDie(ctx, leaderelection.LeaderElectionConfig{
Lock: lock,
// IMPORTANT: you MUST ensure that any code you have that
// is protected by the lease must terminate **before**
// you call cancel. Otherwise, you could have a background
// loop still running and another process could
// get elected before your background loop finished, violating
// the stated goal of the lease.
ReleaseOnCancel: true,
LeaseDuration: time.Duration(run.config.LeaseDuration) * time.Second,
RenewDeadline: time.Duration(run.config.RenewDeadline) * time.Second,
RetryPeriod: time.Duration(run.config.RetryPeriod) * time.Second,
Callbacks: leaderelection.LeaderCallbacks{
OnStartedLeading: run.onStartedLeading,
OnStoppedLeading: run.onStoppedLeading,
OnNewLeader: run.onNewLeader,
},
})
}
func (cluster *Cluster) runEtcdLeaderElectionOrDie(ctx context.Context, run *runConfig) {
etcd.RunElectionOrDie(ctx, &etcd.LeaderElectionConfig{
EtcdConfig: etcd.ClientConfig{Client: run.sm.EtcdClient},
Name: run.config.LeaseName,
MemberID: run.leaseID,
LeaseDurationSeconds: int64(run.config.LeaseDuration),
Callbacks: etcd.LeaderCallbacks{
OnStartedLeading: run.onStartedLeading,
OnStoppedLeading: run.onStoppedLeading,
OnNewLeader: run.onNewLeader,
},
})
}
func (sm *Manager) NodeWatcher(ctxArp context.Context, lb *loadbalancer.IPVSLoadBalancer, port uint16) error {
// Use a restartable watcher, as this should help in the event of etcd or timeout issues
log.Info("Kube-Vip is watching nodes for control-plane labels")
listOptions := metav1.ListOptions{
LabelSelector: "node-role.kubernetes.io/control-plane",
}
rw, err := watchtools.NewRetryWatcherWithContext(ctxArp, "1", &cache.ListWatch{
WatchFunc: func(_ metav1.ListOptions) (watch.Interface, error) {
return sm.RetryWatcherClient.CoreV1().Nodes().Watch(context.Background(), listOptions)
},
})
if err != nil {
return fmt.Errorf("error creating label watcher: %s", err.Error())
}
go func() {
<-sm.SignalChan
log.Info("Received termination, signaling shutdown")
// Cancel the context
rw.Stop()
}()
ch := rw.ResultChan()
// defer rw.Stop()
for event := range ch {
// We need to inspect the event and get ResourceVersion out of it
switch event.Type {
case watch.Added, watch.Modified:
node, ok := event.Object.(*v1.Node)
if !ok {
return fmt.Errorf("unable to parse Kubernetes Node from Annotation watcher")
if deleted {
log.Info("deleted address", "IP", cluster.Network[i].IP(), "interface", cluster.Network[i].Interface())
}
// Find the node IP address (this isn't foolproof)
for x := range node.Status.Addresses {
if node.Status.Addresses[x].Type == v1.NodeInternalIP {
if checkIfNodeIsReady(node) {
err = lb.AddBackend(node.Status.Addresses[x].Address, port)
if err != nil {
log.Error("add IPVS backend", "err", err)
}
} else {
err = lb.RemoveBackend(node.Status.Addresses[x].Address, port)
if err != nil {
log.Error("remove IPVS backend", "err", err)
}
}
}
}
case watch.Deleted:
node, ok := event.Object.(*v1.Node)
if !ok {
return fmt.Errorf("unable to parse Kubernetes Node from Annotation watcher")
}
// Find the node IP address (this isn't foolproof)
for x := range node.Status.Addresses {
if node.Status.Addresses[x].Type == v1.NodeInternalIP {
err = lb.RemoveBackend(node.Status.Addresses[x].Address, port)
if err != nil {
log.Error("Del IPVS backend", "err", err)
}
}
}
log.Info("Node deleted", "name", node.Name)
case watch.Bookmark:
// Un-used
case watch.Error:
log.Error("Error attempting to watch Kubernetes Nodes")
// This round trip allows us to handle unstructured status
errObject := apierrors.FromObject(event.Object)
statusErr, ok := errObject.(*apierrors.StatusError)
if !ok {
log.Error(spew.Sprintf("Received an error which is not *metav1.Status but %#+v", event.Object))
}
status := statusErr.ErrStatus
log.Error("watcher", "status", status)
default:
}
}
log.Info("Exiting Node watcher")
return nil
log.Error("lost leadership, restarting kube-vip")
}
func checkIfNodeIsReady(node *v1.Node) bool {
if node == nil {
return false
}
for _, condition := range node.Status.Conditions {
if condition.Type == v1.NodeReady {
if condition.Status == v1.ConditionTrue {
return true
func (cluster *Cluster) OnNewLeader(identity string, c *kubevip.Config) {
// we're notified when new leader elected
log.Info("New leader", "leader", identity)
// If we're not the new leader and we have VIPs preserved from previous leadership,
// we need to clean them up to avoid conflicts.
if identity != c.NodeName && c.PreserveVIPOnLeadershipLoss {
log.Info("Cleaning up preserved VIPs as another node became leader", "new_leader", identity)
for i := range cluster.Network {
deleted, err := cluster.Network[i].DeleteIP()
if err != nil {
log.Warn("failed to cleanup preserved VIP", "vip", cluster.Network[i].IP(), "err", err)
}
if deleted {
log.Info("cleaned up preserved VIP to avoid conflict", "IP", cluster.Network[i].IP(),
"interface", cluster.Network[i].Interface(), "new_leader", identity)
} else {
log.Debug("VIP was not present on this node", "IP", cluster.Network[i].IP(),
"interface", cluster.Network[i].Interface())
}
}
}
return false
}

View File

@@ -7,15 +7,17 @@ import (
"io/fs"
"net"
"os"
"os/signal"
"strings"
"sync"
"syscall"
"time"
log "log/slog"
"github.com/kube-vip/kube-vip/pkg/arp"
"github.com/kube-vip/kube-vip/pkg/backend"
"github.com/kube-vip/kube-vip/pkg/bgp"
"github.com/kube-vip/kube-vip/pkg/election"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/loadbalancer"
"github.com/kube-vip/kube-vip/pkg/utils"
@@ -27,85 +29,88 @@ import (
"k8s.io/client-go/kubernetes"
)
func (cluster *Cluster) vipService(ctxArp, ctxDNS context.Context, c *kubevip.Config, sm *Manager, bgpServer *bgp.Server, cancelLeaderElection context.CancelFunc) error {
func (cluster *Cluster) StartVipService(ctx context.Context, c *kubevip.Config, em *election.Manager,
bgpServer *bgp.Server, killFunc func()) error {
var err error
// listen for interrupts or the Linux SIGTERM signal and cancel
// our context, which the leader election code will observe and
// step down
signalChan := make(chan os.Signal, 1)
// Add Notification for Userland interrupt
signal.Notify(signalChan, syscall.SIGINT)
var wg sync.WaitGroup
defer wg.Wait()
// Add Notification for SIGTERM (sent from Kubernetes)
signal.Notify(signalChan, syscall.SIGTERM)
wg.Go(func() {
<-ctx.Done()
killFunc()
})
loadbalancers := []*loadbalancer.IPVSLoadBalancer{}
var arpWG sync.WaitGroup
for i := range cluster.Network {
network := cluster.Network[i]
if network.IsDDNS() {
if err := cluster.StartDDNS(ctxDNS, cluster.Network[i]); err != nil {
if err := cluster.StartDDNS(ctx, cluster.Network[i], c.DHCPBackoffAttempts, &wg); err != nil {
log.Error("failed to start DDNS", "err", err)
}
}
if err := network.SetMask(c.VIPSubnet); err != nil {
log.Error("failed to set mask", "subnet", c.VIPSubnet, "err", err)
panic("")
return fmt.Errorf("failed to set mask for subnet %q: %w", c.VIPSubnet, err)
}
// start the dns updater if address is dns
if network.IsDNS() {
log.Info("starting the DNS updater", "address", network.DNSName())
ipUpdater := vip.NewIPUpdater(network)
ipUpdater.Run(ctxDNS)
wg.Go(func() {
ipUpdater.Run(ctx)
})
}
if !c.EnableRoutingTable {
// Normal VIP addition, use skipDAD=false for normal DAD process
if _, err = network.AddIP(false, false); err != nil {
log.Error(err.Error())
return fmt.Errorf("failed to add IP address %s: %w", network.IP(), err)
}
}
if c.EnableBGP {
// Lets advertise the VIP over BGP, the host needs to be passed using CIDR notation
log.Debug("Attempting to advertise over BGP", "address", network.CIDR())
err = bgpServer.AddHost(network.CIDR())
err = bgpServer.AddHost(ctx, network.CIDR())
if err != nil {
log.Error(err.Error())
}
}
if c.EnableLoadBalancer {
lb, err := loadbalancer.NewIPVSLB(network.IP(), c.LoadBalancerPort, c.LoadBalancerForwardingMethod, c.BackendHealthCheckInterval, c.Interface, cancelLeaderElection, signalChan)
lb, err := loadbalancer.NewIPVSLB(ctx, network.IP(), c.LoadBalancerPort, c.LoadBalancerForwardingMethod,
c.BackendHealthCheckInterval, c.Interface, killFunc, &wg)
if err != nil {
log.Error("Error creating IPVS LoadBalancer", "err", err)
return fmt.Errorf("creating IPVS LoadBalance: %w", err)
}
go func() {
err = sm.NodeWatcher(ctxArp, lb, c.Port) //TODO: We're using the ctxARP as the context this will change when rkatz finishes his change
wg.Go(func() {
err = em.NodeWatcher(ctx, lb, c.Port)
if err != nil {
log.Error("Error watching node labels", "err", err)
if errors.Is(err, &utils.PanicError{}) {
killFunc()
}
}
}()
})
loadbalancers = append(loadbalancers, lb)
}
if c.EnableARP {
arpWG.Add(1)
go cluster.layer2Update(ctxArp, network, c, &arpWG)
wg.Go(func() {
cluster.layer2Update(ctx, network, c)
})
}
}
if c.EnableLoadBalancer {
// Shutdown function that will wait on this signal, unless we call it ourselves
<-signalChan
<-ctx.Done()
for _, lb := range loadbalancers {
err = lb.RemoveIPVSLB()
if err != nil {
@@ -128,8 +133,8 @@ func (cluster *Cluster) vipService(ctxArp, ctxDNS context.Context, c *kubevip.Co
ips := []string{}
if nodename != "" {
if ips, err = getNodeIPs(ctxArp, nodename, sm.KubernetesClient); err != nil && !apierrors.IsNotFound(err) {
log.Error("failed to get IP of control-plane nod", "err", err)
if ips, err = getNodeIPs(ctx, nodename, em.KubernetesClient); err != nil && !apierrors.IsNotFound(err) {
log.Error("failed to get IP of control-plane node", "err", err)
}
}
@@ -160,15 +165,7 @@ func (cluster *Cluster) vipService(ctxArp, ctxDNS context.Context, c *kubevip.Co
}
}
stop := make(chan struct{})
// will wait for system interrupt and will send stop signal to backend watch
go func() {
<-signalChan
stop <- struct{}{}
}()
backend.Watch(func() {
backend.Watch(ctx, func() {
for i := range cluster.Network {
network := cluster.Network[i]
networkIP := network.IP()
@@ -229,14 +226,19 @@ func (cluster *Cluster) vipService(ctxArp, ctxDNS context.Context, c *kubevip.Co
deleted, err := network.DeleteIP()
if err != nil {
log.Error("error deleting IP", "err", err)
panic("")
killFunc()
return
}
if deleted {
log.Info("deleted address", "IP", network.IP(), "interface", network.Interface())
}
}
}
}, c.BackendHealthCheckInterval, stop)
}, c.BackendHealthCheckInterval)
}
if c.EnableBGP {
<-ctx.Done()
}
return nil
@@ -265,24 +267,37 @@ func getNodeIPs(ctx context.Context, nodename string, client *kubernetes.Clients
}
// StartLoadBalancerService will start a VIP instance and leave it for kube-proxy to handle
func (cluster *Cluster) StartLoadBalancerService(ctx context.Context, c *kubevip.Config, bgp *bgp.Server, name string, CountRouteReferences func(*netlink.Route) int) {
func (cluster *Cluster) StartLoadBalancerService(ctx context.Context, c *kubevip.Config, bgp *bgp.Server, name string, CountRouteReferences func(*netlink.Route) int, wg *sync.WaitGroup) error {
// use a Go context so we can tell the arp loop code when we
// want to step down
//nolint
ctxArp, cancelArp := context.WithCancel(ctx)
lbCtx, lbCancel := context.WithCancel(ctx)
cluster.stop = make(chan bool, 1)
cluster.completed = make(chan bool, 1)
var lbWg sync.WaitGroup
var arpWG sync.WaitGroup
log.Debug("StartLoadBalancerService")
for i := range cluster.Network {
network := cluster.Network[i]
if network.IsDDNS() {
ddnsReady := make(chan struct{})
lbWg.Go(func() {
// start the DDNS if requested
log.Debug("(svcs) start DDNS", "name", network.DNSName())
if err := cluster.StartDDNS(lbCtx, cluster.Network[i], c.DHCPBackoffAttempts, &lbWg); err != nil {
log.Error("failed to start DDNS", "err", err)
}
close(ddnsReady)
<-lbCtx.Done()
})
<-ddnsReady
}
log.Debug("current ip to process", "ip", network.IP(), "mask", c.VIPSubnet)
if err := network.SetMask(c.VIPSubnet); err != nil {
log.Error("failed to set mask", "subnet", c.VIPSubnet, "err", err)
panic("")
lbCancel()
return utils.NewPanicError(fmt.Sprintf("failed to set mask for subnet %q: %s", c.VIPSubnet, err.Error()))
}
_, err := network.DeleteIP()
if err != nil {
@@ -299,34 +314,52 @@ func (cluster *Cluster) StartLoadBalancerService(ctx context.Context, c *kubevip
}
}
// Normal VIP addition, use skipDAD=false for normal DAD process
if _, err = network.AddIP(false, false); err != nil {
log.Warn(err.Error())
} else {
log.Info("successful add IP")
if !c.EnableRoutingTable && !c.EnableBGP && !c.EnableWireguard {
// Normal VIP addition, use skipDAD=false for normal DAD process
// Note: When WireGuard is enabled, the VIP is added to the tunnel interface
// instead of lo, so we skip adding it here.
if _, err = network.AddIP(false, false); err != nil {
log.Warn(err.Error())
} else {
log.Info("successful add IP")
}
}
if c.EnableARP {
arpWG.Add(1)
go cluster.layer2Update(ctxArp, network, c, &arpWG)
lbWg.Go(func() {
cluster.layer2Update(lbCtx, network, c)
})
}
if c.EnableBGP && (c.EnableLeaderElection || c.EnableServicesElection) {
// Lets advertise the VIP over BGP, the host needs to be passed using CIDR notation
log.Debug("(svcs) attempting to advertise over BGP", "address", network.CIDR())
err = bgp.AddHost(network.CIDR())
err = bgp.AddHost(lbCtx, network.CIDR())
if err != nil {
log.Error(err.Error())
}
}
}
go func() {
<-cluster.stop
// Stop the Arp context if it is running
cancelArp()
wg.Go(func() {
for i := range cluster.Network {
network := cluster.Network[i]
arpWG.Wait() // wait for all cluster ARP/NDP to be finished
// start the dns updater if address is dns
if network.IsDNS() {
log.Info("(svcs) starting the DNS updater", "address", network.DNSName(), "ip", network.IP())
ipUpdater := vip.NewIPUpdater(network)
wg.Go(func() {
ipUpdater.Run(lbCtx)
})
}
}
<-cluster.stop
// Stop the loadbalancer context if it is running
lbCancel()
lbWg.Wait() // wait for all cluster ARP/NDP to be finished
log.Info("[LOADBALANCER] Stopping load balancers", "name", name)
@@ -342,7 +375,6 @@ func (cluster *Cluster) StartLoadBalancerService(ctx context.Context, c *kubevip
}
}
close(cluster.completed)
return
}
for i := range cluster.Network {
@@ -379,24 +411,21 @@ func (cluster *Cluster) StartLoadBalancerService(ctx context.Context, c *kubevip
}
}
}
})
close(cluster.completed)
}()
return nil
}
// Layer2Update, handles the creation of the
func (cluster *Cluster) layer2Update(ctx context.Context, network vip.Network, c *kubevip.Config, arpWG *sync.WaitGroup) {
defer arpWG.Done()
log.Info("layer 2 broadcaster starting")
func (cluster *Cluster) layer2Update(ctx context.Context, network vip.Network, c *kubevip.Config) {
var ndp *vip.NdpResponder
var err error
ipString := network.IP()
if utils.IsIPv6(ipString) {
if network.IPisLinkLocal() {
log.Error("layer2 is link-local can't use NDP", "address", ipString)
} else {
ndp, err = vip.NewNDPResponder(network.Interface())
ndp, err = waitNDPResponder(ctx, network.Interface())
if err != nil {
log.Error("failed to create new NDP Responder", "error", err)
} else {
@@ -407,6 +436,7 @@ func (cluster *Cluster) layer2Update(ctx context.Context, network vip.Network, c
}
}
log.Info("layer 2 broadcaster starting", "IP", network.IP(), "device", network.Interface())
log.Debug("layer 2 update", "ip", ipString, "interface", network.Interface(), "ms", c.ArpBroadcastRate)
arpInstance := arp.NewInstance(network, ndp)
@@ -416,3 +446,30 @@ func (cluster *Cluster) layer2Update(ctx context.Context, network vip.Network, c
log.Debug("ending layer 2 update", "ip", ipString, "interface", network.Interface(), "ms", c.ArpBroadcastRate)
cluster.arpMgr.RemoveOnLeadershipLoss(arpInstance)
}
func waitNDPResponder(ctx context.Context, ifaceName string) (*vip.NdpResponder, error) {
ndp, err := vip.NewNDPResponder(ifaceName)
if err != nil && strings.Contains(err.Error(), "no such device") {
log.Warn("unable to create NDP responder at first try", "interface", ifaceName, "err", err)
ndpCreateCtx, cancel := context.WithTimeout(ctx, 2*time.Minute)
defer cancel()
ticker := time.NewTicker(time.Second)
for {
select {
case <-ndpCreateCtx.Done():
return nil, fmt.Errorf("failed to create NDP responder for interface %q: %w", ifaceName, ndpCreateCtx.Err())
case <-ticker.C:
ndp, err = vip.NewNDPResponder(ifaceName)
if err != nil {
log.Warn("unable to create NDP responder on retry", "interface", ifaceName, "err", err)
} else {
return ndp, nil
}
}
}
} else if err != nil {
return nil, fmt.Errorf("unable to create NDP responder for interface %q: %w", ifaceName, err)
}
return ndp, nil
}

View File

@@ -1,85 +0,0 @@
package cluster
import (
"context"
log "log/slog"
"github.com/kube-vip/kube-vip/pkg/bgp"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/vip"
)
// StartSingleNode will start a single node cluster
func (cluster *Cluster) StartSingleNode(c *kubevip.Config, disableVIP bool) error {
// Start kube-vip as a single node server
// TODO - Split all this code out as a separate function
log.Info("Starting kube-vip as a single node cluster")
log.Info("This node is assuming leadership of the cluster")
cluster.stop = make(chan bool, 1)
cluster.completed = make(chan bool, 1)
for i := range cluster.Network {
if !disableVIP {
deleted, err := cluster.Network[i].DeleteIP()
if err != nil {
log.Warn("Attempted to clean existing VIP", "err", err)
}
if deleted {
log.Info("deleted address", "IP", cluster.Network[i].IP(), "interface", cluster.Network[i].Interface())
}
// Normal VIP addition for single node, use skipDAD=false for normal DAD process
_, err = cluster.Network[i].AddIP(false, false)
if err != nil {
log.Warn(err.Error())
}
}
if c.EnableARP {
// Gratuitous ARP, will broadcast to new MAC <-> IP
err := vip.ARPSendGratuitous(cluster.Network[i].IP(), c.Interface)
if err != nil {
log.Warn(err.Error())
}
}
}
go func() {
<-cluster.stop
if !disableVIP {
for i := range cluster.Network {
log.Info("[VIP] Releasing the VIP", "address", cluster.Network[i].IP())
deleted, err := cluster.Network[i].DeleteIP()
if err != nil {
log.Warn(err.Error())
}
if deleted {
log.Info("deleted address", "IP", cluster.Network[i].IP(), "interface", cluster.Network[i].Interface())
}
}
}
close(cluster.completed)
}()
log.Info("Started Load Balancer and Virtual IP")
return nil
}
func (cluster *Cluster) StartVipService(c *kubevip.Config, sm *Manager, bgp *bgp.Server) error {
// use a Go context so we can tell the arp loop code when we
// want to step down
ctxArp, cancelArp := context.WithCancel(context.Background())
defer cancelArp()
// use a Go context so we can tell the dns loop code when we
// want to step down
ctxDNS, cancelDNS := context.WithCancel(context.Background())
defer cancelDNS()
return cluster.vipService(ctxArp, ctxDNS, c, sm, bgp, nil)
}

View File

@@ -37,5 +37,5 @@ func FindIPAddress(addrName string) (string, string, error) {
}
}
return "", "", fmt.Errorf("Unknown interface [%s]", addrName)
return "", "", fmt.Errorf("unknown interface [%s]", addrName)
}

261
pkg/election/election.go Normal file
View File

@@ -0,0 +1,261 @@
package election
import (
"context"
"errors"
"fmt"
"sync"
"time"
log "log/slog"
"github.com/davecgh/go-spew/spew"
"github.com/kube-vip/kube-vip/pkg/etcd"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/lease"
"github.com/kube-vip/kube-vip/pkg/loadbalancer"
"github.com/kube-vip/kube-vip/pkg/utils"
clientv3 "go.etcd.io/etcd/client/v3"
v1 "k8s.io/api/core/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/watch"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/cache"
"k8s.io/client-go/tools/leaderelection"
"k8s.io/client-go/tools/leaderelection/resourcelock"
watchtools "k8s.io/client-go/tools/watch"
)
type Manager struct {
KubernetesClient *kubernetes.Clientset
RetryWatcherClient *kubernetes.Clientset
// This channel is used to signal a shutdown
EtcdClient *clientv3.Client
}
// NewManager will create a new managing object
func NewManager(config *kubevip.Config, k8sClientset, rwClientset *kubernetes.Clientset) (*Manager, error) {
m := &Manager{}
switch config.LeaderElectionType {
case "kubernetes", "":
if k8sClientset == nil || rwClientset == nil {
return nil, fmt.Errorf("provided nil clientset")
}
m.KubernetesClient = k8sClientset
m.RetryWatcherClient = rwClientset
case "etcd":
client, err := etcd.NewClient(config)
if err != nil {
return nil, err
}
m.EtcdClient = client
default:
return nil, fmt.Errorf("invalid LeaderElectionMode %s not supported", config.LeaderElectionType)
}
return m, nil
}
func RunOrDie(ctx context.Context, run *RunConfig, c *kubevip.Config) error {
switch c.LeaderElectionType {
case "kubernetes", "":
runKubernetesLeaderElectionOrDie(ctx, run)
case "etcd":
if err := runEtcdLeaderElectionOrDie(ctx, run); err != nil {
return err
}
default:
log.Info(fmt.Sprintf("LeaderElectionMode %s not supported, exiting", c.LeaderElectionType))
}
return nil
}
func runKubernetesLeaderElectionOrDie(ctx context.Context, run *RunConfig) {
// we use the Lease lock type since edits to Leases are less common
// and fewer objects in the cluster watch "all Leases".
lock := &resourcelock.LeaseLock{
LeaseMeta: metav1.ObjectMeta{
Name: run.LeaseID.Name(),
Namespace: run.LeaseID.Namespace(),
Annotations: run.LeaseAnnotations,
},
Client: run.Mgr.KubernetesClient.CoordinationV1(),
LockConfig: resourcelock.ResourceLockConfig{
Identity: run.Config.NodeName,
},
}
// start the leader election code loop
leaderelection.RunOrDie(ctx, leaderelection.LeaderElectionConfig{
Lock: lock,
// IMPORTANT: you MUST ensure that any code you have that
// is protected by the lease must terminate **before**
// you call cancel. Otherwise, you could have a background
// loop still running and another process could
// get elected before your background loop finished, violating
// the stated goal of the lease.
ReleaseOnCancel: true,
LeaseDuration: time.Duration(run.Config.LeaseDuration) * time.Second,
RenewDeadline: time.Duration(run.Config.RenewDeadline) * time.Second,
RetryPeriod: time.Duration(run.Config.RetryPeriod) * time.Second,
Callbacks: leaderelection.LeaderCallbacks{
OnStartedLeading: run.OnStartedLeading,
OnStoppedLeading: run.OnStoppedLeading,
OnNewLeader: run.OnNewLeader,
},
})
}
func runEtcdLeaderElectionOrDie(ctx context.Context, run *RunConfig) error {
if err := etcd.RunElectionOrDie(ctx, &etcd.LeaderElectionConfig{
EtcdConfig: etcd.ClientConfig{Client: run.Mgr.EtcdClient},
Name: run.LeaseID.NamespacedName(),
MemberID: run.Config.NodeName,
LeaseDurationSeconds: int64(run.Config.LeaseDuration),
Callbacks: etcd.LeaderCallbacks{
OnStartedLeading: run.OnStartedLeading,
OnStoppedLeading: run.OnStoppedLeading,
OnNewLeader: run.OnNewLeader,
},
}); err != nil {
return fmt.Errorf("etcd leaderelection: %w", err)
}
return nil
}
type Actions interface {
OnStartedLeading(ctx context.Context)
OnStoppedLeading()
OnNewLeader(identity string)
}
type RunConfig struct {
Config *kubevip.Config
LeaseID lease.ID
Mgr *Manager
LeaseAnnotations map[string]string
// onStartedLeading is called when this member starts leading.
OnStartedLeading func(context.Context)
// onStoppedLeading is called when this member stops leading.
OnStoppedLeading func()
// onNewLeader is called when the client observes a leader that is
// not the previously observed leader. This includes the first observed
// leader when the client starts.
OnNewLeader func(identity string)
}
func (em *Manager) NodeWatcher(ctx context.Context, lb *loadbalancer.IPVSLoadBalancer, port uint16) error {
// Use a restartable watcher, as this should help in the event of etcd or timeout issues
log.Info("Kube-Vip is watching nodes for control-plane labels")
listOptions := metav1.ListOptions{
LabelSelector: "node-role.kubernetes.io/control-plane",
}
rw, err := watchtools.NewRetryWatcherWithContext(ctx, "1", &cache.ListWatch{
WatchFunc: func(_ metav1.ListOptions) (watch.Interface, error) {
return em.RetryWatcherClient.CoreV1().Nodes().Watch(ctx, listOptions)
},
})
if err != nil {
return fmt.Errorf("error creating label watcher: %s", err.Error())
}
wg := sync.WaitGroup{}
defer wg.Wait()
wg.Go(func() {
<-ctx.Done()
log.Info("Received termination, signaling shutdown")
// Cancel the context
rw.Stop()
})
ch := rw.ResultChan()
// defer rw.Stop()
for event := range ch {
// We need to inspect the event and get ResourceVersion out of it
switch event.Type {
case watch.Added, watch.Modified:
node, ok := event.Object.(*v1.Node)
if !ok {
return fmt.Errorf("unable to parse Kubernetes Node from Annotation watcher")
}
// Find the node IP address (this isn't foolproof)
for x := range node.Status.Addresses {
if node.Status.Addresses[x].Type == v1.NodeInternalIP {
if checkIfNodeIsReady(node) {
err = lb.AddBackend(node.Status.Addresses[x].Address, port)
if err != nil {
log.Error("add IPVS backend", "err", err)
if errors.Is(err, &utils.PanicError{}) {
return fmt.Errorf("add IPVS backend: %w", err)
}
}
} else {
err = lb.RemoveBackend(node.Status.Addresses[x].Address, port)
if err != nil {
log.Error("remove IPVS backend", "err", err)
}
}
}
}
case watch.Deleted:
node, ok := event.Object.(*v1.Node)
if !ok {
return fmt.Errorf("unable to parse Kubernetes Node from Annotation watcher")
}
// Find the node IP address (this isn't foolproof)
for x := range node.Status.Addresses {
if node.Status.Addresses[x].Type == v1.NodeInternalIP {
err = lb.RemoveBackend(node.Status.Addresses[x].Address, port)
if err != nil {
log.Error("Del IPVS backend", "err", err)
}
}
}
log.Info("Node deleted", "name", node.Name)
case watch.Bookmark:
// Un-used
case watch.Error:
log.Error("Error attempting to watch Kubernetes Nodes")
// This round trip allows us to handle unstructured status
errObject := apierrors.FromObject(event.Object)
statusErr, ok := errObject.(*apierrors.StatusError)
if !ok {
log.Error(spew.Sprintf("Received an error which is not *metav1.Status but %#+v", event.Object))
}
status := statusErr.ErrStatus
log.Error("watcher", "status", status)
default:
}
}
log.Info("Exiting Node watcher")
return nil
}
func checkIfNodeIsReady(node *v1.Node) bool {
if node == nil {
return false
}
for _, condition := range node.Status.Conditions {
if condition.Type == v1.NodeReady {
if condition.Status == v1.ConditionTrue {
return true
}
}
}
return false
}

View File

@@ -4,6 +4,7 @@ import (
"context"
"fmt"
"net"
"sync"
log "log/slog"
@@ -11,10 +12,13 @@ import (
"github.com/kube-vip/kube-vip/pkg/endpoints/providers"
"github.com/kube-vip/kube-vip/pkg/instance"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/lease"
"github.com/kube-vip/kube-vip/pkg/servicecontext"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/kube-vip/kube-vip/pkg/wireguard"
v1 "k8s.io/api/core/v1"
discoveryv1 "k8s.io/api/discovery/v1"
"k8s.io/apimachinery/pkg/watch"
"k8s.io/client-go/kubernetes"
)
type Processor struct {
@@ -26,23 +30,24 @@ type Processor struct {
}
func NewEndpointProcessor(config *kubevip.Config, provider providers.Provider, bgpServer *bgp.Server,
instances *[]*instance.Instance) *Processor {
instances *[]*instance.Instance, leaseMgr *lease.Manager, tunnelMgr *wireguard.TunnelManager) *Processor {
return &Processor{
config: config,
provider: provider,
bgpServer: bgpServer,
instances: instances,
worker: newEndpointWorker(config, provider, bgpServer, instances),
worker: newEndpointWorker(config, provider, bgpServer, instances, leaseMgr, tunnelMgr),
}
}
func (p *Processor) AddOrModify(ctx *servicecontext.Context, event watch.Event,
lastKnownGoodEndpoint *string, service *v1.Service, id string, leaderElectionActive *bool,
serviceFunc func(context.Context, *v1.Service) error,
leaderCtx *context.Context, cancel *context.CancelFunc) (bool, error) {
func (p *Processor) AddOrModify(svcCtx *servicecontext.Context, event watch.Event,
lastKnownGoodEndpoint *string, service *v1.Service, id string,
serviceFunc func(*servicecontext.Context, *v1.Service, *sync.WaitGroup) error, wg *sync.WaitGroup,
clientSet *kubernetes.Clientset,
egressUpdateFunc func(context.Context, *v1.Service) error) (bool, error) {
var err error
if err = p.provider.LoadObject(event.Object, *cancel); err != nil {
if err = p.provider.LoadObject(event.Object, svcCtx.Cancel); err != nil {
return false, fmt.Errorf("[%s] error loading k8s object: %w", p.provider.GetLabel(), err)
}
@@ -64,50 +69,61 @@ func (p *Processor) AddOrModify(ctx *servicecontext.Context, event watch.Event,
// Check that we have local endpoints
if len(endpoints) != 0 {
// Ignore IPv4
if service.Annotations[kubevip.EgressIPv6] == "true" && net.ParseIP(endpoints[0]).To4() != nil {
if service.Annotations[kubevip.EgressIPv6] == "true" && !hasV6(endpoints) {
return true, nil
}
p.updateLastKnownGoodEndpoint(lastKnownGoodEndpoint, endpoints, service, leaderElectionActive, *cancel)
p.updateLastKnownGoodEndpoint(svcCtx, lastKnownGoodEndpoint, endpoints, service)
svcCtx.HasEndpoints.Store(true)
// start leader election if it's enabled and not already started
if !*leaderElectionActive && p.config.EnableServicesElection {
go func() {
*leaderCtx, *cancel = context.WithCancel(ctx.Ctx)
startLeaderElection(*leaderCtx, leaderElectionActive, service, serviceFunc)
}()
if !svcCtx.IsActive && p.config.EnableServicesElection {
wg.Go(func() {
startLeaderElection(svcCtx, service, serviceFunc, wg)
})
}
// There are local endpoints available on the node
if !p.config.EnableServicesElection && !p.config.EnableLeaderElection {
if err := p.worker.processInstance(ctx, service, leaderElectionActive); err != nil {
// Process immediately if:
// - No services/leader election is enabled, OR
// - WireGuard is enabled (it always needs immediate DNAT rule updates)
if (!p.config.EnableServicesElection && !p.config.EnableLeaderElection) || p.config.EnableWireguard {
if err := p.worker.processInstance(svcCtx, service); err != nil {
return false, fmt.Errorf("failed to process non-empty instance: %w", err)
}
}
} else {
svcCtx.HasEndpoints.Store(false)
// There are no local endpoints
p.worker.clear(ctx, lastKnownGoodEndpoint, service, *cancel, leaderElectionActive)
p.worker.clear(svcCtx, lastKnownGoodEndpoint, service)
}
// Set the service accordingly
p.updateAnnotations(service, lastKnownGoodEndpoint)
p.updateAnnotations(service, lastKnownGoodEndpoint, clientSet, egressUpdateFunc)
log.Debug("watcher", "provider",
p.provider.GetLabel(), "service name", service.Name, "namespace", service.Namespace, "endpoints", len(endpoints), "last endpoint", *lastKnownGoodEndpoint, "active leader election", *leaderElectionActive)
p.provider.GetLabel(), "service name", service.Name, "namespace", service.Namespace, "endpoints", len(endpoints), "last endpoint", *lastKnownGoodEndpoint, "active leader election", svcCtx.IsActive)
return false, nil
}
func (p *Processor) Delete(service *v1.Service, id string) error {
if err := p.worker.delete(service, id); err != nil {
func (p *Processor) Delete(ctx context.Context, service *v1.Service, id string) error {
if err := p.worker.delete(ctx, service, id); err != nil {
return fmt.Errorf("[%s] error deleting service: %w", p.provider.GetLabel(), err)
}
return nil
}
func (p *Processor) updateLastKnownGoodEndpoint(lastKnownGoodEndpoint *string, endpoints []string, service *v1.Service, leaderElectionActive *bool, cancel context.CancelFunc) {
func (p *Processor) updateLastKnownGoodEndpoint(svcCtx *servicecontext.Context, lastKnownGoodEndpoint *string, endpoints []string, service *v1.Service) {
// if we haven't populated one, then do so
family := utils.IPv4Family
if service.Annotations[kubevip.EgressIPv6] == "true" {
family = utils.IPv6Family
}
ep := getEndpoint(endpoints, family)
if *lastKnownGoodEndpoint == "" {
*lastKnownGoodEndpoint = endpoints[0]
*lastKnownGoodEndpoint = ep
return
}
@@ -122,44 +138,122 @@ func (p *Processor) updateLastKnownGoodEndpoint(lastKnownGoodEndpoint *string, e
// If the last endpoint no longer exists, we cancel our leader Election, and set another endpoint as last known good
if !stillExists {
p.worker.removeEgress(service, lastKnownGoodEndpoint)
if *leaderElectionActive && (p.config.EnableServicesElection || p.config.EnableLeaderElection) {
if svcCtx.IsActive && (p.config.EnableServicesElection || p.config.EnableLeaderElection) {
log.Warn("existing endpoint has been removed, restarting leaderElection", "provider", p.provider.GetLabel(), "endpoint", *lastKnownGoodEndpoint)
// Stop the existing leaderElection
cancel()
// disable last leaderElection flag
*leaderElectionActive = false
if svcCtx.Lease != nil {
svcCtx.Lease.Cancel()
}
}
// Set our active endpoint to an existing one
*lastKnownGoodEndpoint = endpoints[0]
*lastKnownGoodEndpoint = ep
}
}
func (p *Processor) updateAnnotations(service *v1.Service, lastKnownGoodEndpoint *string) {
func (p *Processor) updateAnnotations(service *v1.Service, lastKnownGoodEndpoint *string,
clientSet *kubernetes.Clientset,
egressUpdateFunc func(context.Context, *v1.Service) error) {
// Set the service accordingly
if service.Annotations[kubevip.Egress] == "true" {
activeEndpointAnnotation := kubevip.ActiveEndpoint
ip := net.ParseIP(*lastKnownGoodEndpoint)
if !p.config.EnableEndpoints && p.provider.GetProtocol() == string(discoveryv1.AddressTypeIPv6) {
activeEndpointAnnotation = kubevip.ActiveEndpointIPv6
// Store old values from ServiceSnapshot to detect if annotation actually changed
// We use the ServiceSnapshot instead of the service parameter because the service parameter
// may have stale annotations if the last update failed
var oldEndpoint, oldEndpointIPv6 string
if p.instances != nil {
serviceInstance := instance.FindServiceInstance(service, *p.instances)
if serviceInstance != nil {
oldEndpoint = serviceInstance.ServiceSnapshot.Annotations[kubevip.ActiveEndpoint]
oldEndpointIPv6 = serviceInstance.ServiceSnapshot.Annotations[kubevip.ActiveEndpointIPv6]
}
}
// Fall back to service annotations if we couldn't find the instance
if oldEndpoint == "" && oldEndpointIPv6 == "" {
oldEndpoint = service.Annotations[kubevip.ActiveEndpoint]
oldEndpointIPv6 = service.Annotations[kubevip.ActiveEndpointIPv6]
}
// Determine which annotation to update based on IP version
var endpoint, endpointIPv6 string
if ip.To4() == nil && !p.config.EnableEndpoints {
// IPv6
endpointIPv6 = *lastKnownGoodEndpoint
endpoint = oldEndpoint // Preserve existing IPv4 if any
} else {
// IPv4
endpoint = *lastKnownGoodEndpoint
endpointIPv6 = oldEndpointIPv6 // Preserve existing IPv6 if any
}
// Check if annotation actually changed
annotationChanged := (oldEndpoint != endpoint) || (oldEndpointIPv6 != endpointIPv6)
if !annotationChanged {
return // Nothing to do
}
// Persist to Kubernetes
ctx := context.Background()
if err := p.provider.UpdateServiceAnnotation(ctx, endpoint, endpointIPv6, service, clientSet); err != nil {
log.Warn("failed to update service annotation", "service", service.Name, "namespace", service.Namespace, "err", err)
return
}
log.Debug("updated active endpoint annotation", "service", service.Name, "namespace", service.Namespace, "endpoint", *lastKnownGoodEndpoint)
// Trigger egress reconfiguration
// For services with leader election, the service watcher doesn't process Modified events
// after initial setup, so we need to directly call the update function
if egressUpdateFunc != nil {
// Create a copy of service with updated annotations
svcCopy := service.DeepCopy()
svcCopy.Annotations[kubevip.ActiveEndpoint] = endpoint
svcCopy.Annotations[kubevip.ActiveEndpointIPv6] = endpointIPv6
if err := egressUpdateFunc(ctx, svcCopy); err != nil {
log.Error("failed to reconfigure egress", "service", service.Name, "namespace", service.Namespace, "err", err)
}
}
service.Annotations[activeEndpointAnnotation] = *lastKnownGoodEndpoint
}
}
func startLeaderElection(ctx context.Context, leaderElectionActive *bool, service *v1.Service, serviceFunc func(context.Context, *v1.Service) error) {
func startLeaderElection(svcCtx *servicecontext.Context, service *v1.Service, serviceFunc func(*servicecontext.Context, *v1.Service, *sync.WaitGroup) error, wg *sync.WaitGroup) {
// This is a blocking function, that will restart (in the event of failure)
for {
// if the context isn't cancelled restart
if ctx.Err() != context.Canceled {
*leaderElectionActive = true
err := serviceFunc(ctx, service)
select {
case <-svcCtx.Ctx.Done():
return
default:
err := serviceFunc(svcCtx, service, wg)
if err != nil {
log.Error(err.Error())
}
*leaderElectionActive = false
} else {
*leaderElectionActive = false
break
}
}
}
func hasV6(endpoints []string) bool {
for _, e := range endpoints {
ip := net.ParseIP(e)
if ip != nil {
if ip.To4() == nil {
return true
}
}
}
return false
}
func getEndpoint(endpoints []string, family string) string {
for _, e := range endpoints {
ip := net.ParseIP(e)
if family == utils.IPv4Family && ip.To4() != nil {
return e
}
if family == utils.IPv6Family && ip.To4() == nil {
return e
}
}
return ""
}

View File

@@ -23,20 +23,19 @@ func newBGP(generic generic, bgpServer *bgp.Server) endpointWorker {
}
}
func (b *BGP) processInstance(ctx *servicecontext.Context, service *v1.Service, leaderElectionActive *bool) error {
func (b *BGP) processInstance(svcCtx *servicecontext.Context, service *v1.Service) error {
if instance := instance.FindServiceInstance(service, *b.instances); instance != nil {
for _, cluster := range instance.Clusters {
for i := range cluster.Network {
if !ctx.IsNetworkConfigured(cluster.Network[i].IP()) {
if !svcCtx.IsNetworkConfigured(cluster.Network[i].IP()) {
log.Debug("attempting to advertise BGP service", "provider", b.provider.GetLabel(), "ip", cluster.Network[i].IP())
err := b.bgpServer.AddHost(cluster.Network[i].CIDR())
err := b.bgpServer.AddHost(svcCtx.Ctx, cluster.Network[i].CIDR())
if err != nil {
log.Error("error adding BGP host", "provider", b.provider.GetLabel(), "err", err)
} else {
log.Info("added BGP host", "provider",
b.provider.GetLabel(), "ip", cluster.Network[i].CIDR(), "service name", service.Name, "namespace", service.Namespace)
ctx.ConfiguredNetworks.Store(cluster.Network[i].IP(), true)
*leaderElectionActive = true
svcCtx.ConfiguredNetworks.Store(cluster.Network[i].IP(), true)
}
}
}
@@ -45,20 +44,19 @@ func (b *BGP) processInstance(ctx *servicecontext.Context, service *v1.Service,
return nil
}
func (b *BGP) clear(ctx *servicecontext.Context, lastKnownGoodEndpoint *string, service *v1.Service, cancel context.CancelFunc, leaderElectionActive *bool) {
func (b *BGP) clear(svcCtx *servicecontext.Context, lastKnownGoodEndpoint *string, service *v1.Service) {
if !b.config.EnableServicesElection && !b.config.EnableLeaderElection {
// If BGP mode is enabled - routes should be deleted
if instance := instance.FindServiceInstance(service, *b.instances); instance != nil {
for _, cluster := range instance.Clusters {
for i := range cluster.Network {
err := b.bgpServer.DelHost(cluster.Network[i].CIDR())
err := b.bgpServer.DelHost(svcCtx.Ctx, cluster.Network[i].CIDR())
if err != nil {
log.Error("deleting BGP host", "provider", b.provider.GetLabel(), "ip", cluster.Network[i].IP(), "err", err)
} else {
log.Info("deleted BGP host", "provider",
b.provider.GetLabel(), "ip", cluster.Network[i].IP(), "service name", service.Name, "namespace", service.Namespace)
ctx.ConfiguredNetworks.Delete(cluster.Network[i].IP())
*leaderElectionActive = false
svcCtx.ConfiguredNetworks.Delete(cluster.Network[i].IP())
}
}
}
@@ -66,14 +64,18 @@ func (b *BGP) clear(ctx *servicecontext.Context, lastKnownGoodEndpoint *string,
}
}
b.clearEgress(lastKnownGoodEndpoint, service, cancel, leaderElectionActive)
b.clearEgress(lastKnownGoodEndpoint, service)
if svcCtx.LeaderCancel != nil {
svcCtx.LeaderCancel()
}
}
func (b *BGP) getEndpoints(service *v1.Service, id string) ([]string, error) {
return b.getAllEndpoints(service, id)
}
func (b *BGP) delete(service *v1.Service, id string) error {
func (b *BGP) delete(ctx context.Context, service *v1.Service, id string) error {
// When no-leader-elecition mode
if !b.config.EnableServicesElection && !b.config.EnableLeaderElection {
// find all existing local endpoints
@@ -84,32 +86,32 @@ func (b *BGP) delete(service *v1.Service, id string) error {
// If there were local endpoints deleted
if len(endpoints) > 0 {
b.deleteAction(service)
b.deleteAction(ctx, service)
}
}
return nil
}
func (b *BGP) deleteAction(service *v1.Service) {
b.clearBGPHosts(service)
func (b *BGP) deleteAction(ctx context.Context, service *v1.Service) {
b.clearBGPHosts(ctx, service)
}
func (b *BGP) clearBGPHosts(service *v1.Service) {
ClearBGPHosts(service, b.instances, b.bgpServer)
func (b *BGP) clearBGPHosts(ctx context.Context, service *v1.Service) {
ClearBGPHosts(ctx, service, b.instances, b.bgpServer)
}
func (b *BGP) setInstanceEndpointsStatus(_ *v1.Service, _ []string) error {
return nil
}
func ClearBGPHosts(service *v1.Service, instances *[]*instance.Instance, bgpServer *bgp.Server) {
func ClearBGPHosts(ctx context.Context, service *v1.Service, instances *[]*instance.Instance, bgpServer *bgp.Server) {
if instance := instance.FindServiceInstance(service, *instances); instance != nil {
ClearBGPHostsByInstance(instance, bgpServer)
ClearBGPHostsByInstance(ctx, instance, bgpServer)
}
}
func ClearBGPHostsByInstance(instance *instance.Instance, bgpServer *bgp.Server) {
func ClearBGPHostsByInstance(ctx context.Context, instance *instance.Instance, bgpServer *bgp.Server) {
if instance == nil {
log.Error("failed to clear BGP host for nil instance")
return
@@ -117,7 +119,7 @@ func ClearBGPHostsByInstance(instance *instance.Instance, bgpServer *bgp.Server)
for _, cluster := range instance.Clusters {
for i := range cluster.Network {
network := cluster.Network[i]
err := bgpServer.DelHost(network.CIDR())
err := bgpServer.DelHost(ctx, network.CIDR())
if err != nil {
log.Error("[endpoint] error deleting BGP host", "err", err)
} else {

View File

@@ -10,22 +10,27 @@ import (
"github.com/kube-vip/kube-vip/pkg/endpoints/providers"
"github.com/kube-vip/kube-vip/pkg/instance"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/lease"
"github.com/kube-vip/kube-vip/pkg/servicecontext"
"github.com/kube-vip/kube-vip/pkg/wireguard"
v1 "k8s.io/api/core/v1"
)
type endpointWorker interface {
processInstance(svcCtx *servicecontext.Context, service *v1.Service, leaderElectionActive *bool) error
clear(svcCtx *servicecontext.Context, lastKnownGoodEndpoint *string, service *v1.Service, cancel context.CancelFunc, leaderElectionActive *bool)
processInstance(svcCtx *servicecontext.Context, service *v1.Service) error
clear(svcCtx *servicecontext.Context, lastKnownGoodEndpoint *string, service *v1.Service)
getEndpoints(service *v1.Service, id string) ([]string, error)
removeEgress(service *v1.Service, lastKnownGoodEndpoint *string)
delete(service *v1.Service, id string) error
delete(ctx context.Context, service *v1.Service, id string) error
setInstanceEndpointsStatus(service *v1.Service, endpoints []string) error
}
func newEndpointWorker(config *kubevip.Config, provider providers.Provider, bgpServer *bgp.Server, instances *[]*instance.Instance) endpointWorker {
generic := newGeneric(config, provider, instances)
func newEndpointWorker(config *kubevip.Config, provider providers.Provider, bgpServer *bgp.Server, instances *[]*instance.Instance, leaseMgr *lease.Manager, tunnelMgr *wireguard.TunnelManager) endpointWorker {
generic := newGeneric(config, provider, instances, leaseMgr)
if config.EnableWireguard {
return newWireguardWorker(config, provider, bgpServer, instances, leaseMgr, tunnelMgr)
}
if config.EnableRoutingTable {
return newRoutingTable(generic)
}
@@ -40,36 +45,37 @@ type generic struct {
config *kubevip.Config
provider providers.Provider
instances *[]*instance.Instance
leaseMgr *lease.Manager
}
func newGeneric(config *kubevip.Config, provider providers.Provider, instances *[]*instance.Instance) generic {
func newGeneric(config *kubevip.Config, provider providers.Provider, instances *[]*instance.Instance, leaseMgr *lease.Manager) generic {
return generic{
config: config,
provider: provider,
instances: instances,
leaseMgr: leaseMgr,
}
}
func (g *generic) processInstance(_ *servicecontext.Context, _ *v1.Service, _ *bool) error {
func (g *generic) processInstance(_ *servicecontext.Context, _ *v1.Service) error {
return nil
}
func (g *generic) clear(_ *servicecontext.Context, lastKnownGoodEndpoint *string, service *v1.Service, cancel context.CancelFunc, leaderElectionActive *bool) {
g.clearEgress(lastKnownGoodEndpoint, service, cancel, leaderElectionActive)
func (g *generic) clear(svcCtx *servicecontext.Context, lastKnownGoodEndpoint *string, service *v1.Service) {
g.clearEgress(lastKnownGoodEndpoint, service)
if svcCtx.LeaderCancel != nil {
svcCtx.LeaderCancel()
}
}
func (g *generic) clearEgress(lastKnownGoodEndpoint *string, service *v1.Service, cancel context.CancelFunc, leaderElectionActive *bool) {
func (g *generic) clearEgress(lastKnownGoodEndpoint *string, service *v1.Service) {
if *lastKnownGoodEndpoint != "" {
log.Warn("existing endpoint has been removed, no remaining endpoints for leaderElection", "provider", g.provider.GetLabel(), "endpoint", lastKnownGoodEndpoint)
log.Warn("existing endpoint has been removed, no remaining endpoints for leaderElection", "provider", g.provider.GetLabel(), "endpoint", lastKnownGoodEndpoint)
if err := egress.Teardown(*lastKnownGoodEndpoint, service.Spec.LoadBalancerIP, service.Namespace, string(service.UID), service.Annotations, g.config.EgressWithNftables); err != nil {
log.Error("error removing redundant egress rules", "err", err)
}
*lastKnownGoodEndpoint = "" // reset endpoint
if g.config.EnableServicesElection || g.config.EnableLeaderElection {
cancel() // stop services watcher
}
*leaderElectionActive = false
}
}
@@ -109,7 +115,7 @@ func (g *generic) getAllEndpoints(service *v1.Service, id string) ([]string, err
func (g *generic) removeEgress(_ *v1.Service, _ *string) {
}
func (g *generic) delete(_ *v1.Service, _ string) error {
func (g *generic) delete(_ context.Context, _ *v1.Service, _ string) error {
return nil
}

View File

@@ -26,7 +26,7 @@ func newRoutingTable(generic generic) endpointWorker {
}
}
func (rt *RoutingTable) processInstance(ctx *servicecontext.Context, service *v1.Service, leaderElectionActive *bool) error {
func (rt *RoutingTable) processInstance(ctx *servicecontext.Context, service *v1.Service) error {
instance := instance.FindServiceInstance(service, *rt.instances)
if instance != nil {
for _, cluster := range instance.Clusters {
@@ -59,7 +59,6 @@ func (rt *RoutingTable) processInstance(ctx *servicecontext.Context, service *v1
rt.provider.GetLabel(), "ip", cluster.Network[i].IP(), "service name", service.Name, "namespace",
service.Namespace, "interface", cluster.Network[i].Interface(), "tableID", rt.config.RoutingTableID)
ctx.ConfiguredNetworks.Store(cluster.Network[i].IP(), true)
*leaderElectionActive = true
}
}
}
@@ -69,7 +68,7 @@ func (rt *RoutingTable) processInstance(ctx *servicecontext.Context, service *v1
return nil
}
func (rt *RoutingTable) clear(svcCtx *servicecontext.Context, lastKnownGoodEndpoint *string, service *v1.Service, cancel context.CancelFunc, leaderElectionActive *bool) {
func (rt *RoutingTable) clear(svcCtx *servicecontext.Context, lastKnownGoodEndpoint *string, service *v1.Service) {
if !rt.config.EnableServicesElection && !rt.config.EnableLeaderElection {
if errs := ClearRoutes(service, rt.instances); len(errs) == 0 {
svcCtx.ConfiguredNetworks.Clear()
@@ -80,7 +79,11 @@ func (rt *RoutingTable) clear(svcCtx *servicecontext.Context, lastKnownGoodEndpo
}
}
rt.clearEgress(lastKnownGoodEndpoint, service, cancel, leaderElectionActive)
rt.clearEgress(lastKnownGoodEndpoint, service)
if svcCtx.LeaderCancel != nil {
svcCtx.LeaderCancel()
}
}
func (rt *RoutingTable) getEndpoints(service *v1.Service, id string) ([]string, error) {
@@ -94,7 +97,7 @@ func (rt *RoutingTable) removeEgress(service *v1.Service, lastKnownGoodEndpoint
}
}
func (rt *RoutingTable) delete(service *v1.Service, id string) error {
func (rt *RoutingTable) delete(_ context.Context, service *v1.Service, id string) error {
// When no-leader-elecition mode
if !rt.config.EnableServicesElection && !rt.config.EnableLeaderElection {
// find all existing local endpoints
@@ -117,7 +120,7 @@ func (rt *RoutingTable) deleteAction(service *v1.Service) {
}
func (rt *RoutingTable) setInstanceEndpointsStatus(service *v1.Service, endpoints []string) error {
instance := instance.FindServiceInstanceWithTimeout(service, *rt.instances)
instance := instance.FindServiceInstance(service, *rt.instances)
if instance == nil {
log.Error("failed to find the instance", "namespace", service.Namespace, "name", service.Name, "uid", service.UID, "provider", rt.provider.GetLabel())
} else {
@@ -126,10 +129,14 @@ func (rt *RoutingTable) setInstanceEndpointsStatus(service *v1.Service, endpoint
// if there are no endpoints set HasEndpoints false just in case
if len(endpoints) < 1 {
c.Network[n].SetHasEndpoints(false)
}
// check if endpoint are available and are of same IP family as service
if len(endpoints) > 0 && ((net.ParseIP(c.Network[n].IP()).To4() == nil) == (net.ParseIP(endpoints[0]).To4() == nil)) {
c.Network[n].SetHasEndpoints(true)
} else {
// check if endpoint are available and are of same IP family as service
for _, ep := range endpoints {
if (net.ParseIP(c.Network[n].IP()).To4() == nil) == (net.ParseIP(ep).To4() == nil) {
c.Network[n].SetHasEndpoints(true)
break
}
}
}
}
}

View File

@@ -0,0 +1,289 @@
package endpoints
import (
"context"
"fmt"
log "log/slog"
"github.com/kube-vip/kube-vip/pkg/bgp"
"github.com/kube-vip/kube-vip/pkg/endpoints/providers"
"github.com/kube-vip/kube-vip/pkg/instance"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/lease"
"github.com/kube-vip/kube-vip/pkg/nftables"
"github.com/kube-vip/kube-vip/pkg/servicecontext"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/kube-vip/kube-vip/pkg/wireguard"
v1 "k8s.io/api/core/v1"
)
// wireguardWorker handles endpoint changes for WireGuard-based services
type wireguardWorker struct {
config *kubevip.Config
provider providers.Provider
bgpServer *bgp.Server
instances *[]*instance.Instance
leaseMgr *lease.Manager
tunnelMgr *wireguard.TunnelManager
}
func newWireguardWorker(config *kubevip.Config, provider providers.Provider, bgpServer *bgp.Server,
instances *[]*instance.Instance, leaseMgr *lease.Manager, tunnelMgr *wireguard.TunnelManager) *wireguardWorker {
return &wireguardWorker{
config: config,
provider: provider,
bgpServer: bgpServer,
instances: instances,
leaseMgr: leaseMgr,
tunnelMgr: tunnelMgr,
}
}
// processInstance updates nftables DNAT rules when endpoints change
// This is called by the endpoint watcher when endpoints are added/modified
func (w *wireguardWorker) processInstance(svcCtx *servicecontext.Context, service *v1.Service) error {
log.Debug("[wireguard] processing instance for endpoint change", "service", service.Name, "namespace", service.Namespace)
// Get the target endpoint for this service
// For ExternalTrafficPolicy=Local, only use local endpoints
// For ExternalTrafficPolicy=Cluster, use all endpoints
var endpoints []string
var err error
if service.Spec.ExternalTrafficPolicy == v1.ServiceExternalTrafficPolicyTypeLocal {
endpoints, err = w.provider.GetLocalEndpoints(w.config.NodeName, w.config)
} else {
endpoints, err = w.provider.GetAllEndpoints()
}
if err != nil {
return fmt.Errorf("failed to get endpoints: %w", err)
}
if len(endpoints) == 0 {
log.Debug("[wireguard] no endpoints available", "service", service.Name)
w.clear(svcCtx, nil, service)
return nil
}
// Find the service processor to call updateServiceWireguardEndpoints
// Note: This requires access to the service processor which we don't have here
// So we'll recreate the DNAT rules directly
// First, clear existing rules
w.clear(svcCtx, nil, service)
// Get service VIPs
serviceIPs, err := utils.FetchServiceIPs(service)
if err != nil {
return fmt.Errorf("failed to get service IPs: %w", err)
}
// Create service identifier
serviceID := utils.SanitizeServiceID(fmt.Sprintf("%s_%s", service.Namespace, service.Name))
log.Info("[wireguard] updating DNAT rules for endpoint change",
"service", service.Name,
"namespace", service.Namespace,
"endpoints", endpoints,
"vips", serviceIPs)
// Update DNAT rules for each port
for _, port := range service.Spec.Ports {
// Determine target port (resolve named ports if necessary)
targetPort := w.provider.ResolvePort(port)
log.Info("[wireguard] resolved port", "service", service.Name, "servicePort", port.Port, "targetPort", targetPort, "targetPortName", port.TargetPort.StrVal)
// Build targets list from all endpoints
targets := make([]nftables.DNATTarget, len(endpoints))
for i, ep := range endpoints {
targets[i] = nftables.DNATTarget{
IP: ep,
Port: uint16(targetPort), //nolint:gosec // Port range validated by Kubernetes
}
}
for _, vip := range serviceIPs {
// Strip CIDR notation if present
vipAddr := utils.StripCIDR(vip)
// Get WireGuard interface name from TunnelManager for this VIP
if w.tunnelMgr == nil {
log.Error("[wireguard] TunnelManager not configured; cannot update DNAT rules",
"service", service.Name,
"namespace", service.Namespace)
return fmt.Errorf("TunnelManager not configured")
}
tunnelConfig := w.tunnelMgr.GetConfigForVIP(vipAddr)
if tunnelConfig == nil {
log.Error("[wireguard] WireGuard interface name not configured; cannot update DNAT rules",
"service", service.Name,
"namespace", service.Namespace,
"vip", vipAddr)
return fmt.Errorf("wireguard interface name not configured for VIP %s", vipAddr)
}
wgInterface := tunnelConfig.InterfaceName
portServiceID := fmt.Sprintf("%s_p%d", serviceID, port.Port)
log.Info("[wireguard] applying DNAT rule with load balancing",
"service", service.Name,
"vip", vipAddr,
"interface", wgInterface,
"sourcePort", port.Port,
"targets", targets,
"chainID", portServiceID)
// Apply the DNAT rule with load balancing across all endpoints
// localEndpoint=true when using ExternalTrafficPolicy=Local, which preserves client source IP
isLocalEndpoint := service.Spec.ExternalTrafficPolicy == v1.ServiceExternalTrafficPolicyTypeLocal
err := nftables.ApplyDNAT(
wgInterface,
vipAddr,
uint16(port.Port), //nolint:gosec // Port range validated by Kubernetes
targets,
portServiceID,
port.Protocol,
isLocalEndpoint,
tunnelConfig.ListenPort,
)
if err != nil {
log.Error("[wireguard] failed to update DNAT rule",
"service", service.Name,
"vip", vipAddr,
"port", port.Port,
"err", err)
continue
}
log.Debug("[wireguard] DNAT rule updated successfully",
"service", service.Name,
"vip", vipAddr,
"port", port.Port,
"targetCount", len(targets))
}
}
return nil
}
// clear removes DNAT rules when no endpoints are available
func (w *wireguardWorker) clear(svcCtx *servicecontext.Context, lastKnownGoodEndpoint *string, service *v1.Service) {
log.Info("[wireguard] clearing DNAT rules (no endpoints)", "service", service.Name, "namespace", service.Namespace)
serviceID := utils.SanitizeServiceID(fmt.Sprintf("%s_%s", service.Namespace, service.Name))
// Get service IPs to determine IPv4 vs IPv6
serviceIPs, _ := utils.FetchServiceIPs(service)
// Delete DNAT chains for each port
for _, port := range service.Spec.Ports {
if port.Protocol != v1.ProtocolTCP && port.Protocol != v1.ProtocolUDP {
continue
}
portServiceID := fmt.Sprintf("%s_p%d", serviceID, port.Port)
// Determine if we have IPv4 or IPv6
hasIPv4, hasIPv6 := false, false
for _, vip := range serviceIPs {
if isIPv6Address(vip) {
hasIPv6 = true
} else {
hasIPv4 = true
}
}
if hasIPv4 {
if err := nftables.DeleteIngressChains(false, portServiceID); err != nil {
log.Warn("[wireguard] failed to delete IPv4 DNAT chains",
"service", service.Name,
"port", port.Port,
"err", err)
}
}
if hasIPv6 {
if err := nftables.DeleteIngressChains(true, portServiceID); err != nil {
log.Warn("[wireguard] failed to delete IPv6 DNAT chains",
"service", service.Name,
"port", port.Port,
"err", err)
}
}
}
if svcCtx.LeaderCancel != nil {
svcCtx.LeaderCancel()
}
}
// getEndpoints retrieves the list of endpoints for a service
// For ExternalTrafficPolicy=Local, only local endpoints are returned
// For ExternalTrafficPolicy=Cluster, all endpoints are returned
func (w *wireguardWorker) getEndpoints(service *v1.Service, id string) ([]string, error) {
var endpoints []string
var err error
if service.Spec.ExternalTrafficPolicy == v1.ServiceExternalTrafficPolicyTypeLocal {
endpoints, err = w.provider.GetLocalEndpoints(id, w.config)
} else {
endpoints, err = w.provider.GetAllEndpoints()
}
if err != nil {
return nil, fmt.Errorf("[wireguard] failed to get endpoints: %w", err)
}
log.Debug("[wireguard] retrieved endpoints", "service", service.Name, "count", len(endpoints), "endpoints", endpoints)
return endpoints, nil
}
// removeEgress is a no-op for WireGuard since egress is handled separately
func (w *wireguardWorker) removeEgress(service *v1.Service, lastKnownGoodEndpoint *string) {
// WireGuard doesn't use egress in the same way as other modes
log.Debug("[wireguard] removeEgress called (no-op)", "service", service.Name)
}
// delete removes all DNAT rules for a service
func (w *wireguardWorker) delete(ctx context.Context, service *v1.Service, id string) error {
log.Info("[wireguard] deleting DNAT rules for service", "service", service.Name, "namespace", service.Namespace)
w.clear(nil, nil, service)
return nil
}
// setInstanceEndpointsStatus updates the endpoint status on the service instance
func (w *wireguardWorker) setInstanceEndpointsStatus(service *v1.Service, endpoints []string) error {
hasEndpoints := len(endpoints) > 0
log.Debug("[wireguard] setting instance endpoint status",
"service", service.Name,
"hasEndpoints", hasEndpoints,
"endpointCount", len(endpoints))
// Find the service instance
for _, inst := range *w.instances {
if inst.ServiceSnapshot == nil {
continue
}
if inst.ServiceSnapshot.UID == service.UID {
// Update the network status for all clusters
for _, cluster := range inst.Clusters {
for i := range cluster.Network {
cluster.Network[i].SetHasEndpoints(hasEndpoints)
}
}
log.Debug("[wireguard] updated instance endpoint status",
"service", service.Name,
"hasEndpoints", hasEndpoints)
return nil
}
}
log.Debug("[wireguard] instance not found for endpoint status update", "service", service.Name)
return nil
}
func isIPv6Address(ip string) bool {
// Strip CIDR notation if present before checking
addr := utils.StripCIDR(ip)
return utils.IsIPv6(addr)
}

View File

@@ -97,11 +97,11 @@ func (ep *Endpoints) GetLocalEndpoints(id string, _ *kubevip.Config) ([]string,
return localEndpoints, nil
}
func (ep *Endpoints) UpdateServiceAnnotation(endpoint string, _ string, service *v1.Service, clientSet *kubernetes.Clientset) error {
func (ep *Endpoints) UpdateServiceAnnotation(ctx context.Context, endpoint string, _ string, service *v1.Service, clientSet *kubernetes.Clientset) error {
retryErr := retry.RetryOnConflict(retry.DefaultRetry, func() error {
// Retrieve the latest version of Deployment before attempting update
// RetryOnConflict uses exponential backoff to avoid exhausting the apiserver
currentService, err := clientSet.CoreV1().Services(service.Namespace).Get(context.TODO(), service.Name, metav1.GetOptions{})
currentService, err := clientSet.CoreV1().Services(service.Namespace).Get(ctx, service.Name, metav1.GetOptions{})
if err != nil {
return err
}
@@ -113,7 +113,7 @@ func (ep *Endpoints) UpdateServiceAnnotation(endpoint string, _ string, service
currentServiceCopy.Annotations[kubevip.ActiveEndpoint] = endpoint
_, err = clientSet.CoreV1().Services(currentService.Namespace).Update(context.TODO(), currentServiceCopy, metav1.UpdateOptions{})
_, err = clientSet.CoreV1().Services(currentService.Namespace).Update(ctx, currentServiceCopy, metav1.UpdateOptions{})
if err != nil {
log.Error("error updating Service Spec", "label", ep.GetLabel(), "name", currentServiceCopy.Name, "err", err)
return err
@@ -132,6 +132,15 @@ func (ep *Endpoints) GetLabel() string {
return ep.label
}
func (ep *Endpoints) GetProtocol() string {
return ""
func (ep *Endpoints) ResolvePort(servicePort v1.ServicePort) int32 {
return ResolvePortWithLookup(servicePort, func(name string) int32 {
for _, subset := range ep.endpoints.Subsets {
for _, p := range subset.Ports {
if p.Name == name {
return p.Port
}
}
}
return 0
})
}

View File

@@ -20,8 +20,10 @@ import (
)
type Endpointslices struct {
label string
endpoints *discoveryv1.EndpointSlice
label string
endpointsv4 []discoveryv1.Endpoint
endpointsv6 []discoveryv1.Endpoint
ports []discoveryv1.EndpointPort
}
func NewEndpointslices() Provider {
@@ -56,27 +58,42 @@ func (ep *Endpointslices) LoadObject(endpoints runtime.Object, cancel context.Ca
cancel()
return fmt.Errorf("[%s] error casting endpoints to v1.Endpoints struct", ep.label)
}
ep.endpoints = eps
if eps.AddressType == discoveryv1.AddressTypeIPv6 {
ep.endpointsv6 = eps.Endpoints
} else {
ep.endpointsv4 = eps.Endpoints
}
// Store ports for resolving named ports
ep.ports = eps.Ports
return nil
}
func (ep *Endpointslices) GetAllEndpoints() ([]string, error) {
result := []string{}
for _, ep := range ep.endpoints.Endpoints {
result = append(result, ep.Addresses...)
for _, e := range ep.endpointsv4 {
result = append(result, e.Addresses...)
}
for _, e := range ep.endpointsv6 {
result = append(result, e.Addresses...)
}
return result, nil
}
func (ep *Endpointslices) GetLocalEndpoints(id string, _ *kubevip.Config) ([]string, error) {
var localEndpoints []string
for _, endpoint := range ep.endpoints.Endpoints {
tmpEps := []discoveryv1.Endpoint{}
tmpEps = append(tmpEps, ep.endpointsv4...)
tmpEps = append(tmpEps, ep.endpointsv6...)
for _, endpoint := range tmpEps {
if endpoint.Conditions.Serving == nil || !*endpoint.Conditions.Serving {
continue
}
for _, address := range endpoint.Addresses {
log.Debug("processing endpoint", "provider", ep.label, "ip", address)
// 1. Compare the Nodename
if endpoint.NodeName != nil && id == *endpoint.NodeName {
if endpoint.Hostname != nil {
@@ -98,11 +115,11 @@ func (ep *Endpointslices) GetLocalEndpoints(id string, _ *kubevip.Config) ([]str
return localEndpoints, nil
}
func (ep *Endpointslices) UpdateServiceAnnotation(endpoint, endpointIPv6 string, service *v1.Service, clientSet *kubernetes.Clientset) error {
func (ep *Endpointslices) UpdateServiceAnnotation(ctx context.Context, endpoint, endpointIPv6 string, service *v1.Service, clientSet *kubernetes.Clientset) error {
retryErr := retry.RetryOnConflict(retry.DefaultRetry, func() error {
// Retrieve the latest version of Deployment before attempting update
// RetryOnConflict uses exponential backoff to avoid exhausting the apiserver
currentService, err := clientSet.CoreV1().Services(service.Namespace).Get(context.TODO(), service.Name, metav1.GetOptions{})
currentService, err := clientSet.CoreV1().Services(service.Namespace).Get(ctx, service.Name, metav1.GetOptions{})
if err != nil {
return err
}
@@ -115,7 +132,7 @@ func (ep *Endpointslices) UpdateServiceAnnotation(endpoint, endpointIPv6 string,
currentServiceCopy.Annotations[kubevip.ActiveEndpoint] = endpoint
currentServiceCopy.Annotations[kubevip.ActiveEndpointIPv6] = endpointIPv6
_, err = clientSet.CoreV1().Services(currentService.Namespace).Update(context.TODO(), currentServiceCopy, metav1.UpdateOptions{})
_, err = clientSet.CoreV1().Services(currentService.Namespace).Update(ctx, currentServiceCopy, metav1.UpdateOptions{})
if err != nil {
log.Error("error updating Service Spec", "provider", ep.label, "service name", currentServiceCopy.Name, "err", err)
return err
@@ -134,6 +151,13 @@ func (ep *Endpointslices) GetLabel() string {
return ep.label
}
func (ep *Endpointslices) GetProtocol() string {
return string(ep.endpoints.AddressType)
func (ep *Endpointslices) ResolvePort(servicePort v1.ServicePort) int32 {
return ResolvePortWithLookup(servicePort, func(name string) int32 {
for _, p := range ep.ports {
if p.Name != nil && *p.Name == name && p.Port != nil {
return *p.Port
}
}
return 0
})
}

View File

@@ -16,7 +16,24 @@ type Provider interface {
GetAllEndpoints() ([]string, error)
GetLocalEndpoints(string, *kubevip.Config) ([]string, error)
GetLabel() string
UpdateServiceAnnotation(string, string, *v1.Service, *kubernetes.Clientset) error
UpdateServiceAnnotation(context.Context, string, string, *v1.Service, *kubernetes.Clientset) error
LoadObject(runtime.Object, context.CancelFunc) error
GetProtocol() string
// ResolvePort resolves a service port to the actual target port.
// For named ports, it looks up the port number from the endpoint.
// For numeric ports, it returns the port as-is.
ResolvePort(servicePort v1.ServicePort) int32
}
// ResolvePortWithLookup is a helper that resolves a service port using a lookup function
// for named ports. This consolidates the common resolution logic.
func ResolvePortWithLookup(servicePort v1.ServicePort, lookupNamedPort func(string) int32) int32 {
if servicePort.TargetPort.IntVal != 0 {
return servicePort.TargetPort.IntVal
}
if servicePort.TargetPort.StrVal != "" {
if port := lookupNamedPort(servicePort.TargetPort.StrVal); port != 0 {
return port
}
}
return servicePort.Port
}

View File

@@ -2,7 +2,9 @@ package etcd
import (
"context"
"fmt"
"hash/fnv"
"sync"
"time"
log "log/slog"
@@ -64,10 +66,11 @@ type ClientConfig struct {
}
// RunElectionOrDie behaves the same way as RunElection but panics if there is an error.
func RunElectionOrDie(ctx context.Context, config *LeaderElectionConfig) {
func RunElectionOrDie(ctx context.Context, config *LeaderElectionConfig) error {
if err := RunElection(ctx, config); err != nil {
panic(err)
return fmt.Errorf("leaderelection error: %w", err)
}
return nil
}
// RunElection starts a client with the provided config or panics.
@@ -116,7 +119,12 @@ func RunElection(ctx context.Context, config *LeaderElectionConfig) error {
leaseTTL: lease.TTL,
}
go m.tryToBeLeader(ctx)
wg := sync.WaitGroup{}
defer wg.Wait()
wg.Go(func() {
m.tryToBeLeader(ctx, &wg)
})
m.watchLeaderChanges(ctx)
return nil
@@ -185,7 +193,7 @@ watcher:
log.Debug("Exiting watcher", "id", m.memberID)
}
func (m *member) tryToBeLeader(ctx context.Context) {
func (m *member) tryToBeLeader(ctx context.Context, wg *sync.WaitGroup) {
if err := m.election.Campaign(ctx, m.memberID); err != nil {
log.Error("Failed trying to become the leader", "err", err)
// Resign just in case we acquired leadership just before failing
@@ -204,7 +212,9 @@ func (m *member) tryToBeLeader(ctx context.Context) {
m.weAreTheLeader <- struct{}{}
// Once we are the leader, start the routine to resign if context is canceled
go m.resignOnCancel(ctx)
wg.Go(func() {
m.resignOnCancel(ctx)
})
// After becoming the leader, we wait for at least a lease TTL to wait for
// the previous leader to detect the new leadership (if there was one) and

View File

@@ -20,13 +20,20 @@ import (
func TestRunElectionWithMemberIDCollision(t *testing.T) {
t.Parallel()
g := NewWithT(t)
ctx := context.Background()
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
cli := client(g)
defer cli.Close()
electionName := randomElectionNameForTest("memberIDConflict")
log.Printf("Election name %s\n", electionName)
memberCtx, cancelMember1 := context.WithCancel(ctx)
// Use a channel to signal when the first member has observed a new leader
// This ensures proper ordering without relying on sleep timing
firstMemberObservedLeader := make(chan struct{})
var firstMemberObservedOnce sync.Once
config := &etcd.LeaderElectionConfig{
EtcdConfig: etcd.ClientConfig{
Client: cli,
@@ -42,6 +49,11 @@ func TestRunElectionWithMemberIDCollision(t *testing.T) {
},
OnNewLeader: func(identity string) {
log.Printf("New leader: %s\n", identity)
// Signal that the first member has observed a leader
// This means the lease has been created
firstMemberObservedOnce.Do(func() {
close(firstMemberObservedLeader)
})
},
OnStoppedLeading: func() {
log.Println("I'm not the leader anymore")
@@ -59,8 +71,18 @@ func TestRunElectionWithMemberIDCollision(t *testing.T) {
go func() {
defer wg.Done()
time.Sleep(time.Millisecond * 50) // make sure the first one becomes leader
g.Expect(etcd.RunElection(ctx, config)).Should(MatchError(ContainSubstring("creating lease")))
// Wait for the first member to observe a leader, which means the lease has been created
select {
case <-firstMemberObservedLeader:
// First member has created the lease, now try to create a conflicting one
case <-time.After(5 * time.Second):
t.Error("timeout waiting for first member to observe leader")
return
}
// Use a cancellable context to prevent hanging if this goroutine unexpectedly succeeds
member2Ctx, cancelMember2 := context.WithTimeout(ctx, 5*time.Second)
defer cancelMember2()
g.Expect(etcd.RunElection(member2Ctx, config)).Should(MatchError(ContainSubstring("creating lease")))
}()
wg.Wait()
@@ -69,7 +91,8 @@ func TestRunElectionWithMemberIDCollision(t *testing.T) {
func TestRunElectionWithTwoMembersAndReelection(t *testing.T) {
t.Parallel()
g := NewWithT(t)
ctx := context.Background()
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
cli := client(g)
defer cli.Close()
@@ -85,7 +108,7 @@ func TestRunElectionWithTwoMembersAndReelection(t *testing.T) {
LeaseDurationSeconds: 1,
}
member1Ctx, _ := context.WithCancel(ctx)
member1Ctx, cancelMember1 := context.WithCancel(ctx)
member2Ctx, cancelMember2 := context.WithCancel(ctx)
config1 := configBase
@@ -101,6 +124,7 @@ func TestRunElectionWithTwoMembersAndReelection(t *testing.T) {
log.Println("Losing the leadership on purpose by stopping renewing the lease")
g.Expect(cliMember1.Lease.Close()).To(Succeed())
log.Println("Member1 leases closed")
cancelMember1()
}
config2 := configBase
@@ -129,6 +153,7 @@ func TestRunElectionWithTwoMembersAndReelection(t *testing.T) {
}()
wg.Wait()
}
func baseCallbacksForName(name string) etcd.LeaderCallbacks {

View File

@@ -29,7 +29,8 @@ const (
func TestMain(m *testing.M) {
logrus.SetLevel(logrus.DebugLevel)
ctx := context.Background()
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
expectSuccess(startEtcd(ctx), "starting etcd")
os.Exit(runTestsWithCleanup(m, func() {

View File

@@ -1,13 +1,14 @@
package instance
import (
"context"
"fmt"
"net"
"slices"
"strconv"
"strings"
"time"
"sync"
"log/slog"
log "log/slog"
"github.com/vishvananda/netlink"
@@ -31,18 +32,27 @@ type Instance struct {
Clusters []*cluster.Cluster
// Service uses DHCP
IsDHCP bool
IsDHCPv4 bool
IsDHCPv6 bool
DHCPInterface string
DHCPInterfaceHwaddr string
DHCPInterfaceIP string
DHCPInterfaceIPv4 string
DHCPInterfaceIPv6 string
DHCPHostname string
DHCPClient *vip.DHCPClient
DHCPv4Client vip.DHCPClient
DHCPv6Client vip.DHCPClient
// External Gateway IP the service is forwarded from
UPNPGatewayIPs []string
// Kubernetes service mapping
ServiceSnapshot *v1.Service
dnsAddresses []string
// AddCalled determined that ActionAdd was already performed for the instance
AddCalled bool
}
type Port struct {
@@ -50,12 +60,14 @@ type Port struct {
Type string
}
func NewInstance(svc *v1.Service, config *kubevip.Config, intfMgr *networkinterface.Manager, arpMgr *arp.Manager) (*Instance, error) {
instanceAddresses, _ := FetchServiceAddresses(svc)
func NewInstance(ctx context.Context, svc *v1.Service, config *kubevip.Config, intfMgr *networkinterface.Manager, arpMgr *arp.Manager, wg *sync.WaitGroup) (*Instance, error) {
instanceAddresses, instanceHostnames := FetchServiceAddresses(svc)
log.Info("new instance", "namespace", svc.Namespace, "service", svc.Name, "addresses", instanceAddresses, "hostnames", instanceHostnames)
var newVips []*kubevip.Config
var link netlink.Link
var err error
var dnsAddresses []string
for _, address := range instanceAddresses {
// Detect if we're using a specific interface for services
@@ -131,7 +143,7 @@ func NewInstance(svc *v1.Service, config *kubevip.Config, intfMgr *networkinterf
if cidrs[0] != "" && cidrs[0] != kubevip.Auto {
subnet = cidrs[0]
} else {
subnet = "32"
subnet = strconv.Itoa(vip.DefaultMaskIPv4)
}
}
} else {
@@ -144,7 +156,7 @@ func NewInstance(svc *v1.Service, config *kubevip.Config, intfMgr *networkinterf
if len(cidrs) > 1 && cidrs[1] != "" && cidrs[1] != kubevip.Auto {
subnet = cidrs[1]
} else {
subnet = "128"
subnet = strconv.Itoa(vip.DefaultMaskIPv6)
}
}
}
@@ -164,6 +176,8 @@ func NewInstance(svc *v1.Service, config *kubevip.Config, intfMgr *networkinterf
ArpBroadcastRate: config.ArpBroadcastRate,
EnableServiceSecurity: config.EnableServiceSecurity,
DNSMode: config.DNSMode,
DHCPMode: config.DHCPMode,
DHCPBackoffAttempts: config.DHCPBackoffAttempts,
DisableServiceUpdates: config.DisableServiceUpdates,
EnableServicesElection: config.EnableServicesElection,
PreserveVIPOnLeadershipLoss: config.PreserveVIPOnLeadershipLoss,
@@ -173,22 +187,78 @@ func NewInstance(svc *v1.Service, config *kubevip.Config, intfMgr *networkinterf
})
}
for _, hostname := range instanceHostnames {
log.Info("hostname", "addr", hostname)
// Detect if we're using a specific interface for services
var svcInterface string
svcInterface = svc.Annotations[kubevip.ServiceInterface] // If the service has a specific interface defined, then use it
// If it is still blank then use the
if svcInterface == "" {
switch config.ServicesInterface {
case "":
svcInterface = config.Interface
default:
svcInterface = config.ServicesInterface
}
}
if link == nil {
if link, err = netlink.LinkByName(svcInterface); err != nil {
return nil, fmt.Errorf("failed to get interface %s: %w", svcInterface, err)
}
if link == nil {
return nil, fmt.Errorf("failed to get interface %s", svcInterface)
}
}
// Generate new Virtual IP configuration
newVips = append(newVips, &kubevip.Config{
VIP: hostname,
Interface: svcInterface,
SingleNode: true,
EnableARP: config.EnableARP,
EnableBGP: config.EnableBGP,
VIPSubnet: config.VIPSubnet,
EnableRoutingTable: config.EnableRoutingTable,
RoutingTableID: config.RoutingTableID,
RoutingTableType: config.RoutingTableType,
RoutingProtocol: config.RoutingProtocol,
ArpBroadcastRate: config.ArpBroadcastRate,
EnableServiceSecurity: config.EnableServiceSecurity,
DNSMode: config.DNSMode,
DHCPMode: config.DHCPMode,
DisableServiceUpdates: config.DisableServiceUpdates,
EnableServicesElection: config.EnableServicesElection,
KubernetesLeaderElection: kubevip.KubernetesLeaderElection{
EnableLeaderElection: config.EnableLeaderElection,
},
})
}
// Create new service
instance := &Instance{
//UID: instanceUID,
//VIPs: instanceAddresses,
ServiceSnapshot: svc,
dnsAddresses: dnsAddresses,
}
// for _, port := range svc.Spec.Ports {
// instance.ExternalPorts = append(instance.ExternalPorts, Port{
// Port: uint16(port.Port), //nolint
// Type: string(port.Protocol),
// })
// }
if svc.Annotations != nil {
instance.DHCPInterfaceHwaddr = svc.Annotations[kubevip.HwAddrKey]
instance.DHCPInterfaceIP = svc.Annotations[kubevip.RequestedIP]
requestedIP := svc.Annotations[kubevip.RequestedIP]
if requestedIP != "" {
requestedIPs := strings.Split(requestedIP, ",")
if len(requestedIPs) > 2 {
return nil, fmt.Errorf("annotation %q cannot request more than one IPv4 and one Ipv6 address", kubevip.RequestedIP)
}
for _, ip := range requestedIPs {
netip := net.ParseIP(ip)
if netip.To4() != nil {
instance.DHCPInterfaceIPv4 = ip
} else {
instance.DHCPInterfaceIPv6 = ip
}
}
}
instance.DHCPHostname = svc.Annotations[kubevip.LoadbalancerHostname]
}
@@ -212,29 +282,82 @@ func NewInstance(svc *v1.Service, config *kubevip.Config, intfMgr *networkinterf
// Create Add configuration to the new service
instance.VIPConfigs = newVips
// If this was purposely created with the address 0.0.0.0,
// If this was purposely created with the address '0.0.0.0', or '::'
// we will create a macvlan on the main interface and a DHCP client
// TODO: Consider how best to handle DHCP with multiple addresses
if len(instanceAddresses) == 1 && instanceAddresses[0] == "0.0.0.0" {
err := instance.startDHCP()
if err != nil {
return nil, err
}
select {
case err := <-instance.DHCPClient.ErrorChannel():
return nil, fmt.Errorf("error starting DHCP for %s/%s: error: %s",
instance.ServiceSnapshot.Namespace, instance.ServiceSnapshot.Name, err)
case ip := <-instance.DHCPClient.IPChannel():
instance.VIPConfigs[0].Interface = instance.DHCPInterface
instance.VIPConfigs[0].VIP = ip
instance.DHCPInterfaceIP = ip
}
if len(instanceAddresses) > 2 && (slices.Contains(instanceAddresses, "0.0.0.0") || slices.Contains(instanceAddresses, "::")) {
return nil, fmt.Errorf("DHCP cannot be used if more than 2 addresses (one IPv4 and one IPv6) were specified")
}
for i := range instance.VIPConfigs {
if instance.VIPConfigs[i].VIP == "0.0.0.0" {
err := instance.startDHCP(ctx, i, config.DHCPBackoffAttempts, wg)
if err != nil {
return nil, err
}
select {
case err := <-instance.DHCPv4Client.ErrorChannel():
return nil, fmt.Errorf("error starting DHCPv4 for %s/%s: error: %s",
instance.ServiceSnapshot.Namespace, instance.ServiceSnapshot.Name, err)
case ip := <-instance.DHCPv4Client.IPChannel():
instance.VIPConfigs[i].Interface = instance.DHCPInterface
instance.VIPConfigs[i].VIP = ip
instance.DHCPInterfaceIPv4 = ip
}
}
if instance.VIPConfigs[i].VIP == "::" {
err := instance.startDHCP(ctx, i, config.DHCPBackoffAttempts, wg)
if err != nil {
return nil, err
}
select {
case err := <-instance.DHCPv6Client.ErrorChannel():
return nil, fmt.Errorf("error starting DHCPv6 for %s/%s: error: %s",
instance.ServiceSnapshot.Namespace, instance.ServiceSnapshot.Name, err)
case ip := <-instance.DHCPv6Client.IPChannel():
instance.VIPConfigs[i].Interface = instance.DHCPInterface
instance.VIPConfigs[i].VIP = ip
instance.DHCPInterfaceIPv6 = ip
}
}
for _, vipConfig := range instance.VIPConfigs {
c, err := cluster.InitCluster(vipConfig, false, intfMgr, arpMgr)
ddnsAnnotation, exists := svc.Annotations[kubevip.ServiceDDNS]
if exists {
instance.VIPConfigs[i].DDNS, err = strconv.ParseBool(ddnsAnnotation)
if err != nil {
log.Error("Failed to add service", "err", err)
return nil, err
}
}
if len(svc.Spec.IPFamilies) > 0 {
if len(svc.Spec.IPFamilies) > 1 {
instance.VIPConfigs[i].DHCPMode = utils.DualFamily
instance.VIPConfigs[i].DNSMode = utils.DualFamily
switch *svc.Spec.IPFamilyPolicy {
case v1.IPFamilyPolicyRequireDualStack:
instance.VIPConfigs[i].IsDualStack = true
instance.VIPConfigs[i].RequireDualStack = true
case v1.IPFamilyPolicyPreferDualStack:
instance.VIPConfigs[i].IsDualStack = true
instance.VIPConfigs[i].RequireDualStack = false
default:
instance.VIPConfigs[i].IsDualStack = false
instance.VIPConfigs[i].RequireDualStack = false
}
} else {
if strings.EqualFold(string(svc.Spec.IPFamilies[0]), utils.IPv4Family) {
instance.VIPConfigs[i].DHCPMode = strings.ToLower(utils.IPv4Family)
instance.VIPConfigs[i].DNSMode = strings.ToLower(utils.IPv4Family)
} else {
instance.VIPConfigs[i].DHCPMode = strings.ToLower(utils.IPv6Family)
instance.VIPConfigs[i].DNSMode = strings.ToLower(utils.IPv6Family)
}
}
}
c, err := cluster.InitCluster(instance.VIPConfigs[i], false, intfMgr, arpMgr)
if err != nil {
log.Error("Failed to add Service %s/%s", svc.Namespace, svc.Name)
log.Error("failed to add service", "err", err)
return nil, err
}
@@ -243,7 +366,7 @@ func NewInstance(svc *v1.Service, config *kubevip.Config, intfMgr *networkinterf
}
instance.Clusters = append(instance.Clusters, c)
log.Info("(svcs) adding VIP", "ip", vipConfig.VIP, "interface", vipConfig.Interface, "namespace", svc.Namespace, "name", svc.Name)
log.Info("(svcs) adding VIP", "ip", instance.VIPConfigs[i].VIP, "interface", instance.VIPConfigs[i].Interface, "namespace", svc.Namespace, "name", svc.Name)
}
return instance, nil
@@ -306,11 +429,11 @@ func getAutoInterfaceName(link netlink.Link, defaultInterface string) string {
return link.Attrs().Name
}
func (i *Instance) startDHCP() error {
if len(i.VIPConfigs) != 1 {
return fmt.Errorf("DHCP requires exactly 1 VIP config, got: %v", len(i.VIPConfigs))
func (i *Instance) startDHCP(ctx context.Context, index int, backoffAttempts uint, wg *sync.WaitGroup) error {
if len(i.VIPConfigs) > 2 {
return fmt.Errorf("DHCP can be used with 2 VIP config maximally, got: %v", len(i.VIPConfigs))
}
parent, err := netlink.LinkByName(i.VIPConfigs[0].Interface)
parent, err := netlink.LinkByName(i.VIPConfigs[index].Interface)
if err != nil {
return fmt.Errorf("error finding VIP Interface, for building DHCP Link : %v", err)
}
@@ -361,34 +484,61 @@ func (i *Instance) startDHCP() error {
log.Info("Using existing macvlan interface for DHCP", "interface", interfaceName)
}
// Default rp_filter setting (https://github.com/kube-vip/kube-vip/issues/1170)
rpfilterSetting := "0"
var initRebootFlag bool
ip := net.ParseIP(i.VIPConfigs[index].VIP)
// Check if we need to set an override rp_filter value for the interface
if i.ServiceSnapshot.Annotations[kubevip.RPFilter] != "" {
// Check the rp_filter value
rpFilter, err := strconv.Atoi(i.ServiceSnapshot.Annotations[kubevip.RPFilter])
if err != nil {
slog.Error("[DHCP] unable to process rp_filter", "value", rpFilter)
} else {
if rpFilter >= 0 && rpFilter < 3 { // Ensure the value is 0,1,2
rpfilterSetting = i.ServiceSnapshot.Annotations[kubevip.RPFilter]
var client vip.DHCPClient
if ip.To4() != nil {
// Default rp_filter setting (https://github.com/kube-vip/kube-vip/issues/1170)
rpfilterSetting := "0"
// Check if we need to set an override rp_filter value for the interface
if i.ServiceSnapshot.Annotations[kubevip.RPFilter] != "" {
// Check the rp_filter value
rpFilter, err := strconv.Atoi(i.ServiceSnapshot.Annotations[kubevip.RPFilter])
if err != nil {
log.Error("[DHCP] unable to process rp_filter", "value", rpFilter)
} else {
slog.Error("[DHCP] rp_filter value not within range 0-2", "value", rpFilter)
if rpFilter >= 0 && rpFilter < 3 { // Ensure the value is 0,1,2
rpfilterSetting = i.ServiceSnapshot.Annotations[kubevip.RPFilter]
} else {
log.Error("[DHCP] rp_filter value not within range 0-2", "value", rpFilter)
}
}
}
}
err = sysctl.WriteProcSys("/proc/sys/net/ipv4/conf/"+interfaceName+"/rp_filter", rpfilterSetting)
if err != nil {
slog.Error("[DHCP] unable to write rp_filter", "value", rpfilterSetting, "err", err)
}
var initRebootFlag bool
if i.DHCPInterfaceIP != "" {
initRebootFlag = true
}
err = sysctl.WriteProcSys("/proc/sys/net/ipv4/conf/"+interfaceName+"/rp_filter", rpfilterSetting)
if err != nil {
log.Error("[DHCP] unable to write rp_filter", "value", rpfilterSetting, "err", err)
}
client := vip.NewDHCPClient(iface, initRebootFlag, i.DHCPInterfaceIP)
if i.DHCPInterfaceIPv4 != "" {
initRebootFlag = true
}
client = vip.NewDHCPv4Client(iface, initRebootFlag, i.DHCPInterfaceIPv4, backoffAttempts)
// Add the client so that we can call it to stop function
i.DHCPv4Client = client
// Set that DHCPv4 is enabled
i.IsDHCPv4 = true
} else {
if i.DHCPInterfaceIPv6 != "" {
initRebootFlag = true
}
client, err = vip.NewDHCPv6Client(iface, parent, initRebootFlag, i.DHCPInterfaceIPv6, backoffAttempts)
if err != nil {
return fmt.Errorf("unable to create client: %w", err)
}
// Add the client so that we can call it to stop function
i.DHCPv6Client = client
// Set that DHCPv6 is enabled
i.IsDHCPv6 = true
}
// Add hostname to dhcp client if annotated
if i.DHCPHostname != "" {
@@ -396,15 +546,15 @@ func (i *Instance) startDHCP() error {
client.WithHostName(i.DHCPHostname)
}
go client.Start()
wg.Go(func() {
if err := client.Start(ctx); err != nil {
log.Error("[instance] DHCP client error: %w")
}
})
// Set that DHCP is enabled
i.IsDHCP = true
// Set the name of the interface so that it can be removed on Service deletion
i.DHCPInterface = interfaceName
i.DHCPInterfaceHwaddr = iface.HardwareAddr.String()
// Add the client so that we can call it to stop function
i.DHCPClient = client
return nil
}
@@ -485,7 +635,6 @@ func FetchServiceAddresses(s *v1.Service) ([]string, []string) {
func FindServiceInstance(svc *v1.Service, instances []*Instance) *Instance {
log.Debug("finding service", "namespace", svc.Namespace, "name", svc.Name, "UID", svc.UID)
for i := range instances {
log.Debug("saved service", "instance", i, "UID", instances[i].ServiceSnapshot.UID)
if instances[i].ServiceSnapshot.UID == svc.UID {
return instances[i]
}
@@ -493,24 +642,3 @@ func FindServiceInstance(svc *v1.Service, instances []*Instance) *Instance {
log.Debug("instance not found", "namespace", svc.Namespace, "name", svc.Name, "UID", svc.UID)
return nil
}
func FindServiceInstanceWithTimeout(svc *v1.Service, instances []*Instance) *Instance {
log.Debug("finding service with timeout", "namespace", svc.Namespace, "name", svc.Name, "UID", svc.UID)
ticker := time.NewTicker(time.Millisecond * 200)
defer ticker.Stop()
to := time.NewTimer(time.Second * 60)
defer to.Stop()
for {
select {
case <-to.C:
return nil
case <-ticker.C:
for i := range instances {
log.Debug("saved service", "instance", i, "UID", instances[i].ServiceSnapshot.UID)
if instances[i].ServiceSnapshot.UID == svc.UID {
return instances[i]
}
}
}
}
}

View File

@@ -57,7 +57,14 @@ const (
// Enable UPNP on a Service
UpnpEnabled = "kube-vip.io/forwardUPNP"
// Set the UPNP lease duration for a specific service using duration format (e.g., "30s", "1h")
UpnpLeaseDuration = "kube-vip.io/upnp-lease-duration"
RPFilter = "kube-vip.io/rp_filter" // Set the return path filter for a specific service interface
// Name of the service lease object
ServiceLease = "kube-vip.io/leaseName"
// Enable DDNS for the service
ServiceDDNS = "kube-vip.io/ddns"
)

View File

@@ -11,6 +11,7 @@ import (
"strings"
"github.com/kube-vip/kube-vip/pkg/detector"
"github.com/kube-vip/kube-vip/pkg/utils"
"sigs.k8s.io/yaml"
)
@@ -25,7 +26,7 @@ func ParseEnvironment(c *Config) error {
if env != "" {
logLevel, err := strconv.ParseInt(env, 10, 32)
if err != nil {
panic("Unable to parse environment variable [vip_loglevel], should be int")
return fmt.Errorf("unable to parse environment variable [vip_loglevel], should be int: %w", err)
}
c.Logging = int32(logLevel)
}
@@ -301,9 +302,6 @@ func ParseEnvironment(c *Config) error {
return err
}
c.PreserveVIPOnLeadershipLoss = b
} else {
// Default to false for backward compatibility
c.PreserveVIPOnLeadershipLoss = false
}
// Wireguard Mode
@@ -379,6 +377,30 @@ func ParseEnvironment(c *Config) error {
c.DNSMode = env
}
// DHCP mode
env = os.Getenv(dhcpMode)
if env != "" {
c.DHCPMode = env
} else {
if c.DNSMode != "first" {
c.DHCPMode = c.DNSMode
} else {
c.DHCPMode = strings.ToLower(utils.IPv4Family)
}
}
// DHCP backoff attempts
env = os.Getenv(dhcpBackoffAttempts)
if env != "" {
tmp, err := strconv.ParseInt(env, 10, 32)
if err != nil {
return err
}
if tmp >= 0 {
c.DHCPBackoffAttempts = uint(tmp)
}
}
// Disable updates for services (status.LoadBalancer.Ingress will not be updated)
env = os.Getenv(disableServiceUpdates)
if env != "" {
@@ -878,6 +900,16 @@ func mergeConfigValues(baseConfig, fileConfig *Config) {
baseConfig.DNSMode = fileConfig.DNSMode
}
// DHCP configuration - mode
if baseConfig.DHCPMode == "" && fileConfig.DHCPMode != "" {
baseConfig.DHCPMode = fileConfig.DHCPMode
}
// DHCP configuration - backoff attempts
if baseConfig.DHCPBackoffAttempts == DefaultDHCPBackoffAttempts && fileConfig.DHCPBackoffAttempts != DefaultDHCPBackoffAttempts {
baseConfig.DHCPBackoffAttempts = fileConfig.DHCPBackoffAttempts
}
// Health check configuration
if baseConfig.HealthCheckPort == 0 && fileConfig.HealthCheckPort != 0 {
baseConfig.HealthCheckPort = fileConfig.HealthCheckPort

View File

@@ -212,6 +212,12 @@ const (
// dnsMode defines mode that DNS lookup will be performed with (first, ipv4, ipv6, dual)
dnsMode = "dns_mode"
// dhcpMode defines mode that DHCP lookup will be performed with (ipv4, ipv6, dual)
dhcpMode = "dhcp_mode"
// dhcpBackoffAttempts defines how many times DHCP client will try to obtain an IP address
dhcpBackoffAttempts = "dhcp_backoff_attempts"
// disableServiceUpdates disables service updating
disableServiceUpdates = "disable_service_updates"

View File

@@ -167,10 +167,10 @@ func GenerateRoleBinding(rolebinding bool, saCfg *applyCoreV1.ServiceAccountAppl
}
// generatePodSpec will take a kube-vip config and generate a Pod spec
func generatePodSpec(c *Config, image, imageVersion string, inCluster bool) *corev1.Pod {
func generatePodSpec(c *Config, image, imageVersion string, inCluster bool) (*corev1.Pod, error) {
imageRef, err := name.NewTag(image, name.WeakValidation, name.WithDefaultTag(imageVersion))
if err != nil {
panic(fmt.Errorf("Cannot parse %q: %w", image, err))
return nil, fmt.Errorf("cannot parse %q: %w", image, err)
}
command := "manager"
@@ -255,6 +255,28 @@ func generatePodSpec(c *Config, image, imageVersion string, inCluster bool) *cor
newEnvironment = append(newEnvironment, dnsModeSelector...)
}
if c.DHCPMode != "" {
// build environment variables
dhcpModeSelector := []corev1.EnvVar{
{
Name: dhcpMode,
Value: c.DHCPMode,
},
}
newEnvironment = append(newEnvironment, dhcpModeSelector...)
}
if c.DHCPBackoffAttempts != DefaultDHCPBackoffAttempts {
// build environment variables
dhcpBackoff := []corev1.EnvVar{
{
Name: dhcpBackoffAttempts,
Value: strconv.FormatUint(uint64(c.DHCPBackoffAttempts), 10),
},
}
newEnvironment = append(newEnvironment, dhcpBackoff...)
}
// If we're doing the hybrid mode
if c.EnableControlPlane {
cp := []corev1.EnvVar{
@@ -583,6 +605,16 @@ func generatePodSpec(c *Config, image, imageVersion string, inCluster bool) *cor
}
}
if c.PreserveVIPOnLeadershipLoss {
preserveVIPOnLeadershipLoss := []corev1.EnvVar{
{
Name: vipPreserveOnLeadershipLoss,
Value: strconv.FormatBool(c.PreserveVIPOnLeadershipLoss),
},
}
newEnvironment = append(newEnvironment, preserveVIPOnLeadershipLoss...)
}
newManifest := &corev1.Pod{
TypeMeta: metav1.TypeMeta{
Kind: "Pod",
@@ -637,18 +669,24 @@ func generatePodSpec(c *Config, image, imageVersion string, inCluster bool) *cor
newManifest.Spec.HostAliases = append(newManifest.Spec.HostAliases, hostAlias)
}
return newManifest
return newManifest, nil
}
// GeneratePodManifestFromConfig will take a kube-vip config and generate a manifest
func GeneratePodManifestFromConfig(c *Config, image, imageVersion string, inCluster bool) string {
newManifest := generatePodSpec(c, image, imageVersion, inCluster)
b, _ := yaml.Marshal(newManifest)
return string(b)
func GeneratePodManifestFromConfig(c *Config, image, imageVersion string, inCluster bool) (string, error) {
newManifest, err := generatePodSpec(c, image, imageVersion, inCluster)
if err != nil {
return "", err
}
b, err := yaml.Marshal(newManifest)
if err != nil {
return "", fmt.Errorf("failed to marshal manifest: %w", err)
}
return string(b), nil
}
// GenerateDaemonsetManifestFromConfig will take a kube-vip config and generate a manifest
func GenerateDaemonsetManifestFromConfig(c *Config, image, imageVersion string, inCluster, taint bool) string {
func GenerateDaemonsetManifestFromConfig(c *Config, image, imageVersion string, inCluster, taint bool) (string, error) {
// Determine where the pod should be deployed
var namespace string
if c.ServiceNamespace != "" {
@@ -657,7 +695,11 @@ func GenerateDaemonsetManifestFromConfig(c *Config, image, imageVersion string,
namespace = metav1.NamespaceSystem
}
podSpec := generatePodSpec(c, image, imageVersion, inCluster).Spec
pod, err := generatePodSpec(c, image, imageVersion, inCluster)
if err != nil {
return "", err
}
newManifest := &appv1.DaemonSet{
TypeMeta: metav1.TypeMeta{
Kind: "DaemonSet",
@@ -684,7 +726,7 @@ func GenerateDaemonsetManifestFromConfig(c *Config, image, imageVersion string,
"app.kubernetes.io/version": imageVersion,
},
},
Spec: podSpec,
Spec: pod.Spec,
},
},
}
@@ -734,5 +776,5 @@ func GenerateDaemonsetManifestFromConfig(c *Config, image, imageVersion string,
delete(m, "status")
b, _ = yaml.Marshal(m)
return string(b)
return string(b), nil
}

View File

@@ -158,6 +158,16 @@ type Config struct {
// DNSMode, this will set the mode DSN lookup will be performed (first, ipv4, ipv6, dual)
DNSMode string `yaml:"dnsDualStackMode"`
// IsDualStack reports if service is DualStack.
IsDualStack bool
// RequireDualStack defines if DualStack is required for the service. Based on service's Spec.ipFamilyPolicy field.
RequireDualStack bool
// DNSMode, this will set the mode DHCP lookup will be performed for DDNS (ipv4, ipv6, dual). By default will be the same as DNSMode.
// If DNSMode is 'first', IPv4 will be used.
DHCPMode string `yaml:"dhcpDualStackMode"`
// DisableServiceUpdates, if true, kube-vip will only advertise service, but it will not update service's Status.LoadBalancer.Ingress slice
DisableServiceUpdates bool `yaml:"disableServiceUpdates"`
@@ -189,6 +199,9 @@ type Config struct {
// ConfigFile defines the path to a JSON/YAML configuration file
ConfigFile string `yaml:"configFile"`
// DHCPBackoffAttempts defaines how many times will DHCP client try to obtain address (unlimited when 0)
DHCPBackoffAttempts uint `yaml:"dhcpBackoffAttempts"`
}
// KubernetesLeaderElection defines all of the settings for Kubernetes KubernetesLeaderElection

View File

@@ -2,4 +2,6 @@ package kubevip
const (
LBClassName = "kube-vip.io/kube-vip-class"
DefaultDHCPBackoffAttempts = 3
)

View File

@@ -3,7 +3,11 @@ package lease
import (
"context"
"fmt"
log "log/slog"
"os"
"strings"
"sync"
"sync/atomic"
"github.com/kube-vip/kube-vip/pkg/kubevip"
v1 "k8s.io/api/core/v1"
@@ -22,106 +26,209 @@ func NewManager() *Manager {
}
}
// Add adds lease or incerements counter if lease is alreay used.
func (m *Manager) Add(service *v1.Service) (*Lease, bool) {
// Add adds lease to the manager.
// It returns three values:
// - lease for the object
// - isNewObject, which reports if it is a new object that is being handled
// - isSharedLease, which is true if object shares the lease with another object
// If object is new but not shared, we should start leaderelection and sync it
// If object is new and shared, we should only sync it as the leaderelection should be already handled
// If object is not new we should do nothing
func (m *Manager) Add(ctx context.Context, id ID) *Lease {
m.lock.Lock()
defer m.lock.Unlock()
_, id := GetName(service)
if _, exist := m.leases[id]; !exist {
ctx, cancel := context.WithCancel(context.Background())
m.leases[id] = newLease(ctx, cancel)
return m.leases[id], true
if _, exists := m.leases[id.NamespacedName()]; !exists {
leaseCtx, leaseCancel := context.WithCancel(ctx)
m.leases[id.NamespacedName()] = newLease(leaseCtx, leaseCancel)
}
m.leases[id].increment()
return m.leases[id], false
return m.leases[id.NamespacedName()]
}
// Delete decrements lease counter and removes the lease if counter equals 0.
func (m *Manager) Delete(service *v1.Service) {
// Delete removes the lease and cancels it if the lease counter equals 0.
func (m *Manager) Delete(id ID, objectName string) {
m.lock.Lock()
defer m.lock.Unlock()
_, id := GetName(service)
if _, exist := m.leases[id]; exist {
m.leases[id].decrement()
if m.leases[id].cnt < 1 {
delete(m.leases, id)
if _, exist := m.leases[id.NamespacedName()]; exist {
m.leases[id.NamespacedName()].delete(objectName)
if m.leases[id.NamespacedName()].cnt.Load() < 1 {
m.leases[id.NamespacedName()].Cancel()
delete(m.leases, id.NamespacedName())
}
}
}
// Get returns lease for the service.
func (m *Manager) Get(service *v1.Service) *Lease {
func (m *Manager) Get(id ID) *Lease {
m.lock.Lock()
defer m.lock.Unlock()
_, id := GetName(service)
if lease, exist := m.leases[id]; exist {
if lease, exist := m.leases[id.NamespacedName()]; exist {
return lease
}
return nil
}
// GetLeaderContext returns leder context for the service.
func (m *Manager) GetLeaderContext(service *v1.Service) context.Context {
m.lock.Lock()
defer m.lock.Unlock()
_, id := GetName(service)
if _, ok := m.leases[id]; !ok {
return nil
}
return m.leases[id].Ctx
}
// Lease holds lease data.
type Lease struct {
cnt uint
Lock *sync.Mutex
Ctx context.Context
Cancel context.CancelFunc
Started chan any
Ctx context.Context
Cancel context.CancelFunc
Started chan any
services sync.Map
cnt atomic.Int64
Elected atomic.Bool
Mtx sync.Mutex
locked bool
}
func newLease(ctx context.Context, cancel context.CancelFunc) *Lease {
return &Lease{
Ctx: ctx,
Cancel: cancel,
cnt: 1,
Lock: new(sync.Mutex),
Started: make(chan any),
}
}
func (l *Lease) increment() {
l.Lock.Lock()
defer l.Lock.Unlock()
l.cnt++
// Add adds the object to the lease and increments counter
// it will return true if object was added
func (l *Lease) Add(name string) bool {
if _, exists := l.services.Load(name); !exists {
l.services.Store(name, true)
l.cnt.Add(1)
return true
}
return false
}
func (l *Lease) decrement() {
l.Lock.Lock()
defer l.Lock.Unlock()
if l.cnt == 0 {
return
}
l.cnt--
if l.cnt < 1 {
l.Cancel()
// delete removes the service from the lease and decrements the counter
func (l *Lease) delete(service string) {
if _, exists := l.services.Load(service); exists {
l.services.Delete(service)
l.cnt.Add(-1)
}
}
// GetName gets lease name and id for the service.
func GetName(service *v1.Service) (string, string) {
serviceLease, exists := service.Annotations[kubevip.ServiceLease]
if !exists || serviceLease == "" {
serviceLease = fmt.Sprintf("kubevip-%s", service.Name)
}
serviceLeaseID := fmt.Sprintf("%s/%s", serviceLease, service.Namespace)
return serviceLease, serviceLeaseID
func (l *Lease) Lock() {
l.Mtx.Lock()
l.locked = true
}
// UsesCommon checks if service uses common lease feature.
func UsesCommon(service *v1.Service) bool {
_, common := service.Annotations[kubevip.ServiceLease]
return common
func (l *Lease) Unlock() {
if l.locked {
l.locked = false
l.Mtx.Unlock()
}
}
// ServiceName gets lease name and id for the service.
func ServiceName(service *v1.Service) (string, string) {
name, exists := service.Annotations[kubevip.ServiceLease]
if !exists || name == "" {
name = fmt.Sprintf("kubevip-%s", service.Name)
}
serviceLeaseParts := strings.Split(name, "/")
namespace := service.Namespace
if len(serviceLeaseParts) > 1 {
namespace = serviceLeaseParts[0]
name = serviceLeaseParts[1]
}
return namespace, name
}
func ServiceNamespacedName(service *v1.Service) string {
return fmt.Sprintf("%s/%s", service.Namespace, service.Name)
}
func ObjectName(id ID, suffix string) string {
return fmt.Sprintf("%s-%s", id.NamespacedName(), suffix)
}
func NamespaceName(lease string, c *kubevip.Config) (string, string) {
leaseName := lease
leasnameParts := strings.Split(lease, "/")
var ns string
var err error
if len(leasnameParts) > 1 {
ns = leasnameParts[0]
leaseName = leasnameParts[1]
} else {
ns, err = returnNamespace()
if err != nil {
log.Warn("unable to auto-detect namespace, dropping to config", "namespace", c.Namespace)
ns = c.Namespace
}
}
return ns, leaseName
}
func returnNamespace() (string, error) {
if data, err := os.ReadFile("/var/run/secrets/kubernetes.io/serviceaccount/namespace"); err == nil {
if ns := strings.TrimSpace(string(data)); len(ns) > 0 {
return ns, nil
}
return "", err
}
return "", fmt.Errorf("unable to find Namespace")
}
type ID interface {
Name() string
Namespace() string
NamespacedName() string
}
type CommonID struct {
namespace string
name string
}
func NewID(leaseType, namespace, name string) ID {
if leaseType == "etcd" {
return newEtcdID(namespace, name)
}
return newKubernetesID(namespace, name)
}
func newKubernetesID(namespace, name string) ID {
return &KubernetesID{
CommonID: CommonID{
namespace: namespace,
name: name,
},
}
}
func newEtcdID(namespace, name string) ID {
return &EtcdID{
CommonID: CommonID{
namespace: namespace,
name: name,
},
}
}
func (c *CommonID) Name() string {
return c.name
}
func (c *CommonID) Namespace() string {
return c.namespace
}
type KubernetesID struct {
CommonID
}
func (k *KubernetesID) NamespacedName() string {
return fmt.Sprintf("%s/%s", k.namespace, k.name)
}
type EtcdID struct {
CommonID
}
func (e *EtcdID) NamespacedName() string {
return fmt.Sprintf("%s-%s", e.namespace, e.name)
}

939
pkg/lease/lease_test.go Normal file
View File

@@ -0,0 +1,939 @@
package lease
import (
"context"
"sync"
"testing"
"time"
"github.com/kube-vip/kube-vip/pkg/kubevip"
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
func createTestService(name, namespace string, annotations map[string]string) *v1.Service {
return &v1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: name,
Namespace: namespace,
Annotations: annotations,
},
}
}
func getSvcID(svc *v1.Service) ID {
namespace, name := ServiceName(svc)
id := NewID("kubernetes", namespace, name)
return id
}
func getSvcData(svc *v1.Service) (context.Context, ID) {
return context.TODO(), getSvcID(svc)
}
const serviceLeaseAnnotation = kubevip.ServiceLease
// TestManager_Add_NewLease tests adding a new service with a new lease
func TestManager_Add_NewLease(t *testing.T) {
mgr := NewManager()
svc := createTestService("test-svc", "default", nil)
leaseID := mgr.Add(getSvcData(svc))
isNew := leaseID.Add(ServiceNamespacedName(svc))
if !isNew {
t.Error("expected isNew to be true for first Add")
}
if leaseID == nil {
t.Fatal("expected lease to be non-nil")
}
if leaseID.Ctx == nil {
t.Error("expected lease context to be non-nil")
}
if leaseID.Cancel == nil {
t.Error("expected lease cancel func to be non-nil")
}
if leaseID.Started == nil {
t.Error("expected lease Started channel to be non-nil")
}
}
// TestManager_Add_ExistingLease tests adding a service with an existing lease
func TestManager_Add_ExistingLease(t *testing.T) {
mgr := NewManager()
svc := createTestService("test-svc", "default", nil)
leaseID1 := mgr.Add(getSvcData(svc))
isNew1 := leaseID1.Add(ServiceNamespacedName(svc))
leaseID2 := mgr.Add(getSvcData(svc))
isNew2 := leaseID2.Add(ServiceNamespacedName(svc))
if !isNew1 {
t.Error("expected first Add to return isNew=true")
}
if isNew2 {
t.Error("expected second Add to return isNew=false")
}
if leaseID1 != leaseID2 {
t.Error("expected same lease to be returned for same service")
}
}
// TestManager_Delete_DecrementCounter tests the decrement counter functionality
func TestManager_Delete_DecrementCounter(t *testing.T) {
mgr := NewManager()
svc := createTestService("test-svc", "default", nil)
// Add twice (simulating adding same service twice)
objectName := ServiceNamespacedName(svc)
ctx1, leaseID1 := getSvcData(svc)
lease1 := mgr.Add(ctx1, leaseID1)
_ = lease1.Add(objectName)
ctx2, leaseID2 := getSvcData(svc)
lease2 := mgr.Add(ctx2, leaseID2)
_ = lease2.Add(objectName)
// Delete once - should remove the lease
mgr.Delete(leaseID1, objectName)
lease := mgr.Get(getSvcID(svc))
if lease != nil {
t.Error("expected lease to be removed after first delete if same service was processed twice")
}
}
// TestManager_Delete_CancelsContext tests the context cancellation on delete
func TestManager_Delete_CancelsContext(t *testing.T) {
mgr := NewManager()
svc := createTestService("test-svc", "default", nil)
objectName := ServiceNamespacedName(svc)
ctx1, leaseID1 := getSvcData(svc)
lease1 := mgr.Add(ctx1, leaseID1)
_ = lease1.Add(objectName)
// Verify context is not cancelled
select {
case <-lease1.Ctx.Done():
t.Fatal("expected context to not be cancelled initially")
default:
// Expected
}
// Delete the lease
mgr.Delete(leaseID1, objectName)
// Verify context is cancelled
select {
case <-lease1.Ctx.Done():
// Expected
case <-time.After(100 * time.Millisecond):
t.Error("expected context to be cancelled after delete")
}
}
// TestManager_Add_AfterDelete_CreatesNewLease tests adding a service after deleting it
func TestManager_Add_AfterDelete_CreatesNewLease(t *testing.T) {
mgr := NewManager()
svc := createTestService("test-svc", "default", nil)
objectName := ServiceNamespacedName(svc)
ctx1, leaseID1 := getSvcData(svc)
lease1 := mgr.Add(ctx1, leaseID1)
_ = lease1.Add(objectName)
mgr.Delete(leaseID1, objectName)
ctx2, leaseID2 := getSvcData(svc)
lease2 := mgr.Add(ctx2, leaseID2)
isNew := lease2.Add(objectName)
if !isNew {
t.Error("expected isNew to be true after delete and re-add")
}
if lease1 == lease2 {
t.Error("expected new lease to be different from old lease")
}
}
// TestManager_Add_DifferentServices tests adding services with different names
func TestManager_Add_DifferentServices(t *testing.T) {
mgr := NewManager()
svc1 := createTestService("svc1", "default", nil)
svc2 := createTestService("svc2", "default", nil)
objectName1 := ServiceNamespacedName(svc1)
ctx1, leaseID1 := getSvcData(svc1)
lease1 := mgr.Add(ctx1, leaseID1)
isNew1 := lease1.Add(objectName1)
objectName2 := ServiceNamespacedName(svc2)
ctx2, leaseID2 := getSvcData(svc2)
lease2 := mgr.Add(ctx2, leaseID2)
isNew2 := lease1.Add(objectName2)
if !isNew1 || !isNew2 {
t.Error("expected both adds to return isNew=true")
}
if lease1 == lease2 {
t.Error("expected different leases for different services")
}
}
// TestManager_Add_SameNameDifferentNamespace tests adding services with the same name but different namespaces
func TestManager_Add_SameNameDifferentNamespace(t *testing.T) {
mgr := NewManager()
svc1 := createTestService("test-svc", "namespace1", nil)
svc2 := createTestService("test-svc", "namespace2", nil)
objectName1 := ServiceNamespacedName(svc1)
ctx1, leaseID1 := getSvcData(svc1)
lease1 := mgr.Add(ctx1, leaseID1)
isNew1 := lease1.Add(objectName1)
objectName2 := ServiceNamespacedName(svc2)
ctx2, leaseID2 := getSvcData(svc2)
lease2 := mgr.Add(ctx2, leaseID2)
isNew2 := lease1.Add(objectName2)
if !isNew1 || !isNew2 {
t.Error("expected both adds to return isNew=true")
}
if lease1 == lease2 {
t.Error("expected different leases for services in different namespaces")
}
}
// TestManager_ConcurrentAccess tests concurrent access to the lease manager
func TestManager_ConcurrentAccess(t *testing.T) {
mgr := NewManager()
svc := createTestService("test-svc", "default", nil)
var wg sync.WaitGroup
const numGoroutines = 100
objectName1 := ServiceNamespacedName(svc)
ctx1, leaseID1 := getSvcData(svc)
lease1 := mgr.Add(ctx1, leaseID1)
added := lease1.Add(objectName1)
if !added {
t.Error("expected lease to be added")
}
// Concurrent adds
for range numGoroutines {
wg.Go(func() {
added := lease1.Add(objectName1)
if added {
t.Error("expected lease to already exist")
}
})
}
wg.Wait()
mgr.Delete(leaseID1, objectName1)
// After a one delete, lease should be gone
lease := mgr.Get(getSvcID(svc))
if lease != nil {
t.Error("expected lease to be removed after all concurrent deletes")
}
}
// TestLease_StartedChannel tests the Started channel behavior
func TestLease_StartedChannel(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
lease := newLease(ctx, cancel)
// Started channel should be open initially
select {
case <-lease.Started:
t.Fatal("expected Started channel to be open initially")
default:
// Expected
}
// Close the channel
close(lease.Started)
// Now it should be closed
select {
case <-lease.Started:
// Expected
default:
t.Error("expected Started channel to be closed after close()")
}
}
// TestGetName_WithoutAnnotation tests with no annotation
func TestGetName_WithoutAnnotation(t *testing.T) {
svc := createTestService("my-service", "my-namespace", nil)
namespace, name := ServiceName(svc)
id := NewID("kubernetes", namespace, name)
expectedName := "kubevip-my-service"
expectedID := "my-namespace/kubevip-my-service"
if id.Name() != expectedName {
t.Errorf("expected name %q, got %q", expectedName, id.Name())
}
if id.NamespacedName() != expectedID {
t.Errorf("expected id %q, got %q", expectedID, id.NamespacedName())
}
}
// TestGetName_WithAnnotation tests with a shared lease annotation
func TestGetName_WithAnnotation(t *testing.T) {
svc := createTestService("my-service", "my-namespace", map[string]string{
serviceLeaseAnnotation: "shared-lease",
})
namespace, name := ServiceName(svc)
id := NewID("kubernetes", namespace, name)
expectedName := "shared-lease"
expectedID := "my-namespace/shared-lease"
if id.Name() != expectedName {
t.Errorf("expected name %q, got %q", expectedName, id.Name())
}
if id.NamespacedName() != expectedID {
t.Errorf("expected id %q, got %q", expectedID, id.NamespacedName())
}
}
// TestGetName_WithAnnotation tests with a shared lease annotation
func TestGetName_WithAnnotationAndOverriddenNamespace(t *testing.T) {
svc := createTestService("my-service", "my-namespace", map[string]string{
serviceLeaseAnnotation: "other-namespace/shared-lease",
})
namespace, name := ServiceName(svc)
id := NewID("kubernetes", namespace, name)
expectedName := "shared-lease"
expectedID := "other-namespace/shared-lease"
expectedNamespace := "other-namespace"
if id.Name() != expectedName {
t.Errorf("expected name %q, got %q", expectedName, id.Name())
}
if id.NamespacedName() != expectedID {
t.Errorf("expected id %q, got %q", expectedID, id.NamespacedName())
}
if id.Namespace() != expectedNamespace {
t.Errorf("expected namespace %q, got %q", expectedNamespace, id.Namespace())
}
}
// TestGetName_WithoutAnnotation_Etcd tests with no annotation
func TestGetName_WithoutAnnotation_Etcd(t *testing.T) {
svc := createTestService("my-service", "my-namespace", nil)
namespace, name := ServiceName(svc)
id := NewID("etcd", namespace, name)
expectedName := "kubevip-my-service"
expectedID := "my-namespace-kubevip-my-service"
if id.Name() != expectedName {
t.Errorf("expected name %q, got %q", expectedName, id.Name())
}
if id.NamespacedName() != expectedID {
t.Errorf("expected id %q, got %q", expectedID, id.NamespacedName())
}
}
// TestGetName_WithAnnotation_Etcd tests with a shared lease annotation
func TestGetName_WithAnnotation_Etcd(t *testing.T) {
svc := createTestService("my-service", "my-namespace", map[string]string{
serviceLeaseAnnotation: "shared-lease",
})
namespace, name := ServiceName(svc)
id := NewID("etcd", namespace, name)
expectedName := "shared-lease"
expectedID := "my-namespace-shared-lease"
if id.Name() != expectedName {
t.Errorf("expected name %q, got %q", expectedName, id.Name())
}
if id.NamespacedName() != expectedID {
t.Errorf("expected id %q, got %q", expectedID, id.NamespacedName())
}
}
// TestGetName_WithAnnotation_Etcd tests with a shared lease annotation
func TestGetName_WithAnnotationAndOverriddenNamespace_Etcd(t *testing.T) {
svc := createTestService("my-service", "my-namespace", map[string]string{
serviceLeaseAnnotation: "other-namespace/shared-lease",
})
namespace, name := ServiceName(svc)
id := NewID("etcd", namespace, name)
expectedName := "shared-lease"
expectedID := "other-namespace-shared-lease"
expectedNamespace := "other-namespace"
if id.Name() != expectedName {
t.Errorf("expected name %q, got %q", expectedName, id.Name())
}
if id.NamespacedName() != expectedID {
t.Errorf("expected id %q, got %q", expectedID, id.NamespacedName())
}
if id.Namespace() != expectedNamespace {
t.Errorf("expected namespace %q, got %q", expectedNamespace, id.Namespace())
}
}
// TestManager_LeaderElectionRestartScenario simulates the bug scenario where
// leadership is lost and the restartable service watcher tries to restart
// the leader election. This test verifies that after deleting the lease,
// a new lease can be created.
func TestManager_LeaderElectionRestartScenario_etcd(t *testing.T) {
mgr := NewManager()
svc := createTestService("traefik", "traefik", nil)
objectName1 := ServiceNamespacedName(svc)
ctx1, leaseID1 := getSvcData(svc)
lease1 := mgr.Add(ctx1, leaseID1)
isNew1 := lease1.Add(objectName1)
if !isNew1 {
t.Fatal("expected first add to return isNew=true")
}
// Simulate leadership acquired - close Started channel
close(lease1.Started)
// Simulate leadership lost - the leader election function should delete the lease
// This is the fix: delete the lease when RunOrDie returns
mgr.Delete(leaseID1, objectName1)
// Verify lease is removed
if mgr.Get(getSvcID(svc)) != nil {
t.Error("expected lease to be removed after delete")
}
// Simulate restartable service watcher calling StartServicesLeaderElection again
ctx2, leaseID2 := getSvcData(svc)
lease2 := mgr.Add(ctx2, leaseID2)
isNew2 := lease1.Add(objectName1)
if !isNew2 {
t.Fatal("expected second add after delete to return isNew=true")
}
// Verify we got a new lease with a fresh Started channel
if lease1 == lease2 {
t.Error("expected new lease to be different from old lease")
}
// Verify the new Started channel is not closed
select {
case <-lease2.Started:
t.Error("expected new lease's Started channel to be open")
default:
// Expected
}
}
// TestManager_CommonLeaseScenario tests the common lease feature where
// multiple services share the same lease.
func TestManager_CommonLeaseScenario(t *testing.T) {
mgr := NewManager()
// Two services sharing the same lease via annotation
sharedLeaseAnnotations := map[string]string{
serviceLeaseAnnotation: "shared-lease",
}
svc1 := createTestService("svc1", "default", sharedLeaseAnnotations)
svc2 := createTestService("svc2", "default", sharedLeaseAnnotations)
// First service gets a new lease
objectName1 := ServiceNamespacedName(svc1)
ctx1, leaseID1 := getSvcData(svc1)
lease1 := mgr.Add(ctx1, leaseID1)
isNew1 := lease1.Add(objectName1)
if !isNew1 {
t.Error("expected first add to return isNew=true")
}
// Simulate first service starting leadership
close(lease1.Started)
objectName2 := ServiceNamespacedName(svc2)
ctx2, leaseID2 := getSvcData(svc2)
lease2 := mgr.Add(ctx2, leaseID2)
isNew2 := lease2.Add(objectName2)
// Second service should get the same lease
if !isNew2 {
t.Error("expected second add with same lease name to return isNew=true")
}
if lease1 != lease2 {
t.Error("expected same lease for services with same lease annotation")
}
// Delete first service - lease should still exist
mgr.Delete(leaseID1, objectName1)
if mgr.Get(getSvcID(svc1)) == nil {
t.Error("expected lease to still exist after first delete")
}
// Delete second service - lease should be removed
mgr.Delete(leaseID2, objectName2)
if mgr.Get(getSvcID(svc2)) != nil {
t.Error("expected lease to be removed after all services deleted")
}
}
// TestManager_RaceCondition_LeaseExistsBeforeDelete tests the scenario where
// a second goroutine calls Add before the first goroutine's defer deletes the lease.
// This simulates the race condition that could cause the gaps in the logs where there is no leader.
func TestManager_RaceCondition_LeaseExistsBeforeDelete(t *testing.T) {
mgr := NewManager()
svc := createTestService("traefik", "traefik", nil)
// Simulate first leader election start
objectName1 := ServiceNamespacedName(svc)
ctx1, leaseID1 := getSvcData(svc)
lease1 := mgr.Add(ctx1, leaseID1)
isNew1 := lease1.Add(objectName1)
if !isNew1 {
t.Fatal("expected first add to return isNew=true")
}
// Simulate leadership acquired - close Started channel
close(lease1.Started)
// Simulate a second goroutine calling Add BEFORE the first goroutine's defer deletes the lease
// This is the race condition scenario
ctx2, leaseID2 := getSvcData(svc)
lease2 := mgr.Add(ctx2, leaseID2)
isNew2 := lease1.Add(objectName1)
if isNew2 {
t.Error("expected second add before delete to return isNew=false")
}
if lease1 != lease2 {
t.Error("expected same lease to be returned")
}
// The Started channel should be closed (from the first run)
select {
case <-lease2.Started:
// Expected - channel is closed
default:
t.Error("expected Started channel to be closed")
}
// Now the first goroutine's defer deletes the lease
mgr.Delete(leaseID1, objectName1)
// The lease should not still exist because same service was processed twice, so we do not increment the counter
if mgr.Get(getSvcID(svc)) != nil {
t.Error("expected lease tonot exist")
}
// Second delete does nothing
mgr.Delete(leaseID2, objectName1)
if mgr.Get(getSvcID(svc)) != nil {
t.Error("expected lease to not exist")
}
}
// TestManager_NonCommonLease_MultipleAdds tests that multiple Adds for a non-common
// lease service increment the counter correctly.
func TestManager_NonCommonLease_MultipleAdds(t *testing.T) {
mgr := NewManager()
svc := createTestService("traefik", "traefik", nil) // No common lease annotation
// First Add
objectName1 := ServiceNamespacedName(svc)
ctx1, leaseID1 := getSvcData(svc)
lease1 := mgr.Add(ctx1, leaseID1)
isNew1 := lease1.Add(objectName1)
if !isNew1 {
t.Error("expected first add to return isNew=true")
}
// Close Started to simulate leadership acquired
close(lease1.Started)
// Second Add (simulating another goroutine or restart attempt)
ctx2, leaseID2 := getSvcData(svc)
lease2 := mgr.Add(ctx2, leaseID2)
isNew2 := lease1.Add(objectName1)
if isNew2 {
t.Error("expected second add to return isNew=false")
}
if lease1 != lease2 {
t.Error("expected same lease")
}
// Third Add
ctx3, leaseID3 := getSvcData(svc)
lease3 := mgr.Add(ctx3, leaseID3)
isNew3 := lease1.Add(objectName1)
if isNew3 {
t.Error("expected third add to return isNew=false")
}
if lease1 != lease3 {
t.Error("expected same lease")
}
// Need one delete to remove the lease, another delete runs do nothing
mgr.Delete(leaseID1, objectName1)
if mgr.Get(getSvcID(svc)) != nil {
t.Error("expected lease to be deleted")
}
mgr.Delete(leaseID2, objectName1)
if mgr.Get(getSvcID(svc)) != nil {
t.Error("expected lease to be deleted")
}
mgr.Delete(leaseID3, objectName1)
if mgr.Get(getSvcID(svc)) != nil {
t.Error("expected lease to be deleted")
}
}
// TestManager_LeaseContextCancelledBeforeStarted tests the scenario where
// the lease context is cancelled before the Started channel is closed.
// This can happen if leadership is never acquired and the context times out.
func TestManager_LeaseContextCancelledBeforeStarted(t *testing.T) {
mgr := NewManager()
svc := createTestService("traefik", "traefik", nil)
// First Add
objectName1 := ServiceNamespacedName(svc)
ctx1, leaseID1 := getSvcData(svc)
lease1 := mgr.Add(ctx1, leaseID1)
isNew1 := lease1.Add(objectName1)
if !isNew1 {
t.Fatal("expected first add to return isNew=true")
}
ctx2, leaseID2 := getSvcData(svc)
lease2 := mgr.Add(ctx2, leaseID2)
isNew2 := lease1.Add(objectName1)
if isNew2 {
t.Error("expected second add to return isNew=false")
}
// Verify Started is not closed yet
select {
case <-lease2.Started:
t.Error("expected Started channel to be open")
default:
// Expected
}
// Cancel the lease context (simulating timeout or leadership loss before acquiring)
lease1.Cancel()
// Verify context is cancelled
select {
case <-lease2.Ctx.Done():
// Expected
case <-time.After(100 * time.Millisecond):
t.Error("expected context to be cancelled")
}
// Delete should still work
mgr.Delete(leaseID1, objectName1)
mgr.Delete(leaseID2, objectName1)
if mgr.Get(getSvcID(svc)) != nil {
t.Error("expected lease to be removed")
}
}
// TestManager_RestartAfterLeaseContextCancelled tests that after the lease
// context is cancelled and the lease is deleted, a new lease can be created.
func TestManager_RestartAfterLeaseContextCancelled(t *testing.T) {
mgr := NewManager()
svc := createTestService("traefik", "traefik", nil)
// First Add
objectName1 := ServiceNamespacedName(svc)
ctx1, leaseID1 := getSvcData(svc)
lease1 := mgr.Add(ctx1, leaseID1)
_ = lease1.Add(objectName1)
// Cancel context before Started is closed
lease1.Cancel()
// Delete the lease
mgr.Delete(leaseID1, objectName1)
// Verify lease is gone
if mgr.Get(getSvcID(svc)) != nil {
t.Error("expected lease to be removed after delete")
}
// Add again - should create new lease
ctx2, leaseID2 := getSvcData(svc)
lease2 := mgr.Add(ctx2, leaseID2)
isNew2 := lease1.Add(objectName1)
if !isNew2 {
t.Error("expected new lease after delete")
}
// Verify new lease has fresh context and Started channel
select {
case <-lease2.Ctx.Done():
t.Error("expected new lease context to be active")
default:
// Expected
}
select {
case <-lease2.Started:
t.Error("expected new lease Started channel to be open")
default:
// Expected
}
}
// TestManager_NonCommonLease_WaitForLeaseContextDone tests the scenario where
// a non-common lease service calls Add while another leader election is running.
// The caller should wait for the lease context to be done before returning.
// This test verifies the fix for the tight spin loop issue.
func TestManager_NonCommonLease_WaitForLeaseContextDone(t *testing.T) {
mgr := NewManager()
svc := createTestService("egress-service", "default", nil) // Non-common lease
// First Add - simulates the first leader election starting
objectName1 := ServiceNamespacedName(svc)
ctx1, leaseID1 := getSvcData(svc)
lease1 := mgr.Add(ctx1, leaseID1)
isNew1 := lease1.Add(objectName1)
if !isNew1 {
t.Fatal("expected first add to return isNew=true")
}
// Simulate leadership acquired
close(lease1.Started)
// Second Add - simulates another goroutine trying to start leader election
// This should return isNew=false
ctx2, leaseID2 := getSvcData(svc)
lease2 := mgr.Add(ctx2, leaseID2)
isNew2 := lease1.Add(objectName1)
if isNew2 {
t.Error("expected second add to return isNew=false")
}
if lease1 != lease2 {
t.Error("expected same lease to be returned")
}
// Verify Started channel is closed (leadership was acquired by first)
select {
case <-lease2.Started:
// Expected - channel is closed
default:
t.Error("expected Started channel to be closed")
}
// In the actual code (leader.go), when isNew=false for non-common lease,
// the code waits on either svcCtx.Ctx.Done() or svcLease.Ctx.Done()
// Here we verify that the lease context gets cancelled when we delete the lease
// Start a goroutine that waits for the lease context to be done
// This simulates what the leader.go code does
waitDone := make(chan struct{})
go func() {
select {
case <-lease2.Ctx.Done():
close(waitDone)
case <-time.After(1 * time.Second):
// Timeout - test will fail
}
}()
// Verify the goroutine is still waiting (lease context not yet cancelled)
select {
case <-waitDone:
t.Fatal("goroutine should still be waiting")
case <-time.After(50 * time.Millisecond):
// Expected - still waiting
}
// Now simulate the first leader election ending (defer deletes the lease)
mgr.Delete(leaseID1, objectName1)
// The lease context should now be cancelled (because counter went to 0)
// But we added twice, so we need to delete twice
mgr.Delete(leaseID2, objectName1)
// Now the goroutine should have completed
select {
case <-waitDone:
// Expected - lease context was cancelled
case <-time.After(200 * time.Millisecond):
t.Error("expected goroutine to complete after lease context cancelled")
}
// Verify lease is removed
if mgr.Get(getSvcID(svc)) != nil {
t.Error("expected lease to be removed")
}
}
// TestManager_NonCommonLease_SpinLoopPrevention tests that the fix prevents
// a tight spin loop when a non-common lease service repeatedly calls Add
// while leader election is running. The key behavior is that when isNew=false,
// the lease context should be used to block until the leader election ends.
func TestManager_NonCommonLease_SpinLoopPrevention(t *testing.T) {
mgr := NewManager()
svc := createTestService("egress-service", "default", nil) // Non-common lease
// First Add - leader election starts
objectName1 := ServiceNamespacedName(svc)
ctx1, leaseID1 := getSvcData(svc)
lease1 := mgr.Add(ctx1, leaseID1)
isNew1 := lease1.Add(objectName1)
if !isNew1 {
t.Fatal("expected first add to return isNew=true")
}
close(lease1.Started)
// Track how many times Add is called in a tight loop
// In the buggy code, this would spin forever
// In the fixed code, Add returns isNew=false and the caller blocks on lease.Ctx.Done()
addCount := 0
done := make(chan struct{})
go func() {
for i := 0; i < 100; i++ {
objectName1 := ServiceNamespacedName(svc)
ctxTmp, leaseIDTmp := getSvcData(svc)
leaseTmp := mgr.Add(ctxTmp, leaseIDTmp)
isNewTmp := leaseTmp.Add(objectName1)
addCount++
if isNewTmp {
// This shouldn't happen while the first lease exists
t.Error("unexpected isNew=true")
break
}
// In the fixed code, we would block here on lease.Ctx.Done()
// For this test, we just verify that isNew=false is returned
// and the same lease is returned each time
if leaseTmp != lease1 {
t.Error("expected same lease")
break
}
}
close(done)
}()
// Wait for the loop to complete
select {
case <-done:
// Expected
case <-time.After(1 * time.Second):
t.Fatal("loop timed out")
}
// All 100 adds should have completed (returning isNew=false)
if addCount != 100 {
t.Errorf("expected 100 adds, got %d", addCount)
}
mgr.Delete(leaseID1, objectName1)
if mgr.Get(getSvcID(svc)) != nil {
t.Error("expected lease to be removed after first delete")
}
}
// TestManager_NonCommonLease_ServiceContextCancellation tests that when
// a service is deleted (svcCtx.Ctx cancelled), the waiting goroutine
// should also unblock. This is the other exit path from the wait.
func TestManager_NonCommonLease_ServiceContextCancellation(t *testing.T) {
mgr := NewManager()
svc := createTestService("egress-service", "default", nil)
// First Add - leader election starts
objectName1 := ServiceNamespacedName(svc)
ctx1, leaseID1 := getSvcData(svc)
lease1 := mgr.Add(ctx1, leaseID1)
_ = lease1.Add(objectName1)
close(lease1.Started)
// Second Add - returns isNew=false
ctx2, leaseID2 := getSvcData(svc)
lease2 := mgr.Add(ctx2, leaseID2)
isNew2 := lease2.Add(objectName1)
if isNew2 {
t.Error("expected isNew=false")
}
// Create a simulated service context
svcCtx, svcCancel := context.WithCancel(context.Background())
// Start a goroutine that waits on either svcCtx or lease context
// This simulates the behavior in leader.go
waitDone := make(chan string)
go func() {
select {
case <-svcCtx.Done():
waitDone <- "svcCtx"
case <-lease2.Ctx.Done():
waitDone <- "leaseCtx"
case <-time.After(1 * time.Second):
waitDone <- "timeout"
}
}()
// Cancel the service context (simulates service deletion)
svcCancel()
// The goroutine should unblock via svcCtx.Done()
select {
case result := <-waitDone:
if result != "svcCtx" {
t.Errorf("expected to unblock via svcCtx, got %s", result)
}
case <-time.After(200 * time.Millisecond):
t.Error("goroutine should have unblocked")
}
}

View File

@@ -17,6 +17,7 @@ import (
"github.com/kube-vip/kube-vip/pkg/backend"
"github.com/kube-vip/kube-vip/pkg/sysctl"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/kube-vip/kube-vip/pkg/vip"
"github.com/vishvananda/netlink"
)
@@ -52,47 +53,53 @@ type IPVSLoadBalancer struct {
lock sync.Mutex
stop chan struct{}
networkInterface string
leaderCancel context.CancelFunc
signal chan os.Signal
killFunc func()
address string
family ipvs.AddressFamily
}
func NewIPVSLB(address string, port uint16, forwardingMethod string, backendHealthCheckInterval int, networkInterface string, leaderCancel context.CancelFunc, signal chan os.Signal) (*IPVSLoadBalancer, error) {
func NewIPVSLB(ctx context.Context, address string, port uint16, forwardingMethod string, backendHealthCheckInterval int,
networkInterface string, killFunc func(), wg *sync.WaitGroup) (*IPVSLoadBalancer, error) {
log.Info("Starting IPVS LoadBalancer", "address", address)
// Create IPVS client
c, err := ipvs.New()
if err != nil {
log.Error("ensure IPVS kernel modules are loaded")
log.Error("Error starting IPVS", "err", err)
panic("")
log.Error("error starting IPVS", "err", err)
return nil, fmt.Errorf("starting IPVS: %w", err)
}
i, err := c.Info()
if err != nil {
log.Error("ensure IPVS kernel modules are loaded")
log.Error("Error retrieving IPVS info", "err", err)
log.Error("error retrieving IPVS info", "err", err)
if errors.Is(err, os.ErrPermission) {
log.Error("no permission to get IPVS info - please ensure that kube-vip is running with proper capabilities/privileged mode")
}
panic("")
return nil, fmt.Errorf("retrieving IPVS: %w", err)
}
log.Info("IPVS Loadbalancer enabled", "version", fmt.Sprintf("%d.%d.%d", i.Version[0], i.Version[1], i.Version[2]))
ip, family := ipAndFamily(address)
if strings.ToLower(forwardingMethod) == "masquerade" {
enableProcSys("/proc/sys/net/ipv4/vs/conntrack", "net.ipv4.vs.conntrack")
if err := enableProcSys("/proc/sys/net/ipv4/vs/conntrack", "net.ipv4.vs.conntrack"); err != nil {
return nil, err
}
if family == ipvs.INET6 {
enableProcSys("/proc/sys/net/ipv6/conf/all/forwarding", "net.ipv6.conf.all.forwarding")
if err := enableProcSys("/proc/sys/net/ipv6/conf/all/forwarding", "net.ipv6.conf.all.forwarding"); err != nil {
return nil, err
}
} else {
enableProcSys("/proc/sys/net/ipv4/ip_forward", "net.ipv4.ip_forward")
if err := enableProcSys("/proc/sys/net/ipv4/ip_forward", "net.ipv4.ip_forward"); err != nil {
return nil, err
}
}
}
netMask := netmask.MaskFrom(31, 32) // For ipv4
netMask := netmask.MaskFrom(31, vip.DefaultMaskIPv4) // For ipv4
if family == ipvs.INET6 {
netMask = netmask.MaskFrom(128, 128) // For ipv6
netMask = netmask.MaskFrom(128, vip.DefaultMaskIPv6) // For ipv6
}
// Generate out API Server LoadBalancer instance
@@ -133,29 +140,29 @@ func NewIPVSLB(address string, port uint16, forwardingMethod string, backendHeal
forwardingMethod: m,
interval: backendHealthCheckInterval,
backendMap: make(backend.Map),
stop: make(chan struct{}),
networkInterface: networkInterface,
leaderCancel: leaderCancel,
signal: signal,
killFunc: killFunc,
address: address,
family: family,
}
go lb.healthCheck()
wg.Go(func() {
lb.healthCheck(ctx)
})
// Return our created load-balancer
return lb, nil
}
func enableProcSys(path, name string) {
func enableProcSys(path, name string) error {
isSet, err := sysctl.EnableProcSys(path)
if err != nil {
log.Error(fmt.Sprintf("ensuring %s enabled", name), "err", err)
panic("")
return fmt.Errorf("ensuring %s enabled: %w", name, err)
}
if isSet {
log.Info(fmt.Sprintf("sysctl set %s to 1", name))
}
return nil
}
func (lb *IPVSLoadBalancer) RemoveIPVSLB() error {
@@ -227,7 +234,7 @@ func (lb *IPVSLoadBalancer) addBackend(address string, port uint16) error {
// Fatal error at this point as IPVS is probably not working
log.Error("Unable to create an IPVS service, ensure IPVS kernel modules are loaded")
log.Error("IPVS service", "err", err)
panic("")
return utils.NewPanicError(fmt.Sprintf("unable to create an IPVS service - %s", err))
}
log.Info("load-Balancer services created", "address", lb.addrString(), "port", lb.Port)
@@ -312,8 +319,8 @@ func ipAndFamily(address string) (netip.Addr, ipvs.AddressFamily) {
return netip.AddrFrom4([4]byte(ipAddr.To4())), ipvs.INET
}
func (lb *IPVSLoadBalancer) healthCheck() {
backend.Watch(func() {
func (lb *IPVSLoadBalancer) healthCheck(ctx context.Context) {
backend.Watch(ctx, func() {
lb.lock.Lock()
defer lb.lock.Unlock()
for backend, oldStatus := range lb.backendMap {
@@ -338,17 +345,13 @@ func (lb *IPVSLoadBalancer) healthCheck() {
lb.backendMap[backend] = newStatus
}
if lb.forwardingMethod == ipvs.Local && !lb.localBackendExists() {
if lb.signal != nil {
close(lb.signal)
}
if lb.leaderCancel != nil {
lb.leaderCancel()
if lb.killFunc != nil {
lb.killFunc()
}
}
}
}
}, lb.interval, lb.stop)
}, lb.interval)
}
func (lb *IPVSLoadBalancer) isLocal(address string) (bool, error) {

View File

@@ -1,30 +0,0 @@
package manager
import (
"github.com/pkg/errors"
"github.com/kube-vip/kube-vip/pkg/cluster"
"github.com/kube-vip/kube-vip/pkg/etcd"
)
func initClusterManager(sm *Manager) (*cluster.Manager, error) {
m := &cluster.Manager{
SignalChan: sm.signalChan,
}
switch sm.config.LeaderElectionType {
case "kubernetes", "":
m.KubernetesClient = sm.clientSet
m.RetryWatcherClient = sm.rwClientSet
case "etcd":
client, err := etcd.NewClient(sm.config)
if err != nil {
return nil, err
}
m.EtcdClient = client
default:
return nil, errors.Errorf("invalid LeaderElectionMode %s not supported", sm.config.LeaderElectionType)
}
return m, nil
}

View File

@@ -2,13 +2,14 @@ package manager
import (
"context"
"errors"
"fmt"
"net/http"
"os"
"os/signal"
"path/filepath"
"strings"
"sync"
"sync/atomic"
"syscall"
"time"
@@ -16,20 +17,25 @@ import (
"github.com/kube-vip/kube-vip/pkg/arp"
"github.com/kube-vip/kube-vip/pkg/bgp"
"github.com/kube-vip/kube-vip/pkg/cluster"
"github.com/kube-vip/kube-vip/pkg/election"
"github.com/kube-vip/kube-vip/pkg/iptables"
"github.com/kube-vip/kube-vip/pkg/k8s"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/lease"
"github.com/kube-vip/kube-vip/pkg/manager/worker"
"github.com/kube-vip/kube-vip/pkg/networkinterface"
"github.com/kube-vip/kube-vip/pkg/nftables"
"github.com/kube-vip/kube-vip/pkg/node"
"github.com/kube-vip/kube-vip/pkg/services"
"github.com/kube-vip/kube-vip/pkg/upnp"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/kube-vip/kube-vip/pkg/vip"
"github.com/prometheus/client_golang/prometheus"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/rest"
)
const plunderLock = "plndr-svcs-lock"
// Manager degines the manager of the load-balancing services
type Manager struct {
clientSet *kubernetes.Clientset
@@ -46,8 +52,7 @@ type Manager struct {
// This channel is used to catch an OS signal and trigger a shutdown
signalChan chan os.Signal
// This channel is used to signal a shutdown
shutdownChan chan struct{}
sigint sync.Once
svcProcessor *services.Processor
@@ -72,6 +77,15 @@ type Manager struct {
// implementation will be decided in constructor
// based on config.EnableNodeLabeling
nodeLabelManager node.LabelManager
// This variable reports if manager is being closed
closing atomic.Bool
// Will be used for leaderelection when required
electionMgr *election.Manager
// Will handle leases
leaseMgr *lease.Manager
}
// New will create a new managing object
@@ -187,8 +201,8 @@ func New(configMap string, config *kubevip.Config) (*Manager, error) {
// Add Notification for SIGTERM (sent from Kubernetes)
signal.Notify(signalChan, syscall.SIGTERM)
// All watchers and other goroutines should have an additional goroutine that blocks on this, to shut things down
shutdownChan := make(chan struct{})
// Add Notification for SIGUSR1 (for configuration dump)
signal.Notify(signalChan, syscall.SIGUSR1)
intfMgr := networkinterface.NewManager()
arpMgr := arp.NewManager(config)
@@ -205,7 +219,15 @@ func New(configMap string, config *kubevip.Config) (*Manager, error) {
}
}
svcProcessor := services.NewServicesProcessor(config, bgpServer, clientset, rwClientSet, shutdownChan, intfMgr, arpMgr, nodeLabelManager)
electionMgr, err := election.NewManager(config, clientset, rwClientSet)
if err != nil {
return nil, fmt.Errorf("creating election manager: %w", err)
}
leaseMgr := lease.NewManager()
svcProcessor := services.NewServicesProcessor(config, bgpServer, clientset, rwClientSet,
intfMgr, arpMgr, nodeLabelManager, electionMgr, leaseMgr)
return &Manager{
clientSet: clientset,
@@ -225,32 +247,20 @@ func New(configMap string, config *kubevip.Config) (*Manager, error) {
Help: "Display state of session by setting metric for label value with current state to 1",
}, []string{"state", "peer"}),
signalChan: signalChan,
shutdownChan: shutdownChan,
svcProcessor: svcProcessor,
intfMgr: intfMgr,
arpMgr: arpMgr,
bgpServer: bgpServer,
nodeLabelManager: nodeLabelManager,
electionMgr: electionMgr,
leaseMgr: leaseMgr,
}, nil
}
// Start will begin the Manager, which will start services and watch the configmap
func (sm *Manager) Start() error {
// listen for interrupts or the Linux SIGTERM signal and cancel
// our context, which the leader election code will observe and
// step down
sm.signalChan = make(chan os.Signal, 1)
// Add Notification for Userland interrupt
signal.Notify(sm.signalChan, syscall.SIGINT)
// Add Notification for SIGTERM (sent from Kubernetes)
signal.Notify(sm.signalChan, syscall.SIGTERM)
// Add Notification for SIGUSR1 (for configuration dump)
signal.Notify(sm.signalChan, syscall.SIGUSR1)
// All watchers and other goroutines should have an additional goroutine that blocks on this, to shut things down
sm.shutdownChan = make(chan struct{})
func (sm *Manager) Start(ctx context.Context) error {
wg := sync.WaitGroup{}
defer wg.Wait()
// HealthCheck
if sm.config.HealthCheckPort != 0 {
@@ -260,7 +270,7 @@ func (sm *Manager) Start() error {
http.HandleFunc("/healthz", func(w http.ResponseWriter, _ *http.Request) {
fmt.Fprintf(w, "OK")
})
go func() {
wg.Go(func() {
server := &http.Server{
Addr: fmt.Sprintf(":%d", sm.config.HealthCheckPort),
ReadHeaderTimeout: 3 * time.Second,
@@ -269,7 +279,7 @@ func (sm *Manager) Start() error {
if err != nil {
log.Error("healthcheck", "unable to start", err)
}
}()
})
}
// on exit, clean up the node labels
@@ -279,22 +289,9 @@ func (sm *Manager) Start() error {
}
}()
// If BGP is enabled then we start a server instance that will broadcast VIPs
if sm.config.EnableBGP {
// If Annotations have been set then we will look them up
err := sm.parseAnnotations()
if err != nil {
return err
}
log.Info("Starting Kube-vip Manager with the BGP engine")
return sm.startBGP()
}
if sm.config.EnableARP || sm.config.EnableWireguard {
if sm.config.EnableUPNP {
clients := upnp.GetConnectionClients(context.TODO())
clients := upnp.GetConnectionClients(ctx)
if len(clients) == 0 {
log.Error("Error Enabling UPNP. No Clients found")
// Set the struct to false so nothing should use it in future
@@ -309,49 +306,144 @@ func (sm *Manager) Start() error {
}
}
// TODO: It would be nice to run the UPNP refresh only on the leader.
go sm.svcProcessor.RefreshUPNPForwards()
wg.Go(func() {
sm.svcProcessor.RefreshUPNPForwards(ctx)
})
}
}
// If ARP is enabled then we start a LeaderElection that will use ARP to advertise VIPs
if sm.config.EnableARP {
log.Info("Starting Kube-vip Manager with the ARP engine")
return sm.startARP(sm.config.NodeName)
// If BGP is enabled then we start a server instance that will broadcast VIPs
if sm.config.EnableBGP {
// If Annotations have been set then we will look them up
err := sm.parseAnnotations(ctx)
if err != nil {
return err
}
}
if sm.config.EnableWireguard {
log.Info("Starting Kube-vip Manager with the Wireguard engine")
return sm.startWireguard(sm.config.NodeName)
return sm.startMode(ctx)
}
// Start will begin the Manager, which will start services and watch the configmap
func (sm *Manager) startMode(ctx context.Context) error {
var cpCluster *cluster.Cluster
var err error
w := worker.New(sm.arpMgr, sm.intfMgr, sm.config, &sm.closing, sm.Kill,
sm.svcProcessor, &sm.mutex, sm.clientSet, sm.bgpServer, sm.bgpSessionInfoGauge,
sm.electionMgr, sm.leaseMgr)
// use a Go context so we can tell the leaderelection code when we
// want to step down
wg := sync.WaitGroup{}
modeCtx, cancel := context.WithCancel(ctx)
defer func() {
// wait for gorutines then cancel context just in case
wg.Wait()
w.Cleanup()
cancel()
log.Info("Shutting down Kube-Vip")
}()
log.Info("starting Kube-vip Manager", "mode", w.Name())
if err := w.Configure(modeCtx, &wg); err != nil {
return fmt.Errorf("failed to configure %s mode: %w", w.Name(), err)
}
if sm.config.EnableRoutingTable {
log.Info("Starting Kube-vip Manager with the Routing Table engine")
return sm.startTableMode(sm.config.NodeName)
if sm.config.EnableControlPlane {
err = w.InitControlPlane()
if err != nil {
return err
}
}
// Shutdown function that will wait on this signal, unless we call it ourselves
wg.Go(func() {
sm.waitForShutdown(modeCtx, cancel, cpCluster)
})
if sm.config.EnableControlPlane {
wg.Go(func() {
w.StartControlPlane(modeCtx, sm.electionMgr)
})
}
if sm.config.EnableServices {
// This will tidy any dangling kube-vip iptables rules
if sm.config.EgressClean {
err := nftables.ClearTables()
if err != nil {
log.Warn("[egress]", "mode", "nftables-internal", "clearing error", err)
} else {
log.Info("[egress]", "mode", "nftables-internal", "tables", "cleared")
}
// TODO: Deprecate the iptables code v1.2.x
err = vip.ClearIPTables(sm.config.EgressWithNftables, sm.config.ServiceNamespace, iptables.ProtocolIPv4)
if err != nil {
log.Info("[egress]", "legacy-iptables", sm.config.EgressWithNftables, "mode", "IPv4", "error", err)
}
err = vip.ClearIPTables(sm.config.EgressWithNftables, sm.config.ServiceNamespace, iptables.ProtocolIPv6)
if err != nil {
log.Info("[egress]", "legacy-iptables", sm.config.EgressWithNftables, "mode", "IPv6", "error", err)
}
}
w.ConfigureServices()
for {
select {
case <-modeCtx.Done():
return nil
default:
if err = w.StartServices(modeCtx); err != nil {
if errors.Is(err, &utils.PanicError{}) {
sm.Kill()
return fmt.Errorf("failed to reconcile services, non-recoverable error: %w", err)
} else {
log.Error("failed to reconcile services, restarting", "error", err)
}
}
}
}
}
log.Error("prematurely exiting Load-balancer as no modes [ARP/BGP/Wireguard] are enabled")
return nil
}
func returnNameSpace() (string, error) {
if data, err := os.ReadFile("/var/run/secrets/kubernetes.io/serviceaccount/namespace"); err == nil {
if ns := strings.TrimSpace(string(data)); len(ns) > 0 {
return ns, nil
}
return "", err
}
return "", fmt.Errorf("unable to find Namespace")
}
func (sm *Manager) parseAnnotations() error {
func (sm *Manager) parseAnnotations(ctx context.Context) error {
if sm.config.Annotations == "" {
log.Debug("No Node annotations to parse")
return nil
}
err := sm.annotationsWatcher()
err := sm.annotationsWatcher(ctx)
if err != nil {
return err
}
return nil
}
func (sm *Manager) waitForShutdown(ctx context.Context, cancel context.CancelFunc, cpCluster *cluster.Cluster) {
for {
sig := <-sm.signalChan
switch sig {
case syscall.SIGUSR1:
log.Info("Received SIGUSR1, dumping configuration")
sm.dumpConfiguration(ctx)
case syscall.SIGINT, syscall.SIGTERM:
sm.closing.Store(true)
log.Info("Received kube-vip termination, signaling shutdown")
if cpCluster != nil {
cpCluster.Stop()
}
// Cancel the context, which will in turn cancel the leadership and all goroutines
cancel()
return
}
}
}
func (sm *Manager) Kill() {
sm.sigint.Do(func() {
sm.signalChan <- syscall.SIGINT
})
}

View File

@@ -1,168 +0,0 @@
package manager
import (
"context"
"syscall"
"time"
log "log/slog"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/tools/leaderelection"
"k8s.io/client-go/tools/leaderelection/resourcelock"
"github.com/kube-vip/kube-vip/pkg/cluster"
"github.com/kube-vip/kube-vip/pkg/iptables"
"github.com/kube-vip/kube-vip/pkg/vip"
)
// Start will begin the Manager, which will start services and watch the configmap
func (sm *Manager) startARP(id string) error {
var cpCluster *cluster.Cluster
var ns string
var err error
// use a Go context so we can tell the leaderelection code when we
// want to step down
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
log.Info("Start ARP/NDP advertisement")
go sm.arpMgr.StartAdvertisement(ctx)
// Shutdown function that will wait on this signal, unless we call it ourselves
go func() {
for {
sig := <-sm.signalChan
switch sig {
case syscall.SIGUSR1:
log.Info("Received SIGUSR1, dumping configuration")
sm.dumpConfiguration()
case syscall.SIGINT, syscall.SIGTERM:
log.Info("Received kube-vip termination, signaling shutdown")
if sm.config.EnableControlPlane {
cpCluster.Stop()
}
// Close all go routines
close(sm.shutdownChan)
// Cancel the context, which will in turn cancel the leadership
cancel()
return
}
}
}()
if sm.config.EnableControlPlane {
cpCluster, err = cluster.InitCluster(sm.config, false, sm.intfMgr, sm.arpMgr)
if err != nil {
return err
}
clusterManager, err := initClusterManager(sm)
if err != nil {
return err
}
go func() {
err := cpCluster.StartCluster(sm.config, clusterManager, nil)
if err != nil {
log.Error("starting control plane", "err", err)
// Trigger the shutdown of this manager instance
sm.signalChan <- syscall.SIGINT
}
}()
// Check if we're also starting the services, if not we can sit and wait on the closing channel and return here
if !sm.config.EnableServices {
<-sm.shutdownChan
log.Info("Shutting down Kube-Vip")
return nil
}
ns = sm.config.Namespace
} else {
ns, err = returnNameSpace()
if err != nil {
log.Warn("unable to auto-detect namespace, dropping to config", "namespace", sm.config.Namespace)
ns = sm.config.Namespace
}
}
// This will tidy any dangling kube-vip iptables rules
if sm.config.EgressClean {
vip.ClearIPTables(sm.config.EgressWithNftables, sm.config.ServiceNamespace, iptables.ProtocolIPv4)
}
// Start a services watcher (all kube-vip pods will watch services), upon a new service
// a lock based upon that service is created that they will all leaderElection on
if sm.config.EnableServicesElection {
log.Info("beginning watching services, leaderelection will happen for every service")
err = sm.svcProcessor.StartServicesWatchForLeaderElection(ctx)
if err != nil {
return err
}
} else {
log.Info("beginning services leadership", "namespace", ns, "lock name", sm.config.ServicesLeaseName, "id", id)
// we use the Lease lock type since edits to Leases are less common
// and fewer objects in the cluster watch "all Leases".
lock := &resourcelock.LeaseLock{
LeaseMeta: metav1.ObjectMeta{
Name: sm.config.ServicesLeaseName,
Namespace: ns,
},
Client: sm.clientSet.CoordinationV1(),
LockConfig: resourcelock.ResourceLockConfig{
Identity: id,
},
}
// start the leader election code loop
leaderelection.RunOrDie(ctx, leaderelection.LeaderElectionConfig{
Lock: lock,
// IMPORTANT: you MUST ensure that any code you have that
// is protected by the lease must terminate **before**
// you call cancel. Otherwise, you could have a background
// loop still running and another process could
// get elected before your background loop finished, violating
// the stated goal of the lease.
ReleaseOnCancel: true,
LeaseDuration: time.Duration(sm.config.LeaseDuration) * time.Second,
RenewDeadline: time.Duration(sm.config.RenewDeadline) * time.Second,
RetryPeriod: time.Duration(sm.config.RetryPeriod) * time.Second,
Callbacks: leaderelection.LeaderCallbacks{
OnStartedLeading: func(ctx context.Context) {
err = sm.svcProcessor.ServicesWatcher(ctx, sm.svcProcessor.SyncServices)
if err != nil {
log.Error("service watcher", "err", err)
panic("") // TODO: - emulating log.fatal here
}
},
OnStoppedLeading: func() {
// we can do cleanup here
sm.mutex.Lock()
defer sm.mutex.Unlock()
log.Info("leader lost", "new leader", id)
sm.svcProcessor.Stop()
log.Error("lost leadership, restarting kube-vip")
panic("") // TODO: - emulating log.fatal here
},
OnNewLeader: func(identity string) {
// we're notified when new leader elected
if sm.config.EnableNodeLabeling {
applyNodeLabel(sm.clientSet, sm.config.Address, id, identity)
}
if identity == id {
// I just got the lock
return
}
log.Info("new leader elected", "new leader", identity)
},
},
})
}
return nil
}

View File

@@ -1,134 +0,0 @@
package manager
import (
"context"
"fmt"
"syscall"
log "log/slog"
"github.com/kube-vip/kube-vip/pkg/bgp"
"github.com/kube-vip/kube-vip/pkg/cluster"
api "github.com/osrg/gobgp/v3/api"
"github.com/prometheus/client_golang/prometheus"
)
// Start will begin the Manager, which will start services and watch the configmap
func (sm *Manager) startBGP() error {
var cpCluster *cluster.Cluster
// var ns string
var err error
if sm.bgpServer == nil {
sm.bgpServer, err = bgp.NewBGPServer(sm.config.BGPConfig)
if err != nil {
return fmt.Errorf("creating BGP server: %w", err)
}
}
log.Info("Starting the BGP server to advertise VIP routes to BGP peers")
if err := sm.bgpServer.Start(func(p *api.WatchEventResponse_PeerEvent) {
ipaddr := p.GetPeer().GetState().GetNeighborAddress()
port := uint64(179)
peerDescription := fmt.Sprintf("%s:%d", ipaddr, port)
for stateName, stateValue := range api.PeerState_SessionState_value {
metricValue := 0.0
if stateValue == int32(p.GetPeer().GetState().GetSessionState().Number()) {
metricValue = 1
}
sm.bgpSessionInfoGauge.With(prometheus.Labels{
"state": stateName,
"peer": peerDescription,
}).Set(metricValue)
}
}); err != nil {
return fmt.Errorf("starting BGP server: %w", err)
}
// use a Go context so we can tell the leaderelection code when we
// want to step down
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
// Defer a function to check if the bgpServer has been created and if so attempt to close it
defer func() {
if sm.bgpServer != nil {
sm.bgpServer.Close()
}
}()
// Shutdown function that will wait on this signal, unless we call it ourselves
go func() {
for {
sig := <-sm.signalChan
switch sig {
case syscall.SIGUSR1:
log.Info("Received SIGUSR1, dumping configuration")
sm.dumpConfiguration()
case syscall.SIGINT, syscall.SIGTERM:
log.Info("Received termination, signaling shutdown")
if sm.config.EnableControlPlane {
if cpCluster != nil {
cpCluster.Stop()
}
}
// Cancel the context, which will in turn cancel the leadership
cancel()
return
}
}
}()
if sm.config.EnableControlPlane {
cpCluster, err = cluster.InitCluster(sm.config, false, sm.intfMgr, sm.arpMgr)
if err != nil {
return err
}
clusterManager, err := initClusterManager(sm)
if err != nil {
return err
}
go func() {
if sm.config.EnableLeaderElection {
err = cpCluster.StartCluster(sm.config, clusterManager, sm.bgpServer)
} else {
err = cpCluster.StartVipService(sm.config, clusterManager, sm.bgpServer)
}
if err != nil {
log.Error("Control Plane", "err", err)
// Trigger the shutdown of this manager instance
sm.signalChan <- syscall.SIGINT
}
}()
// Check if we're also starting the services, if not we can sit and wait on the closing channel and return here
if !sm.config.EnableServices {
<-sm.signalChan
log.Info("Shutting down Kube-Vip")
return nil
}
}
if sm.config.EnableServicesElection {
log.Info("beginning watching services, leaderelection will happen for every service")
err = sm.svcProcessor.StartServicesWatchForLeaderElection(ctx)
if err != nil {
return err
}
} else {
log.Info("beginning watching services without leader election")
err = sm.svcProcessor.ServicesWatcher(ctx, sm.svcProcessor.SyncServices)
if err != nil {
return err
}
}
log.Info("Shutting down Kube-Vip")
return nil
}

View File

@@ -12,7 +12,7 @@ import (
)
// dumpConfiguration prints the current configuration to stdout when SIGUSR1 is received
func (sm *Manager) dumpConfiguration() {
func (sm *Manager) dumpConfiguration(ctx context.Context) {
sm.mutex.Lock()
defer sm.mutex.Unlock()
@@ -29,7 +29,7 @@ func (sm *Manager) dumpConfiguration() {
sm.dumpConfigSection()
sm.dumpBGPSection()
sm.dumpARPSection()
sm.dumpServicesSection()
sm.dumpServicesSection(ctx)
sm.dumpNetworkInterfacesSection()
sm.dumpLeaderElectionSection()
sm.dumpRuntimeSection()
@@ -89,7 +89,7 @@ func (sm *Manager) dumpARPSection() {
fmt.Printf("\n")
}
func (sm *Manager) dumpServicesSection() {
func (sm *Manager) dumpServicesSection(ctx context.Context) {
fmt.Printf("--- SERVICES CONFIGURATION ---\n")
fmt.Printf("Services Enabled: %t\n", sm.config.EnableServices)
if sm.config.EnableServices {
@@ -124,7 +124,7 @@ func (sm *Manager) dumpServicesSection() {
fmt.Println("--- KUBERNETES CONFIGURATION (SERVICES/ENDPOINTSLICES) ---")
fmt.Println("Service Configuration:")
svcList, err := sm.clientSet.CoreV1().Services(v1.NamespaceAll).List(context.TODO(), metav1.ListOptions{})
svcList, err := sm.clientSet.CoreV1().Services(v1.NamespaceAll).List(ctx, metav1.ListOptions{})
if err != nil {
fmt.Println("Unable to retrieve all Services")
} else {
@@ -144,7 +144,7 @@ func (sm *Manager) dumpServicesSection() {
}
fmt.Println("EndpointSlice Configuration (note endpoint names have -XXXXX prefixed):")
epList, err := sm.clientSet.DiscoveryV1().EndpointSlices(v1.NamespaceAll).List(context.TODO(), metav1.ListOptions{})
epList, err := sm.clientSet.DiscoveryV1().EndpointSlices(v1.NamespaceAll).List(ctx, metav1.ListOptions{})
if err != nil {
fmt.Println("Unable to retrieve all EndpointSlices")
} else {

View File

@@ -2,6 +2,7 @@ package manager
import (
"bytes"
"context"
"io"
"os"
"testing"
@@ -36,7 +37,7 @@ func TestDumpConfiguration(t *testing.T) {
r, w, _ := os.Pipe()
os.Stdout = w
mgr.dumpConfiguration()
mgr.dumpConfiguration(context.TODO())
w.Close()
os.Stdout = old

View File

@@ -1,203 +0,0 @@
package manager
import (
"context"
"fmt"
"syscall"
"time"
log "log/slog"
"github.com/kube-vip/kube-vip/pkg/cluster"
"github.com/kube-vip/kube-vip/pkg/endpoints"
"github.com/kube-vip/kube-vip/pkg/iptables"
"github.com/kube-vip/kube-vip/pkg/vip"
"github.com/vishvananda/netlink"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/tools/leaderelection"
"k8s.io/client-go/tools/leaderelection/resourcelock"
)
// Start will begin the Manager, which will start services and watch the configmap
func (sm *Manager) startTableMode(id string) error {
var cpCluster *cluster.Cluster
var err error
// use a Go context so we can tell the leaderelection code when we
// want to step down
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
log.Info("destination for routes", "table", sm.config.RoutingTableID, "protocol", sm.config.RoutingProtocol)
if sm.config.CleanRoutingTable {
go func() {
// we assume that after 10s all services should be configured so we can delete redundant routes
time.Sleep(time.Second * 10)
if err := sm.cleanRoutes(); err != nil {
log.Error("error checking for old routes", "err", err)
}
}()
}
if sm.config.EgressClean {
vip.ClearIPTables(sm.config.EgressWithNftables, sm.config.ServiceNamespace, iptables.ProtocolIPv4)
vip.ClearIPTables(sm.config.EgressWithNftables, sm.config.ServiceNamespace, iptables.ProtocolIPv6)
log.Debug("IPtables rules cleaned on startup")
}
// Shutdown function that will wait on this signal, unless we call it ourselves
go func() {
for {
sig := <-sm.signalChan
switch sig {
case syscall.SIGUSR1:
log.Info("Received SIGUSR1, dumping configuration")
sm.dumpConfiguration()
case syscall.SIGINT, syscall.SIGTERM:
log.Info("Received kube-vip termination, signaling shutdown")
if sm.config.EnableControlPlane {
cpCluster.Stop()
}
// Cancel the context, which will in turn cancel the leadership
cancel()
return
}
}
}()
if sm.config.EnableServices {
log.Debug("starting Services")
ns, err := returnNameSpace()
if err != nil {
log.Warn("unable to auto-detect namespace", "dropping to", sm.config.Namespace)
ns = sm.config.Namespace
}
// Start a services watcher (all kube-vip pods will watch services), upon a new service
// a lock based upon that service is created that they will all leaderElection on
if sm.config.EnableServicesElection {
log.Info("beginning watching services, leaderelection will happen for every service")
err = sm.svcProcessor.StartServicesWatchForLeaderElection(ctx)
if err != nil {
return err
}
} else if sm.config.EnableLeaderElection {
log.Info("beginning services leadership", "namespace", ns, "lock name", plunderLock, "id", id)
// we use the Lease lock type since edits to Leases are less common
// and fewer objects in the cluster watch "all Leases".
lock := &resourcelock.LeaseLock{
LeaseMeta: metav1.ObjectMeta{
Name: plunderLock,
Namespace: ns,
},
Client: sm.clientSet.CoordinationV1(),
LockConfig: resourcelock.ResourceLockConfig{
Identity: id,
},
}
// start the leader election code loop
leaderelection.RunOrDie(ctx, leaderelection.LeaderElectionConfig{
Lock: lock,
// IMPORTANT: you MUST ensure that any code you have that
// is protected by the lease must terminate **before**
// you call cancel. Otherwise, you could have a background
// loop still running and another process could
// get elected before your background loop finished, violating
// the stated goal of the lease.
ReleaseOnCancel: true,
LeaseDuration: time.Duration(sm.config.LeaseDuration) * time.Second,
RenewDeadline: time.Duration(sm.config.RenewDeadline) * time.Second,
RetryPeriod: time.Duration(sm.config.RetryPeriod) * time.Second,
Callbacks: leaderelection.LeaderCallbacks{
OnStartedLeading: func(ctx context.Context) {
err = sm.svcProcessor.ServicesWatcher(ctx, sm.svcProcessor.SyncServices)
if err != nil {
log.Error(err.Error())
panic("")
}
},
OnStoppedLeading: func() {
// we can do cleanup here
sm.mutex.Lock()
defer sm.mutex.Unlock()
log.Info("leader lost", "id", id)
sm.svcProcessor.Stop()
log.Error("lost leadership, restarting kube-vip")
panic("")
},
OnNewLeader: func(identity string) {
// we're notified when new leader elected
if identity == id {
// I just got the lock
return
}
log.Info("new leader elected", "id", identity)
},
},
})
} else {
log.Info("beginning watching services without leader election")
err = sm.svcProcessor.ServicesWatcher(ctx, sm.svcProcessor.SyncServices)
if err != nil {
log.Error("Cannot watch services", "err", err)
} else {
log.Debug("watching services")
}
}
}
if sm.config.EnableControlPlane {
log.Debug("initCluster for ControlPlane")
cpCluster, err = cluster.InitCluster(sm.config, false, sm.intfMgr, sm.arpMgr)
if err != nil {
log.Debug("init of ControlPlane NOT successful")
return fmt.Errorf("cluster initialization error: %w", err)
}
log.Debug("init of ControlPlane successful")
log.Debug("init ClusterManager")
clusterManager, err := initClusterManager(sm)
if err != nil {
log.Debug("init cluster manager NOT successful")
return fmt.Errorf("cluster manager initialization error: %w", err)
}
log.Debug("init ClusterManager successful")
if err := cpCluster.StartVipService(sm.config, clusterManager, nil); err != nil {
log.Error("Control Plane", "err", err)
// Trigger the shutdown of this manager instance
sm.signalChan <- syscall.SIGINT
} else {
log.Debug("start VipServer for cluster manager successful")
}
}
return nil
}
func (sm *Manager) cleanRoutes() error {
sm.mutex.Lock()
defer sm.mutex.Unlock()
routes, err := vip.ListRoutes(sm.config.RoutingTableID, sm.config.RoutingProtocol)
if err != nil {
return fmt.Errorf("error getting routes: %w", err)
}
for i := range routes {
found := false
if sm.config.EnableControlPlane {
found = (routes[i].Dst.IP.String() == sm.config.Address)
} else {
found = endpoints.CountRouteReferences(&routes[i], &sm.svcProcessor.ServiceInstances) > 0
}
if !found {
err = netlink.RouteDel(&(routes[i]))
if err != nil {
log.Error("[route] deletion", "route", routes[i], "err", err)
}
log.Debug("[route] deletion", "route", routes[i])
}
}
return nil
}

View File

@@ -1,131 +0,0 @@
package manager
import (
"context"
"syscall"
"time"
log "log/slog"
"github.com/kube-vip/kube-vip/pkg/wireguard"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/tools/leaderelection"
"k8s.io/client-go/tools/leaderelection/resourcelock"
)
// Start will begin the Manager, which will start services and watch the configmap
func (sm *Manager) startWireguard(id string) error {
var ns string
var err error
// use a Go context so we can tell the leaderelection code when we
// want to step down
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
log.Info("reading wireguard peer configuration from Kubernetes secret")
s, err := sm.clientSet.CoreV1().Secrets(sm.config.Namespace).Get(ctx, "wireguard", metav1.GetOptions{})
if err != nil {
return err
}
// parse all the details needed for Wireguard
peerPublicKey := s.Data["peerPublicKey"]
peerEndpoint := s.Data["peerEndpoint"]
privateKey := s.Data["privateKey"]
// Configure the interface to join the Wireguard VPN
err = wireguard.ConfigureInterface(string(privateKey), string(peerPublicKey), string(peerEndpoint))
if err != nil {
return err
}
// Shutdown function that will wait on this signal, unless we call it ourselves
go func() {
for {
sig := <-sm.signalChan
switch sig {
case syscall.SIGUSR1:
log.Info("Received SIGUSR1, dumping configuration")
sm.dumpConfiguration()
case syscall.SIGINT, syscall.SIGTERM:
log.Info("Received termination, signaling shutdown")
// Cancel the context, which will in turn cancel the leadership
cancel()
return
}
}
}()
ns, err = returnNameSpace()
if err != nil {
log.Warn("unable to auto-detect namespace", "dropping to", sm.config.Namespace)
ns = sm.config.Namespace
}
// Start a services watcher (all kube-vip pods will watch services), upon a new service
// a lock based upon that service is created that they will all leaderElection on
if sm.config.EnableServicesElection {
log.Info("beginning watching services, leaderelection will happen for every service")
err = sm.svcProcessor.StartServicesWatchForLeaderElection(ctx)
if err != nil {
return err
}
} else {
log.Info("beginning services leadership", "namespace", ns, "lock name", plunderLock, "id", id)
// we use the Lease lock type since edits to Leases are less common
// and fewer objects in the cluster watch "all Leases".
lock := &resourcelock.LeaseLock{
LeaseMeta: metav1.ObjectMeta{
Name: plunderLock,
Namespace: ns,
},
Client: sm.clientSet.CoordinationV1(),
LockConfig: resourcelock.ResourceLockConfig{
Identity: id,
},
}
// start the leader election code loop
leaderelection.RunOrDie(ctx, leaderelection.LeaderElectionConfig{
Lock: lock,
// IMPORTANT: you MUST ensure that any code you have that
// is protected by the lease must terminate **before**
// you call cancel. Otherwise, you could have a background
// loop still running and another process could
// get elected before your background loop finished, violating
// the stated goal of the lease.
ReleaseOnCancel: true,
LeaseDuration: time.Duration(sm.config.LeaseDuration) * time.Second,
RenewDeadline: time.Duration(sm.config.RenewDeadline) * time.Second,
RetryPeriod: time.Duration(sm.config.RetryPeriod) * time.Second,
Callbacks: leaderelection.LeaderCallbacks{
OnStartedLeading: func(ctx context.Context) {
err = sm.svcProcessor.ServicesWatcher(ctx, sm.svcProcessor.SyncServices)
if err != nil {
log.Error(err.Error())
panic("")
}
},
OnStoppedLeading: func() {
// we can do cleanup here
sm.mutex.Lock()
defer sm.mutex.Unlock()
log.Info("leader lost", "id", id)
sm.svcProcessor.Stop()
log.Error("lost leadership, restarting kube-vip")
panic("")
},
OnNewLeader: func(identity string) {
// we're notified when new leader elected
if identity == id {
// I just got the lock
return
}
log.Info("new leader elected", "id", identity)
},
},
})
}
return nil
}

View File

@@ -9,7 +9,7 @@ func (sm *Manager) PrometheusCollector() []prometheus.Collector {
collectors = append(collectors, sm.svcProcessor.CountServiceWatchEvent)
}
if sm.bgpServer != nil {
collectors = append(collectors, sm.bgpServer.BGPSessionInfoGauge)
collectors = append(collectors, sm.bgpSessionInfoGauge)
}
return collectors
}

View File

@@ -25,7 +25,7 @@ import (
// This file handles the watching of node annotations for configuration, it will exit once the annotations are
// present
func (sm *Manager) annotationsWatcher() error {
func (sm *Manager) annotationsWatcher(ctx context.Context) error {
// Use a restartable watcher, as this should help in the event of etcd or timeout issues
log.Info("Kube-Vip is waiting for annotation prefix to be present on this node", "prefix", sm.config.Annotations)
@@ -36,7 +36,7 @@ func (sm *Manager) annotationsWatcher() error {
// First we'll check the annotations for the node and if
// they aren't what are expected, we'll drop into the watch until they are
nodeList, err := sm.clientSet.CoreV1().Nodes().List(context.Background(), listOptions)
nodeList, err := sm.clientSet.CoreV1().Nodes().List(ctx, listOptions)
if err != nil {
return err
}
@@ -57,30 +57,19 @@ func (sm *Manager) annotationsWatcher() error {
// they're as needed
log.Warn(err.Error())
// TODO, will need refactoring as part of rikatz work
rw, err := watchtools.NewRetryWatcherWithContext(context.TODO(), node.ResourceVersion, &cache.ListWatch{
rw, err := watchtools.NewRetryWatcherWithContext(ctx, node.ResourceVersion, &cache.ListWatch{
WatchFunc: func(_ metav1.ListOptions) (watch.Interface, error) {
return sm.rwClientSet.CoreV1().Nodes().Watch(context.Background(), listOptions)
return sm.rwClientSet.CoreV1().Nodes().Watch(ctx, listOptions)
},
})
if err != nil {
return fmt.Errorf("error creating annotations watcher: %s", err.Error())
}
exitFunction := make(chan struct{})
go func() {
select {
case <-sm.shutdownChan:
log.Debug("[annotations] shutdown called")
// Stop the retry watcher
rw.Stop()
return
case <-exitFunction:
log.Debug("[annotations] function ending")
// Stop the retry watcher
rw.Stop()
return
}
defer func() {
rw.Stop()
log.Debug("[annotations] watcher stopped")
}()
ch := rw.ResultChan()
@@ -103,7 +92,8 @@ func (sm *Manager) annotationsWatcher() error {
sm.config.BGPConfig = bgpConfig
sm.config.BGPPeerConfig = bgpPeer
rw.Stop()
log.Info("[annotations] exiting Annotations watcher - annotations found")
return nil
case watch.Deleted:
node, ok := event.Object.(*v1.Node)
if !ok {
@@ -130,10 +120,8 @@ func (sm *Manager) annotationsWatcher() error {
default:
}
}
close(exitFunction)
log.Info("Exiting Annotations watcher")
log.Info("[annotations] exiting annotations watcher")
return nil
}
// parseNodeAnnotations parses the annotations on the node and updates the configuration

69
pkg/manager/worker/arp.go Normal file
View File

@@ -0,0 +1,69 @@
package worker
import (
"context"
log "log/slog"
"sync"
"sync/atomic"
"github.com/kube-vip/kube-vip/pkg/arp"
"github.com/kube-vip/kube-vip/pkg/election"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/lease"
"github.com/kube-vip/kube-vip/pkg/networkinterface"
"github.com/kube-vip/kube-vip/pkg/services"
"k8s.io/client-go/kubernetes"
)
type ARP struct {
Common
}
func NewARP(arpMgr *arp.Manager, intfMgr *networkinterface.Manager,
config *kubevip.Config, closing *atomic.Bool, killFunc func(),
svcProcessor *services.Processor, mutex *sync.Mutex, clientSet *kubernetes.Clientset,
electionMgr *election.Manager, leaseMgr *lease.Manager) *ARP {
return &ARP{
Common: *newCommon(arpMgr, intfMgr, config, closing, killFunc,
svcProcessor, mutex, clientSet, electionMgr, leaseMgr),
}
}
func (a *ARP) Configure(ctx context.Context, wg *sync.WaitGroup) error {
log.Info("Start ARP/NDP advertisement Global")
wg.Go(func() {
a.arpMgr.StartAdvertisement(ctx)
})
return nil
}
func (a *ARP) StartControlPlane(ctx context.Context, electionManager *election.Manager) {
err := a.cpCluster.StartCluster(ctx, a.config, electionManager, nil, a.leaseMgr, a.killFunc)
if err != nil {
log.Error("starting control plane", "err", err)
}
// Trigger the shutdown of this manager instance
a.killFunc()
}
func (a *ARP) ConfigureServices() {
}
func (a *ARP) StartServices(ctx context.Context) error {
// Start a services watcher (all kube-vip pods will watch services), upon a new service
// a lock based upon that service is created that they will all leaderElection on
if a.config.EnableServicesElection {
if err := a.PerServiceLeader(ctx); err != nil {
return err
}
} else {
a.GlobalLeader(ctx, a.config.ServicesLeaseName)
}
return nil
}
func (a *ARP) Name() string {
return "ARP"
}

128
pkg/manager/worker/bgp.go Normal file
View File

@@ -0,0 +1,128 @@
package worker
import (
"context"
"fmt"
log "log/slog"
"sync"
"sync/atomic"
"github.com/kube-vip/kube-vip/pkg/arp"
"github.com/kube-vip/kube-vip/pkg/bgp"
"github.com/kube-vip/kube-vip/pkg/election"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/lease"
"github.com/kube-vip/kube-vip/pkg/networkinterface"
"github.com/kube-vip/kube-vip/pkg/services"
api "github.com/osrg/gobgp/v3/api"
"github.com/prometheus/client_golang/prometheus"
"k8s.io/client-go/kubernetes"
)
type BGP struct {
Common
bgpServer *bgp.Server
bgpSessionInfoGauge *prometheus.GaugeVec
}
func NewBGP(arpMgr *arp.Manager, intfMgr *networkinterface.Manager,
config *kubevip.Config, closing *atomic.Bool, killFunc func(),
svcProcessor *services.Processor, mutex *sync.Mutex, clientSet *kubernetes.Clientset,
bgpServer *bgp.Server, bgpSessionInfoGauge *prometheus.GaugeVec,
electionMgr *election.Manager, leaseMgr *lease.Manager) *BGP {
return &BGP{
Common: *newCommon(arpMgr, intfMgr, config, closing, killFunc,
svcProcessor, mutex, clientSet, electionMgr, leaseMgr),
bgpServer: bgpServer,
bgpSessionInfoGauge: bgpSessionInfoGauge,
}
}
func (b *BGP) Configure(ctx context.Context, _ *sync.WaitGroup) error {
var err error
if b.bgpServer == nil {
b.bgpServer, err = bgp.NewBGPServer(b.config.BGPConfig)
if err != nil {
return fmt.Errorf("creating BGP server: %w", err)
}
}
log.Info("Starting the BGP server to advertise VIP routes to BGP peers")
if err := b.bgpServer.Start(ctx, func(p *api.WatchEventResponse_PeerEvent) {
ipaddr := p.GetPeer().GetState().GetNeighborAddress()
port := uint64(179)
peerDescription := fmt.Sprintf("%s:%d", ipaddr, port)
for stateName, stateValue := range api.PeerState_SessionState_value {
metricValue := 0.0
if stateValue == int32(p.GetPeer().GetState().GetSessionState().Number()) {
metricValue = 1
}
b.bgpSessionInfoGauge.With(prometheus.Labels{
"state": stateName,
"peer": peerDescription,
}).Set(metricValue)
}
}); err != nil {
return fmt.Errorf("starting BGP server: %w", err)
}
return nil
}
func (b *BGP) Cleanup() {
// Defer a function to check if the bgpServer has been created and if so attempt to close it
if b.bgpServer != nil {
b.bgpServer.Close()
}
}
func (b *BGP) StartControlPlane(ctx context.Context, electionManager *election.Manager) {
var err error
if b.config.EnableLeaderElection {
err = b.cpCluster.StartCluster(ctx, b.config, electionManager, b.bgpServer, b.leaseMgr, b.killFunc)
} else {
err = b.cpCluster.StartVipService(ctx, b.config, electionManager, b.bgpServer, b.killFunc)
}
if err != nil {
log.Error("Control Plane", "err", err)
// Trigger the shutdown of this manager instance
b.killFunc()
}
}
func (b *BGP) ConfigureServices() {
// No configuration required
}
func (b *BGP) StartServices(ctx context.Context) error {
if b.config.EnableServicesElection {
if err := b.PerServiceLeader(ctx); err != nil {
return err
}
} else {
if err := b.ServicesNoLeader(ctx); err != nil {
return err
}
}
return nil
}
func (b *BGP) ServicesGlobalLeader(ctx context.Context, id string) {
// NOT IMPLEMENTED
}
func (b *BGP) ServicesNoLeader(ctx context.Context) error {
log.Info("beginning watching services without leader election")
err := b.svcProcessor.ServicesWatcher(ctx, b.svcProcessor.SyncServices)
if err != nil {
return err
}
return nil
}
func (b *BGP) Name() string {
return "BGP"
}

View File

@@ -0,0 +1,216 @@
package worker
import (
"context"
"fmt"
log "log/slog"
"sync"
"sync/atomic"
"time"
"github.com/kube-vip/kube-vip/pkg/arp"
"github.com/kube-vip/kube-vip/pkg/cluster"
"github.com/kube-vip/kube-vip/pkg/election"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/lease"
"github.com/kube-vip/kube-vip/pkg/networkinterface"
"github.com/kube-vip/kube-vip/pkg/services"
"k8s.io/client-go/kubernetes"
)
type Common struct {
arpMgr *arp.Manager
cpCluster *cluster.Cluster
intfMgr *networkinterface.Manager
config *kubevip.Config
closing *atomic.Bool
killFunc func()
svcProcessor *services.Processor
mutex *sync.Mutex
clientSet *kubernetes.Clientset
electionMgr *election.Manager
leaseMgr *lease.Manager
}
func newCommon(arpMgr *arp.Manager, intfMgr *networkinterface.Manager,
config *kubevip.Config, closing *atomic.Bool, killFunc func(),
svcProcessor *services.Processor, mutex *sync.Mutex, clientSet *kubernetes.Clientset,
electionMgr *election.Manager, leaseMgr *lease.Manager) *Common {
return &Common{
arpMgr: arpMgr,
intfMgr: intfMgr,
config: config,
closing: closing,
killFunc: killFunc,
svcProcessor: svcProcessor,
mutex: mutex,
clientSet: clientSet,
electionMgr: electionMgr,
leaseMgr: leaseMgr,
}
}
func (c *Common) InitControlPlane() error {
var err error
c.cpCluster, err = cluster.InitCluster(c.config, false, c.intfMgr, c.arpMgr)
if err != nil {
return fmt.Errorf("cluster initialization error: %w", err)
}
return nil
}
func (c *Common) PerServiceLeader(ctx context.Context) error {
log.Info("beginning watching services, leaderelection will happen for every service")
err := c.svcProcessor.StartServicesWatchForLeaderElection(ctx)
if err != nil {
return err
}
return nil
}
func (c *Common) GlobalLeader(ctx context.Context, leaseName string) {
c.runGlobalElection(ctx, c, leaseName, c.config, c.electionMgr)
}
func (c *Common) ServicesNoLeader(ctx context.Context) error {
log.Info("beginning watching services without leader election")
err := c.svcProcessor.ServicesWatcher(ctx, c.svcProcessor.SyncServices)
if err != nil {
return fmt.Errorf("error while watching services: %w", err)
}
return nil
}
func (c *Common) Cleanup() {
// NOT IMPLEMENTED
}
func (c *Common) OnStartedLeading(ctx context.Context) {
err := c.svcProcessor.ServicesWatcher(ctx, c.svcProcessor.SyncServices)
if err != nil {
log.Error("service watcher", "err", err)
c.killFunc()
}
}
func (c *Common) OnStoppedLeading() {
// we can do cleanup here
c.mutex.Lock()
defer c.mutex.Unlock()
log.Info("leader lost", "former leader", c.config.NodeName)
c.svcProcessor.Stop()
log.Error("lost services leadership, restarting kube-vip")
c.killFunc()
}
func (c *Common) OnNewLeader(identity string) {
// we're notified when new leader elected
if c.config.EnableNodeLabeling {
labelCtx, cancel := context.WithTimeout(context.Background(), time.Second*30)
defer cancel()
applyNodeLabel(labelCtx, c.clientSet, c.config.Address, c.config.NodeName, identity)
}
if identity == c.config.NodeName {
// I just got the lock
return
}
log.Info("new leader elected", "new leader", identity)
}
func (c *Common) runGlobalElection(ctx context.Context, a election.Actions, leaseName string,
config *kubevip.Config, electionManager *election.Manager) {
ns, leaseName := lease.NamespaceName(leaseName, config)
leaseID := lease.NewID(config.LeaderElectionType, ns, leaseName)
objectName := lease.ObjectName(leaseID, "svcs0")
// objLease, isNew, isSharedLease := c.leaseMgr.Add(leaseID, objectName)
objLease := c.leaseMgr.Add(ctx, leaseID)
isNew := objLease.Add(objectName)
// this service was already processed so we do not need to do anything
if !isNew {
log.Debug("this election was already done, waiting for it to finish", "lease", c.config.ServicesLeaseName)
// Wait for either the service context or lease context to be done
select {
case <-ctx.Done():
// Service was deleted
c.leaseMgr.Delete(leaseID, objectName)
case <-objLease.Ctx.Done():
// Leader election ended (leadership lost or context cancelled)
}
return
}
objLease.Lock()
defer func() {
objLease.Unlock()
}()
if objLease.Elected.Load() {
objLease.Unlock()
log.Debug("this election was already done, shared lease", "lease", leaseID.Name())
// wait for leader election to start or context to be done
select {
case <-objLease.Started:
case <-objLease.Ctx.Done():
// Lease was cancelled (e.g., leader election ended), return immediately
// This allows the restart loop to create a fresh lease
log.Debug("lease context cancelled before leader election started", "lease", leaseID.Name())
return
}
a.OnStartedLeading(objLease.Ctx)
log.Debug("waiting for lease to finish", "lease", leaseID.Name())
// wait for leaderelection to be finished
<-objLease.Ctx.Done()
// we can do cleanup here
a.OnStoppedLeading()
log.Error("lost leadership, restarting kube-vip", "lease", leaseID.Name())
c.killFunc()
return
}
// For new leases (not shared), ensure cleanup when the leader election ends
// This is critical for the restartable service watcher to be able to restart
// the leader election after leadership loss
defer func() {
// Delete the lease from the manager so subsequent calls can create a fresh lease
// This handles the case where leader election ends due to:
// 1. Leadership loss (e.g., network timeout)
// 2. Context cancellation
// 3. Any other reason RunOrDie returns
c.leaseMgr.Delete(leaseID, objectName)
}()
run := &election.RunConfig{
Config: config,
LeaseID: leaseID,
LeaseAnnotations: map[string]string{},
Mgr: electionManager,
OnStartedLeading: func(ctx context.Context) {
objLease.Elected.Store(true)
objLease.Unlock()
close(objLease.Started)
a.OnStartedLeading(ctx)
},
OnStoppedLeading: func() {
objLease.Elected.Store(false)
a.OnStoppedLeading()
},
OnNewLeader: a.OnNewLeader,
}
if err := election.RunOrDie(ctx, run, config); err != nil {
log.Error("leaderelection failed", "err", err, "id", config.NodeName, "name", leaseID.Name())
}
}

View File

@@ -1,4 +1,4 @@
package manager
package worker
import (
"context"
@@ -21,8 +21,7 @@ type patchStringLabel struct {
// applyNodeLabel add/remove node label `kube-vip.io/has-ip=<VIP-Address>` to/from
// the node where the virtual IP was added to/removed from.
func applyNodeLabel(clientSet *kubernetes.Clientset, address, id, identity string) {
ctx := context.Background()
func applyNodeLabel(ctx context.Context, clientSet *kubernetes.Clientset, address, id, identity string) {
node, err := clientSet.CoreV1().Nodes().Get(ctx, id, metav1.GetOptions{})
if err != nil {
log.Error("can't query node labels", "node", id, "err", err)

113
pkg/manager/worker/table.go Normal file
View File

@@ -0,0 +1,113 @@
package worker
import (
"context"
"fmt"
log "log/slog"
"sync"
"sync/atomic"
"time"
"github.com/kube-vip/kube-vip/pkg/arp"
"github.com/kube-vip/kube-vip/pkg/election"
"github.com/kube-vip/kube-vip/pkg/endpoints"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/lease"
"github.com/kube-vip/kube-vip/pkg/networkinterface"
"github.com/kube-vip/kube-vip/pkg/services"
"github.com/kube-vip/kube-vip/pkg/vip"
"github.com/vishvananda/netlink"
"k8s.io/client-go/kubernetes"
)
type Table struct {
Common
}
func NewTable(arpMgr *arp.Manager, intfMgr *networkinterface.Manager,
config *kubevip.Config, closing *atomic.Bool, killFUnc func(),
svcProcessor *services.Processor, mutex *sync.Mutex, clientSet *kubernetes.Clientset,
electionMgr *election.Manager, leaseMgr *lease.Manager) *Table {
return &Table{
Common: *newCommon(arpMgr, intfMgr, config, closing, killFUnc,
svcProcessor, mutex, clientSet, electionMgr, leaseMgr),
}
}
func (t *Table) Configure(ctx context.Context, wg *sync.WaitGroup) error {
log.Info("destination for routes", "table", t.config.RoutingTableID, "protocol", t.config.RoutingProtocol)
if t.config.CleanRoutingTable {
wg.Go(func() {
// we assume that after 10s all services should be configured so we can delete redundant routes
time.Sleep(time.Second * 10)
if err := t.cleanRoutes(); err != nil {
log.Error("error checking for old routes", "err", err)
}
})
}
return nil
}
func (t *Table) StartControlPlane(ctx context.Context, electionManager *election.Manager) {
if err := t.cpCluster.StartVipService(ctx, t.config, electionManager, nil, t.killFunc); err != nil {
log.Error("Control Plane", "err", err)
// Trigger the shutdown of this manager instance
t.killFunc()
} else {
log.Debug("start VipServer for cluster manager successful")
}
}
func (t *Table) ConfigureServices() {
// No configuration required
}
func (t *Table) StartServices(ctx context.Context) error {
log.Debug("starting Services")
if t.config.EnableServicesElection {
if err := t.PerServiceLeader(ctx); err != nil {
return err
}
} else if t.config.EnableLeaderElection {
t.GlobalLeader(ctx, t.config.ServicesLeaseName)
} else {
if err := t.ServicesNoLeader(ctx); err != nil {
return err
}
}
return nil
}
func (t *Table) Name() string {
return "Routing Table"
}
func (t *Table) cleanRoutes() error {
t.mutex.Lock()
defer t.mutex.Unlock()
routes, err := vip.ListRoutes(t.config.RoutingTableID, t.config.RoutingProtocol)
if err != nil {
return fmt.Errorf("error getting routes: %w", err)
}
for i := range routes {
found := false
if t.config.EnableControlPlane {
found = (routes[i].Dst.IP.String() == t.config.Address)
} else {
found = endpoints.CountRouteReferences(&routes[i], &t.svcProcessor.ServiceInstances) > 0
}
if !found {
err = netlink.RouteDel(&(routes[i]))
if err != nil {
log.Error("[route] deletion", "route", routes[i], "err", err)
}
log.Debug("[route] deletion", "route", routes[i])
}
}
return nil
}

View File

@@ -0,0 +1,274 @@
package worker
import (
"context"
"fmt"
log "log/slog"
"os"
"sync"
"sync/atomic"
"github.com/kube-vip/kube-vip/pkg/arp"
"github.com/kube-vip/kube-vip/pkg/election"
"github.com/kube-vip/kube-vip/pkg/endpoints/providers"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/lease"
"github.com/kube-vip/kube-vip/pkg/networkinterface"
"github.com/kube-vip/kube-vip/pkg/nftables"
"github.com/kube-vip/kube-vip/pkg/services"
"github.com/kube-vip/kube-vip/pkg/sysctl"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/kube-vip/kube-vip/pkg/wireguard"
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/watch"
"k8s.io/client-go/kubernetes"
)
type WireGuard struct {
Common
tunnelMgr *wireguard.TunnelManager
kubeAPIHost string
kubeAPIPort string
endpointWatcherCtx context.Context
endpointWatcherStop context.CancelFunc
endpointWatcherWg sync.WaitGroup
}
func NewWireGuard(arpMgr *arp.Manager, intfMgr *networkinterface.Manager,
config *kubevip.Config, closing *atomic.Bool, killFUnc func(),
svcProcessor *services.Processor, mutex *sync.Mutex, clientSet *kubernetes.Clientset,
electionMgr *election.Manager, leaseMgr *lease.Manager) *WireGuard {
return &WireGuard{
Common: *newCommon(arpMgr, intfMgr, config, closing, killFUnc,
svcProcessor, mutex, clientSet, electionMgr, leaseMgr),
}
}
func (w *WireGuard) Configure(ctx context.Context, _ *sync.WaitGroup) error {
log.Info("reading wireguard tunnel configurations from Kubernetes secret")
tunnelMgr := wireguard.NewTunnelManager()
err := tunnelMgr.LoadConfigurationsFromSecret(ctx, w.clientSet, w.config.Namespace, "wireguard")
if err != nil {
return fmt.Errorf("failed to load WireGuard tunnel configurations: %w", err)
}
// Clean up any stale resources from previous runs (crash recovery for hostNetwork: true)
// Must be called AFTER loading configs so we know which interfaces/ports to clean
if err := tunnelMgr.CleanupStaleResources(); err != nil {
log.Warn("failed to cleanup stale resources", "err", err)
// Continue anyway - the cleanup is best-effort
}
if _, err := sysctl.EnableProcSys("/proc/sys/net/ipv4/conf/all/src_valid_mark"); err != nil {
return fmt.Errorf("net.ipv4.conf.all.src_valid_mark is disabled and could not be enabled %w", err)
}
if _, err := sysctl.EnableProcSys("/proc/sys/net/ipv4/conf/all/route_localnet"); err != nil {
return fmt.Errorf("net.ipv4.conf.all.route_localnet is disabled and could not be enabled %w", err)
}
w.tunnelMgr = tunnelMgr
configuredVIPs := tunnelMgr.ListConfiguredTunnels()
log.Info("loaded WireGuard tunnel configurations", "vips", configuredVIPs)
return nil
}
func (w *WireGuard) InitControlPlane() error {
// Get Kubernetes service IP and port from environment
w.kubeAPIHost = os.Getenv("KUBERNETES_SERVICE_HOST")
w.kubeAPIPort = os.Getenv("KUBERNETES_SERVICE_PORT_HTTPS")
if w.kubeAPIHost == "" || w.kubeAPIPort == "" {
return fmt.Errorf("KUBERNETES_SERVICE_HOST or KUBERNETES_SERVICE_PORT_HTTPS not set")
}
return nil
}
func (w *WireGuard) StartControlPlane(ctx context.Context, electionManager *election.Manager) {
if !w.tunnelMgr.HasConfigForVIP(w.config.VIP) {
log.Error("no WireGuard tunnel configuration found for control plane VIP", "vip", w.config.VIP)
return
}
w.runGlobalElection(ctx, w, w.config.LeaseName, w.config, electionManager)
}
func (w *WireGuard) ConfigureServices() {
w.svcProcessor.TunnelMgr = w.tunnelMgr
}
func (w *WireGuard) StartServices(ctx context.Context) error {
if w.config.EnableServicesElection {
log.Info("beginning watching services, leaderelection will happen for every service")
err := w.svcProcessor.StartServicesWatchForLeaderElection(ctx)
if err != nil {
return err
}
}
return nil
}
func (w *WireGuard) Name() string {
return "WireGuard"
}
func (w *WireGuard) OnStartedLeading(ctx context.Context) {
// Bring up the WireGuard tunnel for control plane VIP
err := w.tunnelMgr.BringUpTunnelForVIP(w.config.VIP)
if err != nil {
log.Error("could not start wireguard tunnel for control plane", "vip", w.config.VIP, "err", err)
_ = w.tunnelMgr.TearDownTunnelForVIP(w.config.VIP)
w.killFunc()
return
}
// Get the tunnel to access its configuration
wg := w.tunnelMgr.GetTunnelForVIP(w.config.VIP)
if wg == nil {
log.Error("failed to get wireguard tunnel after bringing up", "vip", w.config.VIP)
w.killFunc()
return
}
tunnelConfig := w.tunnelMgr.GetConfigForVIP(w.config.VIP)
if tunnelConfig == nil {
log.Error("failed to get tunnel configuration", "vip", w.config.VIP)
_ = w.tunnelMgr.TearDownTunnelForVIP(w.config.VIP)
w.killFunc()
return
}
// Start endpoint watcher - DNAT rules will be applied when endpoints arrive
w.endpointWatcherCtx, w.endpointWatcherStop = context.WithCancel(ctx)
w.endpointWatcherWg.Go(func() {
w.watchKubernetesEndpoints(w.endpointWatcherCtx, tunnelConfig)
})
if w.config.EnableServices && !w.config.EnableServicesElection {
if err := w.svcProcessor.ServicesWatcher(ctx, w.svcProcessor.SyncServices); err != nil {
log.Error("failed to start services watcher", "err", err)
}
}
}
// watchKubernetesEndpoints watches the kubernetes service EndpointSlices for changes
// and updates the DNAT rules when API server endpoints change (e.g., when an API server goes down)
func (w *WireGuard) watchKubernetesEndpoints(ctx context.Context, tunnelConfig *wireguard.TunnelConfig) {
log.Info("starting kubernetes endpoint watcher for control plane")
kubeSvc := &v1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "kubernetes",
Namespace: "default",
},
}
provider := providers.NewEndpointslices()
rw, err := provider.CreateRetryWatcher(ctx, w.clientSet, kubeSvc)
if err != nil {
log.Error("failed to create kubernetes endpoint watcher", "err", err)
return
}
defer rw.Stop()
for event := range rw.ResultChan() {
select {
case <-ctx.Done():
log.Info("kubernetes endpoint watcher stopped")
return
default:
}
switch event.Type {
case watch.Added, watch.Modified, watch.Deleted:
if err := provider.LoadObject(event.Object, func() {}); err != nil {
log.Error("failed to load endpoint object", "err", err)
continue
}
endpoints, _ := provider.GetAllEndpoints()
log.Info("kubernetes endpoints changed, updating DNAT rules", "eventType", event.Type, "endpoints", endpoints)
if err := w.updateControlPlaneDNAT(tunnelConfig, endpoints); err != nil {
log.Error("failed to update control plane DNAT rules", "err", err)
}
case watch.Error:
log.Warn("kubernetes endpoint watch error", "event", event)
}
}
}
// updateControlPlaneDNAT updates the DNAT rules for the control plane with the given endpoints
func (w *WireGuard) updateControlPlaneDNAT(tunnelConfig *wireguard.TunnelConfig, endpoints []string) error {
if len(endpoints) == 0 {
log.Warn("no kubernetes API server endpoints available")
// Don't delete rules - keep routing to last known endpoints
return nil
}
// Build targets with default port 6443
targets := make([]nftables.DNATTarget, len(endpoints))
for i, ep := range endpoints {
targets[i] = nftables.DNATTarget{IP: ep, Port: 6443}
}
vipIP := utils.StripCIDR(w.config.VIP)
err := nftables.ApplyDNAT(
tunnelConfig.InterfaceName,
vipIP,
6443,
targets,
"controlplane",
v1.ProtocolTCP,
false,
tunnelConfig.ListenPort,
)
if err != nil {
return fmt.Errorf("failed to apply updated DNAT rule: %w", err)
}
log.Info("control plane DNAT rules updated", "targetCount", len(targets))
return nil
}
func (w *WireGuard) OnStoppedLeading() {
// we can do cleanup here
w.mutex.Lock()
defer w.mutex.Unlock()
log.Info("leader lost", "id", w.config.NodeName)
// Stop the kubernetes endpoint watcher and wait for it to finish
if w.endpointWatcherStop != nil {
w.endpointWatcherStop()
w.endpointWatcherWg.Wait()
}
log.Info("deleting nftables DNAT chains")
err := nftables.DeleteIngressChains(false, "controlplane")
if err != nil {
log.Error("could not delete DNAT ingress chains", "err", err)
} else {
log.Info("nftables DNAT chains deleted successfully")
}
// Tear down all tunnels (control plane + services)
err = w.tunnelMgr.TearDownAllTunnels()
if err != nil {
log.Error("failed to tear down tunnels", "err", err)
}
if w.config.EnableServices && !w.config.EnableServicesElection {
w.svcProcessor.Stop()
}
log.Error("lost control plane leadership, restarting kube-vip")
w.killFunc()
}
func (w *WireGuard) OnNewLeader(identity string) {
// we're notified when new leader elected
if identity == w.config.NodeName {
// I just got the lock
return
}
// safety check - tear down tunnel if we're not the leader
_ = w.tunnelMgr.TearDownTunnelForVIP(w.config.VIP)
log.Info("new leader elected", "id", identity)
}

View File

@@ -0,0 +1,56 @@
package worker
import (
"context"
"sync"
"sync/atomic"
"github.com/kube-vip/kube-vip/pkg/arp"
"github.com/kube-vip/kube-vip/pkg/bgp"
"github.com/kube-vip/kube-vip/pkg/election"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/lease"
"github.com/kube-vip/kube-vip/pkg/networkinterface"
"github.com/kube-vip/kube-vip/pkg/services"
"github.com/prometheus/client_golang/prometheus"
"k8s.io/client-go/kubernetes"
)
type Worker interface {
Configure(context.Context, *sync.WaitGroup) error
InitControlPlane() error
StartControlPlane(context.Context, *election.Manager)
ConfigureServices()
StartServices(ctx context.Context) error
Name() string
Cleanup()
}
func New(arpMgr *arp.Manager, intfMgr *networkinterface.Manager,
config *kubevip.Config, closing *atomic.Bool, killFunc func(),
svcProcessor *services.Processor, mutex *sync.Mutex, clientSet *kubernetes.Clientset,
bgpServer *bgp.Server, bgpSessionInfoGauge *prometheus.GaugeVec,
electionMgr *election.Manager, leaseMgr *lease.Manager) Worker {
if config.EnableARP {
return NewARP(arpMgr, intfMgr, config, closing, killFunc,
svcProcessor, mutex, clientSet, electionMgr, leaseMgr)
}
if config.EnableBGP {
return NewBGP(arpMgr, intfMgr, config, closing, killFunc,
svcProcessor, mutex, clientSet, bgpServer, bgpSessionInfoGauge,
electionMgr, leaseMgr)
}
if config.EnableRoutingTable {
return NewTable(arpMgr, intfMgr, config, closing, killFunc,
svcProcessor, mutex, clientSet, electionMgr, leaseMgr)
}
if config.EnableWireguard {
return NewWireGuard(arpMgr, intfMgr, config, closing, killFunc,
svcProcessor, mutex, clientSet, electionMgr, leaseMgr)
}
return nil
}

View File

@@ -1,6 +1,7 @@
package networkinterface
import (
log "log/slog"
"sync"
"github.com/vishvananda/netlink"
@@ -23,6 +24,12 @@ func NewManager() *Manager {
func (m *Manager) Get(intf netlink.Link) *Link {
if l, ok := m.interfaces[intf.Attrs().Name]; ok {
updated, err := netlink.LinkByName(l.Intf.Attrs().Name)
if err != nil {
log.Error("failed to get interface %q: %w", l.Intf.Attrs().Name, err)
return nil
}
l.Intf = updated
return l
}
result := &Link{

File diff suppressed because it is too large Load Diff

View File

@@ -69,6 +69,7 @@ func (m *Manager) RemoveLabel(ctx context.Context, svc *corev1.Service) error {
func (m *Manager) CleanUpLabels(timeout time.Duration) error {
log.Debug("cleaning up labels for node", "node", m.nodeName, "timeout", timeout)
// create new context for labels cleanup (independent)
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()

View File

@@ -3,6 +3,9 @@ package servicecontext
import (
"context"
"sync"
"sync/atomic"
"github.com/kube-vip/kube-vip/pkg/lease"
)
type Context struct {
@@ -11,6 +14,9 @@ type Context struct {
IsActive bool
IsWatched bool
ConfiguredNetworks sync.Map
Lease *lease.Lease
HasEndpoints atomic.Bool
LeaderCancel context.CancelFunc
}
func New(ctx context.Context) *Context {

View File

@@ -135,7 +135,7 @@ func checkCIDR(ip, cidr string) (string, error) {
return "", nil
}
func (p *Processor) configureEgress(vipIP, podIP, namespace, serviceUUID string, annotations map[string]string) error {
func (p *Processor) configureEgress(ctx context.Context, vipIP, podIP, namespace, serviceUUID string, annotations map[string]string) error {
var podCidr, serviceCidr string
var autoServiceCIDR, autoPodCIDR string
var discoverErr error
@@ -147,7 +147,7 @@ func (p *Processor) configureEgress(vipIP, podIP, namespace, serviceUUID string,
internalEgress := annotations[kubevip.EgressInternal]
if p.config.EgressPodCidr == "" || p.config.EgressServiceCidr == "" {
autoServiceCIDR, autoPodCIDR, discoverErr = p.AutoDiscoverCIDRs()
autoServiceCIDR, autoPodCIDR, discoverErr = p.AutoDiscoverCIDRs(ctx)
}
if discoverErr != nil {
@@ -224,8 +224,15 @@ func (p *Processor) configureEgress(vipIP, podIP, namespace, serviceUUID string,
}
// Add networks that we specifically should only SNAT for
allowCIDRs := []string{}
if allowedNetworks != "" {
networks := strings.Split(strings.TrimSpace(allowedNetworks), ",") //Remove whitespace characters and then create an array from the CIDRs
allowCIDRs = append(allowCIDRs, networks...)
}
// Apply the SNAT rules
err := nftables.ApplySNAT(podIP, vipIP, serviceUUID, destinationPorts, ignoreCIDRs, utils.IsIPv6(vipIP))
err := nftables.ApplySNAT(podIP, vipIP, serviceUUID, destinationPorts, ignoreCIDRs, allowCIDRs, utils.IsIPv6(vipIP))
if err != nil {
return fmt.Errorf("error performing netlink nftables [%s]", err)
}
@@ -269,9 +276,9 @@ func (p *Processor) configureEgress(vipIP, podIP, namespace, serviceUUID string,
}
}
mask := "/32"
mask := fmt.Sprintf("/%d", vip.DefaultMaskIPv4)
if !utils.IsIPv4(podIP) {
mask = "/128"
mask = fmt.Sprintf("/%d", vip.DefaultMaskIPv6)
}
if allowedNetworks != "" {
@@ -331,12 +338,12 @@ func (p *Processor) configureEgress(vipIP, podIP, namespace, serviceUUID string,
return nil
}
func (p *Processor) AutoDiscoverCIDRs() (serviceCIDR, podCIDR string, err error) {
func (p *Processor) AutoDiscoverCIDRs(ctx context.Context) (serviceCIDR, podCIDR string, err error) {
log.Debug("Trying to automatically discover Service and Pod CIDRs")
options := v1.ListOptions{
LabelSelector: "component=kube-controller-manager",
}
podList, err := p.clientSet.CoreV1().Pods("kube-system").List(context.TODO(), options)
podList, err := p.clientSet.CoreV1().Pods("kube-system").List(ctx, options)
if err != nil {
return "", "", fmt.Errorf("[Egress] Unable to get kube-controller-manager pod: %w", err)
}

View File

@@ -3,15 +3,14 @@ package services
import (
"context"
"fmt"
"time"
"sync"
log "log/slog"
"github.com/kube-vip/kube-vip/pkg/election"
"github.com/kube-vip/kube-vip/pkg/lease"
"github.com/kube-vip/kube-vip/pkg/servicecontext"
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/tools/leaderelection"
"k8s.io/client-go/tools/leaderelection/resourcelock"
)
// The StartServicesWatchForLeaderElection function will start a services watcher, the
@@ -27,7 +26,6 @@ func (p *Processor) StartServicesWatchForLeaderElection(ctx context.Context) err
for i := range cluster.Network {
_ = cluster.Network[i].DeleteRoute()
}
cluster.Stop()
}
}
}
@@ -38,114 +36,145 @@ func (p *Processor) StartServicesWatchForLeaderElection(ctx context.Context) err
}
// The startServicesWatchForLeaderElection function will start a services watcher, the
func (p *Processor) StartServicesLeaderElection(ctx context.Context, service *v1.Service) error {
serviceLease, _ := lease.GetName(service)
log.Info("new leader election", "service", service.Name, "namespace", service.Namespace, "lock_name", serviceLease, "host_id", p.config.NodeName)
// we use the Lease lock type since edits to Leases are less common
// and fewer objects in the cluster watch "all Leases".
lock := &resourcelock.LeaseLock{
LeaseMeta: metav1.ObjectMeta{
Name: serviceLease,
Namespace: service.Namespace,
},
Client: p.clientSet.CoordinationV1(),
LockConfig: resourcelock.ResourceLockConfig{
Identity: p.config.NodeName,
},
func (p *Processor) StartServicesLeaderElection(svcCtx *servicecontext.Context, service *v1.Service, _ *sync.WaitGroup) error {
if svcCtx == nil {
return fmt.Errorf("no context context for service %q with UID %q: nil context", service.Name, service.UID)
}
go func() {
// wait for the service context to end and delete the lease then
<-ctx.Done()
p.leaseMgr.Delete(service)
leaseNamespace, serviceLease := lease.ServiceName(service)
id := lease.NewID(p.config.LeaderElectionType, leaseNamespace, serviceLease)
objectName := lease.ServiceNamespacedName(service)
svcLease := p.leaseMgr.Get(id)
if svcLease == nil {
return fmt.Errorf("no existing lease found for service %q with UID %q", service.Name, service.UID)
}
isNew := svcLease.Add(objectName)
// this service was already processed so we do not need to do anything
if !isNew && svcLease.Elected.Load() {
log.Debug("this service was already handled, waiting for it to finish", "service", service.Name, "uid", service.UID)
// Wait for either the service context or lease context to be done
select {
case <-svcCtx.Ctx.Done():
case <-svcLease.Ctx.Done():
}
return nil
}
svcLease.Lock()
defer func() {
svcLease.Unlock()
}()
svcCtx, err := p.getServiceContext(service.UID)
if err != nil {
return fmt.Errorf("failed to get context for service %q with UID %q: %w", service.Name, service.UID, err)
}
if svcCtx == nil {
return fmt.Errorf("failed to get context for service %q with UID %q: nil context", service.Name, service.UID)
}
wg := sync.WaitGroup{}
defer wg.Wait()
svcCtx.IsActive = true
// Start a goroutine that will delete the lease when the service context is cancelled.
// This is important for proper cleanup when a service is deleted - it ensures that
// the lease context (svcLease.Ctx) gets cancelled, which causes RunOrDie to return.
// Without this, RunOrDie would continue running until leadership is naturally lost.
wg.Go(func() {
<-svcCtx.Ctx.Done()
p.leaseMgr.Delete(id, objectName)
})
svcLease, isNew := p.leaseMgr.Add(service)
// this service is sharing lease
if !isNew {
// this service is sharing lease with another service
if svcLease.Elected.Load() {
svcLease.Unlock()
// wait for leader election to start or context to be done
select {
case <-svcLease.Started:
case <-svcLease.Ctx.Done():
// Lease was cancelled (e.g., leader election ended), return immediately
// This allows the restart loop to create a fresh lease
log.Debug("lease context cancelled before leader election started", "service", service.Name, "uid", service.UID)
svcCtx.IsActive = false
return nil
}
<-svcLease.Started
if lease.UsesCommon(service) {
if err := p.SyncServices(ctx, service); err != nil {
log.Error("service sync", "err", err)
// Common lease handling: sync the service and wait for context cancellation
if !svcCtx.IsActive {
if err := p.SyncServices(svcCtx, service, &wg); err != nil {
log.Error("service sync", "err", err, "uid", service.UID)
svcLease.Cancel()
}
// just block until context is cancelled
<-ctx.Done()
if svcCtx.IsActive {
if err := p.deleteService(service.UID); err != nil {
log.Error("service deletion", "err", err)
}
svcCtx.IsActive = true
}
// Block until service context is cancelled
<-svcCtx.Ctx.Done()
if svcCtx.IsActive {
// we have no context left here so we use a new one
if err := p.deleteService(context.TODO(), service.UID); err != nil {
log.Error("service deletion", "uid", service.UID, "err", err)
}
}
// wait for leaderelection to be finished
<-svcLease.Ctx.Done()
// Mark this service is inactive
svcCtx.IsActive = false
// wait for leaderelection to be finished
<-svcLease.Ctx.Done()
return nil
}
// start the leader election code loop
leaderelection.RunOrDie(svcLease.Ctx, leaderelection.LeaderElectionConfig{
Lock: lock,
// IMPORTANT: you MUST ensure that any code you have that
// is protected by the lease must terminate **before**
// you call cancel. Otherwise, you could have a background
// loop still running and another process could
// get elected before your background loop finished, violating
// the stated goal of the lease.
ReleaseOnCancel: true,
LeaseDuration: time.Duration(p.config.LeaseDuration) * time.Second,
RenewDeadline: time.Duration(p.config.RenewDeadline) * time.Second,
RetryPeriod: time.Duration(p.config.RetryPeriod) * time.Second,
Callbacks: leaderelection.LeaderCallbacks{
OnStartedLeading: func(ctx context.Context) {
// Mark this service as active (as we've started leading)
// we run this in background as it's blocking
if err := p.SyncServices(ctx, service); err != nil {
log.Error("service sync", "err", err)
svcLease.Cancel()
}
close(svcLease.Started)
},
OnStoppedLeading: func() {
// we can do cleanup here
log.Info("leadership lost", "service", service.Name, "leader", p.config.NodeName)
if svcCtx.IsActive {
if err := p.deleteService(service.UID); err != nil {
log.Error("service deletion", "err", err)
}
}
// Mark this service is inactive
svcCtx.IsActive = false
},
OnNewLeader: func(identity string) {
// we're notified when new leader elected
if identity == p.config.NodeName {
// I just got the lock
return
}
log.Info("new leader", "leader", identity)
},
log.Info("new leader election", "service", service.Name, "namespace", service.Namespace, "lock_name", serviceLease, "host_id", p.config.NodeName)
leaderCtx, leaderCancel := context.WithCancel(svcLease.Ctx)
svcCtx.LeaderCancel = leaderCancel
run := election.RunConfig{
Config: p.config,
LeaseID: id,
Mgr: p.electionMgr,
LeaseAnnotations: map[string]string{},
OnStartedLeading: func(_ context.Context) {
svcLease.Elected.Store(true)
svcLease.Unlock()
close(svcLease.Started)
// Mark this service as active (as we've started leading)
// we run this in background as it's blocking
svcCtx.IsActive = true
if err := p.SyncServices(svcCtx, service, &wg); err != nil {
log.Error("service sync", "uid", service.UID, "err", err)
leaderCancel()
}
},
})
log.Info("stopping leader election", "service", service.Name)
OnStoppedLeading: func() {
// we can do cleanup here
svcLease.Elected.Store(false)
log.Info("leadership lost", "service", service.Name, "uid", service.UID, "leader", p.config.NodeName)
if svcCtx.IsActive {
log.Debug("deleting service due to lost leadership", "uid", service.UID)
if err := p.deleteService(svcLease.Ctx, service.UID); err != nil {
log.Error("service deletion", "err", err)
}
}
// Mark this service is inactive
svcCtx.IsActive = false
svcLease.Started = make(chan any)
leaderCancel()
},
OnNewLeader: func(identity string) {
// we're notified when new leader elected
if identity == p.config.NodeName {
// I just got the lock
return
}
log.Info("new leader", "leader", identity, "service", service.Name, "uid", service.UID)
},
}
if err := election.RunOrDie(leaderCtx, &run, p.config); err != nil {
return fmt.Errorf("services election failed: %w", err)
}
log.Info("stopping leader election", "service", service.Name, "uid", service.UID)
return nil
}

View File

@@ -2,6 +2,7 @@ package services
import (
"context"
"errors"
"fmt"
log "log/slog"
"reflect"
@@ -9,6 +10,7 @@ import (
"github.com/kube-vip/kube-vip/pkg/arp"
"github.com/kube-vip/kube-vip/pkg/bgp"
"github.com/kube-vip/kube-vip/pkg/election"
"github.com/kube-vip/kube-vip/pkg/endpoints"
"github.com/kube-vip/kube-vip/pkg/endpoints/providers"
"github.com/kube-vip/kube-vip/pkg/instance"
@@ -16,7 +18,9 @@ import (
"github.com/kube-vip/kube-vip/pkg/lease"
"github.com/kube-vip/kube-vip/pkg/networkinterface"
"github.com/kube-vip/kube-vip/pkg/servicecontext"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/kube-vip/kube-vip/pkg/vip"
"github.com/kube-vip/kube-vip/pkg/wireguard"
"github.com/prometheus/client_golang/prometheus"
"github.com/vishvananda/netlink"
v1 "k8s.io/api/core/v1"
@@ -39,8 +43,6 @@ type Processor struct {
clientSet *kubernetes.Clientset
rwClientSet *kubernetes.Clientset
shutdownChan chan struct{}
// This is a prometheus counter used to count the number of events received
// from the service watcher
CountServiceWatchEvent *prometheus.CounterVec
@@ -52,6 +54,11 @@ type Processor struct {
// nodeLabelManager is the manager for the node labels
nodeLabelManager labelManager
electionMgr *election.Manager
// TunnelMgr manages multiple WireGuard tunnels (one per service VIP)
TunnelMgr *wireguard.TunnelManager
}
// labelManager is the interface for the node label manager to add/remove labels
@@ -61,8 +68,9 @@ type labelManager interface {
}
func NewServicesProcessor(config *kubevip.Config, bgpServer *bgp.Server,
clientSet *kubernetes.Clientset, rwClientSet *kubernetes.Clientset, shutdownChan chan struct{},
intfMgr *networkinterface.Manager, arpMgr *arp.Manager, nodeLabelManager labelManager) *Processor {
clientSet *kubernetes.Clientset, rwClientSet *kubernetes.Clientset,
intfMgr *networkinterface.Manager, arpMgr *arp.Manager, nodeLabelManager labelManager,
electionMgr *election.Manager, leaseMgr *lease.Manager) *Processor {
lbClassFilterFunc := lbClassFilter
if config.LoadBalancerClassLegacyHandling {
lbClassFilterFunc = lbClassFilterLegacy
@@ -75,7 +83,6 @@ func NewServicesProcessor(config *kubevip.Config, bgpServer *bgp.Server,
bgpServer: bgpServer,
clientSet: clientSet,
rwClientSet: rwClientSet,
shutdownChan: shutdownChan,
CountServiceWatchEvent: prometheus.NewCounterVec(prometheus.CounterOpts{
Namespace: "kube_vip",
Subsystem: "manager",
@@ -85,56 +92,67 @@ func NewServicesProcessor(config *kubevip.Config, bgpServer *bgp.Server,
intfMgr: intfMgr,
arpMgr: arpMgr,
leaseMgr: lease.NewManager(),
leaseMgr: leaseMgr,
nodeLabelManager: nodeLabelManager,
electionMgr: electionMgr,
TunnelMgr: wireguard.NewTunnelManager(),
}
}
func (p *Processor) AddOrModify(ctx context.Context, event watch.Event, serviceFunc func(context.Context, *v1.Service) error) (bool, error) {
// log.Debugf("Endpoints for service [%s] have been Created or modified", s.service.ServiceName)
func (p *Processor) AddOrModify(ctx context.Context, event watch.Event, serviceFunc func(*servicecontext.Context, *v1.Service, *sync.WaitGroup) error, wg *sync.WaitGroup) error {
svc, ok := event.Object.(*v1.Service)
if !ok {
return false, fmt.Errorf("unable to parse Kubernetes services from API watcher")
return fmt.Errorf("unable to parse Kubernetes services from API watcher")
}
// We only care about LoadBalancer services
if svc.Spec.Type != v1.ServiceTypeLoadBalancer {
return true, nil
return nil
}
// Check if we ignore this service
if svc.Annotations[kubevip.LoadbalancerIgnore] == "true" {
log.Info("ignore annotation for kube-vip", "service name", svc.Name)
return true, nil
return nil
}
// Check the loadBalancer class
if p.lbClassFilter(svc, p.config) {
return true, nil
return nil
}
svcAddresses, svcHostnames := instance.FetchServiceAddresses(svc)
// We only care about LoadBalancer services that have been allocated an address
if len(svcAddresses) <= 0 {
return true, nil
if len(svcAddresses) <= 0 && len(svcHostnames) <= 0 {
return nil
}
svcInstance := instance.FindServiceInstance(svc, p.ServiceInstances)
var err error
if svcInstance == nil {
svcInstance, err = instance.NewInstance(ctx, svc, p.config, p.intfMgr, p.arpMgr, wg)
if err != nil {
return fmt.Errorf("unalbe to create instance for service %s/%s", svc.Namespace, svc.Name)
}
p.ServiceInstances = append(p.ServiceInstances, svcInstance)
}
_, usesCommonLease := svc.Annotations[kubevip.ServiceLease]
if usesCommonLease && svc.Spec.ExternalTrafficPolicy != v1.ServiceExternalTrafficPolicyTypeCluster {
return false, fmt.Errorf("annotation %q cannot be used with service traffic policy other than %q",
kubevip.ServiceLease, v1.ServiceExternalTrafficPolicyTypeCluster)
return fmt.Errorf("annotation %q cannot be used with service traffic policy other than %q, service %s/%s",
kubevip.ServiceLease, v1.ServiceExternalTrafficPolicyTypeCluster, svc.Namespace, svc.Name)
}
svcCtx, err := p.getServiceContext(svc.UID)
if err != nil {
return false, fmt.Errorf("failed to get service context: %w", err)
return fmt.Errorf("failed to get service context: %w", err)
}
// The modified event should only be triggered if the service has been modified (i.e. moved somewhere else)
if event.Type == watch.Modified {
i := instance.FindServiceInstance(svc, p.ServiceInstances)
shouldGarbageCollect := true
i := svcInstance
shouldGarbageCollect := false
if i != nil {
originalServiceAddresses, originalServiceHostnames := instance.FetchServiceAddresses(i.ServiceSnapshot)
shouldGarbageCollect =
@@ -143,7 +161,13 @@ func (p *Processor) AddOrModify(ctx context.Context, event watch.Event, serviceF
// Service hostnames changed
!reflect.DeepEqual(originalServiceHostnames, svcHostnames) ||
// ExternalTrafficPolicy changed
svc.Spec.ExternalTrafficPolicy != i.ServiceSnapshot.Spec.ExternalTrafficPolicy
svc.Spec.ExternalTrafficPolicy != i.ServiceSnapshot.Spec.ExternalTrafficPolicy ||
// IP stack configuration changed
!reflect.DeepEqual(svc.Spec.IPFamilies, i.ServiceSnapshot.Spec.IPFamilies) ||
*svc.Spec.IPFamilyPolicy != *i.ServiceSnapshot.Spec.IPFamilyPolicy ||
// DDNS was disabled/enabled
svc.Annotations[kubevip.ServiceDDNS] != i.ServiceSnapshot.Annotations[kubevip.ServiceDDNS]
}
if shouldGarbageCollect {
for _, addr := range svcAddresses {
@@ -165,7 +189,7 @@ func (p *Processor) AddOrModify(ctx context.Context, event watch.Event, serviceF
log.Warn("(svcs) waiting for load balancer to finish")
<-svcCtx.Ctx.Done()
if err := p.deleteService(svc.UID); err != nil {
if err := p.deleteService(ctx, svc.UID); err != nil {
log.Error("(svc) unable to remove", "service", svc.UID)
}
// in theory this should never fail
@@ -188,11 +212,47 @@ func (p *Processor) AddOrModify(ctx context.Context, event watch.Event, serviceF
log.Debug("(svcs) has been added/modified with addresses", "service name", svc.Name, "ips", ips, "hostnames", hostnames)
if svcCtx == nil {
svcCtx = servicecontext.New(ctx)
log.Debug("new context for service", "namespace", svc.Namespace, "name", svc.Name)
ns, name := lease.ServiceName(svc)
leaseID := lease.NewID(p.config.LeaderElectionType, ns, name)
lease := p.leaseMgr.Add(ctx, leaseID)
svcCtx = servicecontext.New(lease.Ctx)
p.svcMap.Store(svc.UID, svcCtx)
}
if p.config.EnableServicesElection || // Service Election
// WireGuard services always need endpoint watching for DNAT rule updates
// This is independent of leader election settings (which are for control plane HA)
if p.config.EnableWireguard && !svcCtx.IsWatched {
// Call serviceFunc first to set up the WireGuard tunnel
err = serviceFunc(svcCtx, svc, wg)
if err != nil {
log.Error(err.Error())
if errors.Is(err, &utils.PanicError{}) {
return err
}
}
wg.Go(func() {
defer func() {
if svcCtx != nil {
svcCtx.IsWatched = false
}
}()
// Start endpoint watcher for WireGuard services (uses EndpointSlices by default)
var provider providers.Provider
if p.config.EnableEndpoints {
provider = providers.NewEndpoints()
} else {
provider = providers.NewEndpointslices()
}
if err = p.watchEndpoint(svcCtx, p.config.NodeName, svc, provider); err != nil {
log.Error(err.Error())
}
})
svcCtx.IsWatched = true
} else if p.config.EnableServicesElection || // Service Election
((p.config.EnableRoutingTable || p.config.EnableBGP) && // Routing table mode or BGP
(!p.config.EnableLeaderElection && !p.config.EnableServicesElection)) { // No leaderelection or services election
@@ -203,13 +263,22 @@ func (p *Processor) AddOrModify(ctx context.Context, event watch.Event, serviceF
if !svcCtx.IsWatched {
// background the endpoint watcher
if (p.config.EnableRoutingTable || p.config.EnableBGP) && (!p.config.EnableLeaderElection && !p.config.EnableServicesElection) {
err = serviceFunc(svcCtx.Ctx, svc)
err = serviceFunc(svcCtx, svc, wg)
if err != nil {
log.Error(err.Error())
if errors.Is(err, &utils.PanicError{}) {
return err
}
}
}
go func() {
wg.Go(func() {
defer func() {
if svcCtx != nil {
svcCtx.IsWatched = false
}
}()
if svc.Spec.ExternalTrafficPolicy == v1.ServiceExternalTrafficPolicyTypeLocal {
// Add Endpoint or EndpointSlices watcher
var provider providers.Provider
@@ -222,18 +291,27 @@ func (p *Processor) AddOrModify(ctx context.Context, event watch.Event, serviceF
log.Error(err.Error())
}
}
}()
})
// We're now watching this service
svcCtx.IsWatched = true
}
} else if (p.config.EnableBGP || p.config.EnableRoutingTable) && (!p.config.EnableLeaderElection && !p.config.EnableServicesElection) {
err = serviceFunc(svcCtx.Ctx, svc)
err = serviceFunc(svcCtx, svc, wg)
if err != nil {
log.Error(err.Error())
if errors.Is(err, &utils.PanicError{}) {
return err
}
}
go func() {
wg.Go(func() {
defer func() {
if svcCtx != nil {
svcCtx.IsWatched = false
}
}()
if svc.Spec.ExternalTrafficPolicy == v1.ServiceExternalTrafficPolicyTypeCluster {
// Add Endpoint watcher
var provider providers.Provider
@@ -246,62 +324,72 @@ func (p *Processor) AddOrModify(ctx context.Context, event watch.Event, serviceF
log.Error(err.Error())
}
}
}()
})
// We're now watching this service
svcCtx.IsWatched = true
} else {
go func() {
wg.Go(func() {
for {
select {
case <-svcCtx.Ctx.Done():
log.Warn("(svcs) restartable service watcher ending", "uid", svc.UID)
return
default:
log.Info("(svcs) restartable service watcher starting", "uid", svc.UID)
err = serviceFunc(svcCtx.Ctx, svc)
if err != nil {
log.Error(err.Error())
if !svcCtx.IsActive {
log.Info("(svcs) restartable service watcher starting", "uid", svc.UID)
err = serviceFunc(svcCtx, svc, wg)
if err != nil {
log.Error(err.Error())
if errors.Is(err, &utils.PanicError{}) {
svcCtx.Cancel()
}
}
log.Info("(svcs) restartable service watcher done", "uid", svc.UID)
}
}
}
}()
})
}
} else {
// Increment the waitGroup before the service Func is called (Done is completed in there)
err = serviceFunc(svcCtx.Ctx, svc)
err = serviceFunc(svcCtx, svc, wg)
if err != nil {
log.Error(err.Error())
if errors.Is(err, &utils.PanicError{}) {
return err
}
}
}
svcCtx.IsActive = true
if !p.config.EnableServicesElection {
log.Debug("Service now active", "name", svc.Name, "uid", svc.UID)
svcCtx.IsActive = true
}
}
return false, nil
return nil
}
func (p *Processor) Delete(event watch.Event) (bool, error) {
func (p *Processor) Delete(event watch.Event) error {
svc, ok := event.Object.(*v1.Service)
if !ok {
return false, fmt.Errorf("(svcs) unable to parse Kubernetes services from API watcher")
return fmt.Errorf("(svcs) unable to parse Kubernetes services from API watcher")
}
svcCtx, err := p.getServiceContext(svc.UID)
if err != nil {
return false, fmt.Errorf("(svcs) unable to get context: %w", err)
return fmt.Errorf("(svcs) unable to get context: %w", err)
}
if svcCtx != nil {
// We only care about LoadBalancer services
if svc.Spec.Type != v1.ServiceTypeLoadBalancer {
return true, nil
return nil
}
// We can ignore this service
if svc.Annotations[kubevip.LoadbalancerIgnore] == "true" {
log.Info("(svcs) ignore annotation for kube-vip", "service name", svc.Name)
return true, nil
return nil
}
// If no leader election is enabled, delete routes here
@@ -312,24 +400,26 @@ func (p *Processor) Delete(event watch.Event) (bool, error) {
}
}
// If this is an active service then and additional leaderElection will handle stopping
err = p.deleteService(svc.UID)
if err != nil {
log.Error(err.Error())
if svcCtx.IsActive && !p.config.EnableServicesElection {
// If this is an active service then and additional leaderElection will handle stopping
err = p.deleteService(svcCtx.Ctx, svc.UID)
if err != nil {
log.Error(err.Error())
}
svcCtx.IsActive = false
}
// Calls the cancel function of the context
log.Warn("(svcs) The load balancer was deleted, cancelling context")
svcCtx.IsActive = false
log.Warn("(svcs) The load balancer was deleted, cancelling context", "namespace", svc.Namespace, "name", svc.Name, "uid", svc.UID)
svcCtx.Cancel()
log.Warn("(svcs) waiting for load balancer to finish")
log.Warn("(svcs) waiting for load balancer to finish", "namespace", svc.Namespace, "name", svc.Name, "uid", svc.UID)
<-svcCtx.Ctx.Done()
p.svcMap.Delete(svc.UID)
}
log.Info("(svcs) deleted", "service name", svc.Name, "namespace", svc.Namespace)
return true, nil
return nil
}
func (p *Processor) Stop() {

View File

@@ -3,8 +3,10 @@ package services
import (
"context"
"fmt"
"net"
"slices"
"strings"
"sync"
"time"
log "log/slog"
@@ -23,6 +25,7 @@ import (
"github.com/kube-vip/kube-vip/pkg/endpoints/providers"
"github.com/kube-vip/kube-vip/pkg/instance"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/servicecontext"
"github.com/kube-vip/kube-vip/pkg/upnp"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/kube-vip/kube-vip/pkg/vip"
@@ -34,33 +37,36 @@ const (
ActionDelete ServiceInstanceAction = "delete"
ActionAdd ServiceInstanceAction = "add"
ActionNone ServiceInstanceAction = "none"
// Default UPNP lease requested is 3600 seconds.
defaultUPNPLeaseDuration = 1 * time.Hour
)
func (p *Processor) SyncServices(ctx context.Context, svc *v1.Service) error {
log.Debug("[STARTING] Service Sync", "namespace", svc.Namespace, "name", svc.Name)
func (p *Processor) SyncServices(ctx *servicecontext.Context, svc *v1.Service, wg *sync.WaitGroup) error {
log.Debug("[STARTING] Service Sync", "namespace", svc.Namespace, "name", svc.Name, "uid", svc.UID)
// Iterate through the synchronising services
action := p.getServiceInstanceAction(svc)
action, instance := p.getServiceInstanceAction(svc)
switch action {
case ActionDelete:
// remove the label from the node before deleting the service
if err := p.nodeLabelManager.RemoveLabel(ctx, svc); err != nil {
if err := p.nodeLabelManager.RemoveLabel(ctx.Ctx, svc); err != nil {
return fmt.Errorf("error removing label from node: %w", err)
}
log.Debug("[service] delete", "namespace", svc.Namespace, "name", svc.Name, "uid", svc.UID)
if err := p.deleteService(svc.UID); err != nil {
if err := p.deleteService(ctx.Ctx, svc.UID); err != nil {
return fmt.Errorf("error deleting service %s/%s: %w", svc.Namespace, svc.Name, err)
}
case ActionAdd:
log.Debug("[service] add", "namespace", svc.Namespace, "name", svc.Name, "uid", svc.UID)
if err := p.addService(ctx, svc); err != nil {
if err := p.addService(ctx.Ctx, instance, svc, wg); err != nil {
return fmt.Errorf("error adding service %s/%s: %w", svc.Namespace, svc.Name, err)
}
// add the label to the node after adding the service
if err := p.nodeLabelManager.AddLabel(ctx, svc); err != nil {
if err := p.nodeLabelManager.AddLabel(ctx.Ctx, svc); err != nil {
return fmt.Errorf("error adding label to node: %w", err)
}
case ActionNone:
@@ -70,7 +76,7 @@ func (p *Processor) SyncServices(ctx context.Context, svc *v1.Service) error {
return nil
}
func (p *Processor) getServiceInstanceAction(svc *v1.Service) ServiceInstanceAction {
func (p *Processor) getServiceInstanceAction(svc *v1.Service) (ServiceInstanceAction, *instance.Instance) {
// protect against multiple calls
// get the annotations or legacy values from manual configuration
addresses, hostnames := instance.FetchServiceAddresses(svc)
@@ -81,37 +87,54 @@ func (p *Processor) getServiceInstanceAction(svc *v1.Service) ServiceInstanceAct
for _, instance := range p.ServiceInstances {
if instance != nil && instance.ServiceSnapshot.UID == svc.UID {
if !instance.AddCalled {
return ActionAdd, instance
}
for _, address := range addresses {
// handle the case where the service instance needs to be deleted
if instance.IsDHCP {
if address != "0.0.0.0" && address != "::" {
return ActionDelete
if instance.IsDHCPv4 {
if address != "0.0.0.0" {
return ActionDelete, instance
}
if len(svc.Status.LoadBalancer.Ingress) > 0 && !slices.Contains(statusAddresses, instance.DHCPInterfaceIP) {
return ActionDelete
if len(svc.Status.LoadBalancer.Ingress) > 0 && !slices.Contains(statusAddresses, instance.DHCPInterfaceIPv4) {
return ActionDelete, instance
}
}
if !instance.IsDHCP {
if address == "0.0.0.0" || address == "::" {
return ActionDelete
} else {
if address == "0.0.0.0" {
return ActionDelete, instance
}
if len(svc.Status.LoadBalancer.Ingress) > 0 && !slices.Contains(statusAddresses, address) {
return ActionDelete
return ActionDelete, instance
}
}
if instance.IsDHCPv6 {
if address != "::" {
return ActionDelete, instance
}
if len(svc.Status.LoadBalancer.Ingress) > 0 && !slices.Contains(statusAddresses, instance.DHCPInterfaceIPv6) {
return ActionDelete, instance
}
} else {
if address == "::" {
return ActionDelete, instance
}
if len(svc.Status.LoadBalancer.Ingress) > 0 && !slices.Contains(statusAddresses, address) {
return ActionDelete, instance
}
}
if len(svc.Status.LoadBalancer.Ingress) > 0 && !comparePortsAndPortStatuses(svc) {
return ActionDelete
return ActionDelete, instance
}
}
// If we reach here, it means the service instance matches the service UID and is not a DHCP service, so we can return "no action"
return ActionNone
return ActionNone, instance
}
}
if len(addresses) > 0 || len(hostnames) > 0 {
log.Debug("no matching service instance found", "service", svc.Name, "namespace", svc.Namespace, "addresses", addresses, "hostnames", hostnames)
return ActionAdd // If no matching instance is found, we need to add a new service instance
log.Debug("no matching service instance found", "service", svc.Name, "namespace", svc.Namespace, "uid", svc.UID, "addresses", addresses, "hostnames", hostnames)
return ActionAdd, nil // If no matching instance is found, we need to add a new service instance
}
return ActionNone
return ActionNone, nil
}
func comparePortsAndPortStatuses(svc *v1.Service) bool {
@@ -130,46 +153,92 @@ func comparePortsAndPortStatuses(svc *v1.Service) bool {
return true
}
func (p *Processor) addService(ctx context.Context, svc *v1.Service) error {
func (p *Processor) addService(ctx context.Context, newService *instance.Instance, svc *v1.Service, wg *sync.WaitGroup) error {
// protect against addService while reading
p.mutex.Lock()
defer p.mutex.Unlock()
startTime := time.Now()
newService, err := instance.NewInstance(svc, p.config, p.intfMgr, p.arpMgr)
if err != nil {
return err
var err error
if newService == nil {
newService, err = instance.NewInstance(ctx, svc, p.config, p.intfMgr, p.arpMgr, wg)
if err != nil {
return err
}
p.ServiceInstances = append(p.ServiceInstances, newService)
}
for x := range newService.VIPConfigs {
log.Debug("starting loadbalancer for service", "name", svc.Name, "namespace", svc.Namespace)
newService.Clusters[x].StartLoadBalancerService(ctx, newService.VIPConfigs[x], p.bgpServer, svc.Name, p.CountRouteReferences)
log.Debug("starting loadbalancer for service", "name", svc.Name, "namespace", svc.Namespace, "uid", svc.UID)
if err := newService.Clusters[x].StartLoadBalancerService(ctx, newService.VIPConfigs[x], p.bgpServer, svc.Name, p.CountRouteReferences, wg); err != nil {
return fmt.Errorf("failed to start lb: %w", err)
}
}
p.upnpMap(ctx, newService)
if newService.IsDHCP && len(newService.VIPConfigs) == 1 {
go func() {
for ip := range newService.DHCPClient.IPChannel() {
log.Debug("IP changed", "ip", ip)
newService.VIPConfigs[0].VIP = ip
newService.DHCPInterfaceIP = ip
if !p.config.DisableServiceUpdates {
if err := p.updateStatus(newService); err != nil {
log.Warn("updating svc", "err", err)
}
if newService.IsDHCPv4 {
wg.Go(func() {
index := -1
for i := range newService.VIPConfigs {
ip := net.ParseIP(newService.VIPConfigs[i].VIP)
if ip.To4() != nil {
index = i
break
}
}
log.Debug("IP update channel closed, stopping")
}()
if index == -1 {
log.Error("unable to find proper VIPConfig for the DHCPv4")
} else {
for ip := range newService.DHCPv4Client.IPChannel() {
log.Debug("IP changed", "ip", ip)
newService.VIPConfigs[index].VIP = ip
newService.DHCPInterfaceIPv4 = ip
if !p.config.DisableServiceUpdates {
if err := p.updateStatus(ctx, newService); err != nil {
log.Warn("updating svc", "err", err)
}
}
}
log.Debug("IPv4 update channel closed, stopping")
}
})
}
p.ServiceInstances = append(p.ServiceInstances, newService)
if newService.IsDHCPv6 {
wg.Go(func() {
index := -1
for i := range newService.VIPConfigs {
ip := net.ParseIP(newService.VIPConfigs[i].VIP)
if ip.To4() == nil {
index = i
break
}
}
if index == -1 {
log.Error("unable to find proper VIPConfig for the DHCPv6")
} else {
for ip := range newService.DHCPv4Client.IPChannel() {
log.Debug("IP changed", "ip", ip)
newService.VIPConfigs[index].VIP = ip
newService.DHCPInterfaceIPv6 = ip
if !p.config.DisableServiceUpdates {
if err := p.updateStatus(ctx, newService); err != nil {
log.Warn("updating svc", "err", err)
}
}
}
log.Debug("IPv6 update channel closed, stopping")
}
})
}
if !p.config.DisableServiceUpdates {
log.Debug("[service] update", "namespace", newService.ServiceSnapshot.Namespace, "name", newService.ServiceSnapshot.Name)
if err := p.updateStatus(newService); err != nil {
if err := p.updateStatus(ctx, newService); err != nil {
log.Error("[service] updating status", "namespace", newService.ServiceSnapshot.Namespace, "name", newService.ServiceSnapshot.Name, "err", err)
}
}
@@ -220,7 +289,7 @@ func (p *Processor) addService(ctx context.Context, svc *v1.Service) error {
podIP = svc.Annotations[kubevip.ActiveEndpointIPv6]
err = p.configureEgress(serviceIP, podIP, svc.Namespace, string(svc.UID), svc.Annotations)
err = p.configureEgress(ctx, serviceIP, podIP, svc.Namespace, string(svc.UID), svc.Annotations)
if err != nil {
errList = append(errList, err)
log.Warn("[service] configuring egress IPv6", "service", svc.Name, "namespace", svc.Namespace, "err", err)
@@ -234,7 +303,7 @@ func (p *Processor) addService(ctx context.Context, svc *v1.Service) error {
if !p.config.EnableEndpoints && utils.IsIPv6(serviceIP) {
podIPs = svc.Annotations[kubevip.ActiveEndpointIPv6]
}
err = p.configureEgress(serviceIP, podIPs, svc.Namespace, string(svc.UID), svc.Annotations)
err = p.configureEgress(ctx, serviceIP, podIPs, svc.Namespace, string(svc.UID), svc.Annotations)
if err != nil {
errList = append(errList, err)
log.Warn("[service] configuring egress IPv4", "service", svc.Name, "namespace", svc.Namespace, "err", err)
@@ -248,20 +317,29 @@ func (p *Processor) addService(ctx context.Context, svc *v1.Service) error {
} else {
provider = providers.NewEndpointslices()
}
err = provider.UpdateServiceAnnotation(svc.Annotations[kubevip.ActiveEndpoint], svc.Annotations[kubevip.ActiveEndpointIPv6], svc, p.clientSet)
err = provider.UpdateServiceAnnotation(ctx, svc.Annotations[kubevip.ActiveEndpoint], svc.Annotations[kubevip.ActiveEndpointIPv6], svc, p.clientSet)
if err != nil {
log.Warn("[service] configuring egress", "service", svc.Name, "namespace", svc.Namespace, "err", err)
}
}
}
// Configure WireGuard DNAT rules if WireGuard is enabled
if p.config.EnableWireguard {
log.Debug("[service] configuring WireGuard DNAT rules", "service", svc.Name, "namespace", svc.Namespace)
if err := p.addServiceWireguard(ctx, svc); err != nil {
log.Warn("[service] failed to configure WireGuard DNAT", "service", svc.Name, "namespace", svc.Namespace, "err", err)
// Don't fail the entire service if WireGuard config fails
}
}
finishTime := time.Since(startTime)
log.Info("[service]", "service", svc.Name, "namespace", svc.Namespace, "synchronised in", fmt.Sprintf("%dms", finishTime.Milliseconds()))
return nil
}
func (p *Processor) deleteService(uid types.UID) error {
func (p *Processor) deleteService(ctx context.Context, uid types.UID) error {
// protect multiple calls
p.mutex.Lock()
defer p.mutex.Unlock()
@@ -270,7 +348,7 @@ func (p *Processor) deleteService(uid types.UID) error {
var serviceInstance *instance.Instance
found := false
for x := range p.ServiceInstances {
log.Debug("[service] lookup", "target UID", uid, "found UID ", p.ServiceInstances[x].ServiceSnapshot.UID, "name", p.ServiceInstances[x].ServiceSnapshot.Name, "namespace", p.ServiceInstances[x].ServiceSnapshot.Namespace)
log.Debug("[service] lookup", "target UID", uid, "found UID", p.ServiceInstances[x].ServiceSnapshot.UID, "name", p.ServiceInstances[x].ServiceSnapshot.Name, "namespace", p.ServiceInstances[x].ServiceSnapshot.Namespace)
// Add the running services to the new array
if p.ServiceInstances[x].ServiceSnapshot.UID != uid {
updatedInstances = append(updatedInstances, p.ServiceInstances[x])
@@ -295,7 +373,7 @@ func (p *Processor) deleteService(uid types.UID) error {
}
}
// Determine if this this VIP is shared with other loadbalancers
// Determine if this VIP is shared with other loadbalancers
shared := false
vipSet := make(map[string]interface{})
for x := range updatedInstances {
@@ -314,8 +392,15 @@ func (p *Processor) deleteService(uid types.UID) error {
for x := range serviceInstance.Clusters {
serviceInstance.Clusters[x].Stop()
}
if serviceInstance.IsDHCP {
serviceInstance.DHCPClient.Stop()
if serviceInstance.IsDHCPv4 || serviceInstance.IsDHCPv6 {
if serviceInstance.IsDHCPv4 {
serviceInstance.DHCPv4Client.Stop()
}
if serviceInstance.IsDHCPv6 {
serviceInstance.DHCPv6Client.Stop()
}
macvlan, err := netlink.LinkByName(serviceInstance.DHCPInterface)
if err != nil {
return fmt.Errorf("[service] error finding VIP Interface: %v", err)
@@ -328,7 +413,7 @@ func (p *Processor) deleteService(uid types.UID) error {
}
if p.config.EnableBGP {
endpoints.ClearBGPHostsByInstance(serviceInstance, p.bgpServer)
endpoints.ClearBGPHostsByInstance(ctx, serviceInstance, p.bgpServer)
}
if p.config.EnableRoutingTable && (p.config.EnableLeaderElection || p.config.EnableServicesElection) {
if errs := endpoints.ClearRoutesByInstance(serviceInstance.ServiceSnapshot, serviceInstance, &p.ServiceInstances); len(errs) > 0 {
@@ -353,11 +438,183 @@ func (p *Processor) deleteService(uid types.UID) error {
// Update the service array
p.ServiceInstances = updatedInstances
// Clean up WireGuard DNAT rules if WireGuard is enabled
if p.config.EnableWireguard {
log.Debug("[service] cleaning up WireGuard DNAT rules", "uid", uid, "name", serviceInstance.ServiceSnapshot.Name)
p.deleteServiceWireguard(ctx, serviceInstance.ServiceSnapshot)
}
log.Info("Removed instance from manager", "uid", uid, "name", serviceInstance.ServiceSnapshot.Name, "remaining advertised services", len(p.ServiceInstances))
return nil
}
func (p *Processor) updateEgressConfiguration(ctx context.Context, svc *v1.Service) error {
p.mutex.Lock()
defer p.mutex.Unlock()
i := instance.FindServiceInstance(svc, p.ServiceInstances)
if i == nil {
return fmt.Errorf("service instance not found for %s/%s", svc.Namespace, svc.Name)
}
oldIPv4 := i.ServiceSnapshot.Annotations[kubevip.ActiveEndpoint]
newIPv4 := svc.Annotations[kubevip.ActiveEndpoint]
oldIPv6 := i.ServiceSnapshot.Annotations[kubevip.ActiveEndpointIPv6]
newIPv6 := svc.Annotations[kubevip.ActiveEndpointIPv6]
// Skip update if endpoints haven't changed
if oldIPv4 == newIPv4 && oldIPv6 == newIPv6 {
return nil
}
log.Info("[service] updating egress configuration",
"service", svc.Name,
"namespace", svc.Namespace,
"old_ipv4", oldIPv4,
"new_ipv4", newIPv4,
"old_ipv6", oldIPv6,
"new_ipv6", newIPv6)
// Remove old egress rules if they exist
if oldIPv4 != "" || oldIPv6 != "" {
oldEndpoint := oldIPv4
if oldEndpoint == "" {
oldEndpoint = oldIPv6
}
serviceIPs, _ := instance.FetchServiceAddresses(i.ServiceSnapshot)
for _, serviceIP := range serviceIPs {
if err := egress.Teardown(
oldEndpoint,
serviceIP,
i.ServiceSnapshot.Namespace,
string(i.ServiceSnapshot.UID),
i.ServiceSnapshot.Annotations,
p.config.EgressWithNftables,
); err != nil {
log.Warn("[service] removing old egress rules", "service", svc.Name, "namespace", svc.Namespace, "err", err)
}
}
}
// Apply new egress rules with updated endpoint
serviceIPs, _ := instance.FetchServiceAddresses(svc)
errList := []error{}
// Check if egress should be IPv6
if svc.Annotations[kubevip.EgressIPv6] == "true" {
// Does the service have an active IPv6 endpoint
if newIPv6 != "" {
for _, serviceIP := range serviceIPs {
if !p.config.EnableEndpoints && utils.IsIPv6(serviceIP) {
podIP := newIPv6
err := p.configureEgress(ctx, serviceIP, podIP, svc.Namespace, string(svc.UID), svc.Annotations)
if err != nil {
errList = append(errList, err)
log.Warn("[service] configuring egress IPv6", "service", svc.Name, "namespace", svc.Namespace, "err", err)
}
}
}
}
} else if newIPv4 != "" { // Not expected to be IPv6, so should be an IPv4 address
for _, serviceIP := range serviceIPs {
podIPs := newIPv4
if !p.config.EnableEndpoints && utils.IsIPv6(serviceIP) {
podIPs = newIPv6
}
err := p.configureEgress(ctx, serviceIP, podIPs, svc.Namespace, string(svc.UID), svc.Annotations)
if err != nil {
errList = append(errList, err)
log.Warn("[service] configuring egress IPv4", "service", svc.Name, "namespace", svc.Namespace, "err", err)
}
}
}
if len(errList) > 0 {
return fmt.Errorf("errors configuring egress: %v", errList)
}
// Update the service snapshot to reflect the new state
// NOTE: Do NOT call UpdateServiceAnnotation here - the annotation was already updated
// by the endpoint processor, which is why we're being called in the first place.
// Calling it again would create an infinite loop of Modified events.
// svc is already a DeepCopy from the endpoint processor, so no need to copy again.
i.ServiceSnapshot = svc
log.Info("[service] egress configuration updated successfully", "service", svc.Name, "namespace", svc.Namespace)
return nil
}
// upnpLeaseDurationForService determines the UPNP lease duration for a given service, based on its annotations.
//
// The default lease duration is set to 1 hour, maintaining the default of 3600 seconds that was previously passed. If
// the service has an annotation of [kubevip.UpnpLeaseDuration], the function attempts to parse its value as a
// [time.Duration] using [time.ParseDuration].
//
// If parsing is successful, the lease duration is updated accordingly; otherwise, a warning is logged and the default
// duration is retained.
//
// Overriding the default lease duration can be useful for services that require longer or shorter UPNP port mappings,
// or for buggy UPNP implementations that may not handle renewals correctly. At least one router's implementation
// completely times out the mapping very shortly after creation if it is set to 3600 or 7200, but works fine if 0 is
// used.
//
// This function must therefore explicitly permit duration of 0, and callers and the underlying library must pass that
// value in XML correctly. A duration of 0 indicates to the UPNP gateway that the mapping should be permanent.
//
// It may be useful to update this function to read a global configuration option as well. This helper could also take
// in v1.Service instead of [instance.Instance], but the latter is more convenient for callers.
//
// Example where 0 was observed to stay on the problematic router: miniupnpc's test client, upnpc v2.2.4.
func upnpLeaseDurationForService(s *instance.Instance) time.Duration {
if s == nil || s.ServiceSnapshot == nil || s.ServiceSnapshot.Annotations == nil {
// No warning output. No annotation is unusual but perfectly ok.
return defaultUPNPLeaseDuration
}
// Constant is named `UpnpLeaseDuration` for consistency with `UpnpEnabled`. According to Go naming conventions
// regarding use of acronyms, `UPNPLeaseDuration` would be preferred. Cleanup of both of these is left for a future
// refactor as these are public symbols and might be used elsewhere in the ecosystem.
val, ok := s.ServiceSnapshot.Annotations[kubevip.UpnpLeaseDuration]
if !ok {
// No warning output. No annotation is common and perfectly ok.
return defaultUPNPLeaseDuration
}
if val == "" {
log.Warn("[UPNP] Lease duration annotation is empty, using default of 1 hour", "service", s.ServiceSnapshot.Name)
return defaultUPNPLeaseDuration
}
parsed, err := time.ParseDuration(val)
if err != nil {
log.Warn("[UPNP] Unable to parse lease duration from annotation, using default of 1 hour", "service", s.ServiceSnapshot.Name, "err", err)
return defaultUPNPLeaseDuration
}
if parsed < 0 {
log.Warn("[UPNP] Lease duration from annotation is negative, using default of 1 hour", "service", s.ServiceSnapshot.Name)
return defaultUPNPLeaseDuration
}
return parsed
}
// upnpLeaseDurationForServiceSec returns the UPNP lease duration for a service in uint32 seconds, as expected by the
// helper library. This is a convenience wrapper around [upnpLeaseDurationForService], and in case
// upnpLeaseDurationForService returns a duration that maps to a negative value of seconds or invalid float of seconds,
// it will return the default lease duration in seconds instead. (Technically, it will check for a reasonable range of
// seconds, e.g. ~10 years-ish.)
func upnpLeaseDurationForServiceSec(s *instance.Instance) uint32 {
duration := upnpLeaseDurationForService(s)
seconds := duration.Seconds()
// Check if within range.
if seconds >= 0 && seconds <= float64(10*365*24*60*60) {
return uint32(seconds)
}
return uint32(defaultUPNPLeaseDuration.Seconds())
}
// Set up UPNP forwards for a service
// We first try to use the more modern Pinhole API introduced in UPNPv2 and fall back to UPNPv2 Port Forwarding if no forward was successful
func (p *Processor) upnpMap(ctx context.Context, s *instance.Instance) {
@@ -374,6 +631,10 @@ func (p *Processor) upnpMap(ctx context.Context, s *instance.Instance) {
gateways := upnp.GetGatewayClients(ctx)
// Determine desired UPNP TTL / "lease duration". Passed into the library as integer seconds from now, as the
// underlying XML API wants integer seconds.
leaseDurationSec := upnpLeaseDurationForServiceSec(s)
// Reset Gateway IPs to remove stale addresses
s.UPNPGatewayIPs = make([]string, 0)
@@ -384,9 +645,9 @@ func (p *Processor) upnpMap(ctx context.Context, s *instance.Instance) {
forwardSucessful := false
if gw.WANIPv6FirewallControlClient != nil {
log.Info("[UPNP] Adding map", "vip", vip, "port", port.Port, "service", s.ServiceSnapshot.Name, "gateway", gw.WANIPv6FirewallControlClient.Location)
log.Info("[UPNP] Adding map", "vip", vip, "port", port.Port, "service", s.ServiceSnapshot.Name, "gateway", gw.WANIPv6FirewallControlClient.Location, "leaseDurationSec", leaseDurationSec)
pinholeID, pinholeErr := gw.WANIPv6FirewallControlClient.AddPinholeCtx(ctx, "0.0.0.0", uint16(port.Port), vip, uint16(port.Port), upnp.MapProtocolToIANA(string(port.Protocol)), 3600) //nolint TODO
pinholeID, pinholeErr := gw.WANIPv6FirewallControlClient.AddPinholeCtx(ctx, "0.0.0.0", uint16(port.Port), vip, uint16(port.Port), upnp.MapProtocolToIANA(string(port.Protocol)), leaseDurationSec) //nolint TODO
if pinholeErr == nil {
forwardSucessful = true
log.Info("[UPNP] Service should be accessible externally", "port", port.Port, "pinhold ID", pinholeID)
@@ -397,16 +658,16 @@ func (p *Processor) upnpMap(ctx context.Context, s *instance.Instance) {
}
// Fallback to PortForward
if !forwardSucessful {
log.Info("[UPNP] Adding map", "vip", vip, "port", port.Port, "service", s.ServiceSnapshot.Name)
log.Info("[UPNP] Adding map", "vip", vip, "port", port.Port, "service", s.ServiceSnapshot.Name, "leaseDurationSec", leaseDurationSec)
portMappingErr := gw.ConnectionClient.AddPortMapping("0.0.0.0", uint16(port.Port), strings.ToUpper(string(port.Protocol)), uint16(port.Port), vip, true, s.ServiceSnapshot.Name, 3600) //nolint TODO
portMappingErr := gw.ConnectionClient.AddPortMapping("0.0.0.0", uint16(port.Port), strings.ToUpper(string(port.Protocol)), uint16(port.Port), vip, true, s.ServiceSnapshot.Name, leaseDurationSec) //nolint TODO
if portMappingErr == nil {
ip, err := gw.ConnectionClient.GetExternalIPAddress()
if err != nil {
// Log the error but continue on the off chance the mapping was successful
log.Error("[UPNP] Unable to get external IP address from gateway", "port", port.Port, "err", err)
log.Error("[UPNP] Unable to get external IP address from gateway", "service", s.ServiceSnapshot.Name, "port", port.Port, "err", err)
} else {
log.Info("[UPNP] Service should be accessible externally", "port", port.Port, "externalip", ip)
log.Info("[UPNP] Service should be accessible externally", "service", s.ServiceSnapshot.Name, "port", port.Port, "externalip", ip)
}
forwardSucessful = true
} else {
@@ -430,7 +691,7 @@ func (p *Processor) upnpMap(ctx context.Context, s *instance.Instance) {
s.UPNPGatewayIPs = slices.Compact(s.UPNPGatewayIPs)
}
func (p *Processor) updateStatus(i *instance.Instance) error {
func (p *Processor) updateStatus(ctx context.Context, i *instance.Instance) error {
// let's retry status update every 10ms for 30s
retryConfig := wait.Backoff{
Steps: 3000,
@@ -442,7 +703,7 @@ func (p *Processor) updateStatus(i *instance.Instance) error {
err := retry.OnError(retryConfig, func(error) bool { return true }, func() error {
// Retrieve the latest version of Deployment before attempting update
// RetryOnConflict uses exponential backoff to avoid exhausting the apiserver
currentService, err := p.clientSet.CoreV1().Services(i.ServiceSnapshot.Namespace).Get(context.TODO(), i.ServiceSnapshot.Name, metav1.GetOptions{})
currentService, err := p.clientSet.CoreV1().Services(i.ServiceSnapshot.Namespace).Get(ctx, i.ServiceSnapshot.Name, metav1.GetOptions{})
if err != nil {
return err
}
@@ -457,9 +718,19 @@ func (p *Processor) updateStatus(i *instance.Instance) error {
// Add the current host
currentServiceCopy.Annotations[kubevip.VipHost] = p.config.NodeName
}
if i.DHCPInterfaceHwaddr != "" || i.DHCPInterfaceIP != "" {
if i.DHCPInterfaceHwaddr != "" || i.DHCPInterfaceIPv4 != "" || i.DHCPInterfaceIPv6 != "" {
currentServiceCopy.Annotations[kubevip.HwAddrKey] = i.DHCPInterfaceHwaddr
currentServiceCopy.Annotations[kubevip.RequestedIP] = i.DHCPInterfaceIP
dhcpInterfaceIP := ""
if i.DHCPInterfaceIPv4 != "" {
dhcpInterfaceIP = i.DHCPInterfaceIPv4
if i.DHCPInterfaceIPv6 != "" {
dhcpInterfaceIP += ","
}
}
if i.DHCPInterfaceIPv6 != "" {
dhcpInterfaceIP += i.DHCPInterfaceIPv6
}
currentServiceCopy.Annotations[kubevip.RequestedIP] = dhcpInterfaceIP
}
if currentService.Annotations["development.kube-vip.io/synthetic-api-server-error-on-update"] == "true" {
@@ -468,7 +739,7 @@ func (p *Processor) updateStatus(i *instance.Instance) error {
}
if !cmp.Equal(currentService, currentServiceCopy) {
currentService, err = p.clientSet.CoreV1().Services(currentServiceCopy.Namespace).Update(context.TODO(), currentServiceCopy, metav1.UpdateOptions{})
currentService, err = p.clientSet.CoreV1().Services(currentServiceCopy.Namespace).Update(ctx, currentServiceCopy, metav1.UpdateOptions{})
if err != nil {
log.Error("updating Spec", "service", i.ServiceSnapshot.Name, "err", err)
return err
@@ -487,7 +758,7 @@ func (p *Processor) updateStatus(i *instance.Instance) error {
for _, c := range i.VIPConfigs {
if !utils.IsIP(c.VIP) {
ips, err := utils.LookupHost(c.VIP, p.config.DNSMode)
ips, err := utils.LookupHost(c.VIP, c.DNSMode, *i.ServiceSnapshot.Spec.IPFamilyPolicy == v1.IPFamilyPolicyRequireDualStack)
if err != nil {
return err
}
@@ -515,9 +786,11 @@ func (p *Processor) updateStatus(i *instance.Instance) error {
}
}
}
if !cmp.Equal(currentService.Status.LoadBalancer.Ingress, ingresses) {
log.Debug("LB status", "current", currentService.Status.LoadBalancer.Ingress, "new", ingresses)
if !ingressEqual(currentService.Status.LoadBalancer.Ingress, ingresses) {
currentService.Status.LoadBalancer.Ingress = ingresses
_, err = p.clientSet.CoreV1().Services(currentService.Namespace).UpdateStatus(context.TODO(), currentService, metav1.UpdateOptions{})
log.Debug("updating service status", "namespace", currentService.Namespace, "name", currentService.Name, "uid", currentService.UID)
_, err = p.clientSet.CoreV1().Services(currentService.Namespace).UpdateStatus(ctx, currentService, metav1.UpdateOptions{})
if err != nil && !apierrors.IsInvalid(err) {
log.Error("updating Service", "namespace", i.ServiceSnapshot.Namespace, "name", i.ServiceSnapshot.Name, "err", err)
return err
@@ -529,20 +802,38 @@ func (p *Processor) updateStatus(i *instance.Instance) error {
return err
}
func ingressEqual(a, b []v1.LoadBalancerIngress) bool {
if len(a) != len(b) {
return false
}
for i := range len(a) {
if a[i].IP != b[i].IP || a[i].Hostname != b[i].Hostname ||
!cmp.Equal(a[i].Ports, b[i].Ports) {
return false
}
}
return true
}
func isUPNPEnabled(s *v1.Service) bool {
return metav1.HasAnnotation(s.ObjectMeta, kubevip.UpnpEnabled) && s.Annotations[kubevip.UpnpEnabled] == "true"
}
// Refresh UPNP Port Forwards for all Service Instances registered in the processor
func (p *Processor) RefreshUPNPForwards() {
func (p *Processor) RefreshUPNPForwards(ctx context.Context) {
log.Info("Starting UPNP Port Refresher")
for {
time.Sleep(300 * time.Second)
// Skip logging if no service instances
if len(p.ServiceInstances) == 0 {
continue
}
log.Info("[UPNP] Refreshing Instances", "number of instances", len(p.ServiceInstances))
for i := range p.ServiceInstances {
p.upnpMap(context.TODO(), p.ServiceInstances[i])
if err := p.updateStatus(p.ServiceInstances[i]); err != nil {
p.upnpMap(ctx, p.ServiceInstances[i])
if err := p.updateStatus(ctx, p.ServiceInstances[i]); err != nil {
log.Warn("[UPNP] Error updating service", "ip", p.ServiceInstances[i].ServiceSnapshot.Name, "err", err)
}
}

View File

@@ -0,0 +1,68 @@
package services
import (
"testing"
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"github.com/kube-vip/kube-vip/pkg/instance"
)
// Test_upnpLeaseDurationForService tests whether the default lease duration is used, and whether the annotation
// overrides it correctly.
//
// For simplicity, this table driven test does not cover all edge cases (passing in nil instance, nil service snapshot,
// nil annotations, etc).
func Test_upnpLeaseDurationForService(t *testing.T) {
const annotation = "kube-vip.io/upnp-lease-duration" // Validating value of kubevip.UpnpLeaseDuration.
tcs := []struct {
name string
annotations map[string]string
want int // in seconds
}{
{
name: "No annotation uses default",
annotations: map[string]string{},
want: 3600,
},
{
name: "Valid annotation overrides default",
annotations: map[string]string{
annotation: "2h",
},
want: 7200,
},
{
name: "Valid short annotation overrides default",
annotations: map[string]string{
annotation: "30m",
},
want: 1800,
},
{
name: "Invalid annotation uses default",
annotations: map[string]string{
annotation: "invalid-duration",
},
want: 3600,
},
}
for _, tc := range tcs {
t.Run(tc.name, func(t *testing.T) {
i := &instance.Instance{
ServiceSnapshot: &v1.Service{
ObjectMeta: metav1.ObjectMeta{
Annotations: tc.annotations,
},
},
}
gotDuration := upnpLeaseDurationForService(i)
got := int(gotDuration.Seconds())
if got != tc.want {
t.Errorf("upnpLeaseDurationForService(%+v) = %v, want %v", tc.annotations, got, tc.want)
}
})
}
}

View File

@@ -0,0 +1,170 @@
package services
import (
"context"
"fmt"
log "log/slog"
"github.com/kube-vip/kube-vip/pkg/nftables"
"github.com/kube-vip/kube-vip/pkg/utils"
v1 "k8s.io/api/core/v1"
)
// addServiceWireguard configures a WireGuard tunnel for a service
// The tunnel is brought up here, but DNAT rules are configured by the endpoint watcher
// via wireguardWorker.processInstance() when endpoints become available
func (p *Processor) addServiceWireguard(_ context.Context, svc *v1.Service) error {
if !p.config.EnableWireguard {
return nil
}
// Get service VIPs
serviceIPs, err := utils.FetchServiceIPs(svc)
if err != nil {
return fmt.Errorf("failed to get service IPs for %s/%s: %w", svc.Namespace, svc.Name, err)
}
if len(serviceIPs) == 0 {
return fmt.Errorf("no service IPs found for service %s/%s", svc.Namespace, svc.Name)
}
// For each VIP, bring up the WireGuard tunnel
// DNAT rules will be configured by the endpoint watcher when endpoints are available
var successCount int
var lastErr error
for _, vip := range serviceIPs {
if err := p.setupServiceWireguardTunnel(svc, vip); err != nil {
log.Error("[wireguard] failed to setup tunnel for VIP",
"service", svc.Name,
"namespace", svc.Namespace,
"vip", vip,
"err", err)
lastErr = err
// Continue with other VIPs even if one fails
continue
}
successCount++
}
if successCount == 0 {
return fmt.Errorf("failed to setup WireGuard tunnel for any VIP in service %s/%s: %w", svc.Namespace, svc.Name, lastErr)
}
return nil
}
// setupServiceWireguardTunnel brings up the WireGuard tunnel for a single VIP
// DNAT rules are NOT configured here - they are handled by the endpoint watcher
func (p *Processor) setupServiceWireguardTunnel(svc *v1.Service, vip string) error {
// Check if we have a tunnel configuration for this VIP
if !p.TunnelMgr.HasConfigForVIP(vip) {
return fmt.Errorf("no WireGuard tunnel configuration found for VIP %s", vip)
}
// Get the tunnel configuration to determine the interface name
tunnelConfig := p.TunnelMgr.GetConfigForVIP(vip)
if tunnelConfig == nil {
return fmt.Errorf("failed to get tunnel configuration for VIP %s", vip)
}
// Bring up the WireGuard tunnel for this VIP
if err := p.TunnelMgr.BringUpTunnelForVIP(vip); err != nil {
return fmt.Errorf("failed to bring up WireGuard tunnel for VIP %s: %w", vip, err)
}
log.Info("[wireguard] brought up tunnel for service",
"namespace", svc.Namespace,
"name", svc.Name,
"vip", vip,
"interface", tunnelConfig.InterfaceName)
// DNAT rules will be configured by wireguardWorker.processInstance()
// when the endpoint watcher detects available endpoints
return nil
}
// deleteServiceWireguard removes nftables DNAT rules and tears down WireGuard tunnel for a service
func (p *Processor) deleteServiceWireguard(_ context.Context, svc *v1.Service) {
if !p.config.EnableWireguard {
return
}
serviceID := fmt.Sprintf("%s_%s", svc.Namespace, svc.Name)
serviceID = utils.SanitizeServiceID(serviceID)
log.Info("[wireguard] deleting DNAT rules and tunnel for service",
"namespace", svc.Namespace,
"name", svc.Name,
"serviceID", serviceID)
// Get service IPs
serviceIPs, _ := utils.FetchServiceIPs(svc)
// Delete DNAT chains for each port
for _, port := range svc.Spec.Ports {
if port.Protocol != v1.ProtocolTCP && port.Protocol != v1.ProtocolUDP {
continue
}
portServiceID := fmt.Sprintf("%s_p%d", serviceID, port.Port)
// Try to delete for both IPv4 and IPv6 if we have mixed IPs
hasIPv4 := false
hasIPv6 := false
for _, vip := range serviceIPs {
// Strip CIDR notation before checking IP version
addr := utils.StripCIDR(vip)
if utils.IsIPv6(addr) {
hasIPv6 = true
} else {
hasIPv4 = true
}
}
if hasIPv4 {
if err := nftables.DeleteIngressChains(false, portServiceID); err != nil {
log.Error("[wireguard] failed to delete IPv4 DNAT chains",
"service", svc.Name,
"port", port.Port,
"err", err)
} else {
log.Debug("[wireguard] deleted IPv4 DNAT chains",
"service", svc.Name,
"port", port.Port)
}
}
if hasIPv6 {
if err := nftables.DeleteIngressChains(true, portServiceID); err != nil {
log.Error("[wireguard] failed to delete IPv6 DNAT chains",
"service", svc.Name,
"port", port.Port,
"err", err)
} else {
log.Debug("[wireguard] deleted IPv6 DNAT chains",
"service", svc.Name,
"port", port.Port)
}
}
}
// Tear down the WireGuard tunnel for each VIP
for _, vip := range serviceIPs {
if err := p.TunnelMgr.TearDownTunnelForVIP(vip); err != nil {
log.Error("[wireguard] failed to tear down tunnel",
"service", svc.Name,
"vip", vip,
"err", err)
} else {
log.Info("[wireguard] tore down tunnel",
"service", svc.Name,
"vip", vip)
}
}
log.Info("[wireguard] DNAT rules deleted and tunnels torn down for service",
"namespace", svc.Namespace,
"name", svc.Name)
}

View File

@@ -1,8 +1,8 @@
package services
import (
"context"
"fmt"
"sync"
log "log/slog"
@@ -18,46 +18,26 @@ func (p *Processor) watchEndpoint(svcCtx *servicecontext.Context, id string, ser
log.Info("watching", "provider", provider.GetLabel(), "service_name", service.Name, "namespace", service.Namespace)
// Use a restartable watcher, as this should help in the event of etcd or timeout issues
leaderCtx, cancel := context.WithCancel(svcCtx.Ctx)
defer cancel()
var leaderElectionActive bool
rw, err := provider.CreateRetryWatcher(leaderCtx, p.rwClientSet, service)
rw, err := provider.CreateRetryWatcher(svcCtx.Ctx, p.rwClientSet, service)
if err != nil {
return fmt.Errorf("[%s] error watching endpoints: %w", provider.GetLabel(), err)
}
exitFunction := make(chan struct{})
go func() {
select {
case <-svcCtx.Ctx.Done():
log.Debug("context cancelled", "provider", provider.GetLabel())
// Stop the retry watcher
rw.Stop()
// Cancel the context, which will in turn cancel the leadership
cancel()
return
case <-p.shutdownChan:
log.Debug("shutdown called", "provider", provider.GetLabel())
// Stop the retry watcher
rw.Stop()
// Cancel the context, which will in turn cancel the leadership
cancel()
return
case <-exitFunction:
log.Debug("function ending", "provider", provider.GetLabel())
// Stop the retry watcher
rw.Stop()
// Cancel the context, which will in turn cancel the leadership
cancel()
return
}
wg := sync.WaitGroup{}
defer func() {
rw.Stop()
wg.Wait()
}()
wg.Go(func() {
<-svcCtx.Ctx.Done()
log.Debug("[endpoint watcher] service context cancelled", "provider", provider.GetLabel())
})
ch := rw.ResultChan()
epProcessor := endpoints.NewEndpointProcessor(p.config, provider, p.bgpServer, &p.ServiceInstances)
epProcessor := endpoints.NewEndpointProcessor(p.config, provider, p.bgpServer, &p.ServiceInstances, p.leaseMgr, p.TunnelMgr)
var lastKnownGoodEndpoint string
for event := range ch {
@@ -65,7 +45,7 @@ func (p *Processor) watchEndpoint(svcCtx *servicecontext.Context, id string, ser
switch event.Type {
case watch.Added, watch.Modified:
restart, err := epProcessor.AddOrModify(svcCtx, event, &lastKnownGoodEndpoint, service, id, &leaderElectionActive, p.StartServicesLeaderElection, &leaderCtx, &cancel)
restart, err := epProcessor.AddOrModify(svcCtx, event, &lastKnownGoodEndpoint, service, id, p.StartServicesLeaderElection, &wg, p.clientSet, p.updateEgressConfiguration)
if restart {
continue
} else if err != nil {
@@ -73,14 +53,11 @@ func (p *Processor) watchEndpoint(svcCtx *servicecontext.Context, id string, ser
}
case watch.Deleted:
if err := epProcessor.Delete(service, id); err != nil {
if err := epProcessor.Delete(svcCtx.Ctx, service, id); err != nil {
return fmt.Errorf("[%s] error while processing delete event: %w", provider.GetLabel(), err)
}
// Close the goroutine that will end the retry watcher, then exit the endpoint watcher function
close(exitFunction)
log.Info("stopping watching", "provider", provider.GetLabel(), "service name", service.Name, "namespace", service.Namespace)
log.Info("[endpoint watcher] stopping watching - endpoint object deleted", "provider", provider.GetLabel(), "service name", service.Name, "namespace", service.Namespace)
return nil
case watch.Error:
errObject := apierrors.FromObject(event.Object)
@@ -88,7 +65,6 @@ func (p *Processor) watchEndpoint(svcCtx *servicecontext.Context, id string, ser
log.Error("watch error", "provider", provider.GetLabel(), "err", statusErr)
}
}
close(exitFunction)
log.Info("stopping watching", "provider", provider.GetLabel(), "service name", service.Name, "namespace", service.Namespace)
log.Info("[endpoint watcher] stopping watching", "provider", provider.GetLabel(), "service name", service.Name, "namespace", service.Namespace)
return nil //nolint:govet
}

View File

@@ -2,24 +2,63 @@ package services
import (
"context"
"errors"
"fmt"
"sync"
"time"
log "log/slog"
"github.com/davecgh/go-spew/spew"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/servicecontext"
"github.com/kube-vip/kube-vip/pkg/trafficmirror"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/prometheus/client_golang/prometheus"
v1 "k8s.io/api/core/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/util/wait"
"k8s.io/apimachinery/pkg/watch"
"k8s.io/client-go/tools/cache"
watchtools "k8s.io/client-go/tools/watch"
)
// watchWithAuthRetry retries watchFn with exponential backoff on transient 403 Forbidden
// and 401 Unauthorized errors. On joining control plane nodes with K8s 1.34+, the local
// etcd may still be a learner when kube-vip starts, causing RBAC data to be unavailable.
// These auth errors resolve once etcd is promoted to a full member (typically within seconds).
// Non-auth errors are returned immediately. Context cancellation stops the retry loop.
func watchWithAuthRetry(ctx context.Context, watchFn func(context.Context) (watch.Interface, error)) (watch.Interface, error) {
var w watch.Interface
var lastErr error
err := wait.ExponentialBackoffWithContext(ctx, wait.Backoff{
Duration: 2 * time.Second,
Factor: 2.0,
Jitter: 0.1,
Steps: 10,
Cap: 30 * time.Second,
}, func(ctx context.Context) (bool, error) {
var watchErr error
w, watchErr = watchFn(ctx)
if watchErr == nil {
return true, nil
}
if !apierrors.IsForbidden(watchErr) && !apierrors.IsUnauthorized(watchErr) {
return false, watchErr
}
lastErr = watchErr
log.Warn("(svcs) services watch auth error, retrying", "err", watchErr)
return false, nil
})
if err != nil {
return nil, fmt.Errorf("(svcs) services watch failed after retries: %w (last: %v)", err, lastErr)
}
return w, nil
}
// This function handles the watching of a services endpoints and updates a load balancers endpoint configurations accordingly
func (p *Processor) ServicesWatcher(ctx context.Context, serviceFunc func(context.Context, *v1.Service) error) error {
func (p *Processor) ServicesWatcher(ctx context.Context, serviceFunc func(*servicecontext.Context, *v1.Service, *sync.WaitGroup) error) error {
// first start port mirroring if enabled
if err := p.startTrafficMirroringIfEnabled(); err != nil {
return err
@@ -43,27 +82,27 @@ func (p *Processor) ServicesWatcher(ctx context.Context, serviceFunc func(contex
// Use a restartable watcher, as this should help in the event of etcd or timeout issues
rw, err := watchtools.NewRetryWatcherWithContext(ctx, "1", &cache.ListWatch{
WatchFunc: func(_ metav1.ListOptions) (watch.Interface, error) {
return p.rwClientSet.CoreV1().Services(p.config.ServiceNamespace).Watch(ctx, metav1.ListOptions{})
return watchWithAuthRetry(ctx, func(ctx context.Context) (watch.Interface, error) {
return p.rwClientSet.CoreV1().Services(p.config.ServiceNamespace).Watch(ctx, metav1.ListOptions{})
})
},
})
if err != nil {
return fmt.Errorf("error creating services watcher: %s", err.Error())
}
exitFunction := make(chan struct{})
go func() {
select {
case <-p.shutdownChan:
log.Debug("(svcs) shutdown called")
// Stop the retry watcher
rw.Stop()
return
case <-exitFunction:
log.Debug("(svcs) function ending")
// Stop the retry watcher
rw.Stop()
return
}
}()
wg := sync.WaitGroup{}
defer wg.Wait()
watcherCtx, watcherCancel := context.WithCancel(ctx)
defer watcherCancel()
wg.Go(func() {
<-watcherCtx.Done()
log.Debug("(svcs) watcher context cancelled")
rw.Stop()
p.Stop()
})
ch := rw.ResultChan()
// Used for tracking an active endpoint / pod
@@ -73,20 +112,20 @@ func (p *Processor) ServicesWatcher(ctx context.Context, serviceFunc func(contex
// We need to inspect the event and get ResourceVersion out of it
switch event.Type {
case watch.Added, watch.Modified:
restart, err := p.AddOrModify(ctx, event, serviceFunc)
if restart {
break
}
if err != nil {
return fmt.Errorf("add/modify service error: %w", err)
if err := p.AddOrModify(watcherCtx, event, serviceFunc, &wg); err != nil {
if errors.Is(err, &utils.PanicError{}) {
return fmt.Errorf("add/modify service error: %w", err)
} else {
log.Error("service watcher event failed", "type", event.Type, "error", err)
}
}
case watch.Deleted:
restart, err := p.Delete(event)
if restart {
break
}
if err != nil {
return fmt.Errorf("delete service error: %w", err)
if err := p.Delete(event); err != nil {
if errors.Is(err, &utils.PanicError{}) {
return fmt.Errorf("delete service error: %w", err)
} else {
log.Error("service watcher event failed", "type", event.Type, "error", err)
}
}
case watch.Bookmark:
// Un-used
@@ -105,7 +144,7 @@ func (p *Processor) ServicesWatcher(ctx context.Context, serviceFunc func(contex
default:
}
}
close(exitFunction)
log.Warn("Stopping watching services for type: LoadBalancer in all namespaces")
return nil
}

View File

@@ -0,0 +1,98 @@
package services
import (
"context"
"fmt"
"testing"
"time"
apierrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/runtime/schema"
"k8s.io/apimachinery/pkg/watch"
)
// fakeWatchInterface is a minimal watch.Interface for testing.
type fakeWatchInterface struct {
ch chan watch.Event
}
func newFakeWatchInterface() *fakeWatchInterface {
return &fakeWatchInterface{ch: make(chan watch.Event)}
}
func (f *fakeWatchInterface) Stop() { close(f.ch) }
func (f *fakeWatchInterface) ResultChan() <-chan watch.Event { return f.ch }
func TestWatchWithAuthRetry(t *testing.T) {
svcResource := schema.GroupResource{Resource: "services"}
fw := newFakeWatchInterface()
tcs := []struct {
name string
watchFn func(int) (watch.Interface, error)
wantErr bool
wantAttempts int
}{
{
name: "403 Forbidden retried then succeeds",
watchFn: func(attempt int) (watch.Interface, error) {
if attempt <= 2 {
return nil, apierrors.NewForbidden(svcResource, "", nil)
}
return fw, nil
},
wantAttempts: 3,
},
{
name: "401 Unauthorized retried then succeeds",
watchFn: func(attempt int) (watch.Interface, error) {
if attempt <= 2 {
return nil, apierrors.NewUnauthorized("not authorized yet")
}
return fw, nil
},
wantAttempts: 3,
},
{
name: "non-auth error fails immediately",
watchFn: func(_ int) (watch.Interface, error) {
return nil, fmt.Errorf("connection refused")
},
wantErr: true,
wantAttempts: 1,
},
{
name: "immediate success no retry",
watchFn: func(_ int) (watch.Interface, error) {
return fw, nil
},
wantAttempts: 1,
},
}
for _, tc := range tcs {
t.Run(tc.name, func(t *testing.T) {
attempts := 0
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
w, err := watchWithAuthRetry(ctx, func(_ context.Context) (watch.Interface, error) {
attempts++
return tc.watchFn(attempts)
})
if tc.wantErr && err == nil {
t.Fatal("expected error, got nil")
}
if !tc.wantErr && err != nil {
t.Fatalf("expected success, got: %v", err)
}
if !tc.wantErr && w != fw {
t.Fatal("returned watcher does not match expected")
}
if attempts != tc.wantAttempts {
t.Errorf("expected %d attempts, got %d", tc.wantAttempts, attempts)
}
})
}
}

View File

@@ -2,6 +2,7 @@ package utils
import (
"fmt"
"log/slog"
"net"
"strings"
@@ -15,7 +16,7 @@ const (
)
// LookupHost resolves dnsName and return an IP or an error
func LookupHost(dnsName, dnsMode string) ([]string, error) {
func LookupHost(dnsName, dnsMode string, requireDualStack bool) ([]string, error) {
result, err := net.LookupHost(dnsName)
if err != nil {
return nil, err
@@ -26,7 +27,7 @@ func LookupHost(dnsName, dnsMode string) ([]string, error) {
addrs := []string{}
switch dnsMode {
case strings.ToLower(IPv4Family), strings.ToLower(IPv6Family), DualFamily:
a, err := getIPbyFamily(result, dnsMode)
a, err := getIPbyFamily(result, dnsMode, requireDualStack)
if err != nil {
return nil, err
}
@@ -38,7 +39,7 @@ func LookupHost(dnsName, dnsMode string) ([]string, error) {
return addrs, nil
}
func getIPbyFamily(addresses []string, family string) ([]string, error) {
func getIPbyFamily(addresses []string, family string, requireDualStack bool) ([]string, error) {
var checkers []func(string) bool
families := []string{}
if family == DualFamily || family == strings.ToLower(IPv4Family) {
@@ -54,11 +55,19 @@ func getIPbyFamily(addresses []string, family string) ([]string, error) {
for i, c := range checkers {
addr, err := getIPbyChecker(addresses, c)
if err != nil {
if len(checkers) > 1 && !requireDualStack {
slog.Warn("no address found", "family", families[i])
continue
}
return nil, fmt.Errorf("error getting %s address: %w", families[i], err)
}
addrs = append(addrs, addr)
}
if len(addrs) == 0 {
return nil, fmt.Errorf("no addresses found")
}
return addrs, nil
}

View File

@@ -3,6 +3,9 @@ package utils
import (
"fmt"
"net"
"strings"
v1 "k8s.io/api/core/v1"
)
// FormatIPWithSubnetMask takes a raw IP address and a subnet mask, and returns a formatted string in CIDR notation.
@@ -56,3 +59,74 @@ func IsIPv6CIDR(cidr string) bool {
}
return ip.To4() == nil
}
// StripCIDR removes the CIDR notation (e.g., "/24") from an IP address string.
// If no CIDR notation is present, the original string is returned unchanged.
func StripCIDR(ip string) string {
if idx := strings.Index(ip, "/"); idx >= 0 {
return ip[:idx]
}
return ip
}
// SanitizeServiceID sanitizes a service ID to be valid for nftables chain names.
// Only alphanumeric characters and underscores are allowed; other characters are replaced with underscores.
// The result is truncated to 50 characters to respect nftables name length limits.
func SanitizeServiceID(id string) string {
var result strings.Builder
for _, r := range id {
if (r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') || (r >= '0' && r <= '9') || r == '_' {
result.WriteRune(r)
} else {
result.WriteRune('_')
}
}
sanitized := result.String()
// Ensure it doesn't exceed nftables name length limit
if len(sanitized) > 50 {
sanitized = sanitized[:50]
}
return sanitized
}
// FetchServiceIPs extracts IP addresses from a Kubernetes Service.
// It checks the following sources in order:
// 1. kube-vip.io/loadbalancerIPs annotation (comma-separated list)
// 2. spec.LoadBalancerIP (deprecated but still used)
// 3. status.loadBalancer.ingress
// Returns an error if no IPs are found.
func FetchServiceIPs(service *v1.Service) ([]string, error) {
var ips []string
// Check for loadBalancerIPs annotation first (new style)
if loadBalancerIPs, ok := service.Annotations["kube-vip.io/loadbalancerIPs"]; ok {
for _, ip := range strings.Split(loadBalancerIPs, ",") {
ip = strings.TrimSpace(ip)
if ip != "" {
ips = append(ips, ip)
}
}
}
// Fallback to spec.LoadBalancerIP (deprecated but still used)
if len(ips) == 0 && service.Spec.LoadBalancerIP != "" {
ips = append(ips, service.Spec.LoadBalancerIP)
}
// Check status.loadBalancer.ingress as well
if len(ips) == 0 {
for _, ingress := range service.Status.LoadBalancer.Ingress {
if ingress.IP != "" {
ips = append(ips, ingress.IP)
}
}
}
if len(ips) == 0 {
return nil, fmt.Errorf("no IPs found for service")
}
return ips, nil
}

15
pkg/utils/panic.go Normal file
View File

@@ -0,0 +1,15 @@
package utils
import "fmt"
type PanicError struct {
cause string
}
func (e *PanicError) Error() string {
return fmt.Sprintf("%s - unrecoverable error", e.cause)
}
func NewPanicError(cause string) error {
return &PanicError{cause: cause}
}

View File

@@ -17,7 +17,7 @@ import (
"golang.org/x/sys/unix"
v1 "k8s.io/api/core/v1"
"github.com/kube-vip/kube-vip/pkg/iptables"
iptables "github.com/kube-vip/kube-vip/pkg/iptables"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/utils"
@@ -28,18 +28,21 @@ const (
defaultValidLft = 60
iptablesComment = "%s kube-vip load balancer IP"
iptablesCommentMarkRule = "kube-vip load balancer IP set mark for masquerade"
defaultMaskIPv6 = 128
defaultMaskIPv4 = 32
DefaultMaskIPv4 = 32
DefaultMaskIPv6 = 128
NoLifetime = 0
)
// Network is an interface that enable managing operations for a given IP
type Network interface {
AddIP(precheck bool, skipDAD bool) (bool, error)
AddIP(precheck bool, skipDAD bool, minLifetime ...int) (bool, error)
AddRoute(precheck bool) error
DeleteIP() (bool, error)
DeleteRoute() error
UpdateRoutes() (bool, error)
IsSet() (bool, error)
IsSet() (*netlink.Addr, error)
IP() string
CIDR() string
IPisLinkLocal() bool
@@ -56,6 +59,8 @@ type Network interface {
SetHasEndpoints(value bool)
HasEndpoints() bool
ARPName() string
GetPossibleSubnets() string
DHCPFamily() string
}
// network - This allows network configuration
@@ -82,12 +87,18 @@ type network struct {
ipvsEnabled bool
hasEndpoints bool
possibleSubnets string
// used by DHCP to get address of proper family
dhcpFamily string
}
// NewConfig will attempt to provide an interface to the kernel network configuration
func NewConfig(address string, iface string, loGlobalScope bool, subnet string, isDDNS bool, tableID int, tableType int,
routingProtocol int, dnsMode, forwardMethod, iptablesBackend string,
ipvsEnabled, enableSecurity bool, intfMgr *networkinterface.Manager) ([]Network, error) {
func NewConfig(address string, iface string, loGlobalScope bool, subnet string, isDDNS bool,
dhcpMode string, requireDualStack, isDualStack bool, tableID int, tableType int, routingProtocol int,
dnsMode, forwardMethod, iptablesBackend string, ipvsEnabled, enableSecurity bool,
intfMgr *networkinterface.Manager) ([]Network, error) {
networks := []Network{}
link, err := netlink.LinkByName(iface)
@@ -106,6 +117,8 @@ func NewConfig(address string, iface string, loGlobalScope bool, subnet string,
forwardMethod: forwardMethod,
iptablesBackend: iptablesBackend,
ipvsEnabled: ipvsEnabled,
enableSecurity: enableSecurity,
possibleSubnets: subnet,
}
subnet, err = SelectSubnet(address, subnet)
@@ -138,35 +151,50 @@ func NewConfig(address string, iface string, loGlobalScope bool, subnet string,
} else {
// try to resolve the address
log.Debug("looking up host", "address", address, "dnsMode", dnsMode)
ips, err := utils.LookupHost(address, dnsMode)
if err != nil {
ips, err := utils.LookupHost(address, dnsMode, requireDualStack)
if (dnsMode == utils.DualFamily && isDDNS && isDualStack) || err != nil {
// return early for ddns if no IP is allocated for the domain
// when leader starts, should do get IP from DHCP for the domain
if isDDNS {
result := &network{
link: networkLink,
routeTable: tableID,
routingTableType: tableType,
routingProtocol: routingProtocol,
forwardMethod: forwardMethod,
iptablesBackend: iptablesBackend,
isDDNS: isDDNS,
dnsName: address,
ipvsEnabled: ipvsEnabled,
enableSecurity: enableSecurity,
address: &netlink.Addr{ // create placeholder for the address
IPNet: &net.IPNet{}, // that will be added later in the process
Peer: &net.IPNet{},
},
log.Info("isDDNS true", "dhcpMode", dhcpMode)
if strings.EqualFold(dhcpMode, utils.IPv4Family) || strings.EqualFold(dhcpMode, utils.DualFamily) {
result := &network{
link: networkLink,
routeTable: tableID,
routingTableType: tableType,
routingProtocol: routingProtocol,
forwardMethod: forwardMethod,
iptablesBackend: iptablesBackend,
isDDNS: isDDNS,
dnsName: address,
ipvsEnabled: ipvsEnabled,
enableSecurity: enableSecurity,
possibleSubnets: subnet,
dhcpFamily: utils.IPv4Family,
}
networks = append(networks, result)
}
// set address as deprecated so it isn't used as source address according to RFC 3484
result.address.PreferedLft = 0
if strings.EqualFold(dhcpMode, utils.IPv6Family) || strings.EqualFold(dhcpMode, utils.DualFamily) {
result := &network{
link: networkLink,
routeTable: tableID,
routingTableType: tableType,
routingProtocol: routingProtocol,
forwardMethod: forwardMethod,
iptablesBackend: iptablesBackend,
isDDNS: isDDNS,
dnsName: address,
ipvsEnabled: ipvsEnabled,
enableSecurity: enableSecurity,
possibleSubnets: subnet,
dhcpFamily: utils.IPv6Family,
}
// Also set ValidLft so the netlink library actually sets them
result.address.ValidLft = math.MaxInt
networks = append(networks, result)
}
networks = append(networks, result)
return networks, nil
}
return nil, err
@@ -184,15 +212,12 @@ func NewConfig(address string, iface string, loGlobalScope bool, subnet string,
dnsName: address,
ipvsEnabled: ipvsEnabled,
enableSecurity: enableSecurity,
possibleSubnets: subnet,
}
// we're able to resolve store this as the initial IP
subnets := Split(subnet)
s := subnet
if len(subnets) > 1 {
s = selectSubnet(ip, subnets)
s, err := SelectSubnet(ip, subnet)
if err != nil {
return nil, fmt.Errorf("failed to select subnet: %w", err)
}
if result.address, err = netlink.ParseAddr(fmt.Sprintf("%s/%s", ip, s)); err != nil {
@@ -204,6 +229,11 @@ func NewConfig(address string, iface string, loGlobalScope bool, subnet string,
// set address as deprecated so it isn't used as source address according to RFC 3484
result.address.PreferedLft = 0
result.dhcpFamily = strings.ToLower(utils.IPv6Family)
if net.ParseIP(ip).To4() != nil {
result.dhcpFamily = strings.ToLower(utils.IPv4Family)
}
networks = append(networks, result)
}
@@ -331,18 +361,24 @@ func (configurator *network) UpdateRoutes() (bool, error) {
// AddIP - Add an IP address to the interface
// precheck: if true, check if the IP already exists before adding
// skipDAD: if true, set IFA_F_NODAD flag for IPv6 addresses to skip Duplicate Address Detection
func (configurator *network) AddIP(precheck bool, skipDAD bool) (bool, error) {
func (configurator *network) AddIP(precheck bool, skipDAD bool, minLifetime ...int) (bool, error) {
configurator.link.Lock.Lock()
defer configurator.link.Lock.Unlock()
exists := false
var existing *netlink.Addr
var err error
if precheck {
if exists, err = configurator.IsSet(); err != nil {
if existing, err = configurator.IsSet(); err != nil {
return false, errors.Wrap(err, "could not check if address exists")
}
}
if exists {
lifetime := NoLifetime
if len(minLifetime) > 0 {
lifetime = minLifetime[0]
}
if existing != nil && existing.ValidLft > lifetime {
return false, nil
}
@@ -355,8 +391,9 @@ func (configurator *network) AddIP(precheck bool, skipDAD bool) (bool, error) {
log.Debug("Setting IFA_F_NODAD flag for IPv6 address to skip DAD", "ip", configurator.address.IP.String())
}
log.Debug("replacing IP", "address", configurator.address)
if err := netlink.AddrReplace(configurator.link.Intf, configurator.address); err != nil {
return false, errors.Wrap(err, "could not add ip")
return false, errors.Wrap(err, fmt.Sprintf("could not add ip to device %q", configurator.link.Intf.Attrs().Name))
}
if err := configurator.configureIPTables(); err != nil {
@@ -367,13 +404,13 @@ func (configurator *network) AddIP(precheck bool, skipDAD bool) (bool, error) {
}
func (configurator *network) configureIPTables() error {
if configurator.enableSecurity && !configurator.ignoreSecurity {
if configurator.enableSecurity && !configurator.ignoreSecurity && len(configurator.ports) > 0 {
if err := configurator.addIptablesRulesToLimitTrafficPorts(); err != nil {
return errors.Wrap(err, "could not add iptables rules to limit traffic ports")
}
}
// It seems that masquerading is only reuired with IPv4 for IPVS to work.
// It seems that masquerading is only required with IPv4 for IPVS to work.
if configurator.ipvsEnabled && configurator.forwardMethod == "masquerade" && configurator.address.IP.To4() != nil {
if err := configurator.addIptablesRulesForMasquerade(); err != nil {
return errors.Wrap(err, "could not add iptables rules for masquerade")
@@ -384,12 +421,17 @@ func (configurator *network) configureIPTables() error {
}
func (configurator *network) addIptablesRulesToLimitTrafficPorts() error {
ipt, err := iptables.New()
vip := configurator.address.IP.String()
opt := iptables.IPFamily(iptables.ProtocolIPv4)
if utils.IsIPv6(vip) {
opt = iptables.IPFamily(iptables.ProtocolIPv6)
}
ipt, err := iptables.New(opt)
if err != nil {
return errors.Wrap(err, "could not create iptables client")
}
vip := configurator.address.IP.String()
comment := fmt.Sprintf(iptablesComment, configurator.serviceName)
if err := insertCommonIPTablesRules(ipt, vip, comment); err != nil {
return fmt.Errorf("could not add common iptables rules: %w", err)
@@ -522,7 +564,7 @@ func (configurator *network) DeleteIP() (bool, error) {
}
// Nothing to delete
if !result {
if result == nil {
return false, nil
}
@@ -637,31 +679,31 @@ func addressHasDADFAILEDFlag(address netlink.Addr) bool {
}
// isSet - Check to see if VIP is set
func (configurator *network) IsSet() (result bool, err error) {
func (configurator *network) IsSet() (result *netlink.Addr, err error) {
var addresses []netlink.Addr
if configurator.address == nil {
return false, nil
return nil, nil
}
if configurator.address.Mask == nil {
return false, nil
return nil, nil
}
addresses, err = netlink.AddrList(configurator.link.Intf, 0)
if err != nil {
err = errors.Wrap(err, "could not list addresses")
return false, err
return nil, err
}
for _, address := range addresses {
if address.Equal(*configurator.address) {
return true, nil
return &address, nil
}
}
return false, nil
return nil, nil
}
// SetIP updates the IP that is used
@@ -675,6 +717,25 @@ func (configurator *network) SetIP(ip string) error {
if strings.Contains("/", ip) {
return fmt.Errorf("ip should not contain CIDR notation got: %s", ip)
}
if configurator.address == nil {
log.Debug("possible", "subnets", configurator.possibleSubnets)
subnet, err := SelectSubnet(ip, configurator.possibleSubnets)
if err != nil {
return fmt.Errorf("unable to select subnet for IP %q from %q: %w", ip, subnet, err)
}
// Check if the subnet needs overriding
cidr, err := utils.FormatIPWithSubnetMask(ip, subnet)
if err != nil {
return errors.Wrapf(err, "2 could not format address %q with subnetMask %q", ip, subnet)
}
configurator.address, err = netlink.ParseAddr(cidr)
if err != nil {
return errors.Wrapf(err, "could not parse address %q", cidr)
}
}
ones, _ := configurator.address.Mask.Size()
cidr, err := utils.FormatIPWithSubnetMask(ip, strconv.Itoa(ones))
if err != nil {
@@ -804,11 +865,17 @@ func (configurator *network) SetMask(mask string) error {
selectedMask := mask
var err error
if mask == "" {
return fmt.Errorf("no mask provided")
}
if configurator.IP() != "" {
selectedMask, err = SelectSubnet(configurator.IP(), mask)
if err != nil {
return fmt.Errorf("failed to select mask %q: %w", mask, err)
}
} else if len(strings.Split(mask, ",")) > 1 {
return fmt.Errorf("cannot select mask from %q when IP address is unknown", mask)
}
m, err := strconv.Atoi(selectedMask)
@@ -816,19 +883,18 @@ func (configurator *network) SetMask(mask string) error {
return err
}
size := defaultMaskIPv4
size := DefaultMaskIPv4
family := utils.IPv4Family
if configurator.IP() != "" {
if utils.IsIPv6(configurator.IP()) {
size = defaultMaskIPv6
size = DefaultMaskIPv6
family = utils.IPv6Family
}
if m > size {
return fmt.Errorf("provided CIDR mask '%d' is greater than the highest mask value for the %s family (%d)", m, family, size)
}
}
toSet := net.CIDRMask(m, size)
@@ -857,6 +923,14 @@ func (configurator *network) ARPName() string {
return fmt.Sprintf("%s-%s", configurator.CIDR(), configurator.Interface())
}
func (configurator *network) GetPossibleSubnets() string {
return configurator.possibleSubnets
}
func (configurator *network) DHCPFamily() string {
return configurator.dhcpFamily
}
// SelectSubnet formats an IP address with the appropriate CIDR based on the input.
// The input SubnetMasks can be "32,128" (dual-stack), "32", "128" (SingleStack).
func SelectSubnet(rawIP string, subnetMasks string) (string, error) {

View File

@@ -2,7 +2,8 @@ package vip
import (
"context"
"net"
"fmt"
"sync"
"time"
log "log/slog"
@@ -14,42 +15,43 @@ import (
// for the dDNSHostName
// will return the IP allocated
type DDNSManager interface {
Start() (string, error)
Start(ctx context.Context, wg *sync.WaitGroup) (string, error)
}
type ddnsManager struct {
ctx context.Context
network Network
network Network
backoffAttempts uint
}
// NewDDNSManager returns a newly created Dynamic DNS manager
func NewDDNSManager(ctx context.Context, network Network) DDNSManager {
func NewDDNSManager(network Network, backoffAttempts uint) DDNSManager {
return &ddnsManager{
ctx: ctx,
network: network,
network: network,
backoffAttempts: backoffAttempts,
}
}
// Start will start the dhcpclient routine to keep the lease
// and return the IP it got from DHCP
func (ddns *ddnsManager) Start() (string, error) {
interfaceName := ddns.network.Interface()
iface, err := net.InterfaceByName(interfaceName)
func (ddns *ddnsManager) Start(ctx context.Context, wg *sync.WaitGroup) (string, error) {
client, err := NewDHCPClient(ddns.network, ddns.backoffAttempts)
if err != nil {
return "", err
return "", fmt.Errorf("unable to create DHCP client: %w", err)
}
client := NewDHCPClient(iface, false, "")
client.WithHostName(ddns.network.DDNSHostName())
go client.Start()
wg.Go(func() {
if err := client.Start(ctx); err != nil {
log.Error("[ddns] DHCP client error: %w")
}
})
log.Info("waiting for ip from dhcp")
ip, timeout := "", time.After(1*time.Minute)
select {
case <-ddns.ctx.Done():
case <-ctx.Done():
client.Stop()
return "", errors.New("context cancelled")
case <-timeout:
@@ -67,7 +69,7 @@ func (ddns *ddnsManager) Start() (string, error) {
// start a go routine to stop dhclient when lose leader election
// also to keep read the ip from channel
// so onbound function is unblocked to send the ip
go func(ctx context.Context) {
wg.Go(func() {
for {
select {
case <-ctx.Done():
@@ -78,7 +80,7 @@ func (ddns *ddnsManager) Start() (string, error) {
log.Info("got address from dhcp", "ip", ip)
}
}
}(ddns.ctx)
})
return ip, nil
}

View File

@@ -1,289 +1,39 @@
package vip
// DHCP client implementation that refers to https://www.rfc-editor.org/rfc/rfc2131.html
import (
"context"
"fmt"
"net"
"time"
"strings"
log "log/slog"
"github.com/insomniacslk/dhcp/dhcpv4"
"github.com/insomniacslk/dhcp/dhcpv4/nclient4"
"github.com/jpillora/backoff"
"github.com/kube-vip/kube-vip/pkg/utils"
)
const dhcpClientPort = "68"
const defaultDHCPRenew = time.Hour
const maxBackoffAttempts = 3
// DHCPClient is responsible for maintaining ipv4 lease for one specified interface
type DHCPClient struct {
iface *net.Interface
ddnsHostName string
lease *nclient4.Lease
initRebootFlag bool
requestedIP net.IP
stopChan chan struct{} // used as a signal to release the IP and stop the dhcp client daemon
releasedChan chan struct{} // indicate that the IP has been released
errorChan chan error // indicates there was an error on the IP request
ipChan chan string
type DHCPClient interface {
ErrorChannel() chan error
IPChannel() chan string
Start(ctx context.Context) error
Stop()
WithHostName(hostname string) DHCPClient
}
// NewDHCPClient returns a new DHCP Client.
func NewDHCPClient(iface *net.Interface, initRebootFlag bool, requestedIP string) *DHCPClient {
return &DHCPClient{
iface: iface,
stopChan: make(chan struct{}),
releasedChan: make(chan struct{}),
errorChan: make(chan error),
initRebootFlag: initRebootFlag,
requestedIP: net.ParseIP(requestedIP),
ipChan: make(chan string),
}
}
func (c *DHCPClient) WithHostName(hostname string) *DHCPClient {
c.ddnsHostName = hostname
return c
}
// Stop state-transition process and close dhcp client
func (c *DHCPClient) Stop() {
close(c.ipChan)
close(c.stopChan)
<-c.releasedChan
}
// Gets the IPChannel for consumption
func (c *DHCPClient) IPChannel() chan string {
return c.ipChan
}
// Gets the ErrorChannel for consumption
func (c *DHCPClient) ErrorChannel() chan error {
return c.errorChan
}
// Start state-transition process of dhcp client
//
// -------- -------
//
// | | +-------------------------->| |<-------------------+
// | INIT- | | +-------------------->| INIT | |
// | REBOOT |DHCPNAK/ +---------->| |<---+ |
// | |Restart| | ------- | |
//
// -------- | DHCPNAK/ | | |
// | Discard offer | -/Send DHCPDISCOVER |
//
// -/Send DHCPREQUEST | | |
//
// | | | DHCPACK v | |
// ----------- | (not accept.)/ ----------- | |
//
// | | | Send DHCPDECLINE | | |
// | REBOOTING | | | | SELECTING |<----+ |
// | | | / | | |DHCPOFFER/ |
//
// ----------- | / ----------- | |Collect |
// | | / | | | replies |
//
// DHCPACK/ | / +----------------+ +-------+ |
// Record lease, set| | v Select offer/ |
// timers T1, T2 ------------ send DHCPREQUEST | |
//
// | +----->| | DHCPNAK, Lease expired/ |
// | | | REQUESTING | Halt network |
// DHCPOFFER/ | | | |
// Discard ------------ | |
// | | | | ----------- |
// | +--------+ DHCPACK/ | | |
// | Record lease, set -----| REBINDING | |
// | timers T1, T2 / | | |
// | | DHCPACK/ ----------- |
// | v Record lease, set ^ |
// +----------------> ------- /timers T1,T2 | |
// +----->| |<---+ | |
// | | BOUND |<---+ | |
// DHCPOFFER, DHCPACK, | | | T2 expires/ DHCPNAK/
// DHCPNAK/Discard ------- | Broadcast Halt network
// | | | | DHCPREQUEST |
// +-------+ | DHCPACK/ | |
// T1 expires/ Record lease, set | |
// Send DHCPREQUEST timers T1, T2 | |
// to leasing server | | |
// | ---------- | |
// | | |------------+ |
// +->| RENEWING | |
// | |----------------------------+
// ----------
// Figure: State-transition diagram for DHCP clients
func (c *DHCPClient) Start() {
lease := c.requestWithBackoff()
c.initRebootFlag = false
c.lease = lease
// Set up two ticker to renew/rebind regularly
t1Timeout := c.lease.ACK.IPAddressLeaseTime(defaultDHCPRenew) / 2
t2Timeout := (c.lease.ACK.IPAddressLeaseTime(defaultDHCPRenew) / 8) * 7
log.Debug("dhcp timeouts", "timeout1", t1Timeout, "timeoute2", t2Timeout)
t1, t2 := time.NewTicker(t1Timeout), time.NewTicker(t2Timeout)
for {
select {
case <-t1.C:
// renew is a unicast request of the IP renewal
// A point on renew is: the library does not return the right message (NAK)
// on renew error due to IP Change, but instead it returns a different error
// This way there's not much to do other than log and continue, as the renew error
// may be an offline server, or may be an incorrect package match
lease, err := c.renew()
if err == nil {
c.lease = lease
log.Info("renew", "lease", lease)
t2.Reset(t2Timeout)
} else {
log.Error("renew failed", "err", err)
}
case <-t2.C:
// rebind is just like a request, but forcing to provide a new IP address
lease, err := c.request(true)
if err == nil {
c.lease = lease
log.Info("rebind", "lease", lease)
} else {
if _, ok := err.(*nclient4.ErrNak); !ok {
t1.Stop()
t2.Stop()
log.Error("rebind failed", "err", err)
return
}
log.Warn("ip may have changed", "ip", c.lease.ACK.YourIPAddr, "err", err)
c.initRebootFlag = false
c.lease = c.requestWithBackoff()
}
t1.Reset(t1Timeout)
t2.Reset(t2Timeout)
case <-c.stopChan:
// release is a unicast request of the IP release.
if err := c.release(); err != nil {
log.Error("release lease failed", "lease", lease, "err", err)
} else {
log.Info("release", "lease", lease)
}
t1.Stop()
t2.Stop()
close(c.releasedChan)
return
}
}
}
// --------------------------------------------------------
// | |INIT-REBOOT | RENEWING |REBINDING |
// --------------------------------------------------------
// |broad/unicast |broadcast | unicast |broadcast |
// |server-ip |MUST NOT | MUST NOT |MUST NOT |
// |requested-ip |MUST | MUST NOT |MUST NOT |
// |ciaddr |zero | IP address |IP address|
// --------------------------------------------------------
func (c *DHCPClient) requestWithBackoff() *nclient4.Lease {
backoff := backoff.Backoff{
Factor: 2,
Jitter: true,
Min: 10 * time.Second,
Max: 1 * time.Minute,
func NewDHCPClient(network Network, backoffAttempts uint) (DHCPClient, error) {
interfaceName := network.Interface()
iface, err := net.InterfaceByName(interfaceName)
if err != nil {
return nil, fmt.Errorf("failed to get interface %q: %w", interfaceName, err)
}
var lease *nclient4.Lease
var err error
var client DHCPClient
for {
log.Debug("trying to get a new IP", "attempt", backoff.Attempt())
lease, err = c.request(false)
if strings.EqualFold(network.DHCPFamily(), utils.IPv6Family) {
client, err = NewDHCPv6Client(iface, nil, false, "", backoffAttempts)
if err != nil {
dur := backoff.Duration()
if backoff.Attempt() > maxBackoffAttempts-1 {
errMsg := fmt.Errorf("failed to get an IP address after %d attempts, error %s, giving up", maxBackoffAttempts, err.Error())
log.Error(errMsg.Error())
c.errorChan <- errMsg
c.Stop()
return nil
}
log.Error("request failed", "err", err.Error(), "waiting", dur)
time.Sleep(dur)
continue
return nil, fmt.Errorf("failed to create DHCP client: %w", err)
}
backoff.Reset()
break
} else {
client = NewDHCPv4Client(iface, false, "", backoffAttempts)
}
if c.ipChan != nil {
log.Debug("using channel")
c.ipChan <- lease.ACK.YourIPAddr.String()
}
return lease
}
func (c *DHCPClient) request(rebind bool) (*nclient4.Lease, error) {
dhclient, err := nclient4.New(c.iface.Name)
if err != nil {
return nil, fmt.Errorf("create a client for iface %s failed, error: %w", c.iface.Name, err)
}
defer dhclient.Close()
modifiers := make([]dhcpv4.Modifier, 0)
if c.ddnsHostName != "" {
modifiers = append(modifiers,
dhcpv4.WithOption(dhcpv4.OptHostName(c.ddnsHostName)),
dhcpv4.WithOption(dhcpv4.OptClientIdentifier([]byte(c.ddnsHostName))),
)
}
// if initRebootFlag is set, this means we have an IP already set on c.requestedIP that should be used
if c.initRebootFlag {
log.Debug("init-reboot", "ip", c.requestedIP)
modifiers = append(modifiers, dhcpv4.WithOption(dhcpv4.OptRequestedIPAddress(c.requestedIP)))
}
// if this is a rebind, then the IP we should set is the one that already exists in lease
if rebind {
log.Debug("rebinding", "ip", c.lease.ACK.YourIPAddr)
modifiers = append(modifiers, dhcpv4.WithOption(dhcpv4.OptRequestedIPAddress(c.lease.ACK.YourIPAddr)))
}
return dhclient.Request(context.TODO(), modifiers...)
}
func (c *DHCPClient) release() error {
dhclient, err := nclient4.New(c.iface.Name)
if err != nil {
return fmt.Errorf("create release client failed, error: %w, iface: %s, server ip: %v", err, c.iface.Name, c.lease.ACK.ServerIPAddr)
}
defer dhclient.Close()
// TODO modify lease
return dhclient.Release(c.lease)
}
func (c *DHCPClient) renew() (*nclient4.Lease, error) {
// renew needs a unicast client. This is due to some servers (like dnsmasq) require the exact request coming from the vip interface
dhclient, err := nclient4.New(c.iface.Name,
nclient4.WithUnicast(&net.UDPAddr{IP: c.lease.ACK.YourIPAddr, Port: nclient4.ClientPort}))
if err != nil {
return nil, fmt.Errorf("create renew client failed, error: %w, server ip: %v", err, c.lease.ACK.ServerIPAddr)
}
defer dhclient.Close()
return dhclient.Renew(context.TODO(), c.lease)
return client, nil
}

299
pkg/vip/dhcpv4.go Normal file
View File

@@ -0,0 +1,299 @@
package vip
// DHCP client implementation that refers to https://www.rfc-editor.org/rfc/rfc2131.html
import (
"context"
"fmt"
"net"
"sync"
"time"
log "log/slog"
"github.com/insomniacslk/dhcp/dhcpv4"
"github.com/insomniacslk/dhcp/dhcpv4/nclient4"
"github.com/jpillora/backoff"
)
const dhcpClientPort = "68"
const defaultDHCPRenew = time.Hour
// DHCPv4Client is responsible for maintaining ipv4 lease for one specified interface
type DHCPv4Client struct {
iface *net.Interface
ddnsHostName string
lease *nclient4.Lease
initRebootFlag bool
requestedIP net.IP
stopChan chan struct{} // used as a signal to release the IP and stop the dhcp client daemon
releasedChan chan struct{} // indicate that the IP has been released
errorChan chan error // indicates there was an error on the IP request
ipChan chan string
backoffAttempts uint
stop sync.Once
}
// NewDHCPv4Client returns a new DHCP Client.
func NewDHCPv4Client(iface *net.Interface, initRebootFlag bool, requestedIP string, backoffAttempts uint) *DHCPv4Client {
return &DHCPv4Client{
iface: iface,
stopChan: make(chan struct{}),
releasedChan: make(chan struct{}),
errorChan: make(chan error),
initRebootFlag: initRebootFlag,
requestedIP: net.ParseIP(requestedIP),
ipChan: make(chan string),
backoffAttempts: backoffAttempts,
}
}
func (c *DHCPv4Client) WithHostName(hostname string) DHCPClient {
c.ddnsHostName = hostname
return c
}
// Stop state-transition process and close dhcp client
func (c *DHCPv4Client) Stop() {
c.stop.Do(func() {
close(c.ipChan)
close(c.stopChan)
})
<-c.releasedChan
}
// Gets the IPChannel for consumption
func (c *DHCPv4Client) IPChannel() chan string {
return c.ipChan
}
// Gets the ErrorChannel for consumption
func (c *DHCPv4Client) ErrorChannel() chan error {
return c.errorChan
}
// Start state-transition process of dhcp client
//
// -------- -------
//
// | | +-------------------------->| |<-------------------+
// | INIT- | | +-------------------->| INIT | |
// | REBOOT |DHCPNAK/ +---------->| |<---+ |
// | |Restart| | ------- | |
//
// -------- | DHCPNAK/ | | |
// | Discard offer | -/Send DHCPDISCOVER |
//
// -/Send DHCPREQUEST | | |
//
// | | | DHCPACK v | |
// ----------- | (not accept.)/ ----------- | |
//
// | | | Send DHCPDECLINE | | |
// | REBOOTING | | | | SELECTING |<----+ |
// | | | / | | |DHCPOFFER/ |
//
// ----------- | / ----------- | |Collect |
// | | / | | | replies |
//
// DHCPACK/ | / +----------------+ +-------+ |
// Record lease, set| | v Select offer/ |
// timers T1, T2 ------------ send DHCPREQUEST | |
//
// | +----->| | DHCPNAK, Lease expired/ |
// | | | REQUESTING | Halt network |
// DHCPOFFER/ | | | |
// Discard ------------ | |
// | | | | ----------- |
// | +--------+ DHCPACK/ | | |
// | Record lease, set -----| REBINDING | |
// | timers T1, T2 / | | |
// | | DHCPACK/ ----------- |
// | v Record lease, set ^ |
// +----------------> ------- /timers T1,T2 | |
// +----->| |<---+ | |
// | | BOUND |<---+ | |
// DHCPOFFER, DHCPACK, | | | T2 expires/ DHCPNAK/
// DHCPNAK/Discard ------- | Broadcast Halt network
// | | | | DHCPREQUEST |
// +-------+ | DHCPACK/ | |
// T1 expires/ Record lease, set | |
// Send DHCPREQUEST timers T1, T2 | |
// to leasing server | | |
// | ---------- | |
// | | |------------+ |
// +->| RENEWING | |
// | |----------------------------+
// ----------
// Figure: State-transition diagram for DHCP clients
func (c *DHCPv4Client) Start(ctx context.Context) error {
dhcpCtx, cancel := context.WithCancel(ctx)
defer cancel()
lease := c.requestWithBackoff(dhcpCtx)
c.initRebootFlag = false
c.lease = lease
// Set up two ticker to renew/rebind regularly
t1Timeout := c.lease.ACK.IPAddressLeaseTime(defaultDHCPRenew) / 2
t2Timeout := (c.lease.ACK.IPAddressLeaseTime(defaultDHCPRenew) / 8) * 7
log.Debug("[DHCPv4] timeouts", "timeout1", t1Timeout, "timeout2", t2Timeout)
t1, t2 := time.NewTicker(t1Timeout), time.NewTicker(t2Timeout)
for {
select {
case <-t1.C:
// renew is a unicast request of the IP renewal
// A point on renew is: the library does not return the right message (NAK)
// on renew error due to IP Change, but instead it returns a different error
// This way there's not much to do other than log and continue, as the renew error
// may be an offline server, or may be an incorrect package match
lease, err := c.renew(dhcpCtx)
if err == nil {
c.lease = lease
log.Info("[DHCPv4] renew", "lease", lease)
t2.Reset(t2Timeout)
} else {
log.Error("[DHCPv4] renew failed", "err", err)
}
case <-t2.C:
// rebind is just like a request, but forcing to provide a new IP address
lease, err := c.request(dhcpCtx, true)
if err == nil {
c.lease = lease
log.Info("[DHCPv4] rebind", "lease", lease)
} else {
if _, ok := err.(*nclient4.ErrNak); !ok {
t1.Stop()
t2.Stop()
log.Error("[DHCPv4] rebind failed", "err", err)
}
log.Warn("[DHCPv4] ip may have changed", "ip", c.lease.ACK.YourIPAddr, "err", err)
c.initRebootFlag = false
c.lease = c.requestWithBackoff(dhcpCtx)
}
t1.Reset(t1Timeout)
t2.Reset(t2Timeout)
case <-c.stopChan:
// release is a unicast request of the IP release.
var err error
if err = c.release(); err != nil {
log.Error("[DHCPv4] release lease failed", "lease", lease, "err", err)
} else {
log.Info("[DHCPv4] release", "lease", lease)
}
t1.Stop()
t2.Stop()
close(c.releasedChan)
return err
}
}
}
// --------------------------------------------------------
// | |INIT-REBOOT | RENEWING |REBINDING |
// --------------------------------------------------------
// |broad/unicast |broadcast | unicast |broadcast |
// |server-ip |MUST NOT | MUST NOT |MUST NOT |
// |requested-ip |MUST | MUST NOT |MUST NOT |
// |ciaddr |zero | IP address |IP address|
// --------------------------------------------------------
func (c *DHCPv4Client) requestWithBackoff(ctx context.Context) *nclient4.Lease {
backoff := backoff.Backoff{
Factor: 2,
Jitter: true,
Min: 10 * time.Second,
Max: 1 * time.Minute,
}
var lease *nclient4.Lease
var err error
log.Debug("[DHCPv4]", "attempts", c.backoffAttempts)
for {
log.Debug("[DHCPv4] trying to get a new IP", "attempt", backoff.Attempt()+1)
lease, err = c.request(ctx, false)
if err != nil {
dur := backoff.Duration()
if c.backoffAttempts > 0 && backoff.Attempt() > float64(c.backoffAttempts)-1 {
errMsg := fmt.Errorf("failed to get an IPv4 address after %d attempt(s), giving up, error: %s", c.backoffAttempts, err.Error())
log.Error(fmt.Sprintf("[DHCPv4] %s", errMsg.Error()))
c.errorChan <- errMsg
c.Stop()
return nil
}
log.Error("[DHCPv4] request failed", "attempt", backoff.Attempt(), "err", err.Error(), "waiting", dur)
time.Sleep(dur)
continue
}
backoff.Reset()
break
}
if c.ipChan != nil {
log.Debug("[DHCPv4] using channel")
c.ipChan <- lease.ACK.YourIPAddr.String()
}
return lease
}
func (c *DHCPv4Client) request(ctx context.Context, rebind bool) (*nclient4.Lease, error) {
dhclient, err := nclient4.New(c.iface.Name)
if err != nil {
return nil, fmt.Errorf("create a client for iface %s failed, error: %w", c.iface.Name, err)
}
defer dhclient.Close()
modifiers := make([]dhcpv4.Modifier, 0)
if c.ddnsHostName != "" {
modifiers = append(modifiers,
dhcpv4.WithOption(dhcpv4.OptHostName(c.ddnsHostName)),
dhcpv4.WithOption(dhcpv4.OptClientIdentifier([]byte(c.ddnsHostName))),
)
}
// if initRebootFlag is set, this means we have an IP already set on c.requestedIP that should be used
if c.initRebootFlag {
log.Debug("[DHCPv4] init-reboot", "ip", c.requestedIP)
modifiers = append(modifiers, dhcpv4.WithOption(dhcpv4.OptRequestedIPAddress(c.requestedIP)))
}
// if this is a rebind, then the IP we should set is the one that already exists in lease
if rebind {
log.Debug("[DHCPv4] rebinding", "ip", c.lease.ACK.YourIPAddr)
modifiers = append(modifiers, dhcpv4.WithOption(dhcpv4.OptRequestedIPAddress(c.lease.ACK.YourIPAddr)))
}
return dhclient.Request(ctx, modifiers...)
}
func (c *DHCPv4Client) release() error {
dhclient, err := nclient4.New(c.iface.Name)
if err != nil {
return fmt.Errorf("create release client failed, error: %w, iface: %s, server ip: %v", err, c.iface.Name, c.lease.ACK.ServerIPAddr)
}
defer dhclient.Close()
// TODO modify lease
return dhclient.Release(c.lease)
}
func (c *DHCPv4Client) renew(ctx context.Context) (*nclient4.Lease, error) {
// renew needs a unicast client. This is due to some servers (like dnsmasq) require the exact request coming from the vip interface
dhclient, err := nclient4.New(c.iface.Name,
nclient4.WithUnicast(&net.UDPAddr{IP: c.lease.ACK.YourIPAddr, Port: nclient4.ClientPort}))
if err != nil {
return nil, fmt.Errorf("create renew client failed, error: %w, server ip: %v", err, c.lease.ACK.ServerIPAddr)
}
defer dhclient.Close()
return dhclient.Renew(ctx, c.lease)
}

393
pkg/vip/dhcpv6.go Normal file
View File

@@ -0,0 +1,393 @@
package vip
import (
"context"
"fmt"
log "log/slog"
"net"
"sync"
"sync/atomic"
"time"
"github.com/insomniacslk/dhcp/dhcpv6"
"github.com/insomniacslk/dhcp/dhcpv6/nclient6"
"github.com/insomniacslk/dhcp/iana"
"github.com/jpillora/backoff"
"github.com/vishvananda/netlink"
)
var dhcpv6ClientManager *DHCPv6ClientManager
func init() {
dhcpv6ClientManager = NewDHCPv6ClientManager()
}
type DHCPv6ClientManager struct {
clients map[string]*DHCPv6InternalClient
}
func NewDHCPv6ClientManager() *DHCPv6ClientManager {
return &DHCPv6ClientManager{
clients: map[string]*DHCPv6InternalClient{},
}
}
func (m *DHCPv6ClientManager) Get(iface string) *DHCPv6InternalClient {
c, exists := m.clients[iface]
if !exists {
return nil
}
return c
}
func (m *DHCPv6ClientManager) Add(iface string) (*DHCPv6InternalClient, error) {
c := m.Get(iface)
if c != nil {
c.references.Add(1)
return c, nil
}
client, err := NewDHCPv6InternalClient(iface)
if err != nil {
return nil, err
}
m.clients[iface] = client
return client, nil
}
func (m *DHCPv6ClientManager) Delete(iface string) {
c := m.Get(iface)
if c != nil {
c.references.Add(-1)
ref := c.references.Load()
if ref < 1 {
c.client.Close()
delete(m.clients, iface)
}
}
}
type DHCPv6InternalClient struct {
client *nclient6.Client
references *atomic.Int32
}
func NewDHCPv6InternalClient(iface string) (*DHCPv6InternalClient, error) {
client, err := nclient6.New(iface)
if err != nil {
return nil, fmt.Errorf("failed to create DHCPv6 client for interface %q: %w", iface, err)
}
ref := &atomic.Int32{}
ref.Store(1)
return &DHCPv6InternalClient{
client: client,
references: ref,
}, nil
}
type DHCPv6Client struct {
iface *net.Interface
ddnsHostName string
initRebootFlag bool
requestedIP net.IP
stopChan chan struct{} // used as a signal to release the IP and stop the dhcp client daemon
releasedChan chan struct{} // indicate that the IP has been released
errorChan chan error // indicates there was an error on the IP request
ipChan chan string
ic *DHCPv6InternalClient
addr *dhcpv6.OptIAAddress
backoffAttempts uint
stop sync.Once
}
// NewDHCPv6Client returns a new DHCP6 Client.
func NewDHCPv6Client(iface *net.Interface, parent netlink.Link, initRebootFlag bool, requestedIP string, backoffAttempts uint) (*DHCPv6Client, error) {
name := iface.Name
if parent != nil {
name = parent.Attrs().Name
}
client, err := dhcpv6ClientManager.Add(name)
if err != nil {
return nil, fmt.Errorf("failed to create DHCPv6 client: %w", err)
}
return &DHCPv6Client{
iface: iface,
stopChan: make(chan struct{}),
releasedChan: make(chan struct{}),
errorChan: make(chan error),
initRebootFlag: initRebootFlag,
requestedIP: net.ParseIP(requestedIP),
ipChan: make(chan string),
ic: client,
backoffAttempts: backoffAttempts,
}, nil
}
func (c *DHCPv6Client) WithHostName(hostname string) DHCPClient {
c.ddnsHostName = hostname
return c
}
// Stop state-transition process and close dhcp client
func (c *DHCPv6Client) Stop() {
c.stop.Do(func() {
close(c.ipChan)
close(c.stopChan)
})
<-c.releasedChan
dhcpv6ClientManager.Delete(c.iface.Name)
}
// Gets the IPChannel for consumption
func (c *DHCPv6Client) IPChannel() chan string {
return c.ipChan
}
// Gets the ErrorChannel for consumption
func (c *DHCPv6Client) ErrorChannel() chan error {
return c.errorChan
}
func (c *DHCPv6Client) Start(ctx context.Context) error {
dhcpCtx, cancel := context.WithCancel(ctx)
defer cancel()
addr, err := c.requestWithBackoff(dhcpCtx)
if err != nil {
return fmt.Errorf("DHCPv6 client failed: %w", err)
}
c.addr = addr
c.initRebootFlag = false
// Set up two ticker to renew/rebind regularly
t1Timeout := c.addr.PreferredLifetime / 2
t2Timeout := (c.addr.ValidLifetime / 8) * 7
log.Debug("[DHCPv6] timeouts", "timeout1", t1Timeout, "timeout2", t2Timeout)
t1, t2 := time.NewTicker(t1Timeout), time.NewTicker(t2Timeout)
for {
select {
case <-t1.C:
// renew is a unicast request of the IP renewal
// A point on renew is: the library does not return the right message (NAK)
// on renew error due to IP Change, but instead it returns a different error
// This way there's not much to do other than log and continue, as the renew error
// may be an offline server, or may be an incorrect package match
addr, err := c.renew(dhcpCtx)
if err == nil {
c.addr = addr
log.Info("[DHCPv6] renew", "addr", addr.IPv6Addr.String())
t2.Reset(t2Timeout)
} else {
log.Error("[DHCPv6] renew failed", "err", err)
}
case <-t2.C:
// rebind is just like a request, but forcing to provide a new IP address
addr, err := c.request(dhcpCtx, true)
if err == nil {
c.addr = addr
log.Info("[DHCPv6] rebind", "lease", addr)
} else {
log.Warn("[DHCPv6] ip may have changed", "ip", addr.IPv6Addr.String(), "err", err)
c.initRebootFlag = false
c.addr, err = c.requestWithBackoff(dhcpCtx)
log.Error("[DHCPv6] rebind failed", "err", err)
}
t1.Reset(t1Timeout)
t2.Reset(t2Timeout)
case <-c.stopChan:
// create new context for DHCP cleanup (independent)
dhcpStopCtx, cancel := context.WithCancel(context.Background())
defer cancel()
// IP address release.
var err error
if err = c.release(dhcpStopCtx); err != nil {
log.Error("[DHCPv6] release failed", "err", err)
} else {
log.Info("[DHCPv6] released", "address", c.addr.String())
}
t1.Stop()
t2.Stop()
close(c.releasedChan)
return err
}
}
}
func (c *DHCPv6Client) requestWithBackoff(ctx context.Context) (*dhcpv6.OptIAAddress, error) {
backoff := backoff.Backoff{
Factor: 2,
Jitter: true,
Min: 10 * time.Second,
Max: 1 * time.Minute,
}
var err error
var addr *dhcpv6.OptIAAddress
for {
log.Debug("[DHCPv6] trying to get a new IP", "attempt", backoff.Attempt()+1)
addr, err = c.request(ctx, false)
if err != nil {
dur := backoff.Duration()
if c.backoffAttempts > 0 && backoff.Attempt() > float64(c.backoffAttempts)-1 {
errMsg := fmt.Errorf("failed to get an IPv4 address after %d attempt(s), giving up, error: %s", c.backoffAttempts, err.Error())
log.Error(fmt.Sprintf("[DHCPv6] %s", errMsg.Error()))
c.errorChan <- errMsg
c.Stop()
return nil, fmt.Errorf("failed to get IPv6 address: %w", err)
}
log.Error("[DHCPv6] request failed", "attempt", backoff.Attempt(), "err", err.Error(), "waiting", dur)
time.Sleep(dur)
continue
}
backoff.Reset()
break
}
if c.ipChan != nil {
log.Debug("[DHCPv6] using channel")
c.ipChan <- addr.IPv6Addr.String()
}
return addr, nil
}
func (c *DHCPv6Client) request(ctx context.Context, rebind bool) (*dhcpv6.OptIAAddress, error) {
modifiers := []dhcpv6.Modifier{}
modifiers = append(modifiers, dhcpv6.WithClientID(&dhcpv6.DUIDEN{EnterpriseNumber: 1, EnterpriseIdentifier: []byte(c.ddnsHostName)}))
modifiers = append(modifiers, dhcpv6.WithFQDN(4, c.ddnsHostName))
// if initRebootFlag is set, this means we have an IP already set on c.requestedIP that should be used
if c.initRebootFlag {
log.Debug("[DHCPv6] init-reboot", "ip", c.requestedIP)
addr := dhcpv6.OptIAAddress{
IPv6Addr: c.requestedIP,
}
modifiers = append(modifiers, dhcpv6.WithIANA(addr))
} else if rebind {
if c.addr == nil {
return nil, fmt.Errorf("unable to rebind - current IP unknown")
}
log.Debug("[DHCPv6] rebinding", "ip", c.addr.IPv6Addr)
modifiers = append(modifiers, dhcpv6.WithIANA(*c.addr))
}
var reply *dhcpv6.Message
if rebind || c.initRebootFlag {
request, err := dhcpv6.NewMessage(modifiers...)
if err != nil {
return nil, fmt.Errorf("failed to create rebind message: %w", err)
}
request.MessageType = dhcpv6.MessageTypeRebind
reply, err = c.ic.client.SendAndRead(ctx, c.ic.client.RemoteAddr(), request, nil)
if err != nil {
return nil, fmt.Errorf("rebind error: %w", err)
}
} else {
adv, err := c.ic.client.Solicit(ctx, modifiers...)
if err != nil {
return nil, fmt.Errorf("solicit error: %w", err)
}
request, err := dhcpv6.NewRequestFromAdvertise(adv, modifiers...)
if err != nil {
return nil, fmt.Errorf("unable to create request message: %w", err)
}
request.MessageType = dhcpv6.MessageTypeAdvertise
reply, err = c.ic.client.Request(ctx, request, modifiers...)
if err != nil {
return nil, fmt.Errorf("request error: %w", err)
}
}
if reply == nil {
return nil, fmt.Errorf("invalid request")
}
return getAddress(reply.Options.IANA())
}
func (c *DHCPv6Client) renew(ctx context.Context) (*dhcpv6.OptIAAddress, error) {
modifiers := []dhcpv6.Modifier{}
modifiers = append(modifiers, dhcpv6.WithClientID(&dhcpv6.DUIDEN{EnterpriseNumber: 1, EnterpriseIdentifier: []byte(c.ddnsHostName)}))
modifiers = append(modifiers, dhcpv6.WithFQDN(4, c.ddnsHostName))
modifiers = append(modifiers, dhcpv6.WithOption(&dhcpv6.OptionGeneric{OptionCode: dhcpv6.OptionUnicast}))
adv, err := c.ic.client.Solicit(ctx, modifiers...)
if err != nil {
return nil, fmt.Errorf("solicit error: %w", err)
}
request, err := dhcpv6.NewRequestFromAdvertise(adv)
if err != nil {
return nil, fmt.Errorf("failed to create request message: %w", err)
}
request.MessageType = dhcpv6.MessageTypeRenew
reply, err := c.ic.client.SendAndRead(ctx, c.ic.client.RemoteAddr(), request, nil)
if err != nil {
return nil, fmt.Errorf("failed to send renew: %w", err)
}
return getAddress(reply.Options.IANA())
}
func (c *DHCPv6Client) release(ctx context.Context) error {
modifiers := []dhcpv6.Modifier{}
modifiers = append(modifiers, dhcpv6.WithClientID(&dhcpv6.DUIDEN{EnterpriseNumber: 1, EnterpriseIdentifier: []byte(c.ddnsHostName)}))
modifiers = append(modifiers, dhcpv6.WithFQDN(4, c.ddnsHostName))
adv, err := c.ic.client.Solicit(ctx, modifiers...)
if err != nil {
return fmt.Errorf("solicit error: %w", err)
}
request, err := dhcpv6.NewRequestFromAdvertise(adv)
if err != nil {
return fmt.Errorf("failed to create release message: %w", err)
}
request.MessageType = dhcpv6.MessageTypeRelease
reply, err := c.ic.client.SendAndRead(ctx, c.ic.client.RemoteAddr(), request, nil)
if err != nil {
return fmt.Errorf("failed to send release: %w", err)
}
if reply.Options.Status().StatusCode != iana.StatusSuccess {
return fmt.Errorf("release failed with code %d: %s", reply.Options.Status().StatusCode, reply.Options.Status().StatusMessage)
}
return nil
}
func getAddress(iana []*dhcpv6.OptIANA) (*dhcpv6.OptIAAddress, error) {
if len(iana) < 1 {
return nil, fmt.Errorf("failed to get IANA")
}
if len(iana) < 1 {
return nil, fmt.Errorf("failed to get addresses data")
}
return iana[0].Options.Addresses()[0], nil
}

View File

@@ -9,6 +9,10 @@ import (
"github.com/kube-vip/kube-vip/pkg/utils"
)
const (
defaultInterval = 3
)
// IPUpdater is the interface to plug dns updaters
type IPUpdater interface {
Run(ctx context.Context)
@@ -27,37 +31,35 @@ func NewIPUpdater(vip Network) IPUpdater {
// Run runs the IP updater
func (d *ipUpdater) Run(ctx context.Context) {
go func(ctx context.Context) {
for {
select {
case <-ctx.Done():
log.Info("stop ipUpdater")
return
default:
mode := "ipv4"
if utils.IsIPv6(d.vip.IP()) {
mode = "ipv6"
}
ip, err := utils.LookupHost(d.vip.DNSName(), mode)
if err != nil {
log.Warn("cannot lookup", "name", d.vip.DNSName(), "err", err)
// fallback to renewing the existing IP
ip = []string{d.vip.IP()}
}
log.Debug("setting IP", "address", ip)
if err := d.vip.SetIP(ip[0]); err != nil {
log.Error("setting IP", "address", ip, "err", err)
}
// Normal VIP addition for DNS, use skipDAD=false for normal DAD process
if _, err := d.vip.AddIP(false, false); err != nil {
log.Error("error adding virtual IP", "err", err)
}
for {
select {
case <-ctx.Done():
log.Info("stop ipUpdater")
return
default:
mode := "ipv4"
if utils.IsIPv6(d.vip.IP()) {
mode = "ipv6"
}
time.Sleep(3 * time.Second)
ip, err := utils.LookupHost(d.vip.DNSName(), mode, true)
if err != nil {
log.Warn("cannot lookup", "name", d.vip.DNSName(), "err", err)
// fallback to renewing the existing IP
ip = []string{d.vip.IP()}
}
if err := d.vip.SetIP(ip[0]); err != nil {
log.Error("setting IP", "address", ip, "err", err)
}
// Normal VIP addition for DNS, use skipDAD=false for normal DAD process
if _, err := d.vip.AddIP(true, false, defaultInterval); err != nil {
log.Error("error adding virtual IP", "err", err)
}
}
}(ctx)
time.Sleep(defaultInterval * time.Second)
}
}

View File

@@ -407,16 +407,17 @@ func (e *Egress) findExistingVIP(rules []string, vip string) [][]string {
return foundRules
}
func ClearIPTables(useNftables bool, namespace string, protocol iptables.Protocol) {
func ClearIPTables(useNftables bool, namespace string, protocol iptables.Protocol) error {
i, err := CreateIptablesClient(useNftables, namespace, protocol)
if err != nil {
log.Warn("[egress] Unable to clean any dangling egress rules", "err", err)
log.Warn("[egress] Can be ignored in non iptables release of kube-vip")
log.Debug("[egress] Unable to clean any dangling egress rules", "err", err)
return fmt.Errorf("unable to create client")
} else {
log.Info("[egress] Cleaning any dangling kube-vip egress rules")
cleanErr := i.CleanIPtables()
if cleanErr != nil {
log.Error("Error cleaning rules", "err", cleanErr)
return err
}
}
return nil
}

View File

@@ -21,13 +21,13 @@ type NdpResponder struct {
func NewNDPResponder(ifaceName string) (*NdpResponder, error) {
iface, err := net.InterfaceByName(ifaceName)
if err != nil {
return nil, fmt.Errorf("failed to get interface %q: %v", ifaceName, err)
return nil, fmt.Errorf("failed to get interface %q: %w", ifaceName, err)
}
// Use link-local address as the source IPv6 address for NDP communications.
conn, _, err := ndp.Listen(iface, ndp.LinkLocal)
if err != nil {
return nil, fmt.Errorf("creating NDP responder for %q: %s", iface.Name, err)
return nil, fmt.Errorf("creating NDP responder for %s: %w", iface.Name, err)
}
ret := &NdpResponder{

View File

@@ -5,13 +5,11 @@ import (
"crypto/rand"
"fmt"
"net"
"strconv"
"strings"
"syscall"
log "log/slog"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/pkg/errors"
"github.com/vishvananda/netlink"
)
@@ -27,29 +25,54 @@ func getHostName(dnsName string) string {
}
// GetDefaultGatewayInterface return default gateway interface link
func GetDefaultGatewayInterface() (*net.Interface, error) {
routes, err := netlink.RouteList(nil, syscall.AF_INET)
if err != nil {
return nil, err
func GetDefaultGatewayInterface() (iface *net.Interface, err error) {
// Attempt IPv4 first (usually the default)
if iface, err = getDefaultRoute(syscall.AF_INET); err == nil {
return iface, nil
}
routes6, err := netlink.RouteList(nil, syscall.AF_INET6)
if err != nil {
return nil, err
// If the IPv4 default route is not found, then attempt IPv6 default route.
if iface, err = getDefaultRoute(syscall.AF_INET6); err == nil {
return iface, nil
}
routes = append(routes, routes6...)
return nil, fmt.Errorf("unable to find interface with default route: %w", err)
}
// getDefaultRoute attempts to find the default route for the specified address family.
func getDefaultRoute(family int) (*net.Interface, error) {
// only search for default routes
filter := &netlink.Route{Dst: nil}
mask := netlink.RT_FILTER_DST
routes, err := netlink.RouteListFiltered(family, filter, mask)
if err != nil {
return nil, fmt.Errorf("listing routes: %w", err)
}
for _, route := range routes {
if route.Dst == nil || route.Dst.String() == "0.0.0.0/0" || route.Dst.String() == "::/0" {
if route.LinkIndex <= 0 {
return nil, errors.New("Found default route but could not determine interface")
// double check
if route.Dst != nil && route.Dst.String() != "0.0.0.0/0" && route.Dst.String() != "::/0" {
continue
}
idx := route.LinkIndex
// handle MultiPath
if idx <= 0 && len(route.MultiPath) > 0 {
for _, nh := range route.MultiPath {
if nh.LinkIndex > 0 {
idx = nh.LinkIndex
break
}
}
return net.InterfaceByIndex(route.LinkIndex)
}
if idx > 0 {
return net.InterfaceByIndex(idx)
}
}
return nil, errors.New("Unable to find default route")
return nil, errors.New("default route not found")
}
// MonitorDefaultInterface monitor the default interface and catch the event of the default route
@@ -144,22 +167,3 @@ func GetNonLinkLocalIP(iface *netlink.Link, family int) (string, error) {
return "", fmt.Errorf("failed to find non-local IP on interface: %s", (*iface).Attrs().Name)
}
func selectSubnet(address string, subnets []string) string {
subnet := ""
if utils.IsIPv4(address) {
if subnets[0] != "" {
subnet = subnets[0]
} else {
subnet = strconv.Itoa(defaultMaskIPv4)
}
} else {
if len(subnets) > 1 && subnets[1] != "" {
subnet = subnets[1]
} else {
subnet = strconv.Itoa(defaultMaskIPv6)
}
}
return subnet
}

View File

@@ -0,0 +1,417 @@
package wireguard
import (
"context"
"fmt"
"sync"
"time"
log "log/slog"
"github.com/kube-vip/kube-vip/pkg/nftables"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/vishvananda/netlink"
"gopkg.in/yaml.v3"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
)
// TunnelConfig represents a single WireGuard tunnel configuration
type TunnelConfig struct {
VIP string `yaml:"vip"` // The VIP this tunnel serves (e.g., "10.0.0.100/24")
PrivateKey string `yaml:"privateKey"` // WireGuard private key
PeerPublicKey string `yaml:"peerPublicKey"` // Peer's public key
PeerEndpoint string `yaml:"peerEndpoint"` // Peer endpoint (IP:Port)
AllowedIPs []string `yaml:"allowedIPs"` // Allowed IPs through tunnel
ListenPort int `yaml:"listenPort"` // Local listen port
// Internal fields (not from YAML)
Name string // Unique name for this tunnel (e.g., "tunnel1")
InterfaceName string // Interface name (e.g., "wg0", "wg1")
}
// TunnelManager manages multiple WireGuard tunnels
type TunnelManager struct {
mu sync.RWMutex
tunnels map[string]*WireGuard // key: VIP (without CIDR), value: WireGuard instance
configs map[string]*TunnelConfig // key: VIP (without CIDR), value: TunnelConfig
refCount map[string]int // key: VIP (without CIDR), value: number of consumers using the tunnel
}
// NewTunnelManager creates a new tunnel manager
func NewTunnelManager() *TunnelManager {
return &TunnelManager{
tunnels: make(map[string]*WireGuard),
configs: make(map[string]*TunnelConfig),
refCount: make(map[string]int),
}
}
// CleanupStaleResources removes any stale WireGuard interfaces, nftables rules,
// and policy routing rules from a previous run. This handles crash recovery
// when using hostNetwork: true, where resources persist after pod termination.
// Must be called AFTER LoadConfigurationsFromSecret so we know which interfaces to clean.
func (tm *TunnelManager) CleanupStaleResources() error {
tm.mu.RLock()
defer tm.mu.RUnlock()
log.Info("cleaning up stale WireGuard resources from previous runs")
// Build set of configured interface names
configuredInterfaces := make(map[string]bool)
for _, config := range tm.configs {
configuredInterfaces[config.InterfaceName] = true
}
if len(configuredInterfaces) == 0 {
log.Warn("no tunnel configurations loaded, skipping interface cleanup")
return nil
}
// 1. Clean up stale WireGuard interfaces that match our configured names
links, err := netlink.LinkList()
if err != nil {
return fmt.Errorf("failed to list network interfaces: %w", err)
}
for _, link := range links {
name := link.Attrs().Name
if configuredInterfaces[name] {
log.Info("removing stale WireGuard interface", "interface", name)
if err := netlink.LinkDel(link); err != nil {
log.Warn("failed to delete stale interface", "interface", name, "err", err)
}
}
}
// 2. Clean up stale nftables chains (kube_vip_*)
if err := nftables.CleanupAllChains(); err != nil {
log.Warn("failed to cleanup stale nftables chains", "err", err)
}
// 3. Clean up stale policy routing rules for configured listen ports
for _, config := range tm.configs {
if err := nftables.CleanupPolicyRouting(config.ListenPort); err != nil {
log.Warn("failed to cleanup policy routing", "listenPort", config.ListenPort, "err", err)
}
}
log.Info("stale resource cleanup complete")
return nil
}
// LoadConfigurationsFromSecret loads WireGuard tunnel configurations from a Kubernetes secret
// Secret format (YAML string in tunnels key):
//
// data:
// tunnels: |
// tunnel1:
// vip: 10.0.0.100/24
// privateKey: <key>
// peerPublicKey: <key>
// peerEndpoint: 203.0.113.1:51820
// allowedIPs:
// - 10.0.0.0/24
// listenPort: 51820
// tunnel2:
// vip: 10.0.0.101/24
// privateKey: <key>
// peerPublicKey: <key>
// peerEndpoint: 203.0.113.2:51821
// allowedIPs:
// - 10.0.0.0/24
// listenPort: 51821
func (tm *TunnelManager) LoadConfigurationsFromSecret(ctx context.Context, clientSet *kubernetes.Clientset, namespace, secretName string) error {
secret, err := clientSet.CoreV1().Secrets(namespace).Get(ctx, secretName, metav1.GetOptions{})
if err != nil {
return fmt.Errorf("failed to get secret %s/%s: %w", namespace, secretName, err)
}
tunnelsData, ok := secret.Data["tunnels"]
if !ok {
return fmt.Errorf("secret %s/%s must contain 'tunnels' key", namespace, secretName)
}
return tm.parseTunnelConfig(tunnelsData)
}
// parseTunnelConfig parses the YAML tunnel configuration
func (tm *TunnelManager) parseTunnelConfig(data []byte) error {
tm.mu.Lock()
defer tm.mu.Unlock()
// Parse YAML into a map of tunnel name -> TunnelConfig
var tunnelsMap map[string]*TunnelConfig
if err := yaml.Unmarshal(data, &tunnelsMap); err != nil {
return fmt.Errorf("failed to parse tunnel configuration YAML: %w", err)
}
if len(tunnelsMap) == 0 {
return fmt.Errorf("no tunnel configurations found in secret")
}
// Use the tunnel name from the YAML key as the interface name
// This ensures deterministic interface naming across restarts
for name, config := range tunnelsMap {
config.Name = name
config.InterfaceName = name
if err := tm.addTunnelConfig(config); err != nil {
return fmt.Errorf("invalid tunnel config %s: %w", name, err)
}
}
log.Info("loaded tunnel configurations", "count", len(tm.configs))
return nil
}
// addTunnelConfig adds a tunnel configuration to the manager
func (tm *TunnelManager) addTunnelConfig(config *TunnelConfig) error {
// Validate required fields
if config.VIP == "" {
return fmt.Errorf("vip is required")
}
if config.PrivateKey == "" {
return fmt.Errorf("privateKey is required")
}
if config.PeerPublicKey == "" {
return fmt.Errorf("peerPublicKey is required")
}
if config.PeerEndpoint == "" {
return fmt.Errorf("peerEndpoint is required")
}
if config.ListenPort < 1 || config.ListenPort > 65535 {
return fmt.Errorf("listenPort must be between 1 and 65535")
}
// Extract VIP without CIDR for indexing
vipKey := utils.StripCIDR(config.VIP)
// Check for duplicate VIP
if _, exists := tm.configs[vipKey]; exists {
return fmt.Errorf("duplicate VIP configuration: %s", config.VIP)
}
tm.configs[vipKey] = config
log.Info("added tunnel configuration",
"name", config.Name,
"vip", config.VIP,
"interface", config.InterfaceName,
"listenPort", config.ListenPort)
return nil
}
// GetTunnelForVIP returns the WireGuard tunnel instance for a given VIP
// If the tunnel doesn't exist yet, it returns nil
func (tm *TunnelManager) GetTunnelForVIP(vip string) *WireGuard {
tm.mu.RLock()
defer tm.mu.RUnlock()
vipKey := utils.StripCIDR(vip)
return tm.tunnels[vipKey]
}
// GetConfigForVIP returns the tunnel configuration for a given VIP
func (tm *TunnelManager) GetConfigForVIP(vip string) *TunnelConfig {
tm.mu.RLock()
defer tm.mu.RUnlock()
vipKey := utils.StripCIDR(vip)
return tm.configs[vipKey]
}
// BringUpTunnelForVIP creates and brings up a WireGuard tunnel for the given VIP.
// If the tunnel is already up, it increments the reference count.
// Multiple consumers (control plane, services) can share the same VIP tunnel.
func (tm *TunnelManager) BringUpTunnelForVIP(vip string) error {
tm.mu.Lock()
defer tm.mu.Unlock()
vipKey := utils.StripCIDR(vip)
// Check if already up - increment reference count
if _, exists := tm.tunnels[vipKey]; exists {
tm.refCount[vipKey]++
log.Debug("tunnel already up, incremented reference count", "vip", vip, "refCount", tm.refCount[vipKey])
return nil
}
// Get configuration
config, exists := tm.configs[vipKey]
if !exists {
return fmt.Errorf("no tunnel configuration found for VIP %s", vip)
}
// Create WireGuard configuration
wgCfg := WGConfig{
InterfaceName: config.InterfaceName,
PrivateKey: config.PrivateKey,
PeerPublicKey: config.PeerPublicKey,
PeerEndpoint: config.PeerEndpoint,
Address: config.VIP,
AllowedIPs: config.AllowedIPs,
ListenPort: config.ListenPort,
KeepAlive: 5 * time.Second,
}
// Create and bring up the tunnel
wg := NewWireGuard(wgCfg)
if err := wg.Up(); err != nil {
return fmt.Errorf("failed to bring up tunnel for VIP %s: %w", vip, err)
}
// Setup policy routing for response packets (uses listen port as mark/table)
if err := nftables.SetupPolicyRouting(config.InterfaceName, config.ListenPort); err != nil {
// Tear down the tunnel if policy routing setup fails
_ = wg.Down()
return fmt.Errorf("failed to setup policy routing for VIP %s: %w", vip, err)
}
tm.tunnels[vipKey] = wg
tm.refCount[vipKey] = 1
log.Info("brought up WireGuard tunnel",
"vip", vip,
"interface", config.InterfaceName,
"listenPort", config.ListenPort)
return nil
}
// TearDownTunnelForVIP decrements the reference count for the given VIP tunnel.
// The tunnel is only torn down when the reference count reaches zero.
// This allows multiple consumers (control plane, services) to share the same VIP tunnel.
func (tm *TunnelManager) TearDownTunnelForVIP(vip string) error {
tm.mu.Lock()
defer tm.mu.Unlock()
vipKey := utils.StripCIDR(vip)
wg, exists := tm.tunnels[vipKey]
if !exists {
log.Debug("tunnel not found for teardown", "vip", vip)
return nil
}
// Decrement reference count
tm.refCount[vipKey]--
if tm.refCount[vipKey] > 0 {
log.Debug("tunnel still in use, decremented reference count", "vip", vip, "refCount", tm.refCount[vipKey])
return nil
}
// Get config before cleanup (need interface name and listen port)
config := tm.configs[vipKey]
// Reference count is zero, tear down the tunnel
if err := wg.Down(); err != nil {
log.Error("failed to tear down tunnel", "vip", vip, "err", err)
// Continue to remove from map even if teardown failed
}
// Cleanup nftables infrastructure for this tunnel (both IPv4 and IPv6)
if config != nil {
if err := nftables.CleanupTunnelInfrastructure(config.InterfaceName, false); err != nil {
log.Warn("failed to cleanup IPv4 nftables infrastructure", "interface", config.InterfaceName, "err", err)
}
if err := nftables.CleanupTunnelInfrastructure(config.InterfaceName, true); err != nil {
log.Warn("failed to cleanup IPv6 nftables infrastructure", "interface", config.InterfaceName, "err", err)
}
// Cleanup policy routing rules
if err := nftables.CleanupPolicyRouting(config.ListenPort); err != nil {
log.Warn("failed to cleanup policy routing", "listenPort", config.ListenPort, "err", err)
}
}
delete(tm.tunnels, vipKey)
delete(tm.refCount, vipKey)
log.Info("tore down WireGuard tunnel", "vip", vip)
return nil
}
// TearDownAllTunnels tears down all active tunnels regardless of reference count.
// This is typically called during shutdown.
func (tm *TunnelManager) TearDownAllTunnels() error {
tm.mu.Lock()
defer tm.mu.Unlock()
var errors []error
for vip, wg := range tm.tunnels {
if err := wg.Down(); err != nil {
log.Error("failed to tear down tunnel", "vip", vip, "err", err)
errors = append(errors, err)
}
// Cleanup nftables infrastructure for this tunnel
config := tm.configs[vip]
if config != nil {
if err := nftables.CleanupTunnelInfrastructure(config.InterfaceName, false); err != nil {
log.Warn("failed to cleanup IPv4 nftables infrastructure", "interface", config.InterfaceName, "err", err)
}
if err := nftables.CleanupTunnelInfrastructure(config.InterfaceName, true); err != nil {
log.Warn("failed to cleanup IPv6 nftables infrastructure", "interface", config.InterfaceName, "err", err)
}
// Cleanup policy routing rules
if err := nftables.CleanupPolicyRouting(config.ListenPort); err != nil {
log.Warn("failed to cleanup policy routing", "listenPort", config.ListenPort, "err", err)
}
}
}
tm.tunnels = make(map[string]*WireGuard)
tm.refCount = make(map[string]int)
log.Info("tore down all WireGuard tunnels")
if len(errors) > 0 {
return fmt.Errorf("errors occurred during teardown: %v", errors)
}
return nil
}
// GetRefCount returns the current reference count for a VIP tunnel.
// Returns 0 if the tunnel doesn't exist.
func (tm *TunnelManager) GetRefCount(vip string) int {
tm.mu.RLock()
defer tm.mu.RUnlock()
vipKey := utils.StripCIDR(vip)
return tm.refCount[vipKey]
}
// ListActiveTunnels returns a list of VIPs with active tunnels
func (tm *TunnelManager) ListActiveTunnels() []string {
tm.mu.RLock()
defer tm.mu.RUnlock()
vips := make([]string, 0, len(tm.tunnels))
for vip := range tm.tunnels {
vips = append(vips, vip)
}
return vips
}
// ListConfiguredTunnels returns a list of VIPs with configurations
func (tm *TunnelManager) ListConfiguredTunnels() []string {
tm.mu.RLock()
defer tm.mu.RUnlock()
vips := make([]string, 0, len(tm.configs))
for vip := range tm.configs {
vips = append(vips, vip)
}
return vips
}
// HasConfigForVIP checks if a tunnel configuration exists for the given VIP
func (tm *TunnelManager) HasConfigForVIP(vip string) bool {
tm.mu.RLock()
defer tm.mu.RUnlock()
vipKey := utils.StripCIDR(vip)
_, exists := tm.configs[vipKey]
return exists
}

View File

@@ -3,62 +3,243 @@ package wireguard
import (
"fmt"
"net"
"os"
"time"
log "log/slog"
"github.com/vishvananda/netlink"
"golang.zx2c4.com/wireguard/wgctrl"
"golang.zx2c4.com/wireguard/wgctrl/wgtypes"
)
func ConfigureInterface(priKey, peerPublicKey, endpoint string) error {
type WGConfig struct {
InterfaceName string // e.g., "wg0"
PrivateKey string
PeerPublicKey string
PeerEndpoint string
Address string
AllowedIPs []string // CIDRs allowed through tunnel
ListenPort int
KeepAlive time.Duration
}
client, err := wgctrl.New()
if err != nil {
return fmt.Errorf("failed to open client: %v", err)
type WireGuard struct {
cfg WGConfig
linkIndex *int
client *wgctrl.Client
}
func NewWireGuard(cfg WGConfig) *WireGuard {
return &WireGuard{
cfg: cfg,
}
defer client.Close()
}
pri, err := wgtypes.ParseKey(priKey)
if err != nil {
return fmt.Errorf("failed to generate private key: %v", err)
// Up brings up the interface, configures peers, and sets routes
func (w *WireGuard) Up() error {
log.Info("bringing up wireguard interface", "interface", w.cfg.InterfaceName)
if err := w.createInterface(); err != nil {
_ = w.teardown()
return err
}
pub, err := wgtypes.ParseKey(peerPublicKey) // Should be generated by the remote peer
if err != nil {
return fmt.Errorf("failed to parse public key: %v", err)
if err := w.configurePeer(); err != nil {
_ = w.teardown()
return err
}
//log.Printf("Public Key [%s]", pri.PublicKey())
port := 51820
ka := 20 * time.Second
conf := wgtypes.Config{
PrivateKey: &pri,
ListenPort: &port,
ReplacePeers: true,
Peers: []wgtypes.PeerConfig{{
PublicKey: pub,
Remove: false,
UpdateOnly: false,
Endpoint: &net.UDPAddr{
IP: net.ParseIP(endpoint),
Port: 51820,
},
PersistentKeepaliveInterval: &ka,
ReplaceAllowedIPs: true,
AllowedIPs: []net.IPNet{{
IP: net.ParseIP("10.0.0.0"),
Mask: net.ParseIP("0.0.0.0").DefaultMask(),
}},
}},
}
if err := client.ConfigureDevice("wg0", conf); err != nil {
if os.IsNotExist(err) {
return fmt.Errorf("wg0 doesn't exist [%s]", err)
}
return fmt.Errorf("unknown config error: %v", err)
if err := w.addRoutes(); err != nil {
_ = w.teardown()
return err
}
return nil
}
// Down tears down the interface and routes
func (w *WireGuard) Down() error {
return w.teardown()
}
// CreateInterface creates the wg interface in the current netns if it doesn't exist
func (w *WireGuard) createInterface() error {
// Validate ListenPort is within valid range for uint32
if w.cfg.ListenPort < 0 || w.cfg.ListenPort > 65535 {
return fmt.Errorf("ListenPort %d is out of valid range (0-65535)", w.cfg.ListenPort)
}
genericLink := &netlink.GenericLink{
LinkAttrs: netlink.LinkAttrs{Name: w.cfg.InterfaceName},
LinkType: "wireguard",
}
err := netlink.LinkAdd(genericLink)
if err != nil && !isExistErr(err) {
return fmt.Errorf("failed to create wireguard interface: %v", err)
}
// Set up interface
link, err := netlink.LinkByName(w.cfg.InterfaceName)
if err != nil {
return fmt.Errorf("cannot find interface after creation: %v", err)
}
if err := netlink.LinkSetUp(link); err != nil {
return fmt.Errorf("failed to bring interface up: %v", err)
}
// Add the VIP address to the WireGuard interface.
// When WireGuard mode is active, the VIP is ONLY on the tunnel interface
// (not also on lo), which prevents the kernel from treating incoming packets
// as loopback traffic and ensures proper INPUT chain processing for nftables DNAT.
addr, err := netlink.ParseAddr(w.cfg.Address)
if err != nil {
return fmt.Errorf("failed to parse address: %s, %v", w.cfg.Address, err)
}
err = netlink.AddrAdd(link, addr)
if err != nil && !isExistErr(err) {
return fmt.Errorf("could not add address to link: %s, %v", w.cfg.Address, err)
}
log.Info("assigned VIP address to WireGuard interface", "interface", w.cfg.InterfaceName, "address", w.cfg.Address)
if err = netlink.LinkSetMTU(link, 1420); err != nil {
return fmt.Errorf("failed to set mtu %w", err)
}
// NOTE: We intentionally do NOT add a blanket "not fwmark" rule here.
// That would route ALL node traffic through the WireGuard tunnel.
// Instead, we use connmark in nftables to mark only response packets
// to incoming WireGuard connections, and policy routing routes only
// those marked packets back through the tunnel.
// See nftables.SetupPolicyRouting() and ApplyDNAT() for details.
w.linkIndex = &link.Attrs().Index
log.Info("created link", "interface", w.cfg.InterfaceName)
return nil
}
// ConfigurePeer sets the private key, peer, allowed IPs, endpoint, keepalive
func (w *WireGuard) configurePeer() error {
client, err := wgctrl.New()
if err != nil {
return fmt.Errorf("failed to open wgctrl client: %v", err)
}
privKey, err := wgtypes.ParseKey(w.cfg.PrivateKey)
if err != nil {
return fmt.Errorf("invalid private key: %v", err)
}
pubKey, err := wgtypes.ParseKey(w.cfg.PeerPublicKey)
if err != nil {
return fmt.Errorf("invalid peer public key: %v", err)
}
addr, err := net.ResolveUDPAddr("udp", w.cfg.PeerEndpoint)
if err != nil {
return fmt.Errorf("failed to resolve UDP address: %v", err)
}
peer := wgtypes.PeerConfig{
PublicKey: pubKey,
Endpoint: addr,
PersistentKeepaliveInterval: &w.cfg.KeepAlive,
ReplaceAllowedIPs: true,
}
for _, cidr := range w.cfg.AllowedIPs {
_, ipnet, err := net.ParseCIDR(cidr)
if err != nil {
return fmt.Errorf("invalid AllowedIP %s: %v", cidr, err)
}
peer.AllowedIPs = append(peer.AllowedIPs, *ipnet)
}
conf := wgtypes.Config{
PrivateKey: &privKey,
ListenPort: &w.cfg.ListenPort,
ReplacePeers: true,
Peers: []wgtypes.PeerConfig{peer},
FirewallMark: &w.cfg.ListenPort,
}
if err = client.ConfigureDevice(w.cfg.InterfaceName, conf); err != nil {
return fmt.Errorf("failed to configure wireguard device: %v", err)
}
w.client = client
log.Info("wireguard device configured", "interface", w.cfg.InterfaceName)
return nil
}
// AddRoutes creates the routes for the VIP/public IPs via the wg interface
func (w *WireGuard) addRoutes() error {
for _, cidr := range w.cfg.AllowedIPs {
_, dstNet, err := net.ParseCIDR(cidr)
if err != nil {
log.Error("invalid route CIDR, skipping", "cidr", cidr, "err", err)
continue
}
route := netlink.Route{
LinkIndex: *w.linkIndex,
Table: w.cfg.ListenPort,
Dst: dstNet,
}
log.Info("Added route", "dst", dstNet.String())
if err := netlink.RouteAdd(&route); err != nil && !isExistErr(err) {
return fmt.Errorf("failed to add route %s: %v", cidr, err)
}
}
return nil
}
// RemoveRoutes deletes the previously added routes
func (w *WireGuard) removeRoutes() {
for _, cidr := range w.cfg.AllowedIPs {
_, dstNet, err := net.ParseCIDR(cidr)
if err != nil {
log.Error("invalid route CIDR, skipping", "cidr", cidr, "err", err)
continue
}
if w.linkIndex == nil {
continue
}
route := netlink.Route{
LinkIndex: *w.linkIndex,
Dst: dstNet,
Table: w.cfg.ListenPort,
}
err = netlink.RouteDel(&route) // best effort
if err != nil {
log.Error("failed to remove route", "cidr", cidr, "err", err)
}
}
log.Info("routes removed", "interface", w.cfg.InterfaceName)
}
// Teardown deletes the interface entirely (called on leadership loss)
func (w *WireGuard) teardown() error {
if w.client != nil {
if err := w.client.Close(); err != nil {
return fmt.Errorf("failed to close wgctrl client: %v", err)
}
w.client = nil
}
w.removeRoutes()
if link, err := netlink.LinkByName(w.cfg.InterfaceName); err == nil {
if err = netlink.LinkDel(link); err != nil {
log.Error("failed to delete link", "err", err)
}
}
w.linkIndex = nil
log.Info("tear down complete", "interface", w.cfg.InterfaceName)
return nil
}
// helper: check if error indicates interface/route exists
func isExistErr(err error) bool {
if err == nil {
return false
}
return err.Error() == "file exists" || err.Error() == "Link exists"
}

14
skaffold.yaml Normal file
View File

@@ -0,0 +1,14 @@
apiVersion: skaffold/v4beta13
kind: Config
metadata:
name: kube-vip
build:
local:
useBuildkit: true
artifacts:
- image: ghcr.io/kube-vip/kube-vip
docker:
dockerfile: Dockerfile
manifests:
rawYaml:
- testing/kind/kube-vip-skaffold.yaml

View File

@@ -1,6 +1,6 @@
# syntax=docker/dockerfile:experimental
FROM golang:1.20-alpine as dev
FROM golang:1.25.5-alpine3.23 as dev
RUN apk add --no-cache git ca-certificates
RUN adduser -D appuser
COPY . /src/

View File

@@ -47,6 +47,10 @@ fmt:
docker:
# @docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7,linux/ppc64le,linux/s390x --push -t $(REPOSITORY)/$(TARGET):$(DOCKERTAG) .
@docker buildx build --platform linux/amd64 --push -t $(REPOSITORY)/$(TARGET):$(DOCKERTAG) .
@echo New Multi Architecture Docker image created and pushed
dockerLocal:
@docker buildx build --platform linux/amd64 -t $(REPOSITORY)/$(TARGET):$(VERSION) .
@echo New Multi Architecture Docker image created
simplify:

View File

@@ -1,7 +1,3 @@
module github.com/kube-vip/kube-vip/testing/e2e/servicesClient
go 1.19
require github.com/sirupsen/logrus v1.9.0
require golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8 // indirect
go 1.25.5

View File

@@ -1,15 +0,0 @@
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/sirupsen/logrus v1.9.0 h1:trlNQbNUG3OdDrDil03MCb1H2o9nJ1x4/5LYw7byDE0=
github.com/sirupsen/logrus v1.9.0/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8 h1:0A+M6Uqn+Eje4kHMK80dtF3JCXC4ykBgQG4Fe06QRhQ=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@@ -16,7 +16,8 @@ func main() {
// Lookup environment variables
mode, exists := os.LookupEnv("E2EMODE")
if !exists {
log.Fatal("The environment variable E2ESERVER, was not set")
log.Error("The environment variable E2ESERVER, was not set")
os.Exit(1)
}
switch mode {
@@ -25,14 +26,16 @@ func main() {
fmt.Fprintf(w, "Hello!")
})
log.Info("Starting server at port 80")
log.Info("starting server at port 80")
if err := http.ListenAndServe(":80", nil); err != nil {
log.Fatal(err)
log.Error(err.Error())
os.Exit(1)
}
case strings.ToUpper("CLIENT"):
address, exists := os.LookupEnv("E2EADDRESS")
if !exists {
log.Fatal("The environment variable E2EADDRESS, was not set")
log.Error("the environment variable E2EADDRESS, was not set")
os.Exit(1)
}
ip := net.ParseIP(address)
network := "tcp"
@@ -41,20 +44,20 @@ func main() {
network = "tcp6"
address = fmt.Sprintf("[%s]", address)
port = ":12346" // use a different port for IPv6 incase the IPv4 port is left connected
log.Infoln("Connecting with IPv6")
log.Info("connecting with IPv6")
}
for {
// Connect to e2e endpoint with a second timeout
conn, err := net.DialTimeout(network, address+port, time.Second)
if err != nil {
log.Errorf("Dial failed: %v", err.Error())
log.Error("dial failed", "err", err)
// Wait for a second and connect again
time.Sleep(time.Second)
continue
}
_, err = conn.Write([]byte("The Grid, a digital frontier"))
if err != nil {
log.Errorf("Write data failed: %v ", err.Error())
log.Error("write data failed", "err", err)
// Wait for a second and connect again
time.Sleep(time.Second)
continue
@@ -64,20 +67,20 @@ func main() {
received := make([]byte, 1024)
_, err = conn.Read(received)
if err != nil {
log.Errorf("Read data failed: %v", err.Error())
log.Error("Read data failed", "err", err)
// Wait for a second and connect again
time.Sleep(time.Second)
continue
}
log.Infof("Received message: %s\n", string(received))
log.Info("received", "message", string(received))
conn.Close()
// Wait for a second and connect again
time.Sleep(time.Second)
}
default:
log.Fatalf("Unknown mode [%s]", mode)
log.Error("Unknown mode", "value", mode)
os.Exit(1)
}
}

View File

@@ -0,0 +1,233 @@
//go:build e2e
// +build e2e
package e2e_test
import (
"context"
"os"
"k8s.io/client-go/kubernetes"
"k8s.io/klog/v2"
kindconfigv1alpha4 "sigs.k8s.io/kind/pkg/apis/config/v1alpha4"
"sigs.k8s.io/kind/pkg/log"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/kube-vip/kube-vip/testing/e2e"
)
var _ = Describe("kube-vip BGP when deployed as a regular pod", Ordered, func() {
if Mode == ModeBGP {
var (
logger log.Logger
imagePath string
k8sImagePath string
configPath string
tempDirPathRoot string
)
ctx, cancel := context.WithCancel(context.TODO())
BeforeEach(func() {
klog.SetOutput(GinkgoWriter)
logger = e2e.TestLogger{}
imagePath = os.Getenv("E2E_IMAGE_PATH") // Path to kube-vip image
configPath = os.Getenv("CONFIG_PATH") // path to the api server config
k8sImagePath = os.Getenv("K8S_IMAGE_PATH") // path to the kubernetes image (version for kind)
if configPath == "" {
configPath = "/etc/kubernetes/admin.conf"
}
})
BeforeAll(func() {
var err error
tempDirPathRoot, err = os.MkdirTemp("", "kube-vip-test-arp")
Expect(err).NotTo(HaveOccurred())
})
AfterAll(func() {
if os.Getenv("E2E_KEEP_LOGS") != "true" {
Expect(os.RemoveAll(tempDirPathRoot)).To(Succeed())
}
cancel()
})
Describe("kube-vip BGP functionality", Ordered, func() {
var (
client kubernetes.Interface
clusterName string
tempDirPath string
)
BeforeAll(func() {
networking := &kindconfigv1alpha4.Networking{
IPFamily: kindconfigv1alpha4.IPv4Family,
}
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareClusterForDS(ctx, tempDirPath, "bgp-ds", imagePath, k8sImagePath,
logger, networking, 1, nil, 1)
})
AfterAll(func() {
Eventually(func() error {
return e2e.GetLogs(ctx, client, tempDirPath)
}, "60s", "5s").Should(Succeed())
cleanupCluster(clusterName, ConfigMtx, logger)
})
It(clusterName+" exits gracefully when only control plane is enabled", func() {
manifestValues := &e2e.KubevipManifestValues{
Mode: Mode,
ControlPlaneEnable: "true",
VipElectionEnable: "false",
ImagePath: imagePath,
ConfigPath: configPath,
SvcEnable: "false",
SvcElectionEnable: "false",
EnableEndpointslices: "true",
EnableNodeLabeling: "false",
BGPPeers: "127.0.0.1:1::false",
BGPAS: 2,
}
testDS(ctx, manifestValues, client, utils.IPv4Family, clusterName)
})
It(clusterName+" exits gracefully when only control plane is enabled with leader election", func() {
manifestValues := &e2e.KubevipManifestValues{
Mode: Mode,
ControlPlaneEnable: "true",
VipElectionEnable: "true",
ImagePath: imagePath,
ConfigPath: configPath,
SvcEnable: "false",
SvcElectionEnable: "false",
EnableEndpointslices: "true",
EnableNodeLabeling: "false",
BGPPeers: "127.0.0.1:1::false",
BGPAS: 2,
}
testDS(ctx, manifestValues, client, utils.IPv4Family, clusterName)
})
It(clusterName+" exits gracefully when only services are enabled", func() {
manifestValues := &e2e.KubevipManifestValues{
Mode: Mode,
ControlPlaneEnable: "false",
VipElectionEnable: "false",
ImagePath: imagePath,
ConfigPath: configPath,
SvcEnable: "true",
SvcElectionEnable: "false",
EnableEndpointslices: "true",
EnableNodeLabeling: "false",
BGPPeers: "127.0.0.1:1::false",
BGPAS: 2,
}
testDS(ctx, manifestValues, client, utils.IPv4Family, clusterName)
})
It(clusterName+" exits gracefully when only services are enabled with leader election", func() {
manifestValues := &e2e.KubevipManifestValues{
Mode: Mode,
ControlPlaneEnable: "false",
VipElectionEnable: "false",
ImagePath: imagePath,
ConfigPath: configPath,
SvcEnable: "true",
SvcElectionEnable: "true",
EnableEndpointslices: "true",
EnableNodeLabeling: "false",
BGPPeers: "127.0.0.1:1::false",
BGPAS: 2,
}
testDS(ctx, manifestValues, client, utils.IPv4Family, clusterName)
})
It(clusterName+" exits gracefully when control plane services are enabled", func() {
manifestValues := &e2e.KubevipManifestValues{
Mode: Mode,
ControlPlaneEnable: "true",
VipElectionEnable: "false",
ImagePath: imagePath,
ConfigPath: configPath,
SvcEnable: "true",
SvcElectionEnable: "false",
EnableEndpointslices: "true",
EnableNodeLabeling: "false",
BGPPeers: "127.0.0.1:1::false",
BGPAS: 2,
}
testDS(ctx, manifestValues, client, utils.IPv4Family, clusterName)
})
It(clusterName+" exits gracefully when control plane w/ leader election and services w/o leader election are enabled", func() {
manifestValues := &e2e.KubevipManifestValues{
Mode: Mode,
ControlPlaneEnable: "true",
VipElectionEnable: "true",
ImagePath: imagePath,
ConfigPath: configPath,
SvcEnable: "true",
SvcElectionEnable: "false",
EnableEndpointslices: "true",
EnableNodeLabeling: "false",
BGPPeers: "127.0.0.1:1::false",
BGPAS: 2,
}
testDS(ctx, manifestValues, client, utils.IPv4Family, clusterName)
})
It(clusterName+" exits gracefully when control plane w/o leader election and services w/ leader election are enabled", func() {
manifestValues := &e2e.KubevipManifestValues{
Mode: Mode,
ControlPlaneEnable: "true",
VipElectionEnable: "false",
ImagePath: imagePath,
ConfigPath: configPath,
SvcEnable: "true",
SvcElectionEnable: "true",
EnableEndpointslices: "true",
EnableNodeLabeling: "false",
BGPPeers: "127.0.0.1:1::false",
BGPAS: 2,
}
testDS(ctx, manifestValues, client, utils.IPv4Family, clusterName)
})
It(clusterName+" exits gracefully when control plane and services w/ leader election are enabled", func() {
manifestValues := &e2e.KubevipManifestValues{
Mode: Mode,
ControlPlaneEnable: "true",
VipElectionEnable: "true",
ImagePath: imagePath,
ConfigPath: configPath,
SvcEnable: "true",
SvcElectionEnable: "true",
EnableEndpointslices: "true",
EnableNodeLabeling: "false",
BGPPeers: "127.0.0.1:1::false",
BGPAS: 2,
}
testDS(ctx, manifestValues, client, utils.IPv4Family, clusterName)
})
})
}
})

Some files were not shown because too many files have changed in this diff Show More