Compare commits

...

50 Commits

Author SHA1 Message Date
Cellebyte
8409073e7a Release v1.0.1 2025-10-05 22:56:55 +02:00
dependabot[bot]
6d419f32bc Bump anchore/sbom-action from 0.20.5 to 0.20.6
Bumps [anchore/sbom-action](https://github.com/anchore/sbom-action) from 0.20.5 to 0.20.6.
- [Release notes](https://github.com/anchore/sbom-action/releases)
- [Changelog](https://github.com/anchore/sbom-action/blob/main/RELEASE.md)
- [Commits](https://github.com/anchore/sbom-action/compare/v0.20.5...v0.20.6)

---
updated-dependencies:
- dependency-name: anchore/sbom-action
  dependency-version: 0.20.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-10-05 22:50:14 +02:00
dependabot[bot]
9d68054e9a Bump github.com/spf13/cobra from 1.9.1 to 1.10.1
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 1.9.1 to 1.10.1.
- [Release notes](https://github.com/spf13/cobra/releases)
- [Commits](https://github.com/spf13/cobra/compare/v1.9.1...v1.10.1)

---
updated-dependencies:
- dependency-name: github.com/spf13/cobra
  dependency-version: 1.10.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-10-05 22:49:53 +02:00
dependabot[bot]
5dcfb8742f Bump go.etcd.io/etcd/client/v3 from 3.6.4 to 3.6.5
Bumps [go.etcd.io/etcd/client/v3](https://github.com/etcd-io/etcd) from 3.6.4 to 3.6.5.
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.6.4...v3.6.5)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/client/v3
  dependency-version: 3.6.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-10-05 22:49:00 +02:00
dependabot[bot]
7e6f70b027 Bump github.com/gookit/slog from 0.5.8 to 0.6.0
Bumps [github.com/gookit/slog](https://github.com/gookit/slog) from 0.5.8 to 0.6.0.
- [Release notes](https://github.com/gookit/slog/releases)
- [Commits](https://github.com/gookit/slog/compare/v0.5.8...v0.6.0)

---
updated-dependencies:
- dependency-name: github.com/gookit/slog
  dependency-version: 0.6.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-10-05 22:48:47 +02:00
dependabot[bot]
3e10aa85d0 Bump google.golang.org/grpc from 1.75.0 to 1.75.1
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.75.0 to 1.75.1.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.75.0...v1.75.1)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.75.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-10-05 22:48:25 +02:00
Marcel Fest
16b9f6767e refactored BGP configs, kubevip pkg and more.
* Moved functions from mod pkg vip to pkg utils
* remove the dependency of the kubevip config pkg on the bgp pkg
* introduce BGPConfig and BGPPeer to kubevip package and migrate label from node labeler
* Use the new BGPConfig and BGPPeer type of the kubevip pkg
* Removed utils functions from vip pkg and refactor to call utils instead
* migrate from pkg vip to utils for generic IP funcs
* migrate common annotations and labels into a central place + support for hostnames
* if any change happens we want to reconcile it here
* fix e2e tests which relied also on the vip package instead of utils
* Better debug logging
* Added info if it is egress
* added missing condition to dhcp specific actions
* Disable the service before cancelling the context
* lets try auto,auto instead of /32 and nothing
* Let's reset svcCtx to be nil to ensure a new context when the previous was garbage collected
* Added a space for project wide constants
---------

Signed-off-by: Cellebyte <marcel.fest@live.de>
2025-10-05 22:47:21 +02:00
Alex Krasnov
d8a9727ff5 extend existing e2e tests with node labeling switch (disabled as is)
Signed-off-by: Alex Krasnov <askrasnov@gmail.com>
2025-09-19 12:20:08 +02:00
Alex Krasnov
016a899e60 refactor to interface and separate NoOP node labeler implementation
Signed-off-by: Alex Krasnov <askrasnov@gmail.com>
2025-09-19 12:20:08 +02:00
Alex Krasnov
68b39a83e0 skip node labeling when control-plane mode enabled
Signed-off-by: Alex Krasnov <askrasnov@gmail.com>
2025-09-19 12:20:08 +02:00
Alex Krasnov
93dabff000 fix case with nil clientset (eg, with etcd leader election)
Signed-off-by: Alex Krasnov <askrasnov@gmail.com>
2025-09-19 12:20:08 +02:00
Alex Krasnov
630be48010 add node labels for services
Signed-off-by: Alex Krasnov <askrasnov@gmail.com>
2025-09-19 12:20:08 +02:00
Cellebyte
ea78d291ce use slices.Compact correctly
Signed-off-by: Cellebyte <marcel.fest@live.de>
2025-09-18 17:41:57 +02:00
Cellebyte
2074be2939 use the new vip.LookupHost for manifest creation
Signed-off-by: Cellebyte <marcel.fest@live.de>
2025-09-18 17:41:57 +02:00
dependabot[bot]
7946c17c00 Bump github.com/docker/docker
Bumps [github.com/docker/docker](https://github.com/docker/docker) from 28.3.3+incompatible to 28.4.0+incompatible.
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](https://github.com/docker/docker/compare/v28.3.3...v28.4.0)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-version: 28.4.0+incompatible
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-09-15 18:21:12 +02:00
dependabot[bot]
a0295d6a2f Bump golang.org/x/sync from 0.16.0 to 0.17.0
Bumps [golang.org/x/sync](https://github.com/golang/sync) from 0.16.0 to 0.17.0.
- [Commits](https://github.com/golang/sync/compare/v0.16.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sync
  dependency-version: 0.17.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-09-15 18:20:47 +02:00
dependabot[bot]
d70068b1a0 Bump github.com/onsi/ginkgo/v2 from 2.23.4 to 2.25.3
Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.23.4 to 2.25.3.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/ginkgo/compare/v2.23.4...v2.25.3)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-version: 2.25.3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-09-15 18:20:30 +02:00
dependabot[bot]
89baba07f5 Bump k8s.io/client-go from 0.33.3 to 0.34.1
Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.33.3 to 0.34.1.
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kubernetes/client-go/compare/v0.33.3...v0.34.1)

---
updated-dependencies:
- dependency-name: k8s.io/client-go
  dependency-version: 0.34.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-09-15 18:20:14 +02:00
Patryk Strusiewicz-Surmacki
95995500bc Added common lease for multiple services for all modes and service election for BGP
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2025-09-15 18:19:50 +02:00
Thorsten Bonhagen
b1183e8a93 fix unit test error
Signed-off-by: Thorsten Bonhagen <Thorsten.Bonhagen@tbon.de>
2025-09-15 15:16:03 +02:00
Thorsten Bonhagen
101f722110 rebase
Signed-off-by: Thorsten Bonhagen <Thorsten.Bonhagen@tbon.de>
2025-09-15 15:16:03 +02:00
Thorsten Bonhagen
3f390120c1 remove condition for adding IP in service
orig was not working and inverse condition might not work for all

Signed-off-by: Thorsten Bonhagen <Thorsten.Bonhagen@tbon.de>
2025-09-15 15:16:03 +02:00
Thorsten Bonhagen
7baa8a3141 reformat with go fmt
Signed-off-by: Thorsten Bonhagen <Thorsten.Bonhagen@tbon.de>
2025-09-15 15:16:03 +02:00
Thorsten Bonhagen
6435581674 fix cp and svc not running on same instance
fix table addIP missing
add more debug lines
start service listener before content plane because content plane is blocking main thread

Signed-off-by: Thorsten Bonhagen <Thorsten.Bonhagen@tbon.de>
2025-09-15 15:16:03 +02:00
dependabot[bot]
7eb730c0ef Bump github.com/prometheus/client_golang from 1.22.0 to 1.23.2
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.22.0 to 1.23.2.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.22.0...v1.23.2)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-version: 1.23.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-09-11 23:25:41 +02:00
dependabot[bot]
f6a7aeb130 Bump docker/metadata-action from 5.7.0 to 5.8.0
Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 5.7.0 to 5.8.0.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Commits](https://github.com/docker/metadata-action/compare/v5.7.0...v5.8.0)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-version: 5.8.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-09-11 23:25:32 +02:00
dependabot[bot]
29296a9dc2 Bump actions/setup-go from 5 to 6
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5 to 6.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-09-11 22:56:33 +02:00
dependabot[bot]
4d4a2f0ee1 Bump golang.org/x/sys from 0.33.0 to 0.36.0
Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.33.0 to 0.36.0.
- [Commits](https://github.com/golang/sys/compare/v0.33.0...v0.36.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-version: 0.36.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-09-11 22:56:08 +02:00
dependabot[bot]
e4e398bfcf Bump golang from 1.24.6-alpine3.22 to 1.25.1-alpine3.22
Bumps golang from 1.24.6-alpine3.22 to 1.25.1-alpine3.22.

---
updated-dependencies:
- dependency-name: golang
  dependency-version: 1.25.1-alpine3.22
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-09-11 22:55:33 +02:00
dependabot[bot]
8bd2c26a8f Bump actions/checkout from 4 to 5
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 5.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-09-11 22:54:59 +02:00
dependabot[bot]
bd8f30d67d Bump google.golang.org/grpc from 1.73.0 to 1.75.0
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.73.0 to 1.75.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.73.0...v1.75.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.75.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-09-11 22:54:30 +02:00
dependabot[bot]
a3a429b2b9 Bump anchore/sbom-action from 0.20.4 to 0.20.5
Bumps [anchore/sbom-action](https://github.com/anchore/sbom-action) from 0.20.4 to 0.20.5.
- [Release notes](https://github.com/anchore/sbom-action/releases)
- [Changelog](https://github.com/anchore/sbom-action/blob/main/RELEASE.md)
- [Commits](https://github.com/anchore/sbom-action/compare/v0.20.4...v0.20.5)

---
updated-dependencies:
- dependency-name: anchore/sbom-action
  dependency-version: 0.20.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-09-11 22:54:16 +02:00
dependabot[bot]
4f7ce8a1c8 Bump go.etcd.io/etcd/client/v3 from 3.6.1 to 3.6.4
Bumps [go.etcd.io/etcd/client/v3](https://github.com/etcd-io/etcd) from 3.6.1 to 3.6.4.
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.6.1...v3.6.4)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/client/v3
  dependency-version: 3.6.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-09-11 22:54:01 +02:00
Cellebyte
d2ecf22edd fix images in the kind clusters when manifest is used
Signed-off-by: Cellebyte <marcel.fest@live.de>
2025-09-04 18:54:57 +02:00
Abhishek Pareek
3963172e49 rename var to conform to standards
Signed-off-by: Abhishek Pareek <makeittotop@users.noreply.github.com>
Signed-off-by: Cellebyte <marcel.fest@live.de>
2025-09-04 18:54:57 +02:00
Abhishek Pareek
80c6b0bde4 substitute local image into the ds
Signed-off-by: Abhishek Pareek <makeittotop@users.noreply.github.com>
Signed-off-by: Cellebyte <marcel.fest@live.de>
2025-09-04 18:54:57 +02:00
Abhishek Pareek
4f59df38f3 use a lease Identifier to uniquely identify a svc lease
Signed-off-by: Abhishek Pareek <makeittotop@users.noreply.github.com>
Signed-off-by: Cellebyte <marcel.fest@live.de>
2025-09-04 18:54:57 +02:00
Abhishek Pareek
1a3e6c9d5f uniquely identify a service lease combining namespace + service name
Signed-off-by: Abhishek Pareek <makeittotop@users.noreply.github.com>
Signed-off-by: Cellebyte <marcel.fest@live.de>
2025-09-04 18:54:57 +02:00
Cellebyte
0635ec9e01 Fix linting issues 2025-09-03 22:03:51 +02:00
Yuval Weisz
3fff60a64a fix: Update file permissions in tests to 0600 for security compliance
- Change os.WriteFile permissions from 0644 to 0600 in config_file_test.go
- Addresses GoSec G306 linter requirement for secure file permissions
- Ensures test files are readable/writable by owner only

Signed-off-by: Yuval Weisz <yuval.weisz@vastdata.com>
2025-09-03 15:54:17 +02:00
Yuval Weisz
f05f0469cc test: Add comprehensive tests for configuration file functionality
This commit adds extensive unit tests for the new configuration file features:

Test Coverage:
- LoadConfigFromFile() function with YAML and JSON parsing
- MergeConfigFromFile() function with priority logic testing
- Configuration merging with complex nested structures (BGP, leader election)
- Error handling for invalid files, unsupported formats, and missing files
- Environment variable integration for config_file
- Priority validation: CLI flags > env vars > config file
- Boolean flag merging behavior
- BGP configuration merging
- Leader election configuration merging

Files:
- pkg/kubevip/config_file_test.go: New comprehensive test suite (550+ lines)
- pkg/kubevip/config_generator_test.go: Enhanced with config_file env var tests

Test Structure:
- TestLoadConfigFromFile: Tests JSON/YAML parsing and error cases
- TestMergeConfigFromFile: Tests configuration priority and merging
- TestMergeConfigValues: Tests detailed field-by-field merging logic
- TestParseEnvironmentConfigFile: Tests config_file environment variable

The tests validate all aspects of the configuration file implementation
including edge cases, error scenarios, and complex configuration structures.
All tests follow Go testing best practices with table-driven test patterns.

Signed-off-by: Yuval Weisz <yuval.weisz@vastdata.com>
2025-09-03 15:54:17 +02:00
Yuval Weisz
00337a756b feat: Add --config-file flag and config_file environment variable
This commit implements configuration file support for kube-vip, allowing
users to specify configuration through JSON or YAML files.

Features:
- Added --config-file command line flag that accepts JSON/YAML file paths
- Added config_file environment variable support
- Implemented proper configuration priority: CLI flags > env vars > config file
- Added comprehensive configuration merging for all fields including BGP,
  leader election, routing table, and load balancer settings
- Supports both JSON and YAML file formats with automatic detection
- Maintains full backward compatibility with existing deployments

Changes:
- pkg/kubevip/config_envvar.go: Added configFile constant
- pkg/kubevip/config_types.go: Added ConfigFile field to Config struct
- pkg/kubevip/config_environment.go: Added LoadConfigFromFile() and
  MergeConfigFromFile() functions with comprehensive merging logic
- cmd/kube-vip.go: Integrated config file loading into service and manager commands

Usage:
  kube-vip manager --config-file /etc/kube-vip/config.yaml
  export config_file=/etc/kube-vip/config.json && kube-vip manager

This enhancement significantly improves configuration management for complex
kube-vip deployments while maintaining the existing behavior and patterns.

Signed-off-by: Yuval Weisz <yuval.weisz@vastdata.com>
2025-09-03 15:54:17 +02:00
Thorsten Bonhagen
d3473b5d68 change startup order of ControlPlane and Services
Fix: ControlPlane was blocking main thread so Services could not start in parallel
Signed-off-by: Thorsten Bonhagen <thbo@uhurutec.com>
2025-09-03 15:53:58 +02:00
Patryk Strusiewicz-Surmacki
889d442288 Fixed nil pointer reference when endpoints serving condition is not set
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2025-09-03 15:53:30 +02:00
Daniel Finneran
b2c04c9058 Merge pull request #1240 from hensur/check-snat-error
check conn.Flush error in nftables SNAT
2025-09-01 15:20:38 +01:00
Mark Riedel
0889ebed7d Fix deletion of ConfiguredNetworks entry after deleting BGP host
Signed-off-by: Mark Riedel <5668435+rmarkdev@users.noreply.github.com>
2025-08-25 17:04:10 +02:00
Henning Surmeier
d1430e79e2 check conn.Flush error in nftables SNAT
Signed-off-by: Henning Surmeier <h.surmeier@mittwald.de>
2025-08-18 14:28:03 +02:00
Patryk Strusiewicz-Surmacki
31eca367ab Added logs as CI artifact
Signed-off-by: Patryk Strusiewicz-Surmacki <patryk.pawel.strusiewicz-surmacki@external.telekom.de>
2025-08-15 07:50:51 +02:00
Dan Finneran
bdd353d0fd Unintentionally created duplicate functions that can't be used
Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>
2025-08-14 09:55:16 +02:00
dependabot[bot]
4deb0592f6 Bump golang from 1.24.5-alpine3.22 to 1.24.6-alpine3.22
Bumps golang from 1.24.5-alpine3.22 to 1.24.6-alpine3.22.

---
updated-dependencies:
- dependency-name: golang
  dependency-version: 1.24.6-alpine3.22
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-08-14 09:54:24 +02:00
77 changed files with 4290 additions and 1083 deletions

View File

@@ -22,10 +22,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@v5
with:
ref: ${{ github.ref_name }}
- name: Anchore SBOM Action
uses: anchore/sbom-action@v0.20.4
uses: anchore/sbom-action@v0.20.6
with:
format: cyclonedx-json

View File

@@ -9,14 +9,16 @@ jobs:
matrix:
mode: ["arp", "rt", "bgp"]
fail-fast: true
max-parallel: 1
steps:
- name: Get current date
id: date
run: echo "::set-output name=date::$(date +'%Y-%m-%d-%H-%M')"
- name: Ensure fs wont cause issues
run: sudo sysctl fs.inotify.max_user_instances=8192 && sudo sysctl fs.inotify.max_user_watches=524288
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v5
- name: Install Go
uses: actions/setup-go@v5
uses: actions/setup-go@v6
with:
go-version-file: go.mod
- name: Build image locally
@@ -24,28 +26,46 @@ jobs:
- name: Run Manifest generation tests
run: make manifest-test
- name: Run ARP mode tests v1.29.0 onwards
run: make e2e-tests129-arp
run: E2E_KEEP_LOGS=true make e2e-tests129-arp
if: matrix.mode== 'arp'
- name: Run RT mode tests v1.29.0 onwards
run: make e2e-tests129-rt
run: E2E_KEEP_LOGS=true make e2e-tests129-rt
if: matrix.mode== 'rt'
- name: Get GoBGP binaries
run: make get-gobgp
if: matrix.mode== 'bgp'
- name: Run BGP mode tests v1.29.0 onwards
run: sudo -E PATH=$PATH DOCKER_API_VERSION=1.48 make e2e-tests129-bgp
run: sudo -E PATH=$PATH DOCKER_API_VERSION=1.48 E2E_KEEP_LOGS=true make e2e-tests129-bgp
if: matrix.mode== 'bgp'
- name: Change log directory permissions
run: sudo chmod -R 755 /tmp/kube-vip-test-${{ matrix.mode }}*
if: matrix.mode== 'bgp' && always()
- name: Save logs
uses: actions/upload-artifact@v4
with:
name: e2e-test-logs-${{ matrix.mode }}-${{ steps.date.outputs.date }}
path: /tmp/kube-vip-test-${{ matrix.mode }}*
if: always()
service-e2e-tests:
runs-on: ubuntu-latest
name: E2E service tests
steps:
- name: Get current date
id: date
run: echo "::set-output name=date::$(date +'%Y-%m-%d-%H-%M')"
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v5
- name: Install Go
uses: actions/setup-go@v5
uses: actions/setup-go@v6
with:
go-version-file: go.mod
- name: Build image with iptables
run: make dockerx86ActionIPTables
- name: Run tests
run: DOCKERTAG=action make service-tests
run: DOCKERTAG=action E2E_KEEP_LOGS=true make service-tests
- name: Save logs
uses: actions/upload-artifact@v4
with:
name: services-test-logs-${{ steps.date.outputs.date }}
path: /tmp/kube-vip-service-tests*
if: always()

View File

@@ -12,9 +12,9 @@ jobs:
- name: Install golangci-lint
run: curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.64.8
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v5
- name: Install Go
uses: actions/setup-go@v5
uses: actions/setup-go@v6
with:
go-version-file: go.mod
- name: All checks
@@ -24,9 +24,9 @@ jobs:
name: Unit tests
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v5
- name: Install Go
uses: actions/setup-go@v5
uses: actions/setup-go@v6
with:
go-version-file: go.mod
- name: Run tests
@@ -36,9 +36,9 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v5
- name: Install Go
uses: actions/setup-go@v5
uses: actions/setup-go@v6
with:
go-version-file: go.mod
- name: Run tests
@@ -48,7 +48,7 @@ jobs:
name: Image vulnerability scan
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v5
- name: Build image with iptables
run: make dockerx86ActionIPTables
- name: Run Trivy vulnerability scanner

View File

@@ -38,10 +38,10 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v5
- name: Install Go
uses: actions/setup-go@v5
uses: actions/setup-go@v6
with:
go-version-file: go.mod

View File

@@ -11,7 +11,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@v5
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx

View File

@@ -11,7 +11,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@v5
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
@@ -28,7 +28,7 @@ jobs:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Generate Metadata
uses: docker/metadata-action@v5.7.0
uses: docker/metadata-action@v5.8.0
id: metadata
with:
labels: |

View File

@@ -1,6 +1,6 @@
# syntax=docker/dockerfile:experimental
FROM golang:1.24.5-alpine3.22 as dev
FROM golang:1.25.1-alpine3.22 as dev
RUN apk add --no-cache git ca-certificates make
RUN adduser -D appuser
COPY . /src/

View File

@@ -1,6 +1,6 @@
# syntax=docker/dockerfile:experimental
FROM golang:1.24.5-alpine3.22 as dev
FROM golang:1.25.1-alpine3.22 as dev
RUN apk add --no-cache git make
RUN adduser -D appuser
COPY . /src/

View File

@@ -5,7 +5,7 @@ TARGET := kube-vip
.DEFAULT_GOAL := $(TARGET)
# These will be provided to the target
VERSION := v1.0.0
VERSION := v1.0.1
BUILD := `git rev-parse HEAD`
@@ -15,7 +15,7 @@ TARGETOS=linux
# Use linker flags to provide version/build settings to the target
LDFLAGS=-ldflags "-s -w -X=main.Version=$(VERSION) -X=main.Build=$(BUILD) -extldflags -static"
DOCKERTAG ?= $(VERSION)
REPOSITORY ?= plndr
REPOSITORY ?= docker.io/plndr
.PHONY: all build clean install uninstall simplify check run e2e-tests
@@ -109,22 +109,22 @@ run: install
manifests:
@make build
@mkdir -p ./docs/manifests/$(VERSION)/
@./kube-vip manifest pod --interface eth0 --vip 192.168.0.1 --arp --leaderElection --controlplane --services > ./docs/manifests/$(VERSION)/kube-vip-arp.yaml
@./kube-vip manifest pod --interface eth0 --vip 192.168.0.1 --arp --leaderElection --controlplane --services --enableLoadBalancer > ./docs/manifests/$(VERSION)/kube-vip-arp-lb.yaml
@./kube-vip manifest pod --interface eth0 --vip 192.168.0.1 --bgp --controlplane --services > ./docs/manifests/$(VERSION)/kube-vip-bgp.yaml
@./kube-vip manifest daemonset --interface eth0 --vip 192.168.0.1 --arp --leaderElection --controlplane --services --inCluster > ./docs/manifests/$(VERSION)/kube-vip-arp-ds.yaml
@./kube-vip manifest daemonset --interface eth0 --vip 192.168.0.1 --arp --leaderElection --controlplane --services --inCluster --enableLoadBalancer > ./docs/manifests/$(VERSION)/kube-vip-arp-ds-lb.yaml
@./kube-vip manifest daemonset --interface eth0 --vip 192.168.0.1 --bgp --leaderElection --controlplane --services --inCluster > ./docs/manifests/$(VERSION)/kube-vip-bgp-ds.yaml
@./kube-vip manifest daemonset --interface eth0 --vip 192.168.0.1 --bgp --leaderElection --controlplane --services --inCluster > ./docs/manifests/$(VERSION)/kube-vip-bgp-em-ds.yaml
@./kube-vip manifest pod --interface eth0 --vip 192.168.0.1 --image "$(REPOSITORY)/$(TARGET):$(DOCKERTAG)" --arp --leaderElection --controlplane --services > ./docs/manifests/$(VERSION)/kube-vip-arp.yaml
@./kube-vip manifest pod --interface eth0 --vip 192.168.0.1 --image "$(REPOSITORY)/$(TARGET):$(DOCKERTAG)" --arp --leaderElection --controlplane --services --enableLoadBalancer > ./docs/manifests/$(VERSION)/kube-vip-arp-lb.yaml
@./kube-vip manifest pod --interface eth0 --vip 192.168.0.1 --image "$(REPOSITORY)/$(TARGET):$(DOCKERTAG)" --bgp --controlplane --services > ./docs/manifests/$(VERSION)/kube-vip-bgp.yaml
@./kube-vip manifest daemonset --interface eth0 --vip 192.168.0.1 --image "$(REPOSITORY)/$(TARGET):$(DOCKERTAG)" --arp --leaderElection --controlplane --services --inCluster > ./docs/manifests/$(VERSION)/kube-vip-arp-ds.yaml
@./kube-vip manifest daemonset --interface eth0 --vip 192.168.0.1 --image "$(REPOSITORY)/$(TARGET):$(DOCKERTAG)" --arp --leaderElection --controlplane --services --inCluster --enableLoadBalancer > ./docs/manifests/$(VERSION)/kube-vip-arp-ds-lb.yaml
@./kube-vip manifest daemonset --interface eth0 --vip 192.168.0.1 --image "$(REPOSITORY)/$(TARGET):$(DOCKERTAG)" --bgp --leaderElection --controlplane --services --inCluster > ./docs/manifests/$(VERSION)/kube-vip-bgp-ds.yaml
@./kube-vip manifest daemonset --interface eth0 --vip 192.168.0.1 --image "$(REPOSITORY)/$(TARGET):$(DOCKERTAG)" --bgp --leaderElection --controlplane --services --inCluster > ./docs/manifests/$(VERSION)/kube-vip-bgp-em-ds.yaml
@-rm ./kube-vip
manifest-test:
docker run $(REPOSITORY)/$(TARGET):$(DOCKERTAG) manifest pod --interface eth0 --vip 192.168.0.1 --arp --leaderElection --controlplane --services
docker run $(REPOSITORY)/$(TARGET):$(DOCKERTAG) manifest pod --interface eth0 --vip 192.168.0.1 --arp --leaderElection --controlplane --services --enableLoadBalancer
docker run $(REPOSITORY)/$(TARGET):$(DOCKERTAG) manifest pod --interface eth0 --vip 192.168.0.1 --bgp --controlplane --services
docker run $(REPOSITORY)/$(TARGET):$(DOCKERTAG) manifest daemonset --interface eth0 --vip 192.168.0.1 --arp --leaderElection --controlplane --services --inCluster
docker run $(REPOSITORY)/$(TARGET):$(DOCKERTAG) manifest daemonset --interface eth0 --vip 192.168.0.1 --arp --leaderElection --controlplane --services --inCluster --enableLoadBalancer
docker run $(REPOSITORY)/$(TARGET):$(DOCKERTAG) manifest daemonset --interface eth0 --vip 192.168.0.1 --bgp --leaderElection --controlplane --services --inCluster
docker run $(REPOSITORY)/$(TARGET):$(DOCKERTAG) manifest pod --interface eth0 --vip 192.168.0.1 --image "$(REPOSITORY)/$(TARGET):$(DOCKERTAG)" --arp --leaderElection --controlplane --services
docker run $(REPOSITORY)/$(TARGET):$(DOCKERTAG) manifest pod --interface eth0 --vip 192.168.0.1 --image "$(REPOSITORY)/$(TARGET):$(DOCKERTAG)" --arp --leaderElection --controlplane --services --enableLoadBalancer
docker run $(REPOSITORY)/$(TARGET):$(DOCKERTAG) manifest pod --interface eth0 --vip 192.168.0.1 --image "$(REPOSITORY)/$(TARGET):$(DOCKERTAG)" --bgp --controlplane --services
docker run $(REPOSITORY)/$(TARGET):$(DOCKERTAG) manifest daemonset --interface eth0 --vip 192.168.0.1 --image "$(REPOSITORY)/$(TARGET):$(DOCKERTAG)" --arp --leaderElection --controlplane --services --inCluster
docker run $(REPOSITORY)/$(TARGET):$(DOCKERTAG) manifest daemonset --interface eth0 --vip 192.168.0.1 --image "$(REPOSITORY)/$(TARGET):$(DOCKERTAG)" --arp --leaderElection --controlplane --services --inCluster --enableLoadBalancer
docker run $(REPOSITORY)/$(TARGET):$(DOCKERTAG) manifest daemonset --interface eth0 --vip 192.168.0.1 --image "$(REPOSITORY)/$(TARGET):$(DOCKERTAG)" --bgp --leaderElection --controlplane --services --inCluster
unit-tests:
go test ./...
@@ -166,7 +166,7 @@ kind-quick:
kubectl create configmap --namespace kube-system kubevip --from-literal range-global=172.18.100.10-172.18.100.30
kubectl apply -f https://raw.githubusercontent.com/kube-vip/kube-vip-cloud-provider/main/manifest/kube-vip-cloud-controller.yaml
kind load docker-image --name kube-vip $(REPOSITORY)/$(TARGET):$(DOCKERTAG)
docker run --network host --rm $(REPOSITORY)/$(TARGET):$(DOCKERTAG) manifest daemonset --services --inCluster --arp --servicesElection --interface eth0 | kubectl apply -f -
docker run --network host --rm $(REPOSITORY)/$(TARGET):$(DOCKERTAG) manifest daemonset --services --inCluster --image "$(REPOSITORY)/$(TARGET):$(DOCKERTAG)" --arp --servicesElection --interface eth0 | kubectl apply -f -
kind-reload:
kind load docker-image $(REPOSITORY)/$(TARGET):$(DOCKERTAG) --name kube-vip

View File

@@ -58,14 +58,14 @@ var kubeKubeadmInit = &cobra.Command{
// Ensure there is an address to generate the CIDR from
if initConfig.VIPSubnet == "" && initConfig.Address != "" {
initConfig.VIPSubnet, err = GenerateCidrRange(initConfig.Address)
initConfig.VIPSubnet, err = GenerateCidrRange(initConfig.Address, initConfig.DNSMode)
if err != nil {
log.Error("generating VIPSubnet", "err", err)
return
}
}
cfg := kubevip.GeneratePodManifestFromConfig(&initConfig, Release.Version, inCluster)
cfg := kubevip.GeneratePodManifestFromConfig(&initConfig, image, Release.Version, inCluster)
fmt.Println(cfg) // output manifest to stdout
},
}
@@ -103,14 +103,14 @@ var kubeKubeadmJoin = &cobra.Command{
// Ensure there is an address to generate the CIDR from
if initConfig.VIPSubnet == "" && initConfig.Address != "" {
initConfig.VIPSubnet, err = GenerateCidrRange(initConfig.Address)
initConfig.VIPSubnet, err = GenerateCidrRange(initConfig.Address, initConfig.DNSMode)
if err != nil {
log.Error("generating VIPSubnet", "err", err)
return
}
}
cfg := kubevip.GeneratePodManifestFromConfig(&initConfig, Release.Version, inCluster)
cfg := kubevip.GeneratePodManifestFromConfig(&initConfig, image, Release.Version, inCluster)
fmt.Println(cfg) // output manifest to stdout
},
}

View File

@@ -19,6 +19,7 @@ var taint, role, rolebinding bool
func init() {
kubeManifest.PersistentFlags().BoolVar(&inCluster, "inCluster", false, "Use the incluster token to authenticate to Kubernetes")
kubeManifest.PersistentFlags().StringVar(&image, "image", "ghcr.io/kube-vip/kube-vip", "Define a hardcoded image with or without tag for the manifest")
kubeManifestDaemon.PersistentFlags().BoolVar(&taint, "taint", false, "Taint the manifest for only running on control planes")
kubeManifestRbac.PersistentFlags().BoolVar(&role, "role", false, "Generate only a Role inside the serviceNamespace access")
kubeManifestRbac.PersistentFlags().BoolVar(&rolebinding, "rolebinding", false, "Generate only a RoleBinding for namespaced access")
@@ -59,14 +60,14 @@ var kubeManifestPod = &cobra.Command{
// Ensure there is an address to generate the CIDR from
if initConfig.VIPSubnet == "" && initConfig.Address != "" {
initConfig.VIPSubnet, err = GenerateCidrRange(initConfig.Address)
initConfig.VIPSubnet, err = GenerateCidrRange(initConfig.Address, initConfig.DNSMode)
if err != nil {
log.Error("config parse", "err", err)
return
}
}
cfg := kubevip.GeneratePodManifestFromConfig(&initConfig, Release.Version, inCluster)
cfg := kubevip.GeneratePodManifestFromConfig(&initConfig, image, Release.Version, inCluster)
fmt.Println(cfg) // output manifest to stdout
},
}
@@ -92,14 +93,14 @@ var kubeManifestDaemon = &cobra.Command{
// Ensure there is an address to generate the CIDR from
if initConfig.VIPSubnet == "" && initConfig.Address != "" {
initConfig.VIPSubnet, err = GenerateCidrRange(initConfig.Address)
initConfig.VIPSubnet, err = GenerateCidrRange(initConfig.Address, initConfig.DNSMode)
if err != nil {
log.Error("config parse", "err", err)
return
}
}
cfg := kubevip.GenerateDaemonsetManifestFromConfig(&initConfig, Release.Version, inCluster, taint)
cfg := kubevip.GenerateDaemonsetManifestFromConfig(&initConfig, image, Release.Version, inCluster, taint)
fmt.Println(cfg) // output manifest to stdout
},
}
@@ -126,7 +127,7 @@ var kubeManifestRbac = &cobra.Command{
// Ensure there is an address to generate the CIDR from
if initConfig.VIPSubnet == "" && initConfig.Address != "" {
initConfig.VIPSubnet, err = GenerateCidrRange(initConfig.Address)
initConfig.VIPSubnet, err = GenerateCidrRange(initConfig.Address, initConfig.DNSMode)
if err != nil {
log.Error("generating VIPSubnet", "err", err)
return

View File

@@ -6,6 +6,7 @@ import (
"net"
"net/http"
"os"
"slices"
"strings"
"time"
@@ -19,9 +20,13 @@ import (
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/manager"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/kube-vip/kube-vip/pkg/vip"
)
// Is an option to set the image
var image string
// Is kube-vip running within cluster
var inCluster bool
@@ -120,11 +125,11 @@ func init() {
// Extended behaviour flags
kubeVipCmd.PersistentFlags().BoolVar(&initConfig.EnableServicesElection, "servicesElection", false, "Enable leader election per kubernetes service")
kubeVipCmd.PersistentFlags().BoolVar(&initConfig.LoadBalancerClassOnly, "lbClassOnly", false, "Enable load balancing only for services with LoadBalancerClass \"kube-vip.io/kube-vip-class\"")
kubeVipCmd.PersistentFlags().StringVar(&initConfig.LoadBalancerClassName, "lbClassName", "kube-vip.io/kube-vip-class", "Name of load balancer class for kube-VIP, defaults to \"kube-vip.io/kube-vip-class\"")
kubeVipCmd.PersistentFlags().BoolVar(&initConfig.LoadBalancerClassOnly, "lbClassOnly", false, fmt.Sprintf("Enable load balancing only for services with LoadBalancerClass %q", kubevip.LBClassName))
kubeVipCmd.PersistentFlags().StringVar(&initConfig.LoadBalancerClassName, "lbClassName", kubevip.LBClassName, fmt.Sprintf("Name of load balancer class for kube-VIP, defaults to %q", kubevip.LBClassName))
kubeVipCmd.PersistentFlags().BoolVar(&initConfig.LoadBalancerClassLegacyHandling, "lbClassNameLegacyHandling", true, "Use legacy LoadBalancer class name handling (e.g. accepting services both with empty and non-empty class)")
kubeVipCmd.PersistentFlags().BoolVar(&initConfig.EnableServiceSecurity, "onlyAllowTrafficServicePorts", false, "Only allow traffic to service ports, others will be dropped, defaults to false")
kubeVipCmd.PersistentFlags().BoolVar(&initConfig.EnableNodeLabeling, "enableNodeLabeling", false, "Enable leader node labeling with \"kube-vip.io/has-ip=<VIP address>\", defaults to false")
kubeVipCmd.PersistentFlags().BoolVar(&initConfig.EnableNodeLabeling, "enableNodeLabeling", false, fmt.Sprintf("Enable leader node labeling with %q, defaults to false", kubevip.HasIP))
kubeVipCmd.PersistentFlags().StringVar(&initConfig.ServicesLeaseName, "servicesLeaseName", "plndr-svcs-lock", "Name of the lease that is used for leader election for services (in arp mode)")
kubeVipCmd.PersistentFlags().StringVar(&initConfig.DNSMode, "dnsMode", "first", "Name of the mode that DNS lookup will be performed (first, ipv4, ipv6, dual)")
kubeVipCmd.PersistentFlags().BoolVar(&initConfig.DisableServiceUpdates, "disableServiceUpdates", false, "If true, kube-vip will process services as usual, but will not update service's Status.LoadBalancer.Ingress slice")
@@ -144,6 +149,9 @@ func init() {
// Kubernetes client specific flags
kubeVipCmd.PersistentFlags().StringVar(&initConfig.K8sConfigFile, "k8sConfigPath", "/etc/kubernetes/admin.conf", "Path to the configuration file used with the Kubernetes client")
// Configuration file flag
kubeVipCmd.PersistentFlags().StringVar(&initConfig.ConfigFile, "config-file", "", "Path to a JSON/YAML configuration file to load settings from")
kubeVipCmd.AddCommand(kubeKubeadm)
kubeVipCmd.AddCommand(kubeManifest)
kubeVipCmd.AddCommand(kubeVipManager)
@@ -186,7 +194,16 @@ var kubeVipService = &cobra.Command{
Short: "Start the Virtual IP / Load balancer as a service within a Kubernetes cluster",
Run: func(cmd *cobra.Command, args []string) { //nolint TODO
// parse environment variables, these will overwrite anything loaded or flags
// Load configuration from file if specified (lowest priority)
if initConfig.ConfigFile != "" {
err := kubevip.MergeConfigFromFile(&initConfig, initConfig.ConfigFile)
if err != nil {
log.Error("loading config file", "err", err)
return
}
}
// parse environment variables, these will overwrite anything loaded from config file
err := kubevip.ParseEnvironment(&initConfig)
if err != nil {
log.Error("parsing env", "err", err)
@@ -214,7 +231,7 @@ var kubeVipService = &cobra.Command{
// Ensure there is an address to generate the CIDR from
if initConfig.VIPSubnet == "" && initConfig.Address != "" {
initConfig.VIPSubnet, err = GenerateCidrRange(initConfig.Address)
initConfig.VIPSubnet, err = GenerateCidrRange(initConfig.Address, initConfig.DNSMode)
if err != nil {
log.Error("generating CIDR", "err", err)
return
@@ -241,7 +258,16 @@ var kubeVipManager = &cobra.Command{
Use: "manager",
Short: "Start the kube-vip manager",
Run: func(cmd *cobra.Command, args []string) { //nolint TODO
// parse environment variables, these will overwrite anything loaded or flags
// Load configuration from file if specified (lowest priority)
if initConfig.ConfigFile != "" {
err := kubevip.MergeConfigFromFile(&initConfig, initConfig.ConfigFile)
if err != nil {
log.Error("loading config file", "err", err)
return
}
}
// parse environment variables, these will overwrite anything loaded from config file
err := kubevip.ParseEnvironment(&initConfig)
if err != nil {
log.Error("parsing environment", "err", err)
@@ -258,7 +284,7 @@ var kubeVipManager = &cobra.Command{
// Ensure there is an address to generate the CIDR from
if initConfig.VIPSubnet == "" && initConfig.Address != "" {
initConfig.VIPSubnet, err = GenerateCidrRange(initConfig.Address)
initConfig.VIPSubnet, err = GenerateCidrRange(initConfig.Address, initConfig.DNSMode)
if err != nil {
log.Error("No interface is specified for kube-vip to bind to")
return
@@ -448,26 +474,37 @@ func servePrometheusHTTPServer(ctx context.Context, config PrometheusHTTPServerC
}
}
func GenerateCidrRange(address string) (string, error) {
func GenerateCidrRange(address string, dnsMode string) (string, error) {
var cidrs []string
addresses := strings.Split(address, ",")
for _, a := range addresses {
ip := net.ParseIP(a)
if ip == nil {
ips, err := net.LookupIP(a)
// we probably are a DNS name
ips, err := utils.LookupHost(a, dnsMode)
if len(ips) == 0 || err != nil {
return "", fmt.Errorf("invalid IP address: %s from [%s], %v", a, address, err)
}
ip = ips[0]
}
if ip.To4() != nil {
cidrs = append(cidrs, "32")
for _, addr := range ips {
ip = net.ParseIP(addr)
if ip.To4() != nil {
cidrs = append(cidrs, "32")
} else {
cidrs = append(cidrs, "128")
}
}
} else {
cidrs = append(cidrs, "128")
if ip.To4() != nil {
cidrs = append(cidrs, "32")
} else {
cidrs = append(cidrs, "128")
}
}
}
// compact as DNS could have a lot of addresses
slices.Sort(cidrs)
cidrs = slices.Compact(cidrs)
slices.Reverse(cidrs)
return strings.Join(cidrs, ","), nil
}

93
go.mod
View File

@@ -5,51 +5,52 @@ go 1.24.4
require (
github.com/cloudflare/ipvs v0.11.0
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc
github.com/docker/docker v28.3.3+incompatible
github.com/docker/docker v28.4.0+incompatible
github.com/florianl/go-conntrack v0.4.0
github.com/google/go-cmp v0.7.0
github.com/google/go-containerregistry v0.20.6
github.com/google/nftables v0.3.0
github.com/gookit/slog v0.5.8
github.com/gookit/slog v0.6.0
github.com/huin/goupnp v1.3.0
github.com/insomniacslk/dhcp v0.0.0-20241224095048-b56fa0d5f25d
github.com/jpillora/backoff v1.0.0
github.com/mdlayher/ndp v1.1.0
github.com/onsi/ginkgo/v2 v2.23.4
github.com/onsi/gomega v1.37.0
github.com/onsi/ginkgo/v2 v2.25.3
github.com/onsi/gomega v1.38.2
github.com/osrg/gobgp/v3 v3.37.0
github.com/pkg/errors v0.9.1
github.com/prometheus/client_golang v1.22.0
github.com/prometheus/client_golang v1.23.2
github.com/sirupsen/logrus v1.9.3
github.com/spf13/cobra v1.9.1
github.com/stretchr/testify v1.10.0
github.com/spf13/cobra v1.10.1
github.com/stretchr/testify v1.11.1
github.com/vishvananda/netlink v1.3.1
go.etcd.io/etcd/api/v3 v3.6.1
go.etcd.io/etcd/client/pkg/v3 v3.6.1
go.etcd.io/etcd/client/v3 v3.6.1
go.etcd.io/etcd/api/v3 v3.6.5
go.etcd.io/etcd/client/pkg/v3 v3.6.5
go.etcd.io/etcd/client/v3 v3.6.5
go.uber.org/zap v1.27.0
golang.org/x/exp v0.0.0-20250103183323-7d7fa50e5329
golang.org/x/sync v0.15.0
golang.org/x/sys v0.33.0
golang.org/x/sync v0.17.0
golang.org/x/sys v0.36.0
golang.zx2c4.com/wireguard/wgctrl v0.0.0-20241231184526-a9ab2273dd10
google.golang.org/grpc v1.73.0
google.golang.org/protobuf v1.36.6
k8s.io/api v0.33.3
k8s.io/apimachinery v0.33.3
k8s.io/client-go v0.33.3
google.golang.org/grpc v1.75.1
google.golang.org/protobuf v1.36.8
k8s.io/api v0.34.1
k8s.io/apimachinery v0.34.1
k8s.io/client-go v0.34.1
k8s.io/klog/v2 v2.130.1
sigs.k8s.io/kind v0.29.0
sigs.k8s.io/yaml v1.4.0
sigs.k8s.io/yaml v1.6.0
)
require (
al.essio.dev/pkg/shellescape v1.5.1 // indirect
github.com/BurntSushi/toml v1.4.0 // indirect
github.com/Microsoft/go-winio v0.4.14 // indirect
github.com/Masterminds/semver/v3 v3.4.0 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/containerd/errdefs v1.0.0 // indirect
github.com/containerd/errdefs/pkg v0.3.0 // indirect
github.com/containerd/log v0.1.0 // indirect
github.com/coreos/go-semver v0.3.1 // indirect
github.com/coreos/go-systemd/v22 v22.5.0 // indirect
github.com/dgryski/go-farm v0.0.0-20240924180020-3414d57e47da // indirect
@@ -59,11 +60,11 @@ require (
github.com/docker/go-units v0.5.0 // indirect
github.com/eapache/channels v1.1.0 // indirect
github.com/eapache/queue v1.1.0 // indirect
github.com/emicklei/go-restful/v3 v3.12.1 // indirect
github.com/emicklei/go-restful/v3 v3.12.2 // indirect
github.com/evanphx/json-patch/v5 v5.9.0 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/fsnotify/fsnotify v1.8.0 // indirect
github.com/fxamacker/cbor/v2 v2.7.0 // indirect
github.com/fxamacker/cbor/v2 v2.9.0 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-openapi/jsonpointer v0.21.0 // indirect
@@ -72,12 +73,12 @@ require (
github.com/go-task/slim-sprig/v3 v3.0.0 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/gnostic-models v0.6.9 // indirect
github.com/google/gnostic-models v0.7.0 // indirect
github.com/google/pprof v0.0.0-20250403155104-27863c87afa6 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/gookit/color v1.5.4 // indirect
github.com/gookit/goutil v0.6.18 // indirect
github.com/gookit/gsr v0.1.0 // indirect
github.com/gookit/color v1.6.0 // indirect
github.com/gookit/goutil v0.7.1 // indirect
github.com/gookit/gsr v0.1.1 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.1 // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
@@ -94,11 +95,10 @@ require (
github.com/mdlayher/socket v0.5.1 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/sys/atomicwriter v0.1.0 // indirect
github.com/moby/sys/sequential v0.6.0 // indirect
github.com/moby/term v0.5.2 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/morikuni/aec v1.0.0 // indirect
github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.1 // indirect
@@ -106,16 +106,16 @@ require (
github.com/pelletier/go-toml/v2 v2.2.3 // indirect
github.com/pierrec/lz4/v4 v4.1.22 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/prometheus/client_model v0.6.1 // indirect
github.com/prometheus/common v0.62.0 // indirect
github.com/prometheus/procfs v0.15.1 // indirect
github.com/prometheus/client_model v0.6.2 // indirect
github.com/prometheus/common v0.66.1 // indirect
github.com/prometheus/procfs v0.16.1 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/sagikazarmark/locafero v0.6.0 // indirect
github.com/sagikazarmark/slog-shim v0.1.0 // indirect
github.com/sourcegraph/conc v0.3.0 // indirect
github.com/spf13/afero v1.11.0 // indirect
github.com/spf13/cast v1.7.1 // indirect
github.com/spf13/pflag v1.0.6 // indirect
github.com/spf13/pflag v1.0.9 // indirect
github.com/spf13/viper v1.19.0 // indirect
github.com/subosito/gotenv v1.6.0 // indirect
github.com/tj/go-spin v1.1.0 // indirect
@@ -127,32 +127,33 @@ require (
github.com/xlab/pkgconfig v0.0.0-20170226114623-cea12a0fd245 // indirect
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 // indirect
go.opentelemetry.io/otel v1.37.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.37.0 // indirect
go.opentelemetry.io/otel/metric v1.37.0 // indirect
go.opentelemetry.io/otel/sdk v1.37.0 // indirect
go.opentelemetry.io/otel/trace v1.37.0 // indirect
go.uber.org/automaxprocs v1.6.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
golang.org/x/crypto v0.39.0 // indirect
golang.org/x/mod v0.25.0 // indirect
golang.org/x/net v0.41.0 // indirect
go.yaml.in/yaml/v2 v2.4.2 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/crypto v0.41.0 // indirect
golang.org/x/mod v0.27.0 // indirect
golang.org/x/net v0.43.0 // indirect
golang.org/x/oauth2 v0.30.0 // indirect
golang.org/x/term v0.32.0 // indirect
golang.org/x/text v0.26.0 // indirect
golang.org/x/term v0.34.0 // indirect
golang.org/x/text v0.28.0 // indirect
golang.org/x/time v0.9.0 // indirect
golang.org/x/tools v0.33.0 // indirect
golang.org/x/tools v0.36.0 // indirect
golang.zx2c4.com/wireguard v0.0.0-20231211153847-12269c276173 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250603155806-513f23925822 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250603155806-513f23925822 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250707201910-8d1bb00bc6a7 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250707201910-8d1bb00bc6a7 // indirect
gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/kube-openapi v0.0.0-20250318190949-c8a335a9a2ff // indirect
k8s.io/utils v0.0.0-20241210054802-24370beab758 // indirect
k8s.io/kube-openapi v0.0.0-20250710124328-f3f2b991d03b // indirect
k8s.io/utils v0.0.0-20250604170112-4c0f3b243397 // indirect
modernc.org/cc/v4 v4.24.4 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/opt v0.1.4 // indirect
@@ -161,5 +162,5 @@ require (
modernc.org/token v1.1.0 // indirect
sigs.k8s.io/json v0.0.0-20241014173422-cfa47c3a1cc8 // indirect
sigs.k8s.io/randfill v1.0.0 // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.6.0 // indirect
sigs.k8s.io/structured-merge-diff/v6 v6.3.0 // indirect
)

196
go.sum
View File

@@ -6,8 +6,10 @@ github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03
github.com/BurntSushi/toml v0.4.1/go.mod h1:CxXYINrC8qIiEnFrOxCa7Jy5BFHlXnUU2pbicEuybxQ=
github.com/BurntSushi/toml v1.4.0 h1:kuoIxZQy2WRRk1pttg9asf+WVv6tWQuBNVmK8+nqPr0=
github.com/BurntSushi/toml v1.4.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
github.com/Microsoft/go-winio v0.4.14 h1:+hMXMk01us9KgxGb7ftKQt2Xpf5hH/yky+TDA+qxleU=
github.com/Microsoft/go-winio v0.4.14/go.mod h1:qXqCSQ3Xa7+6tgxaGTIe4Kpcdsi+P8jBhyzoq1bpyYA=
github.com/Masterminds/semver/v3 v3.4.0 h1:Zog+i5UMtVoCU8oKka5P7i9q9HgrJeGzI9SA1Xbatp0=
github.com/Masterminds/semver/v3 v3.4.0/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lprf4nC11bSS5beM=
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/cenkalti/backoff/v5 v5.0.2 h1:rIfFVxEf1QsI7E1ZHfp/B4DF/6QBAUhmgkxc0H7Zss8=
@@ -39,8 +41,8 @@ github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5Qvfr
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
github.com/dlclark/regexp2 v1.11.4 h1:rPYF9/LECdNymJufQKmri9gV604RvvABwgOA8un7yAo=
github.com/dlclark/regexp2 v1.11.4/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/docker/docker v28.3.3+incompatible h1:Dypm25kh4rmk49v1eiVbsAtpAsYURjYkaKubwuBdxEI=
github.com/docker/docker v28.3.3+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v28.4.0+incompatible h1:KVC7bz5zJY/4AZe/78BIvCnPsLaC9T/zh72xnlrTTOk=
github.com/docker/docker v28.4.0+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/go-connections v0.5.0 h1:USnMq7hx7gwdVZq1L49hLXaFtUdTADjXGp+uj1Br63c=
github.com/docker/go-connections v0.5.0/go.mod h1:ov60Kzw0kKElRwhNs9UlUHAE/F9Fe6GLaXnqyDdmEXc=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
@@ -51,8 +53,8 @@ github.com/eapache/channels v1.1.0 h1:F1taHcn7/F0i8DYqKXJnyhJcVpp2kgFcNePxXtnyu4
github.com/eapache/channels v1.1.0/go.mod h1:jMm2qB5Ubtg9zLd+inMZd2/NUvXgzmWXsDaLyQIGfH0=
github.com/eapache/queue v1.1.0 h1:YOEu7KNc61ntiQlcEeUIoDTJ2o8mQznoNvUhiigpIqc=
github.com/eapache/queue v1.1.0/go.mod h1:6eCeP0CKFpHLu8blIFXhExK/dRa7WDZfr6jVFPTqq+I=
github.com/emicklei/go-restful/v3 v3.12.1 h1:PJMDIM/ak7btuL8Ex0iYET9hxM3CI2sjZtzpL63nKAU=
github.com/emicklei/go-restful/v3 v3.12.1/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc=
github.com/emicklei/go-restful/v3 v3.12.2 h1:DhwDP0vY3k8ZzE0RunuJy8GhNpPL6zqLkDf9B/a0/xU=
github.com/emicklei/go-restful/v3 v3.12.2/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc=
github.com/evanphx/json-patch/v5 v5.9.0 h1:kcBlZQbplgElYIlo/n1hJbls2z/1awpXxpRi0/FOJfg=
github.com/evanphx/json-patch/v5 v5.9.0/go.mod h1:VNkHZ/282BpEyt/tObQO8s5CMPmYYq14uClGH4abBuQ=
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
@@ -64,8 +66,8 @@ github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHk
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/fsnotify/fsnotify v1.8.0 h1:dAwr6QBTBZIkG8roQaJjGof0pp0EeF+tNV7YBP3F/8M=
github.com/fsnotify/fsnotify v1.8.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
github.com/fxamacker/cbor/v2 v2.7.0 h1:iM5WgngdRBanHcxugY4JySA0nk1wZorNOpTgCMedv5E=
github.com/fxamacker/cbor/v2 v2.7.0/go.mod h1:pxXPTn3joSm21Gbwsv0w9OSA2y1HFR9qXEeXQVeNoDQ=
github.com/fxamacker/cbor/v2 v2.9.0 h1:NpKPmjDBgUfBms6tr6JZkTHtfFGcMKsw3eGcmD/sapM=
github.com/fxamacker/cbor/v2 v2.9.0/go.mod h1:vM4b+DJCtHn+zz7h3FFp/hDAI9WNWCsZj23V5ytsSxQ=
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
@@ -86,8 +88,8 @@ github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/gnostic-models v0.6.9 h1:MU/8wDLif2qCXZmzncUQ/BOfxWfthHi63KqpoNbWqVw=
github.com/google/gnostic-models v0.6.9/go.mod h1:CiWsm0s6BSQd1hRn8/QmxqB6BesYcbSZxsz9b0KuDBw=
github.com/google/gnostic-models v0.7.0 h1:qwTtogB15McXDaNqTZdzPJRHvaVJlAl+HVQnLmJEJxo=
github.com/google/gnostic-models v0.7.0/go.mod h1:whL5G0m6dmc5cPxKc5bdKdEN3UjI7OUGxBlw57miDrQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
@@ -95,9 +97,10 @@ github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/go-containerregistry v0.20.6 h1:cvWX87UxxLgaH76b4hIvya6Dzz9qHB31qAwjAohdSTU=
github.com/google/go-containerregistry v0.20.6/go.mod h1:T0x8MuoAoKX/873bkeSfLD2FAkwCDf9/HZgsFJ02E2Y=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/nftables v0.3.0 h1:bkyZ0cbpVeMHXOrtlFc8ISmfVqq5gPJukoYieyVmITg=
github.com/google/nftables v0.3.0/go.mod h1:BCp9FsrbF1Fn/Yu6CLUc9GGZFw/+hsxfluNXXmxBfRM=
@@ -107,14 +110,16 @@ github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 h1:El6M4kTTCOh6aBiKaU
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510/go.mod h1:pupxD2MaaD3pAXIBCelhxNneeOaAeabZDe5s4K6zSpQ=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gookit/color v1.5.4 h1:FZmqs7XOyGgCAxmWyPslpiok1k05wmY3SJTytgvYFs0=
github.com/gookit/color v1.5.4/go.mod h1:pZJOeOS8DM43rXbp4AZo1n9zCU2qjpcRko0b6/QJi9w=
github.com/gookit/goutil v0.6.18 h1:MUVj0G16flubWT8zYVicIuisUiHdgirPAkmnfD2kKgw=
github.com/gookit/goutil v0.6.18/go.mod h1:AY/5sAwKe7Xck+mEbuxj0n/bc3qwrGNe3Oeulln7zBA=
github.com/gookit/gsr v0.1.0 h1:0gadWaYGU4phMs0bma38t+Do5OZowRMEVlHv31p0Zig=
github.com/gookit/gsr v0.1.0/go.mod h1:7wv4Y4WCnil8+DlDYHBjidzrEzfHhXEoFjEA0pPPWpI=
github.com/gookit/slog v0.5.8 h1:XZCeHLQvvOZWcSUDZcqxXITsL9+d1ESsKZoASBmK1lI=
github.com/gookit/slog v0.5.8/go.mod h1:s0ViFOY/IgUuT4MDPF0l9x5/npcciy8pL4xwWZadnoc=
github.com/gookit/assert v0.1.1 h1:lh3GcawXe/p+cU7ESTZ5Ui3Sm/x8JWpIis4/1aF0mY0=
github.com/gookit/assert v0.1.1/go.mod h1:jS5bmIVQZTIwk42uXl4lyj4iaaxx32tqH16CFj0VX2E=
github.com/gookit/color v1.6.0 h1:JjJXBTk1ETNyqyilJhkTXJYYigHG24TM9Xa2M1xAhRA=
github.com/gookit/color v1.6.0/go.mod h1:9ACFc7/1IpHGBW8RwuDm/0YEnhg3dwwXpoMsmtyHfjs=
github.com/gookit/goutil v0.7.1 h1:AaFJPN9mrdeYBv8HOybri26EHGCC34WJVT7jUStGJsI=
github.com/gookit/goutil v0.7.1/go.mod h1:vJS9HXctYTCLtCsZot5L5xF+O1oR17cDYO9R0HxBmnU=
github.com/gookit/gsr v0.1.1 h1:TaHD3M7qa6lcAf9D2J4mGNg+QjgDtD1bw7uctF8RXOM=
github.com/gookit/gsr v0.1.1/go.mod h1:7wv4Y4WCnil8+DlDYHBjidzrEzfHhXEoFjEA0pPPWpI=
github.com/gookit/slog v0.6.0 h1:KEQxOJxbTtk7oyqah6nJOEKjOdI0z5qoqkX7I6G65g4=
github.com/gookit/slog v0.6.0/go.mod h1:hPlpNi/WIcGmkEjHzQTS7s5JZkHmmnGy9sYo6csa08s=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.1 h1:X5VWvz21y3gzm9Nw/kaUeku/1+uBhcekkmy4IkffJww=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.1/go.mod h1:Zanoh4+gvIgluNqcfMVTJueD4wSS5hT7zTt4Mrutd90=
github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4=
@@ -151,7 +156,6 @@ github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
@@ -209,16 +213,17 @@ github.com/moby/term v0.5.2/go.mod h1:d3djjFCrjnB+fl8NJux+EJzu0msscUP+f8it8hPkFL
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee h1:W5t00kpgFdJifH4BDsTlE89Zl93FEloxaWZfGcifgq8=
github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/onsi/ginkgo/v2 v2.23.4 h1:ktYTpKJAVZnDT4VjxSbiBenUjmlL/5QkBEocaWXiQus=
github.com/onsi/ginkgo/v2 v2.23.4/go.mod h1:Bt66ApGPBFzHyR+JO10Zbt0Gsp4uWxu5mIOTusL46e8=
github.com/onsi/gomega v1.37.0 h1:CdEG8g0S133B4OswTDC/5XPSzE1OeP29QOioj2PID2Y=
github.com/onsi/gomega v1.37.0/go.mod h1:8D9+Txp43QWKhM24yyOBEdpkzN8FvJyAwecBgsU4KU0=
github.com/onsi/ginkgo/v2 v2.25.3 h1:Ty8+Yi/ayDAGtk4XxmmfUy4GabvM+MegeB4cDLRi6nw=
github.com/onsi/ginkgo/v2 v2.25.3/go.mod h1:43uiyQC4Ed2tkOzLsEYm7hnrb7UJTWHYNsuy3bG/snE=
github.com/onsi/gomega v1.38.2 h1:eZCjf2xjZAqe+LeWvKb5weQ+NcPwX84kqJ0cZNxok2A=
github.com/onsi/gomega v1.38.2/go.mod h1:W2MJcYxRGV63b418Ai34Ud0hEdTVXq9NW9+Sx6uXf3k=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040=
@@ -233,7 +238,6 @@ github.com/pelletier/go-toml/v2 v2.2.3 h1:YmeHyLY8mFWbdkNWwpr+qIL2bEqT0o95WSdkNH
github.com/pelletier/go-toml/v2 v2.2.3/go.mod h1:MfCQTFTvCcUyyvvwm1+G6H/jORL20Xlb6rzQu9GuUkc=
github.com/pierrec/lz4/v4 v4.1.22 h1:cKFw6uJDK+/gfw5BcDL0JL5aBsAFdsIT18eRtLj7VIU=
github.com/pierrec/lz4/v4 v4.1.22/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
@@ -241,24 +245,23 @@ github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRI
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prashantv/gostub v1.1.0 h1:BTyx3RfQjRHnUWaGF9oQos79AlQ5k8WNktv7VGvVH4g=
github.com/prashantv/gostub v1.1.0/go.mod h1:A5zLQHz7ieHGG7is6LLXLz7I8+3LZzsrV0P1IAHhP5U=
github.com/prometheus/client_golang v1.22.0 h1:rb93p9lokFEsctTys46VnV1kLCDpVZ0a/Y92Vm0Zc6Q=
github.com/prometheus/client_golang v1.22.0/go.mod h1:R7ljNsLXhuQXYZYtw6GAE9AZg8Y7vEW5scdCXrWRXC0=
github.com/prometheus/client_model v0.6.1 h1:ZKSh/rekM+n3CeS952MLRAdFwIKqeY8b62p8ais2e9E=
github.com/prometheus/client_model v0.6.1/go.mod h1:OrxVMOVHjw3lKMa8+x6HeMGkHMQyHDk9E3jmP2AmGiY=
github.com/prometheus/common v0.62.0 h1:xasJaQlnWAeyHdUBeGjXmutelfJHWMRr+Fg4QszZ2Io=
github.com/prometheus/common v0.62.0/go.mod h1:vyBcEuLSvWos9B1+CyL7JZ2up+uFzXhkqml0W5zIY1I=
github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc=
github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk=
github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o=
github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg=
github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=
github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs=
github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA=
github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg=
github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/rogpeppe/go-internal v1.13.1 h1:KvO1DLK/DRN07sQ1LQKScxyZJuNnedQ5/wKSR38lUII=
github.com/rogpeppe/go-internal v1.13.1/go.mod h1:uMEvuHeurkdAXX61udpOXGD/AzZDWNMNyH2VO9fmH0o=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/sagikazarmark/locafero v0.6.0 h1:ON7AQg37yzcRPU69mt7gwhFEBwxI6P9T4Qu3N51bwOk=
github.com/sagikazarmark/locafero v0.6.0/go.mod h1:77OmuIc6VTraTXKXIs/uvUxKGUXjE1GbemJYHqdNjX0=
github.com/sagikazarmark/slog-shim v0.1.0 h1:diDBnUNK9N/354PgrxMywXnAwEr1QZcOr6gto+ugjYE=
github.com/sagikazarmark/slog-shim v0.1.0/go.mod h1:SrcSrq8aKtyuqEI1uvTDTK1arOWRIczQRv+GVI1AkeQ=
github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q=
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/sourcegraph/conc v0.3.0 h1:OQTbbt6P72L20UqAkXXuLOj79LfEanQ+YQFNpLA9ySo=
@@ -267,21 +270,19 @@ github.com/spf13/afero v1.11.0 h1:WJQKhtpdm3v2IzqG8VMqrr6Rf3UYpEF239Jy9wNepM8=
github.com/spf13/afero v1.11.0/go.mod h1:GH9Y3pIexgf1MTIWtNGyogA5MwRIDXGUr+hbWNoBjkY=
github.com/spf13/cast v1.7.1 h1:cuNEagBQEHWN1FnbGEjCXL2szYEXqfJPbP2HNUaca9Y=
github.com/spf13/cast v1.7.1/go.mod h1:ancEpBxwJDODSW/UG4rDrAqiKolqNNh2DX3mk86cAdo=
github.com/spf13/cobra v1.9.1 h1:CXSaggrXdbHK9CF+8ywj8Amf7PBRmPCOJugH954Nnlo=
github.com/spf13/cobra v1.9.1/go.mod h1:nDyEzZ8ogv936Cinf6g1RU9MRY64Ir93oCnqb9wxYW0=
github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o=
github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/cobra v1.10.1 h1:lJeBwCfmrnXthfAupyUTzJ/J4Nc1RsHC/mSRU2dll/s=
github.com/spf13/cobra v1.10.1/go.mod h1:7SmJGaTHFVBY0jW4NXGluQoLvhqFQM+6XSKD+P4XaB0=
github.com/spf13/pflag v1.0.9 h1:9exaQaMOCwffKiiiYk6/BndUBv+iRViNW+4lEMi0PvY=
github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/viper v1.19.0 h1:RWq5SEjt8o25SROyN3z2OrDB9l7RPd3lwTWU8EcEdcI=
github.com/spf13/viper v1.19.0/go.mod h1:GQUN9bilAbhU/jgc1bKs99f/suXKeUMct8Adx5+Ntkg=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
github.com/tj/go-spin v1.1.0 h1:lhdWZsvImxvZ3q1C5OIB7d72DuOwP4O2NdBg9PyzNds=
@@ -305,16 +306,16 @@ github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJu
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.4.0/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
go.etcd.io/etcd/api/v3 v3.6.1 h1:yJ9WlDih9HT457QPuHt/TH/XtsdN2tubyxyQHSHPsEo=
go.etcd.io/etcd/api/v3 v3.6.1/go.mod h1:lnfuqoGsXMlZdTJlact3IB56o3bWp1DIlXPIGKRArto=
go.etcd.io/etcd/client/pkg/v3 v3.6.1 h1:CxDVv8ggphmamrXM4Of8aCC8QHzDM4tGcVr9p2BSoGk=
go.etcd.io/etcd/client/pkg/v3 v3.6.1/go.mod h1:aTkCp+6ixcVTZmrJGa7/Mc5nMNs59PEgBbq+HCmWyMc=
go.etcd.io/etcd/client/v3 v3.6.1 h1:KelkcizJGsskUXlsxjVrSmINvMMga0VWwFF0tSPGEP0=
go.etcd.io/etcd/client/v3 v3.6.1/go.mod h1:fCbPUdjWNLfx1A6ATo9syUmFVxqHH9bCnPLBZmnLmMY=
go.etcd.io/etcd/api/v3 v3.6.5 h1:pMMc42276sgR1j1raO/Qv3QI9Af/AuyQUW6CBAWuntA=
go.etcd.io/etcd/api/v3 v3.6.5/go.mod h1:ob0/oWA/UQQlT1BmaEkWQzI0sJ1M0Et0mMpaABxguOQ=
go.etcd.io/etcd/client/pkg/v3 v3.6.5 h1:Duz9fAzIZFhYWgRjp/FgNq2gO1jId9Yae/rLn3RrBP8=
go.etcd.io/etcd/client/pkg/v3 v3.6.5/go.mod h1:8Wx3eGRPiy0qOFMZT/hfvdos+DjEaPxdIDiCDUv/FQk=
go.etcd.io/etcd/client/v3 v3.6.5 h1:yRwZNFBx/35VKHTcLDeO7XVLbCBFbPi+XV4OC3QJf2U=
go.etcd.io/etcd/client/v3 v3.6.5/go.mod h1:ZqwG/7TAFZ0BJ0jXRPoJjKQJtbFo/9NIY8uoFFKcCyo=
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0 h1:jq9TW8u3so/bN+JPT166wjOI6/vQPF6Xe7nMNIltagk=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0/go.mod h1:p8pYQP+m5XfbZm9fxtSKAbM6oIllS7s2AfxrChvc7iw=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 h1:F7Jx+6hwnZ41NSFTO5q4LYDtJRXBf2PD0rNBkeB/lus=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0/go.mod h1:UHB22Z8QsdRDrnAtX4PntOl36ajSxcdUMt1sF7Y6E7Q=
go.opentelemetry.io/otel v1.37.0 h1:9zhNfelUvx0KBfu/gb+ZgeAfAgtWrfHJZcAqFC228wQ=
go.opentelemetry.io/otel v1.37.0/go.mod h1:ehE/umFRLnuLa/vSccNq9oS1ErUlkkK71gMcN34UG8I=
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.37.0 h1:Ahq7pZmv87yiyn3jeFz/LekZmPLLdKejuO3NcK9MssM=
@@ -325,8 +326,8 @@ go.opentelemetry.io/otel/metric v1.37.0 h1:mvwbQS5m0tbmqML4NqK+e3aDiO02vsf/Wgbsd
go.opentelemetry.io/otel/metric v1.37.0/go.mod h1:04wGrZurHYKOc+RKeye86GwKiTb9FKm1WHtO+4EVr2E=
go.opentelemetry.io/otel/sdk v1.37.0 h1:ItB0QUqnjesGRvNcmAcU0LyvkVyGJ2xftD29bWdDvKI=
go.opentelemetry.io/otel/sdk v1.37.0/go.mod h1:VredYzxUvuo2q3WRcDnKDjbdvmO0sCzOvVAiY+yUkAg=
go.opentelemetry.io/otel/sdk/metric v1.35.0 h1:1RriWBmCKgkeHEhM7a2uMjMUfP7MsOF5JpUCaEqEI9o=
go.opentelemetry.io/otel/sdk/metric v1.35.0/go.mod h1:is6XYCUMpcKi+ZsOvfluY5YstFnhW0BidkR+gL+qN+w=
go.opentelemetry.io/otel/sdk/metric v1.37.0 h1:90lI228XrB9jCMuSdA0673aubgRobVZFhbjxHHspCPc=
go.opentelemetry.io/otel/sdk/metric v1.37.0/go.mod h1:cNen4ZWfiD37l5NhS+Keb5RXVWZWpRE+9WyVCpbo5ps=
go.opentelemetry.io/otel/trace v1.37.0 h1:HLdcFNbRQBE2imdSEgm/kwqmQj1Or1l/7bW6mxVK7z4=
go.opentelemetry.io/otel/trace v1.37.0/go.mod h1:TlgrlQ+PtQO5XFerSPUYG0JSgGyryXewPGyayAWSBS0=
go.opentelemetry.io/proto/otlp v1.7.0 h1:jX1VolD6nHuFzOYso2E73H85i92Mv8JQYk0K9vz09os=
@@ -339,19 +340,23 @@ go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8=
go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI=
go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU=
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.39.0 h1:SHs+kF4LP+f+p14esP5jAoDpHU8Gu/v9lFRK6IT5imM=
golang.org/x/crypto v0.39.0/go.mod h1:L+Xg3Wf6HoL4Bn4238Z6ft6KfEpN0tJGo53AAPC632U=
golang.org/x/crypto v0.41.0 h1:WKYxWedPGCTVVl5+WHSSrOBT0O8lx32+zxmHxijgXp4=
golang.org/x/crypto v0.41.0/go.mod h1:pO5AFd7FA68rFak7rOAGVuygIISepHftHnr8dr6+sUc=
golang.org/x/exp v0.0.0-20250103183323-7d7fa50e5329 h1:9kj3STMvgqy3YA4VQXBrN7925ICMxD5wzMRcgA30588=
golang.org/x/exp v0.0.0-20250103183323-7d7fa50e5329/go.mod h1:qj5a5QZpwLU2NLQudwIN5koi3beDhSAlJwa67PuM98c=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.5.1/go.mod h1:5OXOZSfqPIIbmVBIIKWRFfZjPR0E5r58TLhUjH0a2Ro=
golang.org/x/mod v0.25.0 h1:n7a+ZbQKQA/Ysbyb0/6IbB1H/X41mKgbhfv7AfG/44w=
golang.org/x/mod v0.25.0/go.mod h1:IXM97Txy2VM4PJ3gI61r1YEk/gAj6zAHN3AdZt6S9Ww=
golang.org/x/mod v0.27.0 h1:kb+q2PyFnEADO2IEF935ehFUXlWiNjJWtRNgBLSfbxQ=
golang.org/x/mod v0.27.0/go.mod h1:rWI627Fq0DEoudcK+MBkNkCe0EetEaDSwJJkCcjpazc=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
@@ -372,22 +377,20 @@ golang.org/x/net v0.0.0-20210928044308-7d9f5e0b762b/go.mod h1:9nx3DQGgdP8bBQD5qx
golang.org/x/net v0.0.0-20211020060615-d418f374d309/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20211209124913-491a49abca63/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20220107192237-5cfca573fb4d/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.41.0 h1:vBTly1HeNPEn3wtREYfy4GZ/NECgw2Cnl+nK6Nz3uvw=
golang.org/x/net v0.41.0/go.mod h1:B/K4NNqkfmg07DQYrbwvSluqCJOOXwUjeb/5lOisjbA=
golang.org/x/net v0.43.0 h1:lat02VYK2j4aLzMzecihNvTlJNQUq316m2Mr9rnM6YE=
golang.org/x/net v0.43.0/go.mod h1:vhO1fvI4dGsIjh73sWfUVjj3N7CA9WkKJNQm2svM6Jg=
golang.org/x/oauth2 v0.30.0 h1:dnDm7JmhM45NNpd8FDDeLhK6FwqbOf4MLCM9zb1BOHI=
golang.org/x/oauth2 v0.30.0/go.mod h1:B++QgG3ZKulg6sRPGD/mqlHQs5rB3Ml9erfeDY7xKlU=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.15.0 h1:KWH3jNZsfyT6xfAfKiz6MRNmd46ByHDYaZ7KSkCtdW8=
golang.org/x/sync v0.15.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug=
golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190411185658-b44545bcd369/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190826190057-c7b8b68b1456/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191008105621-543471e840be/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -414,16 +417,16 @@ golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.10.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw=
golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/sys v0.36.0 h1:KVRy2GtZBrk1cBYA7MKu5bEZFxQk4NIDV6RLVcC8o0k=
golang.org/x/sys v0.36.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.32.0 h1:DR4lr0TjUs3epypdhTOkMmuF5CDFJ/8pOnbzMZPQ7bg=
golang.org/x/term v0.32.0/go.mod h1:uZG1FhGx848Sqfsq4/DlJr3xGGsYMu/L5GW4abiaEPQ=
golang.org/x/term v0.34.0 h1:O/2T7POpk0ZZ7MAzMeWFSg6S5IpWd/RXDlM9hgM3DR4=
golang.org/x/term v0.34.0/go.mod h1:5jC53AEywhIVebHgPVeg0mj8OD3VO9OzclacVrqpaAw=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.26.0 h1:P42AVeLghgTYr4+xUnTRKDMqpar+PtX7KWuNQL21L8M=
golang.org/x/text v0.26.0/go.mod h1:QK15LZJUUQVJxhz7wXgxSy/CJaTFjd0G+YLonydOVQA=
golang.org/x/text v0.28.0 h1:rhazDwis8INMIwQ4tpjLDzUhx6RlXqZNPEM0huQojng=
golang.org/x/text v0.28.0/go.mod h1:U8nCwOR8jO/marOQ0QbDiOngZVEBB7MAiitBuMjXiNU=
golang.org/x/time v0.9.0 h1:EsRrnYcQiGH+5FfbgvV4AP7qEZstoyrHB0DzarOQ4ZY=
golang.org/x/time v0.9.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
@@ -432,8 +435,8 @@ golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roY
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=
golang.org/x/tools v0.1.7/go.mod h1:LGqMHiF4EqQNHR1JncWGqT5BVaXmza+X+BDGol+dOxo=
golang.org/x/tools v0.33.0 h1:4qz2S3zmRxbGIhDIAgjxvFutSvH5EfnsYrRBj0UI0bc=
golang.org/x/tools v0.33.0/go.mod h1:CIJMaWEY88juyUfo7UbgPqbC8rU2OqfAV1h2Qp0oMYI=
golang.org/x/tools v0.36.0 h1:kWS0uv/zsvHEle1LbV5LE8QujrxB3wfQyxHfhOk0Qkg=
golang.org/x/tools v0.36.0/go.mod h1:WBDiHKJK8YgLHlcQPYQzNCkUxUypCaa5ZegCVutKm+s=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@@ -442,14 +445,16 @@ golang.zx2c4.com/wireguard v0.0.0-20231211153847-12269c276173 h1:/jFs0duh4rdb8uI
golang.zx2c4.com/wireguard v0.0.0-20231211153847-12269c276173/go.mod h1:tkCQ4FQXmpAgYVh++1cq16/dH4QJtmvpRv19DWGAHSA=
golang.zx2c4.com/wireguard/wgctrl v0.0.0-20241231184526-a9ab2273dd10 h1:3GDAcqdIg1ozBNLgPy4SLT84nfcBjr6rhGtXYtrkWLU=
golang.zx2c4.com/wireguard/wgctrl v0.0.0-20241231184526-a9ab2273dd10/go.mod h1:T97yPqesLiNrOYxkwmhMI0ZIlJDm+p0PMR8eRVeR5tQ=
google.golang.org/genproto/googleapis/api v0.0.0-20250603155806-513f23925822 h1:oWVWY3NzT7KJppx2UKhKmzPq4SRe0LdCijVRwvGeikY=
google.golang.org/genproto/googleapis/api v0.0.0-20250603155806-513f23925822/go.mod h1:h3c4v36UTKzUiuaOKQ6gr3S+0hovBtUrXzTG/i3+XEc=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250603155806-513f23925822 h1:fc6jSaCT0vBduLYZHYrBBNY4dsWuvgyff9noRNDdBeE=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250603155806-513f23925822/go.mod h1:qQ0YXyHHx3XkvlzUtpXDkS29lDSafHMZBAZDc03LQ3A=
google.golang.org/grpc v1.73.0 h1:VIWSmpI2MegBtTuFt5/JWy2oXxtjJ/e89Z70ImfD2ok=
google.golang.org/grpc v1.73.0/go.mod h1:50sbHOUqWoCQGI8V2HQLJM0B+LMlIUjNSZmow7EVBQc=
google.golang.org/protobuf v1.36.6 h1:z1NpPI8ku2WgiWnf+t9wTPsn6eP1L7ksHUlkfLvd9xY=
google.golang.org/protobuf v1.36.6/go.mod h1:jduwjTPXsFjZGTmRluh+L6NjiWu7pchiJ2/5YcXBHnY=
gonum.org/v1/gonum v0.16.0 h1:5+ul4Swaf3ESvrOnidPp4GZbzf0mxVQpDCYUQE7OJfk=
gonum.org/v1/gonum v0.16.0/go.mod h1:fef3am4MQ93R2HHpKnLk4/Tbh/s0+wqD5nfa6Pnwy4E=
google.golang.org/genproto/googleapis/api v0.0.0-20250707201910-8d1bb00bc6a7 h1:FiusG7LWj+4byqhbvmB+Q93B/mOxJLN2DTozDuZm4EU=
google.golang.org/genproto/googleapis/api v0.0.0-20250707201910-8d1bb00bc6a7/go.mod h1:kXqgZtrWaf6qS3jZOCnCH7WYfrvFjkC51bM8fz3RsCA=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250707201910-8d1bb00bc6a7 h1:pFyd6EwwL2TqFf8emdthzeX+gZE1ElRq3iM8pui4KBY=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250707201910-8d1bb00bc6a7/go.mod h1:qQ0YXyHHx3XkvlzUtpXDkS29lDSafHMZBAZDc03LQ3A=
google.golang.org/grpc v1.75.1 h1:/ODCNEuf9VghjgO3rqLcfg8fiOP0nSluljWFlDxELLI=
google.golang.org/grpc v1.75.1/go.mod h1:JtPAzKiq4v1xcAB2hydNlWI2RnF85XXcV0mhKXr2ecQ=
google.golang.org/protobuf v1.36.8 h1:xHScyCOEuuwZEc6UtSOvPbAT4zRh0xcNRYekJwfqyMc=
google.golang.org/protobuf v1.36.8/go.mod h1:fuxRtAxBytpl4zzqUh6/eyUujkJdNiuEkXntxiD/uRU=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
@@ -468,18 +473,18 @@ gotest.tools/v3 v3.4.0 h1:ZazjZUfuVeZGLAmlKKuyv3IKP5orXcwtOwDQH6YVr6o=
gotest.tools/v3 v3.4.0/go.mod h1:CtbdzLSsqVhDgMtKsx03ird5YTGB3ar27v0u/yKBW5g=
honnef.co/go/tools v0.2.1/go.mod h1:lPVVZ2BS5TfnjLyizF7o7hv7j9/L+8cZY2hLyjP9cGY=
honnef.co/go/tools v0.2.2/go.mod h1:lPVVZ2BS5TfnjLyizF7o7hv7j9/L+8cZY2hLyjP9cGY=
k8s.io/api v0.33.3 h1:SRd5t//hhkI1buzxb288fy2xvjubstenEKL9K51KBI8=
k8s.io/api v0.33.3/go.mod h1:01Y/iLUjNBM3TAvypct7DIj0M0NIZc+PzAHCIo0CYGE=
k8s.io/apimachinery v0.33.3 h1:4ZSrmNa0c/ZpZJhAgRdcsFcZOw1PQU1bALVQ0B3I5LA=
k8s.io/apimachinery v0.33.3/go.mod h1:BHW0YOu7n22fFv/JkYOEfkUYNRN0fj0BlvMFWA7b+SM=
k8s.io/client-go v0.33.3 h1:M5AfDnKfYmVJif92ngN532gFqakcGi6RvaOF16efrpA=
k8s.io/client-go v0.33.3/go.mod h1:luqKBQggEf3shbxHY4uVENAxrDISLOarxpTKMiUuujg=
k8s.io/api v0.34.1 h1:jC+153630BMdlFukegoEL8E/yT7aLyQkIVuwhmwDgJM=
k8s.io/api v0.34.1/go.mod h1:SB80FxFtXn5/gwzCoN6QCtPD7Vbu5w2n1S0J5gFfTYk=
k8s.io/apimachinery v0.34.1 h1:dTlxFls/eikpJxmAC7MVE8oOeP1zryV7iRyIjB0gky4=
k8s.io/apimachinery v0.34.1/go.mod h1:/GwIlEcWuTX9zKIg2mbw0LRFIsXwrfoVxn+ef0X13lw=
k8s.io/client-go v0.34.1 h1:ZUPJKgXsnKwVwmKKdPfw4tB58+7/Ik3CrjOEhsiZ7mY=
k8s.io/client-go v0.34.1/go.mod h1:kA8v0FP+tk6sZA0yKLRG67LWjqufAoSHA2xVGKw9Of8=
k8s.io/klog/v2 v2.130.1 h1:n9Xl7H1Xvksem4KFG4PYbdQCQxqc/tTUyrgXaOhHSzk=
k8s.io/klog/v2 v2.130.1/go.mod h1:3Jpz1GvMt720eyJH1ckRHK1EDfpxISzJ7I9OYgaDtPE=
k8s.io/kube-openapi v0.0.0-20250318190949-c8a335a9a2ff h1:/usPimJzUKKu+m+TE36gUyGcf03XZEP0ZIKgKj35LS4=
k8s.io/kube-openapi v0.0.0-20250318190949-c8a335a9a2ff/go.mod h1:5jIi+8yX4RIb8wk3XwBo5Pq2ccx4FP10ohkbSKCZoK8=
k8s.io/utils v0.0.0-20241210054802-24370beab758 h1:sdbE21q2nlQtFh65saZY+rRM6x6aJJI8IUa1AmH/qa0=
k8s.io/utils v0.0.0-20241210054802-24370beab758/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0=
k8s.io/kube-openapi v0.0.0-20250710124328-f3f2b991d03b h1:MloQ9/bdJyIu9lb1PzujOPolHyvO06MXG5TUIj2mNAA=
k8s.io/kube-openapi v0.0.0-20250710124328-f3f2b991d03b/go.mod h1:UZ2yyWbFTpuhSbFhv24aGNOdoRdJZgsIObGBUaYVsts=
k8s.io/utils v0.0.0-20250604170112-4c0f3b243397 h1:hwvWFiBzdWw1FhfY1FooPn3kzWuJ8tmbZBHi4zVsl1Y=
k8s.io/utils v0.0.0-20250604170112-4c0f3b243397/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0=
modernc.org/cc/v4 v4.24.4 h1:TFkx1s6dCkQpd6dKurBNmpo+G8Zl4Sq/ztJ+2+DEsh0=
modernc.org/cc/v4 v4.24.4/go.mod h1:uVtb5OGqUKpoLWhqwNQo/8LwvoiEBLvZXIQ/SmO6mL0=
modernc.org/ccorpus2 v1.5.2 h1:Ui+4tc58mf/W+2arcYCJR903y3zl3ecsI7Fpaaqozyw=
@@ -500,10 +505,9 @@ sigs.k8s.io/json v0.0.0-20241014173422-cfa47c3a1cc8 h1:gBQPwqORJ8d8/YNZWEjoZs7np
sigs.k8s.io/json v0.0.0-20241014173422-cfa47c3a1cc8/go.mod h1:mdzfpAEoE6DHQEN0uh9ZbOCuHbLK5wOm7dK4ctXE9Tg=
sigs.k8s.io/kind v0.29.0 h1:3TpCsyh908IkXXpcSnsMjWdwdWjIl7o9IMZImZCWFnI=
sigs.k8s.io/kind v0.29.0/go.mod h1:ldWQisw2NYyM6k64o/tkZng/1qQW7OlzcN5a8geJX3o=
sigs.k8s.io/randfill v0.0.0-20250304075658-069ef1bbf016/go.mod h1:XeLlZ/jmk4i1HRopwe7/aU3H5n1zNUcX6TM94b3QxOY=
sigs.k8s.io/randfill v1.0.0 h1:JfjMILfT8A6RbawdsK2JXGBR5AQVfd+9TbzrlneTyrU=
sigs.k8s.io/randfill v1.0.0/go.mod h1:XeLlZ/jmk4i1HRopwe7/aU3H5n1zNUcX6TM94b3QxOY=
sigs.k8s.io/structured-merge-diff/v4 v4.6.0 h1:IUA9nvMmnKWcj5jl84xn+T5MnlZKThmUW1TdblaLVAc=
sigs.k8s.io/structured-merge-diff/v4 v4.6.0/go.mod h1:dDy58f92j70zLsuZVuUX5Wp9vtxXpaZnkPGWeqDfCps=
sigs.k8s.io/yaml v1.4.0 h1:Mk1wCc2gy/F0THH0TAp1QYyJNzRm2KCLy3o5ASXVI5E=
sigs.k8s.io/yaml v1.4.0/go.mod h1:Ejl7/uTz7PSA4eKMyQCUTnhZYNmLIl+5c2lQPGR2BPY=
sigs.k8s.io/structured-merge-diff/v6 v6.3.0 h1:jTijUJbW353oVOd9oTlifJqOGEkUw2jB/fXCbTiQEco=
sigs.k8s.io/structured-merge-diff/v6 v6.3.0/go.mod h1:M3W8sfWvn2HhQDIbGWj3S099YozAsymCo/wrT5ohRUE=
sigs.k8s.io/yaml v1.6.0 h1:G8fkbMSAFqgEFgh4b1wmtzDnioxFCUgTZhlbj5P9QYs=
sigs.k8s.io/yaml v1.6.0/go.mod h1:796bPqUfzR/0jLAl6XjHl3Ck7MiyVv8dbTdyT3/pMf4=

View File

@@ -2,16 +2,18 @@ package arp
import (
"context"
"fmt"
log "log/slog"
"sync"
"time"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/kube-vip/kube-vip/pkg/vip"
)
type Manager struct {
instances map[string]*Instance
instances sync.Map
config *kubevip.Config
}
@@ -24,8 +26,7 @@ type Instance struct {
func NewManager(config *kubevip.Config) *Manager {
return &Manager{
instances: make(map[string]*Instance),
config: config,
config: config,
}
}
@@ -42,10 +43,14 @@ func (i *Instance) Name() string {
}
func (m *Manager) Insert(instance *Instance) {
i, ok := m.instances[instance.Name()]
if !ok {
log.Info("inserting ARP/NDP instance", "name", instance.Name())
m.instances[instance.Name()] = instance
i, err := m.get(instance.Name())
if err != nil {
log.Error("[ARP manager] unable to insert instance", "err", err)
return
}
if i == nil {
log.Info("[ARP manager] inserting ARP/NDP instance", "name", instance.Name())
m.instances.Store(instance.Name(), instance)
} else {
i.mu.Lock()
defer i.mu.Unlock()
@@ -54,20 +59,35 @@ func (m *Manager) Insert(instance *Instance) {
}
func (m *Manager) Remove(instance *Instance) {
if i, ok := m.instances[instance.Name()]; ok {
i, err := m.get(instance.Name())
if err != nil {
log.Error("[ARP manager] unable to remove the instance", "err", err)
return
}
if i != nil {
i.mu.Lock()
defer i.mu.Unlock()
if i.counter > 1 {
i.counter--
} else {
log.Info("removing ARP/NDP instance", "name", instance.Name())
delete(m.instances, instance.Name())
log.Info("[ARP manager] removing ARP/NDP instance", "name", instance.Name())
if _, err := instance.network.DeleteIP(); err != nil {
log.Error("failed to delete IP", "address", instance.network.IP(), "err", err)
}
m.instances.Delete(instance.Name())
}
} else {
log.Warn("[ARP manager] unable to remove the instance - instance not found", "name", instance.Name())
}
}
func (m *Manager) Count(name string) int {
if i, ok := m.instances[name]; ok {
i, err := m.get(name)
if err != nil {
log.Error("[ARP manager] unable to count instance", "err", err)
return -1
}
if i != nil {
i.mu.Lock()
defer i.mu.Unlock()
return i.counter
@@ -76,26 +96,46 @@ func (m *Manager) Count(name string) int {
}
func (m *Manager) StartAdvertisement(ctx context.Context) {
log.Info("Starting ARP/NDP advertisement")
log.Info("[ARP manager] starting ARP/NDP advertisement")
for {
select {
case <-ctx.Done(): // if cancel() execute
return
default:
for _, instance := range m.instances {
if instance.counter > 0 {
ensureIPAndSendGratuitous(instance)
m.instances.Range(func(_ any, instance any) bool {
if i, ok := instance.(*Instance); ok {
if i.counter > 0 {
ensureIPAndSendGratuitous(i)
} else {
// this instance should not be advertised - delete the IP just in case...
if _, err := i.network.DeleteIP(); err != nil {
log.Error("[ARP manager] failed to delete IP", "address", i.network.IP(), "err", err)
}
}
}
}
return true
})
}
if m.config.ArpBroadcastRate < 500 {
log.Error("arp broadcast rate is too low", "rate (ms)", m.config.ArpBroadcastRate, "setting to (ms)", "3000")
log.Warn("[ARP manager] arp broadcast rate is too low", "rate (ms)", m.config.ArpBroadcastRate, "setting to (ms)", "3000")
m.config.ArpBroadcastRate = 3000
}
time.Sleep(time.Duration(m.config.ArpBroadcastRate) * time.Millisecond)
}
}
func (m *Manager) get(name string) (*Instance, error) {
i, exists := m.instances.Load(name)
if !exists {
return nil, nil
}
inst, ok := i.(*Instance)
if !ok {
return nil, fmt.Errorf("value for name %q is not of Instance pointer type", name)
}
return inst, nil
}
// ensureIPAndSendGratuitous - adds IP to the interface if missing, and send
// either a gratuitous ARP or gratuitous NDP. Re-adds the interface if it is IPv6
// and in a dadfailed state.
@@ -112,22 +152,16 @@ func ensureIPAndSendGratuitous(instance *Instance) {
}
if deleted {
log.Info("deleted and recreating address", "IP", ipString, "interface", iface)
// if _, err := instance.network.AddIP(false); err != nil {
// log.Error("failed to recreate address", "IP", ipString, "interface", iface)
// }
}
}
// Ensure the address exists on the interface before attempting to ARP
// if instance.network.HasEndpoints() {
if added, err := instance.network.AddIP(true); err != nil {
log.Warn(err.Error())
} else if added {
log.Warn("Re-applied the VIP configuration", "ip", ipString, "interface", iface)
}
// }
if vip.IsIPv6(ipString) {
if utils.IsIPv6(ipString) {
// Gratuitous NDP, will broadcast new MAC <-> IPv6 address
if instance.ndp == nil {
log.Error("NDP responder was not created")
@@ -145,5 +179,4 @@ func ensureIPAndSendGratuitous(instance *Instance) {
log.Warn(err.Error())
}
}
}

View File

@@ -8,7 +8,6 @@ import (
"github.com/kube-vip/kube-vip/pkg/k8s"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/kube-vip/kube-vip/pkg/vip"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/rest"
)
@@ -31,7 +30,7 @@ func (e *Entry) Check() bool {
// homeConfigPath := filepath.Join(os.Getenv("HOME"), ".kube", "config")
var k8sAddr string
if vip.IsIPv4(e.Addr) {
if utils.IsIPv4(e.Addr) {
k8sAddr = fmt.Sprintf("%s:%v", e.Addr, e.Port)
} else {
k8sAddr = fmt.Sprintf("[%s]:%v", e.Addr, e.Port)

View File

@@ -4,18 +4,19 @@ import (
"context"
"fmt"
"net"
"strconv"
"strings"
//nolint
"github.com/kube-vip/kube-vip/pkg/vip"
"github.com/kube-vip/kube-vip/pkg/kubevip"
api "github.com/osrg/gobgp/v3/api"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/osrg/gobgp/v3/pkg/server"
"google.golang.org/protobuf/types/known/anypb"
)
// AddPeer will add peers to the BGP configuration
func (b *Server) AddPeer(peer Peer) (err error) {
func (b *Server) AddPeer(peer kubevip.BGPPeer) (err error) {
p := &api.Peer{
Conf: &api.PeerConf{
NeighborAddress: peer.Address,
@@ -66,9 +67,9 @@ func (b *Server) AddPeer(peer Peer) (err error) {
},
}
peer.setMpbgpOptions(b.c)
peer.SetMpbgpOptions(b.c)
ipv4Address, ipv6Address, err := peer.findMpbgpAddresses(p, b.c)
ipv4Address, ipv6Address, err := peer.FindMpbgpAddresses(p, b.c)
if err != nil {
return fmt.Errorf("failed to get MP-BGP addresses: %w", err)
}
@@ -76,7 +77,7 @@ func (b *Server) AddPeer(peer Peer) (err error) {
mask := "128"
address := ipv4Address
family := api.Family_AFI_IP
if vip.IsIPv4(p.Conf.NeighborAddress) {
if utils.IsIPv4(p.Conf.NeighborAddress) {
mask = "32"
address = ipv6Address
family = api.Family_AFI_IP6
@@ -171,104 +172,6 @@ func (b *Server) getPath(ip net.IP) (path *api.Path) {
return
}
// ParseBGPPeerConfig - take a string and parses it into an array of peers
func ParseBGPPeerConfig(config string) (bgpPeers []Peer, err error) {
peers := strings.Split(config, ",")
if len(peers) == 0 {
return nil, fmt.Errorf("no BGP Peer configurations found")
}
for x := range peers {
peerStr := peers[x]
config := strings.Split(peerStr, "/")
peerStr = config[0]
if peerStr == "" {
continue
}
isV6Peer := peerStr[0] == '['
address := ""
if isV6Peer {
addressEndPos := strings.IndexByte(peerStr, ']')
if addressEndPos == -1 {
return nil, fmt.Errorf("no matching ] found for IPv6 BGP Peer")
}
address = peerStr[1:addressEndPos]
peerStr = peerStr[addressEndPos+1:]
}
peer := strings.Split(peerStr, ":")
if len(peer) < 2 {
return nil, fmt.Errorf("mandatory peering params <host>:<AS> incomplete")
}
if !isV6Peer {
address = peer[0]
}
ASNumber, err := strconv.ParseUint(peer[1], 10, 32)
if err != nil {
return nil, fmt.Errorf("BGP Peer AS format error [%s]", peer[1])
}
password := ""
if len(peer) >= 3 {
password = peer[2]
}
multiHop := false
if len(peer) >= 4 {
multiHop, err = strconv.ParseBool(peer[3])
if err != nil {
return nil, fmt.Errorf("BGP MultiHop format error (true/false) [%s]", peer[1])
}
}
var port uint64
if len(peer) >= 5 {
port, err = strconv.ParseUint(peer[4], 10, 16)
if err != nil {
return nil, fmt.Errorf("BGP Peer AS format error [%s]", peer[1])
}
} else {
port = 179
}
var mpbgpNexthop, mpbgpIPv4, mpbgpIPv6 string
if len(config) > 1 {
configData := strings.Split(config[1], ";")
for _, cfg := range configData {
c := strings.Split(cfg, "=")
switch c[0] {
case "mpbgp_nexthop":
mpbgpNexthop = c[1]
case "mpbgp_ipv4":
mpbgpIPv4 = c[1]
case "mpbgp_ipv6":
mpbgpIPv6 = c[1]
default:
return nil, fmt.Errorf("peer configuration parameter '%s' is not supported", c[0])
}
}
}
peerConfig := Peer{
Address: address,
AS: uint32(ASNumber),
Port: uint16(port),
Password: password,
MultiHop: multiHop,
MpbgpNexthop: mpbgpNexthop,
MpbgpIPv4: mpbgpIPv4,
MpbgpIPv6: mpbgpIPv6,
}
bgpPeers = append(bgpPeers, peerConfig)
}
return
}
func insertPolicy(s *server.BgpServer, address string, p *api.Peer, family api.Family_Afi) error {
familyType := "v4"
if family == api.Family_AFI_IP6 {

View File

@@ -3,6 +3,8 @@ package bgp
import (
"reflect"
"testing"
"github.com/kube-vip/kube-vip/pkg/kubevip"
)
func TestParseBGPPeerConfig(t *testing.T) {
@@ -12,13 +14,13 @@ func TestParseBGPPeerConfig(t *testing.T) {
tests := []struct {
name string
args args
wantBgpPeers []Peer
wantBgpPeers []kubevip.BGPPeer
wantErr bool
}{
{
name: "IPv4, default port",
args: args{config: "192.168.0.10:65000::false,192.168.0.11:65000::false"},
wantBgpPeers: []Peer{
wantBgpPeers: []kubevip.BGPPeer{
{Address: "192.168.0.10", Port: 179, AS: 65000, MultiHop: false},
{Address: "192.168.0.11", Port: 179, AS: 65000, MultiHop: false},
},
@@ -26,7 +28,7 @@ func TestParseBGPPeerConfig(t *testing.T) {
{
name: "IPv4, different port",
args: args{config: "192.168.0.10:65000::false:180,192.168.0.11:65000::false:190"},
wantBgpPeers: []Peer{
wantBgpPeers: []kubevip.BGPPeer{
{Address: "192.168.0.10", Port: 180, AS: 65000, MultiHop: false},
{Address: "192.168.0.11", Port: 190, AS: 65000, MultiHop: false},
},
@@ -34,14 +36,14 @@ func TestParseBGPPeerConfig(t *testing.T) {
{
name: "IPv6, multi-protocol",
args: args{config: "[fd00:1111:2222:3333:c7d9:7235:6bf7:5d52]:65501::false/mpbgp_nexthop=auto_sourceif"},
wantBgpPeers: []Peer{
wantBgpPeers: []kubevip.BGPPeer{
{Address: "fd00:1111:2222:3333:c7d9:7235:6bf7:5d52", Port: 179, AS: 65501, MultiHop: false, MpbgpNexthop: "auto_sourceif"},
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
gotBgpPeers, err := ParseBGPPeerConfig(tt.args.config)
gotBgpPeers, err := kubevip.ParseBGPPeerConfig(tt.args.config)
if (err != nil) != tt.wantErr {
t.Errorf("ParseBGPPeerConfig() error = %v, wantErr %v", err, tt.wantErr)
return

View File

@@ -7,13 +7,24 @@ import (
log "log/slog"
"github.com/kube-vip/kube-vip/pkg/kubevip"
api "github.com/osrg/gobgp/v3/api"
gobgp "github.com/osrg/gobgp/v3/pkg/server"
"github.com/prometheus/client_golang/prometheus"
)
// Server manages a server object
type Server struct {
s *gobgp.BgpServer
c *kubevip.BGPConfig
// This is a prometheus gauge indicating the state of the sessions.
// 1 means "ESTABLISHED", 0 means "NOT ESTABLISHED"
BGPSessionInfoGauge *prometheus.GaugeVec
}
// NewBGPServer takes a configuration and returns a running BGP server instance
func NewBGPServer(c *Config) (b *Server, err error) {
func NewBGPServer(c kubevip.BGPConfig) (b *Server, err error) {
if c.AS == 0 {
return nil, fmt.Errorf("you need to provide AS")
}
@@ -28,7 +39,7 @@ func NewBGPServer(c *Config) (b *Server, err error) {
b = &Server{
s: gobgp.NewBgpServer(),
c: c,
c: &c,
BGPSessionInfoGauge: prometheus.NewGaugeVec(prometheus.GaugeOpts{
Namespace: "kube_vip",

View File

@@ -151,7 +151,7 @@ func (cluster *Cluster) StartCluster(c *kubevip.Config, sm *Manager, bgpServer *
if c.EnableBGP && bgpServer == nil {
// Lets start BGP
log.Info("Starting the BGP server to advertise VIP routes to VGP peers")
bgpServer, err = bgp.NewBGPServer(&c.BGPConfig)
bgpServer, err = bgp.NewBGPServer(c.BGPConfig)
if err != nil {
log.Error("new BGP server", "err", err)
}

View File

@@ -18,6 +18,7 @@ import (
"github.com/kube-vip/kube-vip/pkg/bgp"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/loadbalancer"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/kube-vip/kube-vip/pkg/vip"
"github.com/vishvananda/netlink"
corev1 "k8s.io/api/core/v1"
@@ -173,6 +174,7 @@ func (cluster *Cluster) vipService(ctxArp, ctxDNS context.Context, c *kubevip.Co
log.Error("failed to check IP type", "IP", networkIP, "error", err)
continue
}
log.Debug("current ip to process", "ip", networkIP)
backendMap := &backendMapV4
if isNetworkV6 {
@@ -180,7 +182,9 @@ func (cluster *Cluster) vipService(ctxArp, ctxDNS context.Context, c *kubevip.Co
}
for entry := range *backendMap {
log.Debug("entry.Check() for entry", "entry", entry)
if entry.Check() {
log.Debug("entry.Check() true")
_, err = network.AddIP(true)
if err != nil {
log.Error("error adding address", "err", err)
@@ -268,8 +272,10 @@ func (cluster *Cluster) StartLoadBalancerService(ctx context.Context, c *kubevip
var arpWG sync.WaitGroup
log.Debug("StartLoadBalancerService")
for i := range cluster.Network {
network := cluster.Network[i]
log.Debug("current ip to process", "ip", network.IP())
if err := network.SetMask(c.VIPSubnet); err != nil {
log.Error("failed to set mask", "subnet", c.VIPSubnet, "err", err)
panic("")
@@ -278,16 +284,21 @@ func (cluster *Cluster) StartLoadBalancerService(ctx context.Context, c *kubevip
if err != nil {
log.Warn("attempted to clean existing VIP", "err", err)
}
log.Debug("config flags", "enable_routing_table", c.EnableRoutingTable, "enable_leader_election", c.EnableLeaderElection, "enable_services_election", c.EnableServicesElection)
if c.EnableRoutingTable && (c.EnableLeaderElection || c.EnableServicesElection) {
err = network.AddRoute(false)
if err != nil {
log.Warn(err.Error())
} else {
log.Info("successful add Route")
}
} else if !c.EnableRoutingTable {
if _, err = network.AddIP(false); err != nil {
log.Warn(err.Error())
}
}
if _, err = network.AddIP(false); err != nil {
log.Warn(err.Error())
} else {
log.Info("successful add IP")
}
if c.EnableARP {
@@ -330,7 +341,7 @@ func (cluster *Cluster) StartLoadBalancerService(ctx context.Context, c *kubevip
return
}
for i := range cluster.Network {
if c.EnableARP && cluster.arpMgr.Count(cluster.Network[i].ARPName()) > 0 {
if c.EnableARP && cluster.arpMgr.Count(cluster.Network[i].ARPName()) > 1 {
continue
}
log.Info("[VIP] Deleting VIP", "ip", cluster.Network[i].IP())
@@ -354,7 +365,7 @@ func (cluster *Cluster) layer2Update(ctx context.Context, network vip.Network, c
var ndp *vip.NdpResponder
var err error
ipString := network.IP()
if vip.IsIPv6(ipString) {
if utils.IsIPv6(ipString) {
if network.IPisLinkLocal() {
log.Error("layer2 is link-local can't use NDP", "address", ipString)

View File

@@ -7,6 +7,7 @@ import (
"github.com/kube-vip/kube-vip/pkg/iptables"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/nftables"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/kube-vip/kube-vip/pkg/vip"
)
@@ -19,7 +20,7 @@ func Teardown(podIP, vipIP, namespace, serviceUUID string, annotations map[strin
protocol := iptables.ProtocolIPv4
IPv6 := false
if vip.IsIPv6(podIP) {
if utils.IsIPv6(podIP) {
protocol = iptables.ProtocolIPv6
IPv6 = true
}

View File

@@ -57,7 +57,7 @@ func (b *BGP) clear(ctx *servicecontext.Context, lastKnownGoodEndpoint *string,
} else {
log.Info("deleted BGP host", "provider",
b.provider.GetLabel(), "ip", cluster.Network[i].IP(), "service name", service.Name, "namespace", service.Namespace)
ctx.ConfiguredNetworks.Delete(cluster.Network[i])
ctx.ConfiguredNetworks.Delete(cluster.Network[i].IP())
*leaderElectionActive = false
}
}
@@ -110,6 +110,10 @@ func ClearBGPHosts(service *v1.Service, instances *[]*instance.Instance, bgpServ
}
func ClearBGPHostsByInstance(instance *instance.Instance, bgpServer *bgp.Server) {
if instance == nil {
log.Error("failed to clear BGP host for nil instance")
return
}
for _, cluster := range instance.Clusters {
for i := range cluster.Network {
network := cluster.Network[i]

View File

@@ -117,9 +117,9 @@ func (rt *RoutingTable) deleteAction(service *v1.Service) {
}
func (rt *RoutingTable) setInstanceEndpointsStatus(service *v1.Service, endpoints []string) error {
instance := instance.FindServiceInstance(service, *rt.instances)
instance := instance.FindServiceInstanceWithTimeout(service, *rt.instances)
if instance == nil {
log.Error("failed to find the instance", "service", service.UID, "provider", rt.provider.GetLabel())
log.Error("failed to find the instance", "namespace", service.Namespace, "name", service.Name, "uid", service.UID, "provider", rt.provider.GetLabel())
} else {
for _, c := range instance.Clusters {
for n := range c.Network {
@@ -140,23 +140,34 @@ func (rt *RoutingTable) setInstanceEndpointsStatus(service *v1.Service, endpoint
func ClearRoutes(service *v1.Service, instances *[]*instance.Instance) []error {
errs := []error{}
if instance := instance.FindServiceInstance(service, *instances); instance != nil {
for _, cluster := range instance.Clusters {
for i := range cluster.Network {
route := cluster.Network[i].PrepareRoute()
// check if route we are about to delete is not referenced by more than one service
if CountRouteReferences(route, instances) <= 1 {
err := cluster.Network[i].DeleteRoute()
if err != nil && !errors.Is(err, syscall.ESRCH) {
log.Error("failed to delete route", "ip", cluster.Network[i].IP(), "err", err)
errs = append(errs, err)
}
log.Debug("deleted route", "ip",
cluster.Network[i].IP(), "service name", service.Name, "namespace", service.Namespace, "interface", cluster.Network[i].Interface())
if svcInst := instance.FindServiceInstance(service, *instances); svcInst != nil {
clearErrs := ClearRoutesByInstance(service, svcInst, instances)
errs = append(errs, clearErrs...)
}
return errs
}
func ClearRoutesByInstance(service *v1.Service, svcInst *instance.Instance, instances *[]*instance.Instance) []error {
if svcInst == nil {
return []error{fmt.Errorf("failed to remove routes for nil instance of service %s/%s, uid: %s", service.Namespace, service.Name, service.UID)}
}
errs := []error{}
for _, cluster := range svcInst.Clusters {
for i := range cluster.Network {
route := cluster.Network[i].PrepareRoute()
// check if route we are about to delete is not referenced by more than one service
if CountRouteReferences(route, instances) <= 1 {
err := cluster.Network[i].DeleteRoute()
if err != nil && !errors.Is(err, syscall.ESRCH) {
log.Error("failed to delete route", "ip", cluster.Network[i].IP(), "err", err)
errs = append(errs, err)
}
log.Debug("deleted route", "ip",
cluster.Network[i].IP(), "service name", service.Name, "namespace", service.Namespace, "interface", cluster.Network[i].Interface())
}
}
}
return errs
}

View File

@@ -71,7 +71,7 @@ func (ep *Endpointslices) GetAllEndpoints() ([]string, error) {
func (ep *Endpointslices) GetLocalEndpoints(id string, _ *kubevip.Config) ([]string, error) {
var localEndpoints []string
for _, endpoint := range ep.endpoints.Endpoints {
if !*endpoint.Conditions.Serving {
if endpoint.Conditions.Serving == nil || !*endpoint.Conditions.Serving {
continue
}
for _, address := range endpoint.Addresses {

View File

@@ -221,7 +221,7 @@ func (m *member) tryToBeLeader(ctx context.Context) {
func (m *member) resignOnCancel(ctx context.Context) {
<-ctx.Done()
if err := m.election.Resign(m.client.Ctx()); err != nil {
if err := m.election.Resign(m.client.Ctx()); err != nil && !errors.Is(err, context.Canceled) {
log.Error("Failed to resign after the context was canceled", "err", err)
}
}

View File

@@ -32,7 +32,7 @@ func TestRunElectionWithMemberIDCollision(t *testing.T) {
Client: cli,
},
Name: electionName,
MemberID: "my-host",
MemberID: randomElectionNameForTest("my-host"),
LeaseDurationSeconds: 1,
Callbacks: etcd.LeaderCallbacks{
OnStartedLeading: func(ctx context.Context) {
@@ -90,19 +90,21 @@ func TestRunElectionWithTwoMembersAndReelection(t *testing.T) {
config1 := configBase
config1.EtcdConfig.Client = cliMember1
config1.MemberID = "my-host"
config1.MemberID = randomElectionNameForTest("my-host")
uniqueID := rand.Uint64()
config1.MemberUniqueID = &uniqueID
config1.Callbacks = baseCallbacksForName(config1.MemberID)
syncMembers := make(chan (any))
config1.Callbacks.OnStartedLeading = func(_ context.Context) {
log.Println("I'm my-host, the new leader!!!!")
close(syncMembers)
log.Println("Losing the leadership on purpose by stopping renewing the lease")
g.Expect(cliMember1.Lease.Close()).To(Succeed())
log.Println("Member1 leases closed")
}
config2 := configBase
config2.MemberID = "my-other-host"
config2.MemberID = randomElectionNameForTest("my-other-host")
config2.Callbacks = baseCallbacksForName(config2.MemberID)
config2.Callbacks.OnStartedLeading = func(_ context.Context) {
log.Println("I'm my-other-host, the new leader!!!!")
@@ -116,14 +118,14 @@ func TestRunElectionWithTwoMembersAndReelection(t *testing.T) {
go func() {
defer wg.Done()
g.Expect(etcd.RunElection(member1Ctx, &config1)).To(Succeed())
log.Println("Member1 routine done")
log.Printf("%s routine done\n", config1.MemberID)
}()
go func() {
defer wg.Done()
time.Sleep(time.Millisecond * 50) // Make sure member1 becomes leader
<-syncMembers
g.Expect(etcd.RunElection(member2Ctx, &config2)).To(Succeed())
log.Println("Member2 routine done")
log.Printf("%s routine done\n", config2.MemberID)
}()
wg.Wait()

View File

@@ -5,6 +5,7 @@ import (
"net"
"strconv"
"strings"
"time"
"log/slog"
log "log/slog"
@@ -17,6 +18,7 @@ import (
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/networkinterface"
"github.com/kube-vip/kube-vip/pkg/sysctl"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/kube-vip/kube-vip/pkg/vip"
)
@@ -49,8 +51,7 @@ type Port struct {
}
func NewInstance(svc *v1.Service, config *kubevip.Config, intfMgr *networkinterface.Manager, arpMgr *arp.Manager) (*Instance, error) {
instanceAddresses := FetchServiceAddresses(svc)
//instanceUID := string(svc.UID)
instanceAddresses, _ := FetchServiceAddresses(svc)
var newVips []*kubevip.Config
var link netlink.Link
@@ -120,7 +121,7 @@ func NewInstance(svc *v1.Service, config *kubevip.Config, intfMgr *networkinterf
subnet := ""
var err error
if vip.IsIPv4(address) {
if utils.IsIPv4(address) {
if ipv4AutoSubnet {
subnet, err = autoFindSubnet(link, address)
if err != nil {
@@ -407,76 +408,108 @@ func (i *Instance) startDHCP() error {
return nil
}
// FetchIngressAddresses tries to get the addresses from status.loadBalancerIP
func FetchLoadBalancerIngressAddresses(s *v1.Service) []string {
// FetchLoadBalancerIngressAddresses tries to get the addresses from status.loadBalancerIP
func FetchLoadBalancerIngress(s *v1.Service) ([]string, []string) {
// If the service has no status, return empty
lbStatusAddresses := []string{}
lbStatusHostnames := []string{}
if len(s.Status.LoadBalancer.Ingress) == 0 {
return lbStatusAddresses
return lbStatusAddresses, lbStatusHostnames
}
for _, ingress := range s.Status.LoadBalancer.Ingress {
if ingress.IP != "" {
lbStatusAddresses = append(lbStatusAddresses, ingress.IP)
}
// TODO: Handle hostname if needed
if ingress.Hostname != "" {
lbStatusHostnames = append(lbStatusHostnames, ingress.Hostname)
}
}
return lbStatusAddresses
return lbStatusAddresses, lbStatusHostnames
}
// FetchServiceAddresses tries to get the addresses from annotations
// kube-vip.io/loadbalancerIPs, then from spec.loadbalancerIP
func FetchServiceAddresses(s *v1.Service) []string {
func FetchServiceAddresses(s *v1.Service) ([]string, []string) {
annotationAvailable := false
if s.Annotations != nil {
if v, annotationAvailable := s.Annotations[kubevip.LoadbalancerIPAnnotation]; annotationAvailable {
ips := strings.Split(v, ",")
var trimmedIPs []string
for _, ip := range ips {
trimmedIPs = append(trimmedIPs, strings.TrimSpace(ip))
var trimmedHostnames []string
for _, a := range ips {
a = strings.TrimSpace(a)
ip := net.ParseIP(a)
if ip == nil {
// this is probably a DNS name
trimmedHostnames = append(trimmedHostnames, a)
} else {
trimmedIPs = append(trimmedIPs, ip.String())
}
}
return trimmedIPs
return trimmedIPs, trimmedHostnames
}
}
lbStatusAddresses := []string{}
lbStatusHostnames := []string{}
if !annotationAvailable {
if len(s.Status.LoadBalancer.Ingress) > 0 {
for _, ingress := range s.Status.LoadBalancer.Ingress {
lbStatusAddresses = append(lbStatusAddresses, ingress.IP)
lbStatusAddresses, lbStatusHostnames = FetchLoadBalancerIngress(s)
}
// Spec.LoadBalancerIP legacy handling
// if the loadBalancerIP is different from Status.LoadBalancer.Ingress IPs
// return the legacy LB as spec wins over status.
if lbIP := net.ParseIP(s.Spec.LoadBalancerIP); lbIP != nil && len(lbStatusAddresses) > 0 {
isLbIPv4 := utils.IsIPv4(s.Spec.LoadBalancerIP)
for _, a := range lbStatusAddresses {
if lbStatusIP := net.ParseIP(a); lbStatusIP != nil && lbIP != nil && utils.IsIPv4(a) == isLbIPv4 && !lbIP.Equal(lbStatusIP) {
return []string{s.Spec.LoadBalancerIP}, []string{}
}
}
}
lbIP := net.ParseIP(s.Spec.LoadBalancerIP)
isLbIPv4 := vip.IsIPv4(s.Spec.LoadBalancerIP)
if len(lbStatusAddresses) > 0 {
for _, a := range lbStatusAddresses {
if lbStatusIP := net.ParseIP(a); lbStatusIP != nil && lbIP != nil && vip.IsIPv4(a) == isLbIPv4 && !lbIP.Equal(lbStatusIP) {
return []string{s.Spec.LoadBalancerIP}
}
}
return lbStatusAddresses
if len(lbStatusAddresses) > 0 || len(lbStatusHostnames) > 0 {
return lbStatusAddresses, lbStatusHostnames
}
if s.Spec.LoadBalancerIP != "" {
return []string{s.Spec.LoadBalancerIP}
return []string{s.Spec.LoadBalancerIP}, []string{}
}
return []string{}
return []string{}, []string{}
}
func FindServiceInstance(svc *v1.Service, instances []*Instance) *Instance {
log.Debug("finding service", "UID", svc.UID)
log.Debug("finding service", "namespace", svc.Namespace, "name", svc.Name, "UID", svc.UID)
for i := range instances {
log.Debug("saved service", "instance", i, "UID", instances[i].ServiceSnapshot.UID)
if instances[i].ServiceSnapshot.UID == svc.UID {
return instances[i]
}
}
log.Debug("instance not found", "namespace", svc.Namespace, "name", svc.Name, "UID", svc.UID)
return nil
}
func FindServiceInstanceWithTimeout(svc *v1.Service, instances []*Instance) *Instance {
log.Debug("finding service with timeout", "namespace", svc.Namespace, "name", svc.Name, "UID", svc.UID)
ticker := time.NewTicker(time.Millisecond * 200)
defer ticker.Stop()
to := time.NewTimer(time.Second * 60)
defer to.Stop()
for {
select {
case <-to.C:
return nil
case <-ticker.C:
for i := range instances {
log.Debug("saved service", "instance", i, "UID", instances[i].ServiceSnapshot.UID)
if instances[i].ServiceSnapshot.UID == svc.UID {
return instances[i]
}
}
}
}
}

View File

@@ -40,10 +40,24 @@ const (
// Flush the conntrack rules (remove existing sessions) once Egress is configured
FlushContrack = "kube-vip.io/flush-conntrack"
// Configure LoadBalancer IPs instead of relying on a controller
LoadbalancerIPAnnotation = "kube-vip.io/loadbalancerIPs"
LoadbalancerHostname = "kube-vip.io/loadbalancerHostname"
ServiceInterface = "kube-vip.io/serviceInterface"
UpnpEnabled = "kube-vip.io/forwardUPNP"
// Ignore the LoadBalancer Service
LoadbalancerIgnore = "kube-vip.io/ignore"
// Used to configure DHCP with a Hostname
LoadbalancerHostname = "kube-vip.io/loadbalancerHostname"
// Define an interface name to bind the address of the LoadBalancer to
ServiceInterface = "kube-vip.io/serviceInterface"
ServiceSecurityIgnore = "kube-vip.io/ignore-service-security"
// Enable UPNP on a Service
UpnpEnabled = "kube-vip.io/forwardUPNP"
RPFilter = "kube-vip.io/rp_filter" // Set the return path filter for a specific service interface
ServiceLease = "kube-vip.io/leaseName"
)

View File

@@ -1,18 +1,19 @@
package bgp
package kubevip
import (
"fmt"
"net"
"strconv"
"strings"
"github.com/kube-vip/kube-vip/pkg/vip"
"github.com/kube-vip/kube-vip/pkg/utils"
api "github.com/osrg/gobgp/v3/api"
gobgp "github.com/osrg/gobgp/v3/pkg/server"
"github.com/prometheus/client_golang/prometheus"
"github.com/vishvananda/netlink"
)
// Peer defines a BGP Peer
type Peer struct {
type BGPPeer struct {
Address string
Port uint16
AS uint32
@@ -23,21 +24,131 @@ type Peer struct {
MpbgpIPv6 string
}
func (p *Peer) setMpbgpOptions(server *Config) {
if p.MpbgpNexthop == "" {
p.MpbgpNexthop = server.MpbgpNexthop
}
// Config defines the BGP server configuration
type BGPConfig struct {
AS uint32
RouterID string
SourceIP string
SourceIF string
MpbgpNexthop string
MpbgpIPv4 string
MpbgpIPv6 string
if p.MpbgpIPv4 == "" {
p.MpbgpIPv4 = server.MpbgpIPv4
}
HoldTime uint64
KeepaliveInterval uint64
if p.MpbgpIPv6 == "" {
p.MpbgpIPv6 = server.MpbgpIPv6
}
Peers []BGPPeer
Zebra ZebraConfig
}
func (p *Peer) findMpbgpAddresses(ap *api.Peer, server *Config) (string, string, error) {
// Defines Zebra connection configuration. More on the topic - https://github.com/osrg/gobgp/blob/master/docs/sources/zebra.md#configuration
type ZebraConfig struct {
Enabled bool
URL string
Version uint32
SoftwareName string
}
// ParseBGPPeerConfig - take a string and parses it into an array of peers
func ParseBGPPeerConfig(config string) (bgpPeers []BGPPeer, err error) {
peers := strings.Split(config, ",")
if len(peers) == 0 {
return nil, fmt.Errorf("no BGP Peer configurations found")
}
for x := range peers {
peerStr := peers[x]
config := strings.Split(peerStr, "/")
peerStr = config[0]
if peerStr == "" {
continue
}
isV6Peer := peerStr[0] == '['
address := ""
if isV6Peer {
addressEndPos := strings.IndexByte(peerStr, ']')
if addressEndPos == -1 {
return nil, fmt.Errorf("no matching ] found for IPv6 BGP Peer")
}
address = peerStr[1:addressEndPos]
peerStr = peerStr[addressEndPos+1:]
}
peer := strings.Split(peerStr, ":")
if len(peer) < 2 {
return nil, fmt.Errorf("mandatory peering params <host>:<AS> incomplete")
}
if !isV6Peer {
address = peer[0]
}
ASNumber, err := strconv.ParseUint(peer[1], 10, 32)
if err != nil {
return nil, fmt.Errorf("BGP Peer AS format error [%s]", peer[1])
}
password := ""
if len(peer) >= 3 {
password = peer[2]
}
multiHop := false
if len(peer) >= 4 {
multiHop, err = strconv.ParseBool(peer[3])
if err != nil {
return nil, fmt.Errorf("BGP MultiHop format error (true/false) [%s]", peer[1])
}
}
var port uint64
if len(peer) >= 5 {
port, err = strconv.ParseUint(peer[4], 10, 16)
if err != nil {
return nil, fmt.Errorf("BGP Peer AS format error [%s]", peer[1])
}
} else {
port = 179
}
var mpbgpNexthop, mpbgpIPv4, mpbgpIPv6 string
if len(config) > 1 {
configData := strings.Split(config[1], ";")
for _, cfg := range configData {
c := strings.Split(cfg, "=")
switch c[0] {
case "mpbgp_nexthop":
mpbgpNexthop = c[1]
case "mpbgp_ipv4":
mpbgpIPv4 = c[1]
case "mpbgp_ipv6":
mpbgpIPv6 = c[1]
default:
return nil, fmt.Errorf("peer configuration parameter '%s' is not supported", c[0])
}
}
}
peerConfig := BGPPeer{
Address: address,
AS: uint32(ASNumber),
Port: uint16(port),
Password: password,
MultiHop: multiHop,
MpbgpNexthop: mpbgpNexthop,
MpbgpIPv4: mpbgpIPv4,
MpbgpIPv6: mpbgpIPv6,
}
bgpPeers = append(bgpPeers, peerConfig)
}
return
}
func (p *BGPPeer) FindMpbgpAddresses(ap *api.Peer, server *BGPConfig) (string, string, error) {
var ipv4Address, ipv6Address string
switch p.MpbgpNexthop {
case "fixed":
@@ -64,20 +175,20 @@ func (p *Peer) findMpbgpAddresses(ap *api.Peer, server *Config) (string, string,
ap.Transport.LocalAddress = server.SourceIP
// Resolve the local interface by SourceIP
iface, err := vip.GetInterfaceByIP(server.SourceIP)
iface, err := utils.GetInterfaceByIP(server.SourceIP)
if err != nil {
return "", "", fmt.Errorf("failed to get interface by IP: %v", err)
}
if vip.IsIPv4(server.SourceIP) {
if utils.IsIPv4(server.SourceIP) {
// Get the non link-local IPv6 address on that interface
ipv6Address, err = vip.GetNonLinkLocalIP(iface, netlink.FAMILY_V6)
ipv6Address, err = utils.GetNonLinkLocalIP(iface, netlink.FAMILY_V6)
if err != nil {
return "", "", fmt.Errorf("failed to get non link-local IPv6 address: %v", err)
}
} else {
// Get the non link-local IPv4 address on that interface
ipv4Address, err = vip.GetNonLinkLocalIP(iface, netlink.FAMILY_V4)
ipv4Address, err = utils.GetNonLinkLocalIP(iface, netlink.FAMILY_V4)
if err != nil {
return "", "", fmt.Errorf("failed to get non link-local IPv4 address: %v", err)
}
@@ -91,13 +202,13 @@ func (p *Peer) findMpbgpAddresses(ap *api.Peer, server *Config) (string, string,
}
// Get the non link-local IPv4 address on that interface
ipv4Address, err = vip.GetNonLinkLocalIP(&iface, netlink.FAMILY_V4)
ipv4Address, err = utils.GetNonLinkLocalIP(&iface, netlink.FAMILY_V4)
if err != nil {
return "", "", fmt.Errorf("failed to get non link-local IPv4 address: %v", err)
}
// Get the non link-local IPv6 address on that interface
ipv6Address, err = vip.GetNonLinkLocalIP(&iface, netlink.FAMILY_V6)
ipv6Address, err = utils.GetNonLinkLocalIP(&iface, netlink.FAMILY_V6)
if err != nil {
return "", "", fmt.Errorf("failed to get non link-local IPv6 address: %v", err)
}
@@ -108,38 +219,16 @@ func (p *Peer) findMpbgpAddresses(ap *api.Peer, server *Config) (string, string,
return ipv4Address, ipv6Address, nil
}
// Config defines the BGP server configuration
type Config struct {
AS uint32
RouterID string
SourceIP string
SourceIF string
MpbgpNexthop string
MpbgpIPv4 string
MpbgpIPv6 string
func (p *BGPPeer) SetMpbgpOptions(server *BGPConfig) {
if p.MpbgpNexthop == "" {
p.MpbgpNexthop = server.MpbgpNexthop
}
HoldTime uint64
KeepaliveInterval uint64
if p.MpbgpIPv4 == "" {
p.MpbgpIPv4 = server.MpbgpIPv4
}
Peers []Peer
Zebra ZebraConfig
}
// Defines Zebra connection configuration. More on the topic - https://github.com/osrg/gobgp/blob/master/docs/sources/zebra.md#configuration
type ZebraConfig struct {
Enabled bool
URL string
Version uint32
SoftwareName string
}
// Server manages a server object
type Server struct {
s *gobgp.BgpServer
c *Config
// This is a prometheus gauge indicating the state of the sessions.
// 1 means "ESTABLISHED", 0 means "NOT ESTABLISHED"
BGPSessionInfoGauge *prometheus.GaugeVec
if p.MpbgpIPv6 == "" {
p.MpbgpIPv6 = server.MpbgpIPv6
}
}

View File

@@ -6,11 +6,12 @@ import (
"math"
"math/bits"
"os"
"path/filepath"
"strconv"
"strings"
"github.com/kube-vip/kube-vip/pkg/bgp"
"github.com/kube-vip/kube-vip/pkg/detector"
"sigs.k8s.io/yaml"
)
// ParseEnvironment - will popultate the configuration from environment variables
@@ -422,7 +423,7 @@ func ParseEnvironment(c *Config) error {
// Peer AS
env = os.Getenv(bgpPeers)
if env != "" {
peers, err := bgp.ParseBGPPeerConfig(env)
peers, err := ParseBGPPeerConfig(env)
if err != nil {
return err
}
@@ -667,5 +668,282 @@ func ParseEnvironment(c *Config) error {
c.EgressClean = b
}
// check for configuration file path
env = os.Getenv(configFile)
if env != "" {
c.ConfigFile = env
}
return nil
}
// LoadConfigFromFile loads configuration from a JSON or YAML file
func LoadConfigFromFile(configFilePath string) (*Config, error) {
if configFilePath == "" {
return nil, fmt.Errorf("config file path is empty")
}
// Check if file exists
if _, err := os.Stat(configFilePath); os.IsNotExist(err) {
return nil, fmt.Errorf("config file does not exist: %s", configFilePath)
}
// Read file content
data, err := os.ReadFile(configFilePath)
if err != nil {
return nil, fmt.Errorf("failed to read config file %s: %v", configFilePath, err)
}
var config Config
ext := strings.ToLower(filepath.Ext(configFilePath))
switch ext {
case ".json":
err = json.Unmarshal(data, &config)
if err != nil {
return nil, fmt.Errorf("failed to parse JSON config file %s: %v", configFilePath, err)
}
case ".yaml", ".yml":
err = yaml.Unmarshal(data, &config)
if err != nil {
return nil, fmt.Errorf("failed to parse YAML config file %s: %v", configFilePath, err)
}
default:
return nil, fmt.Errorf("unsupported config file format %s. Supported formats: .json, .yaml, .yml", ext)
}
return &config, nil
}
// MergeConfigFromFile merges configuration loaded from file with existing config
// Priority: command line flags > environment variables > config file
func MergeConfigFromFile(c *Config, configFilePath string) error {
if configFilePath == "" {
return nil // No config file specified, nothing to merge
}
fileConfig, err := LoadConfigFromFile(configFilePath)
if err != nil {
return err
}
// Merge file config with existing config
// Only set values from file if they haven't been set by flags or env vars
mergeConfigValues(c, fileConfig)
return nil
}
// mergeConfigValues merges values from fileConfig into baseConfig
// Only overwrites zero values in baseConfig
func mergeConfigValues(baseConfig, fileConfig *Config) {
// Basic configuration
if baseConfig.Logging == 0 && fileConfig.Logging != 0 {
baseConfig.Logging = fileConfig.Logging
}
// Network configuration
if baseConfig.Interface == "" && fileConfig.Interface != "" {
baseConfig.Interface = fileConfig.Interface
}
if baseConfig.ServicesInterface == "" && fileConfig.ServicesInterface != "" {
baseConfig.ServicesInterface = fileConfig.ServicesInterface
}
if baseConfig.VIP == "" && fileConfig.VIP != "" {
baseConfig.VIP = fileConfig.VIP
}
if baseConfig.VIPSubnet == "" && fileConfig.VIPSubnet != "" {
baseConfig.VIPSubnet = fileConfig.VIPSubnet
}
if baseConfig.Address == "" && fileConfig.Address != "" {
baseConfig.Address = fileConfig.Address
}
if baseConfig.Port == 0 && fileConfig.Port != 0 {
baseConfig.Port = fileConfig.Port
}
if baseConfig.NodeName == "" && fileConfig.NodeName != "" {
baseConfig.NodeName = fileConfig.NodeName
}
// Boolean flags - only merge if not explicitly set
if !baseConfig.EnableARP && fileConfig.EnableARP {
baseConfig.EnableARP = fileConfig.EnableARP
}
if !baseConfig.EnableBGP && fileConfig.EnableBGP {
baseConfig.EnableBGP = fileConfig.EnableBGP
}
if !baseConfig.EnableWireguard && fileConfig.EnableWireguard {
baseConfig.EnableWireguard = fileConfig.EnableWireguard
}
if !baseConfig.EnableRoutingTable && fileConfig.EnableRoutingTable {
baseConfig.EnableRoutingTable = fileConfig.EnableRoutingTable
}
if !baseConfig.EnableControlPlane && fileConfig.EnableControlPlane {
baseConfig.EnableControlPlane = fileConfig.EnableControlPlane
}
if !baseConfig.DetectControlPlane && fileConfig.DetectControlPlane {
baseConfig.DetectControlPlane = fileConfig.DetectControlPlane
}
if !baseConfig.EnableServices && fileConfig.EnableServices {
baseConfig.EnableServices = fileConfig.EnableServices
}
if !baseConfig.EnableServicesElection && fileConfig.EnableServicesElection {
baseConfig.EnableServicesElection = fileConfig.EnableServicesElection
}
if !baseConfig.EnableNodeLabeling && fileConfig.EnableNodeLabeling {
baseConfig.EnableNodeLabeling = fileConfig.EnableNodeLabeling
}
if !baseConfig.EnableLoadBalancer && fileConfig.EnableLoadBalancer {
baseConfig.EnableLoadBalancer = fileConfig.EnableLoadBalancer
}
if !baseConfig.DDNS && fileConfig.DDNS {
baseConfig.DDNS = fileConfig.DDNS
}
if !baseConfig.SingleNode && fileConfig.SingleNode {
baseConfig.SingleNode = fileConfig.SingleNode
}
if !baseConfig.StartAsLeader && fileConfig.StartAsLeader {
baseConfig.StartAsLeader = fileConfig.StartAsLeader
}
// Service configuration
if baseConfig.Namespace == "" && fileConfig.Namespace != "" {
baseConfig.Namespace = fileConfig.Namespace
}
if baseConfig.ServiceNamespace == "" && fileConfig.ServiceNamespace != "" {
baseConfig.ServiceNamespace = fileConfig.ServiceNamespace
}
if baseConfig.ServicesLeaseName == "" && fileConfig.ServicesLeaseName != "" {
baseConfig.ServicesLeaseName = fileConfig.ServicesLeaseName
}
// LoadBalancer configuration
if baseConfig.LoadBalancerPort == 0 && fileConfig.LoadBalancerPort != 0 {
baseConfig.LoadBalancerPort = fileConfig.LoadBalancerPort
}
if baseConfig.LoadBalancerForwardingMethod == "" && fileConfig.LoadBalancerForwardingMethod != "" {
baseConfig.LoadBalancerForwardingMethod = fileConfig.LoadBalancerForwardingMethod
}
if baseConfig.LoadBalancerClassName == "" && fileConfig.LoadBalancerClassName != "" {
baseConfig.LoadBalancerClassName = fileConfig.LoadBalancerClassName
}
// Routing Table configuration
if baseConfig.RoutingTableID == 0 && fileConfig.RoutingTableID != 0 {
baseConfig.RoutingTableID = fileConfig.RoutingTableID
}
if baseConfig.RoutingTableType == 0 && fileConfig.RoutingTableType != 0 {
baseConfig.RoutingTableType = fileConfig.RoutingTableType
}
if baseConfig.RoutingProtocol == 0 && fileConfig.RoutingProtocol != 0 {
baseConfig.RoutingProtocol = fileConfig.RoutingProtocol
}
// BGP configuration
mergeBGPConfig(&baseConfig.BGPConfig, &fileConfig.BGPConfig)
// Kubernetes configuration
if baseConfig.K8sConfigFile == "" && fileConfig.K8sConfigFile != "" {
baseConfig.K8sConfigFile = fileConfig.K8sConfigFile
}
// Leader Election configuration
mergeLeaderElectionConfig(&baseConfig.KubernetesLeaderElection, &fileConfig.KubernetesLeaderElection)
// Prometheus configuration
if baseConfig.PrometheusHTTPServer == "" && fileConfig.PrometheusHTTPServer != "" {
baseConfig.PrometheusHTTPServer = fileConfig.PrometheusHTTPServer
}
// DNS configuration
if baseConfig.DNSMode == "" && fileConfig.DNSMode != "" {
baseConfig.DNSMode = fileConfig.DNSMode
}
// Health check configuration
if baseConfig.HealthCheckPort == 0 && fileConfig.HealthCheckPort != 0 {
baseConfig.HealthCheckPort = fileConfig.HealthCheckPort
}
// Egress configuration
if baseConfig.EgressPodCidr == "" && fileConfig.EgressPodCidr != "" {
baseConfig.EgressPodCidr = fileConfig.EgressPodCidr
}
if baseConfig.EgressServiceCidr == "" && fileConfig.EgressServiceCidr != "" {
baseConfig.EgressServiceCidr = fileConfig.EgressServiceCidr
}
// Mirror configuration
if baseConfig.MirrorDestInterface == "" && fileConfig.MirrorDestInterface != "" {
baseConfig.MirrorDestInterface = fileConfig.MirrorDestInterface
}
// Iptables configuration
if baseConfig.IptablesBackend == "" && fileConfig.IptablesBackend != "" {
baseConfig.IptablesBackend = fileConfig.IptablesBackend
}
// Backend health check interval
if baseConfig.BackendHealthCheckInterval == 0 && fileConfig.BackendHealthCheckInterval != 0 {
baseConfig.BackendHealthCheckInterval = fileConfig.BackendHealthCheckInterval
}
// ARP broadcast rate
if baseConfig.ArpBroadcastRate == 0 && fileConfig.ArpBroadcastRate != 0 {
baseConfig.ArpBroadcastRate = fileConfig.ArpBroadcastRate
}
// Annotations
if baseConfig.Annotations == "" && fileConfig.Annotations != "" {
baseConfig.Annotations = fileConfig.Annotations
}
// Load balancers slice
if len(baseConfig.LoadBalancers) == 0 && len(fileConfig.LoadBalancers) > 0 {
baseConfig.LoadBalancers = fileConfig.LoadBalancers
}
}
// mergeBGPConfig merges BGP configuration
func mergeBGPConfig(base, file *BGPConfig) {
if base.RouterID == "" && file.RouterID != "" {
base.RouterID = file.RouterID
}
if base.AS == 0 && file.AS != 0 {
base.AS = file.AS
}
if base.SourceIF == "" && file.SourceIF != "" {
base.SourceIF = file.SourceIF
}
if base.SourceIP == "" && file.SourceIP != "" {
base.SourceIP = file.SourceIP
}
if base.HoldTime == 0 && file.HoldTime != 0 {
base.HoldTime = file.HoldTime
}
if base.KeepaliveInterval == 0 && file.KeepaliveInterval != 0 {
base.KeepaliveInterval = file.KeepaliveInterval
}
if len(base.Peers) == 0 && len(file.Peers) > 0 {
base.Peers = file.Peers
}
}
// mergeLeaderElectionConfig merges leader election configuration
func mergeLeaderElectionConfig(base, file *KubernetesLeaderElection) {
if base.LeaseName == "" && file.LeaseName != "" {
base.LeaseName = file.LeaseName
}
if base.LeaseDuration == 0 && file.LeaseDuration != 0 {
base.LeaseDuration = file.LeaseDuration
}
if base.RenewDeadline == 0 && file.RenewDeadline != 0 {
base.RenewDeadline = file.RenewDeadline
}
if base.RetryPeriod == 0 && file.RetryPeriod != 0 {
base.RetryPeriod = file.RetryPeriod
}
if len(base.LeaseAnnotations) == 0 && len(file.LeaseAnnotations) > 0 {
base.LeaseAnnotations = file.LeaseAnnotations
}
}

View File

@@ -234,4 +234,7 @@ const (
// egressClean enables egress cleaning on kube-vip's start
egressClean = "egress_clean"
// configFile defines the path to a JSON/YAML configuration file
configFile = "config_file"
)

View File

@@ -0,0 +1,559 @@
package kubevip
import (
"os"
"path/filepath"
"testing"
)
func TestLoadConfigFromFile(t *testing.T) {
// Create temporary directory for test files
tmpDir, err := os.MkdirTemp("", "kube-vip-config-test")
if err != nil {
t.Fatalf("Failed to create temp dir: %v", err)
}
defer os.RemoveAll(tmpDir)
tests := []struct {
name string
filename string
content string
expectedConfig *Config
wantErr bool
errContains string
}{
{
name: "Valid YAML config",
filename: "config.yaml",
content: `
logging: 2
enableARP: true
enableControlPlane: true
enableServices: true
address: "192.168.1.100"
port: 6443
interface: "eth0"
namespace: "kube-system"
vipSubnet: "192.168.1.0/24"
leaseName: "test-lease"
leaseDuration: 15
renewDeadline: 10
retryPeriod: 2
prometheusHTTPServer: ":2112"
`,
expectedConfig: &Config{
Logging: 2,
EnableARP: true,
EnableControlPlane: true,
EnableServices: true,
Address: "192.168.1.100",
Port: 6443,
Interface: "eth0",
Namespace: "kube-system",
VIPSubnet: "192.168.1.0/24",
PrometheusHTTPServer: ":2112",
KubernetesLeaderElection: KubernetesLeaderElection{
LeaseName: "test-lease",
LeaseDuration: 15,
RenewDeadline: 10,
RetryPeriod: 2,
},
},
wantErr: false,
},
{
name: "Valid JSON config",
filename: "config.json",
content: `{
"logging": 3,
"enableBGP": true,
"enableServices": true,
"address": "10.0.0.100",
"port": 8443,
"interface": "ens192",
"namespace": "kube-system",
"loadBalancers": [
{
"name": "control-plane",
"ports": [
{
"type": "TCP",
"port": 6443
}
],
"bindToVip": true,
"forwardingMethod": "local"
}
]
}`,
expectedConfig: &Config{
Logging: 3,
EnableBGP: true,
EnableServices: true,
Address: "10.0.0.100",
Port: 8443,
Interface: "ens192",
Namespace: "kube-system",
LoadBalancers: []LoadBalancer{
{
Name: "control-plane",
Ports: []Port{
{
Type: "TCP",
Port: 6443,
},
},
BindToVip: true,
ForwardingMethod: "local",
},
},
},
wantErr: false,
},
{
name: "Complex BGP config",
filename: "bgp-config.yaml",
content: `
enableBGP: true
bgpConfig:
routerID: "192.168.1.1"
as: 65000
sourceIF: "eth0"
holdTime: 60
keepaliveInterval: 20
peers:
- address: "192.168.1.2"
as: 65001
port: 179
multiHop: false
- address: "192.168.1.3"
as: 65002
port: 179
multiHop: true
`,
expectedConfig: &Config{
EnableBGP: true,
BGPConfig: BGPConfig{
RouterID: "192.168.1.1",
AS: 65000,
SourceIF: "eth0",
HoldTime: 60,
KeepaliveInterval: 20,
Peers: []BGPPeer{
{
Address: "192.168.1.2",
AS: 65001,
Port: 179,
MultiHop: false,
},
{
Address: "192.168.1.3",
AS: 65002,
Port: 179,
MultiHop: true,
},
},
},
},
wantErr: false,
},
{
name: "Invalid JSON",
filename: "invalid.json",
content: `{"logging": 2, "invalid": }`,
wantErr: true,
errContains: "failed to parse JSON config file",
},
{
name: "Invalid YAML",
filename: "invalid.yaml",
content: "logging: 2\ninvalid: [unclosed",
wantErr: true,
errContains: "failed to parse YAML config file",
},
{
name: "Unsupported format",
filename: "config.txt",
content: "logging=2",
wantErr: true,
errContains: "unsupported config file format",
},
{
name: "Empty path",
filename: "",
content: "",
wantErr: true,
errContains: "config file path is empty",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var filePath string
if tt.filename != "" {
filePath = filepath.Join(tmpDir, tt.filename)
if err := os.WriteFile(filePath, []byte(tt.content), 0600); err != nil {
t.Fatalf("Failed to write test file: %v", err)
}
}
config, err := LoadConfigFromFile(filePath)
if tt.wantErr {
if err == nil {
t.Errorf("LoadConfigFromFile() expected error, got nil")
return
}
if tt.errContains != "" && !containsString(err.Error(), tt.errContains) {
t.Errorf("LoadConfigFromFile() error = %v, expected to contain %v", err, tt.errContains)
}
return
}
if err != nil {
t.Errorf("LoadConfigFromFile() unexpected error = %v", err)
return
}
if config == nil {
t.Errorf("LoadConfigFromFile() returned nil config")
return
}
// Compare key fields
if config.Logging != tt.expectedConfig.Logging {
t.Errorf("Logging = %v, expected %v", config.Logging, tt.expectedConfig.Logging)
}
if config.EnableARP != tt.expectedConfig.EnableARP {
t.Errorf("EnableARP = %v, expected %v", config.EnableARP, tt.expectedConfig.EnableARP)
}
if config.EnableBGP != tt.expectedConfig.EnableBGP {
t.Errorf("EnableBGP = %v, expected %v", config.EnableBGP, tt.expectedConfig.EnableBGP)
}
if config.Address != tt.expectedConfig.Address {
t.Errorf("Address = %v, expected %v", config.Address, tt.expectedConfig.Address)
}
if config.Port != tt.expectedConfig.Port {
t.Errorf("Port = %v, expected %v", config.Port, tt.expectedConfig.Port)
}
if config.Interface != tt.expectedConfig.Interface {
t.Errorf("Interface = %v, expected %v", config.Interface, tt.expectedConfig.Interface)
}
// Test BGP config if present
if tt.expectedConfig.EnableBGP {
if config.BGPConfig.RouterID != tt.expectedConfig.BGPConfig.RouterID {
t.Errorf("BGPConfig.RouterID = %v, expected %v", config.BGPConfig.RouterID, tt.expectedConfig.BGPConfig.RouterID)
}
if config.BGPConfig.AS != tt.expectedConfig.BGPConfig.AS {
t.Errorf("BGPConfig.AS = %v, expected %v", config.BGPConfig.AS, tt.expectedConfig.BGPConfig.AS)
}
if len(config.BGPConfig.Peers) != len(tt.expectedConfig.BGPConfig.Peers) {
t.Errorf("BGPConfig.Peers length = %v, expected %v", len(config.BGPConfig.Peers), len(tt.expectedConfig.BGPConfig.Peers))
}
}
// Test LoadBalancers if present
if len(tt.expectedConfig.LoadBalancers) > 0 {
if len(config.LoadBalancers) != len(tt.expectedConfig.LoadBalancers) {
t.Errorf("LoadBalancers length = %v, expected %v", len(config.LoadBalancers), len(tt.expectedConfig.LoadBalancers))
}
}
})
}
}
func TestMergeConfigFromFile(t *testing.T) {
// Create temporary directory for test files
tmpDir, err := os.MkdirTemp("", "kube-vip-merge-test")
if err != nil {
t.Fatalf("Failed to create temp dir: %v", err)
}
defer os.RemoveAll(tmpDir)
// Create test config file
configFile := filepath.Join(tmpDir, "test-config.yaml")
configContent := `
logging: 3
enableARP: true
enableServices: true
address: "192.168.1.200"
port: 6443
interface: "eth1"
namespace: "test-namespace"
leaseName: "file-lease"
leaseDuration: 20
prometheusHTTPServer: ":3000"
`
if err := os.WriteFile(configFile, []byte(configContent), 0600); err != nil {
t.Fatalf("Failed to write test config file: %v", err)
}
tests := []struct {
name string
baseConfig *Config
configFilePath string
expected *Config
wantErr bool
}{
{
name: "Merge with empty base config",
baseConfig: &Config{},
configFilePath: configFile,
expected: &Config{
Logging: 3,
EnableARP: true,
EnableServices: true,
Address: "192.168.1.200",
Port: 6443,
Interface: "eth1",
Namespace: "test-namespace",
PrometheusHTTPServer: ":3000",
KubernetesLeaderElection: KubernetesLeaderElection{
LeaseName: "file-lease",
LeaseDuration: 20,
},
},
wantErr: false,
},
{
name: "Merge respects existing values (priority test)",
baseConfig: &Config{
Logging: 5, // Should not be overridden
Port: 8443, // Should not be overridden
Interface: "eth0", // Should not be overridden
},
configFilePath: configFile,
expected: &Config{
Logging: 5, // From base (higher priority)
EnableARP: true, // From file
EnableServices: true, // From file
Address: "192.168.1.200", // From file
Port: 8443, // From base (higher priority)
Interface: "eth0", // From base (higher priority)
Namespace: "test-namespace", // From file
PrometheusHTTPServer: ":3000", // From file
KubernetesLeaderElection: KubernetesLeaderElection{
LeaseName: "file-lease", // From file
LeaseDuration: 20, // From file
},
},
wantErr: false,
},
{
name: "Empty config file path",
baseConfig: &Config{
Logging: 1,
},
configFilePath: "",
expected: &Config{
Logging: 1, // Unchanged
},
wantErr: false,
},
{
name: "Non-existent config file",
baseConfig: &Config{
Logging: 1,
},
configFilePath: "/non/existent/file.yaml",
expected: &Config{
Logging: 1,
},
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := MergeConfigFromFile(tt.baseConfig, tt.configFilePath)
if tt.wantErr {
if err == nil {
t.Errorf("MergeConfigFromFile() expected error, got nil")
}
return
}
if err != nil {
t.Errorf("MergeConfigFromFile() unexpected error = %v", err)
return
}
// Compare key fields
if tt.baseConfig.Logging != tt.expected.Logging {
t.Errorf("Logging = %v, expected %v", tt.baseConfig.Logging, tt.expected.Logging)
}
if tt.baseConfig.EnableARP != tt.expected.EnableARP {
t.Errorf("EnableARP = %v, expected %v", tt.baseConfig.EnableARP, tt.expected.EnableARP)
}
if tt.baseConfig.Address != tt.expected.Address {
t.Errorf("Address = %v, expected %v", tt.baseConfig.Address, tt.expected.Address)
}
if tt.baseConfig.Port != tt.expected.Port {
t.Errorf("Port = %v, expected %v", tt.baseConfig.Port, tt.expected.Port)
}
if tt.baseConfig.Interface != tt.expected.Interface {
t.Errorf("Interface = %v, expected %v", tt.baseConfig.Interface, tt.expected.Interface)
}
if tt.baseConfig.Namespace != tt.expected.Namespace {
t.Errorf("Namespace = %v, expected %v", tt.baseConfig.Namespace, tt.expected.Namespace)
}
if tt.baseConfig.PrometheusHTTPServer != tt.expected.PrometheusHTTPServer {
t.Errorf("PrometheusHTTPServer = %v, expected %v", tt.baseConfig.PrometheusHTTPServer, tt.expected.PrometheusHTTPServer)
}
})
}
}
func TestMergeConfigValues(t *testing.T) {
tests := []struct {
name string
baseConfig *Config
fileConfig *Config
expectedBase *Config
}{
{
name: "Merge basic configuration",
baseConfig: &Config{
Logging: 5, // Should not be overridden
Port: 0, // Should be overridden
},
fileConfig: &Config{
Logging: 2,
Port: 6443,
Interface: "eth0",
Address: "192.168.1.100",
},
expectedBase: &Config{
Logging: 5, // From base (non-zero)
Port: 6443, // From file (base was zero)
Interface: "eth0", // From file (base was empty)
Address: "192.168.1.100", // From file (base was empty)
},
},
{
name: "Merge boolean flags",
baseConfig: &Config{
EnableARP: true, // Should not be overridden
},
fileConfig: &Config{
EnableARP: false, // Should not override true
EnableBGP: true, // Should be set
EnableServices: true, // Should be set
EnableWireguard: false, // Should not be set (false doesn't override false)
},
expectedBase: &Config{
EnableARP: true, // From base (true has priority)
EnableBGP: true, // From file
EnableServices: true, // From file
EnableWireguard: false, // Remains false
},
},
{
name: "Merge BGP configuration",
baseConfig: &Config{
BGPConfig: BGPConfig{
RouterID: "1.1.1.1", // Should not be overridden
},
},
fileConfig: &Config{
BGPConfig: BGPConfig{
RouterID: "2.2.2.2", // Should not override
AS: 65000, // Should be set
SourceIF: "eth0", // Should be set
HoldTime: 30, // Should be set
KeepaliveInterval: 10, // Should be set
},
},
expectedBase: &Config{
BGPConfig: BGPConfig{
RouterID: "1.1.1.1", // From base (non-empty)
AS: 65000, // From file (base was zero)
SourceIF: "eth0", // From file (base was empty)
HoldTime: 30, // From file (base was zero)
KeepaliveInterval: 10, // From file (base was zero)
},
},
},
{
name: "Merge leader election configuration",
baseConfig: &Config{
KubernetesLeaderElection: KubernetesLeaderElection{
LeaseName: "base-lease", // Should not be overridden
},
},
fileConfig: &Config{
KubernetesLeaderElection: KubernetesLeaderElection{
LeaseName: "file-lease", // Should not override
LeaseDuration: 15, // Should be set
RenewDeadline: 10, // Should be set
RetryPeriod: 2, // Should be set
},
},
expectedBase: &Config{
KubernetesLeaderElection: KubernetesLeaderElection{
LeaseName: "base-lease", // From base (non-empty)
LeaseDuration: 15, // From file (base was zero)
RenewDeadline: 10, // From file (base was zero)
RetryPeriod: 2, // From file (base was zero)
},
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
mergeConfigValues(tt.baseConfig, tt.fileConfig)
// Compare results
if tt.baseConfig.Logging != tt.expectedBase.Logging {
t.Errorf("Logging = %v, expected %v", tt.baseConfig.Logging, tt.expectedBase.Logging)
}
if tt.baseConfig.Port != tt.expectedBase.Port {
t.Errorf("Port = %v, expected %v", tt.baseConfig.Port, tt.expectedBase.Port)
}
if tt.baseConfig.Interface != tt.expectedBase.Interface {
t.Errorf("Interface = %v, expected %v", tt.baseConfig.Interface, tt.expectedBase.Interface)
}
if tt.baseConfig.EnableARP != tt.expectedBase.EnableARP {
t.Errorf("EnableARP = %v, expected %v", tt.baseConfig.EnableARP, tt.expectedBase.EnableARP)
}
if tt.baseConfig.EnableBGP != tt.expectedBase.EnableBGP {
t.Errorf("EnableBGP = %v, expected %v", tt.baseConfig.EnableBGP, tt.expectedBase.EnableBGP)
}
if tt.baseConfig.BGPConfig.RouterID != tt.expectedBase.BGPConfig.RouterID {
t.Errorf("BGPConfig.RouterID = %v, expected %v", tt.baseConfig.BGPConfig.RouterID, tt.expectedBase.BGPConfig.RouterID)
}
if tt.baseConfig.BGPConfig.AS != tt.expectedBase.BGPConfig.AS {
t.Errorf("BGPConfig.AS = %v, expected %v", tt.baseConfig.BGPConfig.AS, tt.expectedBase.BGPConfig.AS)
}
if tt.baseConfig.KubernetesLeaderElection.LeaseName != tt.expectedBase.KubernetesLeaderElection.LeaseName {
t.Errorf("KubernetesLeaderElection.LeaseName = %v, expected %v", tt.baseConfig.KubernetesLeaderElection.LeaseName, tt.expectedBase.KubernetesLeaderElection.LeaseName)
}
})
}
}
func TestLoadConfigFromFile_FileNotExists(t *testing.T) {
_, err := LoadConfigFromFile("/non/existent/path/config.yaml")
if err == nil {
t.Error("LoadConfigFromFile() expected error for non-existent file, got nil")
}
if !containsString(err.Error(), "config file does not exist") {
t.Errorf("LoadConfigFromFile() error = %v, expected to contain 'config file does not exist'", err)
}
}
// Helper function to check if a string contains a substring
func containsString(str, substr string) bool {
return len(str) >= len(substr) && (str == substr || len(substr) == 0 ||
(len(substr) > 0 && func() bool {
for i := 0; i <= len(str)-len(substr); i++ {
if str[i:i+len(substr)] == substr {
return true
}
}
return false
}()))
}

View File

@@ -5,6 +5,7 @@ import (
"log"
"strconv"
"github.com/google/go-containerregistry/pkg/name"
appv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -13,7 +14,6 @@ import (
applyCoreV1 "k8s.io/client-go/applyconfigurations/core/v1"
applyMetaV1 "k8s.io/client-go/applyconfigurations/meta/v1"
applyRbacV1 "k8s.io/client-go/applyconfigurations/rbac/v1"
"sigs.k8s.io/yaml"
)
@@ -167,7 +167,11 @@ func GenerateRoleBinding(rolebinding bool, saCfg *applyCoreV1.ServiceAccountAppl
}
// generatePodSpec will take a kube-vip config and generate a Pod spec
func generatePodSpec(c *Config, imageVersion string, inCluster bool) *corev1.Pod {
func generatePodSpec(c *Config, image, imageVersion string, inCluster bool) *corev1.Pod {
imageRef, err := name.NewTag(image, name.WeakValidation, name.WithDefaultTag(imageVersion))
if err != nil {
panic(fmt.Errorf("Cannot parse %q: %w", image, err))
}
command := "manager"
// Determine where the pods should be living (for multi-tenancy)
@@ -592,7 +596,7 @@ func generatePodSpec(c *Config, imageVersion string, inCluster bool) *corev1.Pod
Containers: []corev1.Container{
{
Name: "kube-vip",
Image: fmt.Sprintf("ghcr.io/kube-vip/kube-vip:%s", imageVersion),
Image: imageRef.Name(),
ImagePullPolicy: corev1.PullIfNotPresent,
SecurityContext: securityContext,
Args: []string{
@@ -637,14 +641,14 @@ func generatePodSpec(c *Config, imageVersion string, inCluster bool) *corev1.Pod
}
// GeneratePodManifestFromConfig will take a kube-vip config and generate a manifest
func GeneratePodManifestFromConfig(c *Config, imageVersion string, inCluster bool) string {
newManifest := generatePodSpec(c, imageVersion, inCluster)
func GeneratePodManifestFromConfig(c *Config, image, imageVersion string, inCluster bool) string {
newManifest := generatePodSpec(c, image, imageVersion, inCluster)
b, _ := yaml.Marshal(newManifest)
return string(b)
}
// GenerateDaemonsetManifestFromConfig will take a kube-vip config and generate a manifest
func GenerateDaemonsetManifestFromConfig(c *Config, imageVersion string, inCluster, taint bool) string {
func GenerateDaemonsetManifestFromConfig(c *Config, image, imageVersion string, inCluster, taint bool) string {
// Determine where the pod should be deployed
var namespace string
if c.ServiceNamespace != "" {
@@ -653,7 +657,7 @@ func GenerateDaemonsetManifestFromConfig(c *Config, imageVersion string, inClust
namespace = metav1.NamespaceSystem
}
podSpec := generatePodSpec(c, imageVersion, inCluster).Spec
podSpec := generatePodSpec(c, image, imageVersion, inCluster).Spec
newManifest := &appv1.DaemonSet{
TypeMeta: metav1.TypeMeta{
Kind: "DaemonSet",

View File

@@ -1,6 +1,9 @@
package kubevip
import "testing"
import (
"os"
"testing"
)
func TestParseEnvironment(t *testing.T) {
@@ -9,8 +12,8 @@ func TestParseEnvironment(t *testing.T) {
c *Config
wantErr bool
}{
{"", nil, false},
{"", &Config{Interface: "eth0", ServicesInterface: "eth1"}, false},
{"nil config", nil, false},
{"basic config", &Config{Interface: "eth0", ServicesInterface: "eth1"}, false},
}
for _, tt := range tests {
t.Logf("%v", tt.c)
@@ -21,3 +24,53 @@ func TestParseEnvironment(t *testing.T) {
})
}
}
func TestParseEnvironmentConfigFile(t *testing.T) {
// Save original environment
originalConfigFile := os.Getenv("config_file")
defer func() {
if originalConfigFile != "" {
os.Setenv("config_file", originalConfigFile)
} else {
os.Unsetenv("config_file")
}
}()
tests := []struct {
name string
envValue string
expectedConfig string
}{
{
name: "config_file environment variable set",
envValue: "/etc/kube-vip/config.yaml",
expectedConfig: "/etc/kube-vip/config.yaml",
},
{
name: "config_file environment variable empty",
envValue: "",
expectedConfig: "",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Set environment variable
if tt.envValue != "" {
os.Setenv("config_file", tt.envValue)
} else {
os.Unsetenv("config_file")
}
config := &Config{}
err := ParseEnvironment(config)
if err != nil {
t.Errorf("ParseEnvironment() unexpected error = %v", err)
}
if config.ConfigFile != tt.expectedConfig {
t.Errorf("ConfigFile = %v, expected %v", config.ConfigFile, tt.expectedConfig)
}
})
}
}

View File

@@ -1,9 +1,5 @@
package kubevip
import (
"github.com/kube-vip/kube-vip/pkg/bgp"
)
// Config defines all of the settings for the Kube-Vip Pod
type Config struct {
// Logging, settings
@@ -128,8 +124,8 @@ type Config struct {
CleanRoutingTable bool `yaml:"cleanRoutingTable"`
// BGP Configuration
BGPConfig bgp.Config
BGPPeerConfig bgp.Peer
BGPConfig BGPConfig
BGPPeerConfig BGPPeer
BGPPeers []string
// LoadBalancers are the various services we can load balance over
@@ -186,6 +182,9 @@ type Config struct {
// EgressClean, enables egress cleaning on Kube-vip's start
EgressClean bool `yaml:"egressClean"`
// ConfigFile defines the path to a JSON/YAML configuration file
ConfigFile string `yaml:"configFile"`
}
// KubernetesLeaderElection defines all of the settings for Kubernetes KubernetesLeaderElection

5
pkg/kubevip/constants.go Normal file
View File

@@ -0,0 +1,5 @@
package kubevip
const (
LBClassName = "kube-vip.io/kube-vip-class"
)

7
pkg/kubevip/labels.go Normal file
View File

@@ -0,0 +1,7 @@
package kubevip
const (
// label used on nodes, which announce the LoadBalancer IP
HasIP = "kube-vip.io/has-ip"
HasIPJSONPath = `kube-vip.io~1has-ip`
)

127
pkg/lease/lease.go Normal file
View File

@@ -0,0 +1,127 @@
package lease
import (
"context"
"fmt"
"sync"
"github.com/kube-vip/kube-vip/pkg/kubevip"
v1 "k8s.io/api/core/v1"
)
// Manager is used to manage leases.
type Manager struct {
leases map[string]*Lease
lock sync.Mutex
}
// NewManager creates new lease manager.
func NewManager() *Manager {
return &Manager{
leases: make(map[string]*Lease),
}
}
// Add adds lease or incerements counter if lease is alreay used.
func (m *Manager) Add(service *v1.Service) (*Lease, bool) {
m.lock.Lock()
defer m.lock.Unlock()
_, id := GetName(service)
if _, exist := m.leases[id]; !exist {
ctx, cancel := context.WithCancel(context.Background())
m.leases[id] = newLease(ctx, cancel)
return m.leases[id], true
}
m.leases[id].increment()
return m.leases[id], false
}
// Delete decrements lease counter and removes the lease if counter equals 0.
func (m *Manager) Delete(service *v1.Service) {
m.lock.Lock()
defer m.lock.Unlock()
_, id := GetName(service)
if _, exist := m.leases[id]; exist {
m.leases[id].decrement()
if m.leases[id].cnt < 1 {
delete(m.leases, id)
}
}
}
// Get returns lease for the service.
func (m *Manager) Get(service *v1.Service) *Lease {
m.lock.Lock()
defer m.lock.Unlock()
_, id := GetName(service)
if lease, exist := m.leases[id]; exist {
return lease
}
return nil
}
// GetLeaderContext returns leder context for the service.
func (m *Manager) GetLeaderContext(service *v1.Service) context.Context {
m.lock.Lock()
defer m.lock.Unlock()
_, id := GetName(service)
if _, ok := m.leases[id]; !ok {
return nil
}
return m.leases[id].Ctx
}
// Lease holds lease data.
type Lease struct {
cnt uint
Lock *sync.Mutex
Ctx context.Context
Cancel context.CancelFunc
Started chan any
}
func newLease(ctx context.Context, cancel context.CancelFunc) *Lease {
return &Lease{
Ctx: ctx,
Cancel: cancel,
cnt: 1,
Lock: new(sync.Mutex),
Started: make(chan any),
}
}
func (l *Lease) increment() {
l.Lock.Lock()
defer l.Lock.Unlock()
l.cnt++
}
func (l *Lease) decrement() {
l.Lock.Lock()
defer l.Lock.Unlock()
if l.cnt == 0 {
return
}
l.cnt--
if l.cnt < 1 {
l.Cancel()
}
}
// GetName gets lease name and id for the service.
func GetName(service *v1.Service) (string, string) {
serviceLease, exists := service.Annotations[kubevip.ServiceLease]
if !exists || serviceLease == "" {
serviceLease = fmt.Sprintf("kubevip-%s", service.Name)
}
serviceLeaseID := fmt.Sprintf("%s/%s", serviceLease, service.Namespace)
return serviceLease, serviceLeaseID
}
// UsesCommon checks if service uses common lease feature.
func UsesCommon(service *v1.Service) bool {
_, common := service.Annotations[kubevip.ServiceLease]
return common
}

View File

@@ -16,7 +16,7 @@ import (
"github.com/cloudflare/ipvs/netmask"
"github.com/kube-vip/kube-vip/pkg/backend"
"github.com/kube-vip/kube-vip/pkg/sysctl"
"github.com/kube-vip/kube-vip/pkg/vip"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/vishvananda/netlink"
)
@@ -358,7 +358,7 @@ func (lb *IPVSLoadBalancer) isLocal(address string) (bool, error) {
}
family := netlink.FAMILY_V6
if vip.IsIPv4(address) {
if utils.IsIPv4(address) {
family = netlink.FAMILY_V4
}

View File

@@ -19,6 +19,7 @@ import (
"github.com/kube-vip/kube-vip/pkg/k8s"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/networkinterface"
"github.com/kube-vip/kube-vip/pkg/node"
"github.com/kube-vip/kube-vip/pkg/services"
"github.com/kube-vip/kube-vip/pkg/upnp"
"github.com/kube-vip/kube-vip/pkg/utils"
@@ -66,6 +67,11 @@ type Manager struct {
// This tracks VIPs and performs ARP/NDP advertisement.
arpMgr *arp.Manager
// This tracks node labels and performs label management
// implementation will be decided in constructor
// based on config.EnableNodeLabeling
nodeLabelManager node.LabelManager
}
// New will create a new managing object
@@ -187,15 +193,19 @@ func New(configMap string, config *kubevip.Config) (*Manager, error) {
intfMgr := networkinterface.NewManager()
arpMgr := arp.NewManager(config)
// create the node label manager
// constructor will decide if it should be a noop or not
nodeLabelManager := node.NewManager(config, clientset)
var bgpServer *bgp.Server
if config.EnableBGP {
bgpServer, err = bgp.NewBGPServer(&config.BGPConfig)
bgpServer, err = bgp.NewBGPServer(config.BGPConfig)
if err != nil {
return nil, fmt.Errorf("creating BGP server: %w", err)
}
}
svcProcessor := services.NewServicesProcessor(config, bgpServer, clientset, rwClientSet, shutdownChan, intfMgr, arpMgr)
svcProcessor := services.NewServicesProcessor(config, bgpServer, clientset, rwClientSet, shutdownChan, intfMgr, arpMgr, nodeLabelManager)
return &Manager{
clientSet: clientset,
@@ -214,12 +224,13 @@ func New(configMap string, config *kubevip.Config) (*Manager, error) {
Name: "bgp_session_info",
Help: "Display state of session by setting metric for label value with current state to 1",
}, []string{"state", "peer"}),
signalChan: signalChan,
shutdownChan: shutdownChan,
svcProcessor: svcProcessor,
intfMgr: intfMgr,
arpMgr: arpMgr,
bgpServer: bgpServer,
signalChan: signalChan,
shutdownChan: shutdownChan,
svcProcessor: svcProcessor,
intfMgr: intfMgr,
arpMgr: arpMgr,
bgpServer: bgpServer,
nodeLabelManager: nodeLabelManager,
}, nil
}
@@ -258,6 +269,13 @@ func (sm *Manager) Start() error {
}()
}
// on exit, clean up the node labels
defer func() {
if err := sm.nodeLabelManager.CleanUpLabels(10 * time.Second); err != nil {
log.Error("CleanUpNodeLabels", "unable to cleanup node labels", err)
}
}()
// If BGP is enabled then we start a server instance that will broadcast VIPs
if sm.config.EnableBGP {

View File

@@ -20,7 +20,7 @@ func (sm *Manager) startBGP() error {
var err error
if sm.bgpServer == nil {
sm.bgpServer, err = bgp.NewBGPServer(&sm.config.BGPConfig)
sm.bgpServer, err = bgp.NewBGPServer(sm.config.BGPConfig)
if err != nil {
return fmt.Errorf("creating BGP server: %w", err)
}
@@ -105,9 +105,18 @@ func (sm *Manager) startBGP() error {
}
}
err = sm.svcProcessor.ServicesWatcher(ctx, sm.svcProcessor.SyncServices)
if err != nil {
return err
if sm.config.EnableServicesElection {
log.Info("beginning watching services, leaderelection will happen for every service")
err = sm.svcProcessor.StartServicesWatchForLeaderElection(ctx)
if err != nil {
return err
}
} else {
log.Info("beginning watching services without leader election")
err = sm.svcProcessor.ServicesWatcher(ctx, sm.svcProcessor.SyncServices)
if err != nil {
return err
}
}
log.Info("Shutting down Kube-Vip")

View File

@@ -57,23 +57,8 @@ func (sm *Manager) startTableMode(id string) error {
cancel()
}()
if sm.config.EnableControlPlane {
cpCluster, err = cluster.InitCluster(sm.config, false, sm.intfMgr, sm.arpMgr)
if err != nil {
return fmt.Errorf("cluster initialization error: %w", err)
}
clusterManager, err := initClusterManager(sm)
if err != nil {
return fmt.Errorf("cluster manager initialization error: %w", err)
}
if err := cpCluster.StartVipService(sm.config, clusterManager, nil); err != nil {
log.Error("Control Plane", "err", err)
// Trigger the shutdown of this manager instance
sm.signalChan <- syscall.SIGINT
}
} else {
if sm.config.EnableServices {
log.Debug("starting Services")
ns, err := returnNameSpace()
if err != nil {
log.Warn("unable to auto-detect namespace", "dropping to", sm.config.Namespace)
@@ -149,9 +134,34 @@ func (sm *Manager) startTableMode(id string) error {
err = sm.svcProcessor.ServicesWatcher(ctx, sm.svcProcessor.SyncServices)
if err != nil {
log.Error("Cannot watch services", "err", err)
} else {
log.Debug("watching services")
}
}
}
if sm.config.EnableControlPlane {
log.Debug("initCluster for ControlPlane")
cpCluster, err = cluster.InitCluster(sm.config, false, sm.intfMgr, sm.arpMgr)
if err != nil {
log.Debug("init of ControlPlane NOT successful")
return fmt.Errorf("cluster initialization error: %w", err)
}
log.Debug("init of ControlPlane successful")
log.Debug("init ClusterManager")
clusterManager, err := initClusterManager(sm)
if err != nil {
log.Debug("init cluster manager NOT successful")
return fmt.Errorf("cluster manager initialization error: %w", err)
}
log.Debug("init ClusterManager successful")
if err := cpCluster.StartVipService(sm.config, clusterManager, nil); err != nil {
log.Error("Control Plane", "err", err)
// Trigger the shutdown of this manager instance
sm.signalChan <- syscall.SIGINT
} else {
log.Debug("start VipServer for cluster manager successful")
}
}
return nil
}

View File

@@ -7,16 +7,12 @@ import (
log "log/slog"
"github.com/kube-vip/kube-vip/pkg/kubevip"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/kubernetes"
)
const (
nodeLabelIndex = "kube-vip.io/has-ip"
nodeLabelJSONPath = `kube-vip.io~1has-ip`
)
type patchStringLabel struct {
Op string `json:"op"`
Path string `json:"path"`
@@ -35,8 +31,8 @@ func applyNodeLabel(clientSet *kubernetes.Clientset, address, id, identity strin
log.Debug(fmt.Sprintf("node %s labels: %+v", id, node.Labels))
value, ok := node.Labels[nodeLabelIndex]
path := fmt.Sprintf("/metadata/labels/%s", nodeLabelJSONPath)
value, ok := node.Labels[kubevip.HasIP]
path := fmt.Sprintf("/metadata/labels/%s", kubevip.HasIPJSONPath)
log.Debug(fmt.Sprintf("Received identity: %s - id: %s", identity, id))
if ok && value == address {
log.Debug(fmt.Sprintf("removing node label `has-ip=%s` on %s", address, id))

View File

@@ -10,7 +10,7 @@ import (
log "log/slog"
"github.com/kube-vip/kube-vip/pkg/bgp"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/davecgh/go-spew/spew"
v1 "k8s.io/api/core/v1"
@@ -151,8 +151,8 @@ func (sm *Manager) annotationsWatcher() error {
// * `<info>` is the relevant information, such as `node-asn` or `peer-ip`
// * `{{n}}` is the number of the peer, always starting with `0`
// * kube-vip is only designed to manage one peer, just look for {{n}} == 0
func parseBgpAnnotations(bgpConfig bgp.Config, node *v1.Node, prefix string) (bgp.Config, bgp.Peer, error) {
bgpPeer := bgp.Peer{}
func parseBgpAnnotations(bgpConfig kubevip.BGPConfig, node *v1.Node, prefix string) (kubevip.BGPConfig, kubevip.BGPPeer, error) {
bgpPeer := kubevip.BGPPeer{}
nodeASN := ""
for k, v := range node.Annotations {
@@ -216,7 +216,7 @@ func parseBgpAnnotations(bgpConfig bgp.Config, node *v1.Node, prefix string) (bg
peerIPs := strings.Split(peerIPString, ",")
bgpConfig.Peers = make([]bgp.Peer, 0, len(peerIPs))
bgpConfig.Peers = make([]kubevip.BGPPeer, 0, len(peerIPs))
for _, peerIP := range peerIPs {
ipAddr := strings.TrimSpace(peerIP)

View File

@@ -4,7 +4,7 @@ import (
"reflect"
"testing"
"github.com/kube-vip/kube-vip/pkg/bgp"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/stretchr/testify/assert"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -15,7 +15,7 @@ func TestParseBgpAnnotations(t *testing.T) {
ObjectMeta: metav1.ObjectMeta{Name: "test", Annotations: map[string]string{}},
}
bgpConfigBase := bgp.Config{
bgpConfigBase := kubevip.BGPConfig{
HoldTime: 15,
KeepaliveInterval: 5,
}
@@ -54,7 +54,7 @@ func TestParseBgpAnnotations(t *testing.T) {
t.Fatal("Parsing BGP annotations should return nil when minimum config is met")
}
bgpPeers := []bgp.Peer{
bgpPeers := []kubevip.BGPPeer{
{Address: "10.0.0.1", AS: uint32(64000), Password: "password"},
{Address: "10.0.0.2", AS: uint32(64000), Password: "password"},
{Address: "10.0.0.3", AS: uint32(64000), Password: "password"},
@@ -77,7 +77,7 @@ func TestParseNewBgpAnnotations(t *testing.T) {
ObjectMeta: metav1.ObjectMeta{Name: "test", Annotations: map[string]string{}},
}
bgpConfigBase := bgp.Config{
bgpConfigBase := kubevip.BGPConfig{
HoldTime: 15,
KeepaliveInterval: 5,
}
@@ -99,7 +99,7 @@ func TestParseNewBgpAnnotations(t *testing.T) {
t.Fatalf("Parsing BGP annotations should return nil when minimum config is met [%v]", err)
}
bgpPeers := []bgp.Peer{
bgpPeers := []kubevip.BGPPeer{
{Address: "10.0.0.1", AS: uint32(64000), Password: "password"},
{Address: "10.0.0.2", AS: uint32(64000), Password: "password"},
{Address: "10.0.0.3", AS: uint32(64000), Password: "password"},
@@ -121,15 +121,15 @@ func Test_parseBgpAnnotations(t *testing.T) {
tests := []struct {
name string
args args
want bgp.Config
want1 bgp.Peer
want kubevip.BGPConfig
want1 kubevip.BGPPeer
wantErr bool
}{
// TODO: Add test cases.
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, got1, err := parseBgpAnnotations(bgp.Config{}, tt.args.node, tt.args.prefix)
got, got1, err := parseBgpAnnotations(kubevip.BGPConfig{}, tt.args.node, tt.args.prefix)
if (err != nil) != tt.wantErr {
t.Errorf("parseBgpAnnotations() error = %v, wantErr %v", err, tt.wantErr)
return

View File

@@ -20,7 +20,6 @@ const (
)
func ApplySNAT(podIP, vipIP, service, destinationPorts string, ignoreCIDR []string, IPv6 bool) error {
conn, err := nftables.New()
if err != nil {
return err
@@ -41,9 +40,11 @@ func ApplySNAT(podIP, vipIP, service, destinationPorts string, ignoreCIDR []stri
}
}
slog.Debug("[egress]", "Creating Chain for service", service, "IPv6", IPv6)
// These don't return errors, so not 100% sure how to guarantee things were created
conn.AddChain(GetSNatChain(IPv6, service))
conn.Flush()
err = conn.Flush()
if err != nil {
return err
}
// Create our nftables rule
rule, err := CreateRule(podIP, vipIP, service, destinationPorts, ignoreCIDR, conn, IPv6)
if err != nil {

View File

@@ -0,0 +1,147 @@
package labeler
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
log "log/slog"
"github.com/kube-vip/kube-vip/pkg/instance"
"github.com/pkg/errors"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/kubernetes"
)
const (
// labelName is the name of the label that will be added to the node
// it is prefix for the label key before "/"
labelName = "service-provided.kube-vip.io"
)
// labelOperation is the operation to perform on the node labels
type labelOperation string
// labelOperation constants
const (
labelOperationRemove labelOperation = "remove"
labelOperationAdd labelOperation = "add"
)
// NewManager creates a new Label Manager for the given node
func NewManager(nodeName string, clientSet *kubernetes.Clientset) *Manager {
return &Manager{
nodeName: nodeName,
clientSet: clientSet,
}
}
// Manager is the label Manager for the node
type Manager struct {
// nodeName is the name of the node to manage
nodeName string
// clientSet is the Kubernetes client set to use
clientSet *kubernetes.Clientset
}
// AddLabel a new label to the node
func (m *Manager) AddLabel(ctx context.Context, svc *corev1.Service) error {
log.Debug("[service] add label to node", "namespace", svc.Namespace, "name", svc.Name)
labelKey, labelValue := generateNodeLabelKeyValue(svc)
return m.patchNode(ctx, labelOperationAdd, map[string]string{labelKey: labelValue})
}
// RemoveLabel a label from the node
func (m *Manager) RemoveLabel(ctx context.Context, svc *corev1.Service) error {
log.Debug("[service] delete label from node", "namespace", svc.Namespace, "name", svc.Name)
labelKey, _ := generateNodeLabelKeyValue(svc)
return m.patchNode(ctx, labelOperationRemove, map[string]string{labelKey: ""})
}
// clean up the node labels
func (m *Manager) CleanUpLabels(timeout time.Duration) error {
log.Debug("cleaning up labels for node", "node", m.nodeName, "timeout", timeout)
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
// get the node
node, err := m.clientSet.CoreV1().Nodes().Get(ctx, m.nodeName, metav1.GetOptions{})
if err != nil {
return errors.Wrapf(err, "failed to get node %s", m.nodeName)
}
// collect all labels with the prefix to remove
labels := map[string]string{}
for k := range node.Labels {
if strings.HasPrefix(k, labelName) {
labels[k] = ""
}
}
if len(labels) == 0 {
log.Debug("no labels to remove for node", "node", m.nodeName)
return nil
}
// patch the node with the labels to remove
return m.patchNode(ctx, labelOperationRemove, labels)
}
// generateNodeLabelKeyValue generates a label key and value for the given service
func generateNodeLabelKeyValue(svc *corev1.Service) (string, string) {
addresses, _ := instance.FetchServiceAddresses(svc)
return fmt.Sprintf("%s/%s.%s", labelName, svc.Name, svc.Namespace), strings.Join(addresses, ",")
}
// patchNode patches the node with the given labels
func (m *Manager) patchNode(ctx context.Context, operation labelOperation, labels map[string]string) error {
type patchStringLabel struct {
Op string `json:"op"`
Path string `json:"path"`
Value string `json:"value"`
}
patchLabels := []patchStringLabel{}
// generate the patch
for k, v := range labels {
patchLabels = append(patchLabels, patchStringLabel{
Op: string(operation),
// replace all slashes with ~1
Path: fmt.Sprintf("/metadata/labels/%s", strings.ReplaceAll(k, "/", "~1")),
Value: v,
})
}
patchData, err := json.Marshal(patchLabels)
if err != nil {
log.Debug("node patch marshaling failed", "err", err, "labels", labels, "patch", patchLabels)
return errors.Wrapf(err, "node patch marshaling failed for labels %v", labels)
}
log.Debug("patching node",
"node", m.nodeName,
"patch", string(patchData),
"operation", operation,
"labels", labels,
"clientSetNil", m.clientSet == nil)
if m.clientSet == nil {
return errors.New("kubernetes client is not initialized")
}
// patch node
node, err := m.clientSet.CoreV1().Nodes().Patch(ctx, m.nodeName, types.JSONPatchType, patchData, metav1.PatchOptions{})
if err != nil {
log.Debug("node patching failed", "err", err, "patchData", patchData)
return errors.Wrapf(err, "node patching failed with patch %s", string(patchData))
}
log.Debug("updated", "node", m.nodeName, "labels", node.Labels)
return nil
}

View File

@@ -0,0 +1,27 @@
package noop
import (
"context"
"time"
corev1 "k8s.io/api/core/v1"
)
// NewManager creates a new NoOp label manager
func NewManager() *Manager {
return &Manager{}
}
type Manager struct{}
func (m *Manager) AddLabel(_ context.Context, _ *corev1.Service) error {
return nil
}
func (m *Manager) RemoveLabel(_ context.Context, _ *corev1.Service) error {
return nil
}
func (m *Manager) CleanUpLabels(_ time.Duration) error {
return nil
}

49
pkg/node/types.go Normal file
View File

@@ -0,0 +1,49 @@
package node
import (
"context"
"time"
log "log/slog"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/node/labeler"
"github.com/kube-vip/kube-vip/pkg/node/noop"
v1 "k8s.io/api/core/v1"
"k8s.io/client-go/kubernetes"
)
// LabelManager is the interface for the node label manager
type LabelManager interface {
// AddLabel adds a label to the node for the given service
AddLabel(ctx context.Context, svc *v1.Service) error
// RemoveLabel removes the label from the node for the given service
RemoveLabel(ctx context.Context, svc *v1.Service) error
// CleanUpLabels removes all labels from the node
CleanUpLabels(timeout time.Duration) error
}
// NewManager creates a new Label Manager for the given node
// NoOp implementation is returned if node labeling is disabled, or if
// running in control plane mode, or if the client is not ready
func NewManager(config *kubevip.Config, clientSet *kubernetes.Clientset) LabelManager {
if !config.EnableNodeLabeling {
return noop.NewManager()
}
if config.EnableControlPlane {
log.Debug("Skip node labeling, control plane mode enabled")
return noop.NewManager()
}
if clientSet == nil {
log.Debug("Skip node labeling, client is not ready")
return noop.NewManager()
}
log.Debug("Node labeling enabled")
return labeler.NewManager(config.NodeName, clientSet)
}

View File

@@ -1,36 +0,0 @@
package services
import (
"context"
"sync"
)
type Context struct {
Ctx context.Context
Cancel context.CancelFunc
IsActive bool
IsWatched bool
ConfiguredNetworks sync.Map
}
func NewContext(ctx context.Context) *Context {
svcCtx, svcCancel := context.WithCancel(ctx)
return &Context{
Ctx: svcCtx,
Cancel: svcCancel,
}
}
func (ctx *Context) HasConfiguredNetworks() bool {
cnt := 0
ctx.ConfiguredNetworks.Range(func(_ any, _ any) bool {
cnt++
return cnt < 1
})
return cnt > 0
}
func (ctx *Context) IsNetworkConfigured(ip string) bool {
_, exists := ctx.ConfiguredNetworks.Load(ip)
return exists
}

View File

@@ -13,6 +13,7 @@ import (
"github.com/kube-vip/kube-vip/pkg/iptables"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/nftables"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/kube-vip/kube-vip/pkg/vip"
v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
@@ -77,12 +78,12 @@ func (p *Processor) nftablesCheck() error {
func getSameFamilyCidr(sourceCidrs, ip string) string { //Todo: not sure how this ever worked
cidrs := strings.Split(sourceCidrs, ",")
isV6 := vip.IsIPv6(ip)
isV6 := utils.IsIPv6(ip)
matchingFamily := []string{}
for _, cidr := range cidrs {
// Is the ip an IPv6 address
if isV6 {
if vip.IsIPv6CIDR(cidr) {
if utils.IsIPv6CIDR(cidr) {
matchingFamily = append(matchingFamily, cidr)
selectedCIDR, err := checkCIDR(ip, cidr)
if err != nil {
@@ -94,7 +95,7 @@ func getSameFamilyCidr(sourceCidrs, ip string) string { //Todo: not sure how thi
}
}
} else {
if vip.IsIPv4CIDR(cidr) {
if utils.IsIPv4CIDR(cidr) {
matchingFamily = append(matchingFamily, cidr)
selectedCidr, err := checkCIDR(ip, cidr)
if err != nil {
@@ -163,7 +164,7 @@ func (p *Processor) configureEgress(vipIP, podIP, namespace, serviceUUID string,
if podCidr == "" {
// There's no default IPv6 pod CIDR, therefore we silently back off if CIDR s not specified.
if !vip.IsIPv4(podIP) {
if !utils.IsIPv4(podIP) {
return fmt.Errorf("error with the CIDR [%s]", podIP)
}
podCidr = defaultPodCIDR
@@ -179,7 +180,7 @@ func (p *Processor) configureEgress(vipIP, podIP, namespace, serviceUUID string,
if serviceCidr == "" {
// There's no default IPv6 service CIDR, therefore we silently back off if CIDR s not specified.
if !vip.IsIPv4(vipIP) {
if !utils.IsIPv4(vipIP) {
return nil
}
serviceCidr = defaultServiceCIDR
@@ -188,23 +189,23 @@ func (p *Processor) configureEgress(vipIP, podIP, namespace, serviceUUID string,
log.Info("[Egress]", "podCIDR", podCidr, "serviceCIDR", serviceCidr, "vip", serviceCidr, "pod", podIP)
// checking if all addresses are of the same IP family
if vip.IsIPv4(podIP) != vip.IsIPv4CIDR(podCidr) {
if utils.IsIPv4(podIP) != utils.IsIPv4CIDR(podCidr) {
log.Error("[Egress] family is not matching. Backing off...", "pod", podIP, "podCIDR", podCidr)
return nil
}
if vip.IsIPv4(vipIP) != vip.IsIPv4CIDR(serviceCidr) {
if utils.IsIPv4(vipIP) != utils.IsIPv4CIDR(serviceCidr) {
log.Error("[Egress] family is not matching. Backing off...", "pod", podIP, "serviceCIDR", serviceCidr)
return nil
}
if vip.IsIPv4(vipIP) != vip.IsIPv4(podIP) {
if utils.IsIPv4(vipIP) != utils.IsIPv4(podIP) {
log.Error("[Egress] family is not matching. Backing off...", "pod", podIP, "vipIP", vipIP)
return nil
}
protocol := iptables.ProtocolIPv4
if vip.IsIPv6(vipIP) {
if utils.IsIPv6(vipIP) {
protocol = iptables.ProtocolIPv6
}
@@ -224,53 +225,7 @@ func (p *Processor) configureEgress(vipIP, podIP, namespace, serviceUUID string,
}
// Apply the SNAT rules
err := nftables.ApplySNAT(podIP, vipIP, serviceUUID, destinationPorts, ignoreCIDRs, vip.IsIPv6(vipIP))
if err != nil {
return fmt.Errorf("error performing netlink nftables [%s]", err)
}
return nil
}
// Use the internal egress implementation
if internalEgress != "" {
// Create an array of CIDRs that we wont SNAT to.
ignoreCIDRs := []string{
podCidr,
serviceCidr,
}
// Ad anny specifically denied networks
if deniedNetworks != "" {
networks := strings.Split(strings.TrimSpace(deniedNetworks), ",") //Remove whitespace characters and then create an array from the CIDRs
ignoreCIDRs = append(ignoreCIDRs, networks...)
}
// Apply the SNAT rules
err := nftables.ApplySNAT(podIP, vipIP, serviceUUID, destinationPorts, ignoreCIDRs, vip.IsIPv6(vipIP))
if err != nil {
return fmt.Errorf("error performing netlink nftables [%s]", err)
}
return nil
}
// Use the internal egress implementation
if internalEgress != "" {
// Create an array of CIDRs that we wont SNAT to.
ignoreCIDRs := []string{
podCidr,
serviceCidr,
}
// Ad anny specifically denied networks
if deniedNetworks != "" {
networks := strings.Split(strings.TrimSpace(deniedNetworks), ",") //Remove whitespace characters and then create an array from the CIDRs
ignoreCIDRs = append(ignoreCIDRs, networks...)
}
// Apply the SNAT rules
err := nftables.ApplySNAT(podIP, vipIP, serviceUUID, destinationPorts, ignoreCIDRs, vip.IsIPv6(vipIP))
err := nftables.ApplySNAT(podIP, vipIP, serviceUUID, destinationPorts, ignoreCIDRs, utils.IsIPv6(vipIP))
if err != nil {
return fmt.Errorf("error performing netlink nftables [%s]", err)
}
@@ -315,7 +270,7 @@ func (p *Processor) configureEgress(vipIP, podIP, namespace, serviceUUID string,
}
mask := "/32"
if !vip.IsIPv4(podIP) {
if !utils.IsIPv4(podIP) {
mask = "/128"
}

View File

@@ -3,25 +3,17 @@ package services
import (
"context"
"fmt"
"sync"
"time"
log "log/slog"
"github.com/kube-vip/kube-vip/pkg/lease"
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/tools/leaderelection"
"k8s.io/client-go/tools/leaderelection/resourcelock"
)
var (
svcLocks map[string]*sync.Mutex
)
func init() {
svcLocks = make(map[string]*sync.Mutex)
}
// The StartServicesWatchForLeaderElection function will start a services watcher, the
func (p *Processor) StartServicesWatchForLeaderElection(ctx context.Context) error {
err := p.ServicesWatcher(ctx, p.StartServicesLeaderElection)
@@ -29,12 +21,14 @@ func (p *Processor) StartServicesWatchForLeaderElection(ctx context.Context) err
return err
}
for _, instance := range p.ServiceInstances {
for _, cluster := range instance.Clusters {
for i := range cluster.Network {
_ = cluster.Network[i].DeleteRoute()
if p.config.EnableRoutingTable {
for _, instance := range p.ServiceInstances {
for _, cluster := range instance.Clusters {
for i := range cluster.Network {
_ = cluster.Network[i].DeleteRoute()
}
cluster.Stop()
}
cluster.Stop()
}
}
@@ -45,7 +39,7 @@ func (p *Processor) StartServicesWatchForLeaderElection(ctx context.Context) err
// The startServicesWatchForLeaderElection function will start a services watcher, the
func (p *Processor) StartServicesLeaderElection(ctx context.Context, service *v1.Service) error {
serviceLease := fmt.Sprintf("kubevip-%s", service.Name)
serviceLease, _ := lease.GetName(service)
log.Info("new leader election", "service", service.Name, "namespace", service.Namespace, "lock_name", serviceLease, "host_id", p.config.NodeName)
// we use the Lease lock type since edits to Leases are less common
// and fewer objects in the cluster watch "all Leases".
@@ -59,15 +53,12 @@ func (p *Processor) StartServicesLeaderElection(ctx context.Context, service *v1
Identity: p.config.NodeName,
},
}
childCtx, childCancel := context.WithCancel(ctx)
defer childCancel()
if _, ok := svcLocks[serviceLease]; !ok {
svcLocks[serviceLease] = new(sync.Mutex)
}
svcLocks[serviceLease].Lock()
defer svcLocks[serviceLease].Unlock()
go func() {
// wait for the service context to end and delete the lease then
<-ctx.Done()
p.leaseMgr.Delete(service)
}()
svcCtx, err := p.getServiceContext(service.UID)
if err != nil {
@@ -79,8 +70,40 @@ func (p *Processor) StartServicesLeaderElection(ctx context.Context, service *v1
svcCtx.IsActive = true
svcLease, isNew := p.leaseMgr.Add(service)
// this service is sharing lease
if !isNew {
// wait for leader election to start or context to be done
select {
case <-svcLease.Started:
case <-svcLease.Ctx.Done():
svcCtx.IsActive = false
return nil
}
<-svcLease.Started
if lease.UsesCommon(service) {
if err := p.SyncServices(ctx, service); err != nil {
log.Error("service sync", "err", err)
svcLease.Cancel()
}
// just block until context is cancelled
<-ctx.Done()
if svcCtx.IsActive {
if err := p.deleteService(service.UID); err != nil {
log.Error("service deletion", "err", err)
}
}
}
// wait for leaderelection to be finished
<-svcLease.Ctx.Done()
// Mark this service is inactive
svcCtx.IsActive = false
return nil
}
// start the leader election code loop
leaderelection.RunOrDie(childCtx, leaderelection.LeaderElectionConfig{
leaderelection.RunOrDie(svcLease.Ctx, leaderelection.LeaderElectionConfig{
Lock: lock,
// IMPORTANT: you MUST ensure that any code you have that
// is protected by the lease must terminate **before**
@@ -98,8 +121,9 @@ func (p *Processor) StartServicesLeaderElection(ctx context.Context, service *v1
// we run this in background as it's blocking
if err := p.SyncServices(ctx, service); err != nil {
log.Error("service sync", "err", err)
childCancel()
svcLease.Cancel()
}
close(svcLease.Started)
},
OnStoppedLeading: func() {
// we can do cleanup here

View File

@@ -13,6 +13,7 @@ import (
"github.com/kube-vip/kube-vip/pkg/endpoints/providers"
"github.com/kube-vip/kube-vip/pkg/instance"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/lease"
"github.com/kube-vip/kube-vip/pkg/networkinterface"
"github.com/kube-vip/kube-vip/pkg/servicecontext"
"github.com/kube-vip/kube-vip/pkg/vip"
@@ -46,11 +47,22 @@ type Processor struct {
intfMgr *networkinterface.Manager
arpMgr *arp.Manager
leaseMgr *lease.Manager
// nodeLabelManager is the manager for the node labels
nodeLabelManager labelManager
}
// labelManager is the interface for the node label manager to add/remove labels
type labelManager interface {
AddLabel(ctx context.Context, svc *v1.Service) error
RemoveLabel(ctx context.Context, svc *v1.Service) error
}
func NewServicesProcessor(config *kubevip.Config, bgpServer *bgp.Server,
clientSet *kubernetes.Clientset, rwClientSet *kubernetes.Clientset, shutdownChan chan struct{},
intfMgr *networkinterface.Manager, arpMgr *arp.Manager) *Processor {
intfMgr *networkinterface.Manager, arpMgr *arp.Manager, nodeLabelManager labelManager) *Processor {
lbClassFilterFunc := lbClassFilter
if config.LoadBalancerClassLegacyHandling {
lbClassFilterFunc = lbClassFilterLegacy
@@ -71,8 +83,10 @@ func NewServicesProcessor(config *kubevip.Config, bgpServer *bgp.Server,
Help: "Count all events fired by the service watcher categorised by event type",
}, []string{"type"}),
intfMgr: intfMgr,
arpMgr: arpMgr,
intfMgr: intfMgr,
arpMgr: arpMgr,
leaseMgr: lease.NewManager(),
nodeLabelManager: nodeLabelManager,
}
}
@@ -89,7 +103,7 @@ func (p *Processor) AddOrModify(ctx context.Context, event watch.Event, serviceF
}
// Check if we ignore this service
if svc.Annotations["kube-vip.io/ignore"] == "true" {
if svc.Annotations[kubevip.LoadbalancerIgnore] == "true" {
log.Info("ignore annotation for kube-vip", "service name", svc.Name)
return true, nil
}
@@ -99,13 +113,19 @@ func (p *Processor) AddOrModify(ctx context.Context, event watch.Event, serviceF
return true, nil
}
svcAddresses := instance.FetchServiceAddresses(svc)
svcAddresses, svcHostnames := instance.FetchServiceAddresses(svc)
// We only care about LoadBalancer services that have been allocated an address
if len(svcAddresses) <= 0 {
return true, nil
}
_, usesCommonLease := svc.Annotations[kubevip.ServiceLease]
if usesCommonLease && svc.Spec.ExternalTrafficPolicy != v1.ServiceExternalTrafficPolicyTypeCluster {
return false, fmt.Errorf("annotation %q cannot be used with service traffic policy other than %q",
kubevip.ServiceLease, v1.ServiceExternalTrafficPolicyTypeCluster)
}
svcCtx, err := p.getServiceContext(svc.UID)
if err != nil {
return false, fmt.Errorf("failed to get service context: %w", err)
@@ -114,15 +134,20 @@ func (p *Processor) AddOrModify(ctx context.Context, event watch.Event, serviceF
// The modified event should only be triggered if the service has been modified (i.e. moved somewhere else)
if event.Type == watch.Modified {
i := instance.FindServiceInstance(svc, p.ServiceInstances)
var originalService []string
shouldGarbageCollect := true
if i != nil {
originalService = instance.FetchServiceAddresses(i.ServiceSnapshot)
shouldGarbageCollect = !reflect.DeepEqual(originalService, svcAddresses)
originalServiceAddresses, originalServiceHostnames := instance.FetchServiceAddresses(i.ServiceSnapshot)
shouldGarbageCollect =
// Service addresses changed
!reflect.DeepEqual(originalServiceAddresses, svcAddresses) ||
// Service hostnames changed
!reflect.DeepEqual(originalServiceHostnames, svcHostnames) ||
// ExternalTrafficPolicy changed
svc.Spec.ExternalTrafficPolicy != i.ServiceSnapshot.Spec.ExternalTrafficPolicy
}
if shouldGarbageCollect {
for _, addr := range svcAddresses {
// log.Debugf("(svcs) Retreiving local addresses, to ensure that this modified address doesn't exist: %s", addr)
// log.Debugf("(svcs) Retrieving local addresses, to ensure that this modified address doesn't exist: %s", addr)
f, err := vip.GarbageCollect(p.config.Interface, addr, p.intfMgr)
if err != nil {
log.Error("(svcs) cleaning existing address error", "err", err)
@@ -131,30 +156,23 @@ func (p *Processor) AddOrModify(ctx context.Context, event watch.Event, serviceF
log.Warn("(svcs) already found existing config", "address", addr, "adapter", p.config.Interface)
}
}
}
// This service has been modified, but it was also active.
if svcCtx != nil && svcCtx.IsActive {
if i != nil {
originalService := instance.FetchServiceAddresses(i.ServiceSnapshot)
newService := instance.FetchServiceAddresses(svc)
if !reflect.DeepEqual(originalService, newService) {
// This service has been modified, but it was also active.
if svcCtx != nil && svcCtx.IsActive {
log.Warn("(svcs) The load balancer has changed, cancelling original load balancer")
//Set it to inactive
svcCtx.IsActive = false
svcCtx.Cancel()
log.Warn("(svcs) waiting for load balancer to finish")
<-svcCtx.Ctx.Done()
// Calls the cancel function of the context
if svcCtx != nil {
log.Warn("(svcs) The load balancer has changed, cancelling original load balancer")
svcCtx.Cancel()
log.Warn("(svcs) waiting for load balancer to finish")
<-svcCtx.Ctx.Done()
}
err = p.deleteService(svc.UID)
if err != nil {
log.Error("(svc) unable to remove", "service", svc.UID)
}
p.svcMap.Delete(svc.UID)
if err := p.deleteService(svc.UID); err != nil {
log.Error("(svc) unable to remove", "service", svc.UID)
}
// in theory this should never fail
p.svcMap.Delete(svc.UID)
// Reset the the svcCtx when it was garbage collected
// As the next function will create a new context when nil
svcCtx = nil
}
}
}
@@ -166,7 +184,8 @@ func (p *Processor) AddOrModify(ctx context.Context, event watch.Event, serviceF
//
if svcCtx == nil || svcCtx != nil && !svcCtx.IsActive {
log.Debug("(svcs) has been added/modified with addresses", "service name", svc.Name, "ip", instance.FetchServiceAddresses(svc))
ips, hostnames := instance.FetchServiceAddresses(svc)
log.Debug("(svcs) has been added/modified with addresses", "service name", svc.Name, "ips", ips, "hostnames", hostnames)
if svcCtx == nil {
svcCtx = servicecontext.New(ctx)
@@ -266,21 +285,22 @@ func (p *Processor) AddOrModify(ctx context.Context, event watch.Event, serviceF
func (p *Processor) Delete(event watch.Event) (bool, error) {
svc, ok := event.Object.(*v1.Service)
if !ok {
return false, fmt.Errorf("unable to parse Kubernetes services from API watcher")
return false, fmt.Errorf("(svcs) unable to parse Kubernetes services from API watcher")
}
svcCtx, err := p.getServiceContext(svc.UID)
if err != nil {
return false, fmt.Errorf("(svcs) unable to get context: %w", err)
}
if svcCtx != nil && svcCtx.IsActive {
if svcCtx != nil {
// We only care about LoadBalancer services
if svc.Spec.Type != v1.ServiceTypeLoadBalancer {
return true, nil
}
// We can ignore this service
if svc.Annotations["kube-vip.io/ignore"] == "true" {
log.Info("(svcs)ignore annotation for kube-vip", "service name", svc.Name)
if svc.Annotations[kubevip.LoadbalancerIgnore] == "true" {
log.Info("(svcs) ignore annotation for kube-vip", "service name", svc.Name)
return true, nil
}
@@ -300,20 +320,13 @@ func (p *Processor) Delete(event watch.Event) (bool, error) {
// Calls the cancel function of the context
log.Warn("(svcs) The load balancer was deleted, cancelling context")
svcCtx.IsActive = false
svcCtx.Cancel()
log.Warn("(svcs) waiting for load balancer to finish")
<-svcCtx.Ctx.Done()
p.svcMap.Delete(svc.UID)
}
if p.config.EnableLeaderElection && !p.config.EnableServicesElection {
if p.config.EnableBGP {
endpoints.ClearBGPHosts(svc, &p.ServiceInstances, p.bgpServer)
} else if p.config.EnableRoutingTable {
endpoints.ClearRoutes(svc, &p.ServiceInstances)
}
}
log.Info("(svcs) deleted", "service name", svc.Name, "namespace", svc.Namespace)
return true, nil

View File

@@ -12,6 +12,7 @@ import (
"github.com/google/go-cmp/cmp"
"github.com/vishvananda/netlink"
v1 "k8s.io/api/core/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
"k8s.io/apimachinery/pkg/util/wait"
@@ -23,6 +24,7 @@ import (
"github.com/kube-vip/kube-vip/pkg/instance"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/upnp"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/kube-vip/kube-vip/pkg/vip"
)
@@ -42,26 +44,38 @@ func (p *Processor) SyncServices(ctx context.Context, svc *v1.Service) error {
action := p.getServiceInstanceAction(svc)
switch action {
case ActionDelete:
log.Debug("[service] delete", "namespace", svc.Namespace, "name", svc.Name)
// remove the label from the node before deleting the service
if err := p.nodeLabelManager.RemoveLabel(ctx, svc); err != nil {
return fmt.Errorf("error removing label from node: %w", err)
}
log.Debug("[service] delete", "namespace", svc.Namespace, "name", svc.Name, "uid", svc.UID)
if err := p.deleteService(svc.UID); err != nil {
return fmt.Errorf("error deleting service %s/%s: %w", svc.Namespace, svc.Name, err)
}
case ActionAdd:
log.Debug("[service] add", "namespace", svc.Namespace, "name", svc.Name)
log.Debug("[service] add", "namespace", svc.Namespace, "name", svc.Name, "uid", svc.UID)
if err := p.addService(ctx, svc); err != nil {
return fmt.Errorf("error adding service %s/%s: %w", svc.Namespace, svc.Name, err)
}
// add the label to the node after adding the service
if err := p.nodeLabelManager.AddLabel(ctx, svc); err != nil {
return fmt.Errorf("error adding label to node: %w", err)
}
case ActionNone:
log.Debug("[service] no action", "namespace", svc.Namespace, "name", svc.Name)
log.Debug("[service] no action", "namespace", svc.Namespace, "name", svc.Name, "uid", svc.UID)
}
log.Debug("[FINISHED] Service Sync", "namespace", svc.Namespace, "name", svc.Name)
log.Debug("[FINISHED] Service Sync", "namespace", svc.Namespace, "name", svc.Name, "uid", svc.UID)
return nil
}
func (p *Processor) getServiceInstanceAction(svc *v1.Service) ServiceInstanceAction {
// protect against multiple calls
addresses := instance.FetchServiceAddresses(svc)
ingressIPs := instance.FetchLoadBalancerIngressAddresses(svc)
// get the annotations or legacy values from manual configuration
addresses, hostnames := instance.FetchServiceAddresses(svc)
// get the status information of the LB Service
statusAddresses, _ := instance.FetchLoadBalancerIngress(svc)
p.mutex.Lock()
defer p.mutex.Unlock()
@@ -70,22 +84,22 @@ func (p *Processor) getServiceInstanceAction(svc *v1.Service) ServiceInstanceAct
for _, address := range addresses {
// handle the case where the service instance needs to be deleted
if instance.IsDHCP {
if address != "0.0.0.0" {
if address != "0.0.0.0" && address != "::" {
return ActionDelete
}
if len(svc.Status.LoadBalancer.Ingress) > 0 && !slices.Contains(ingressIPs, instance.DHCPInterfaceIP) {
if len(svc.Status.LoadBalancer.Ingress) > 0 && !slices.Contains(statusAddresses, instance.DHCPInterfaceIP) {
return ActionDelete
}
}
if !instance.IsDHCP {
if address == "0.0.0.0" {
if address == "0.0.0.0" || address == "::" {
return ActionDelete
}
if len(svc.Status.LoadBalancer.Ingress) > 0 && !slices.Contains(ingressIPs, address) {
if len(svc.Status.LoadBalancer.Ingress) > 0 && !slices.Contains(statusAddresses, address) {
return ActionDelete
}
}
if !comparePortsAndPortStatuses(svc) {
if len(svc.Status.LoadBalancer.Ingress) > 0 && !comparePortsAndPortStatuses(svc) {
return ActionDelete
}
}
@@ -93,8 +107,8 @@ func (p *Processor) getServiceInstanceAction(svc *v1.Service) ServiceInstanceAct
return ActionNone
}
}
if len(addresses) > 0 {
log.Debug("No matching service instance found", "service", svc.Name, "namespace", svc.Namespace, "addresses", addresses)
if len(addresses) > 0 || len(hostnames) > 0 {
log.Debug("no matching service instance found", "service", svc.Name, "namespace", svc.Namespace, "addresses", addresses, "hostnames", hostnames)
return ActionAdd // If no matching instance is found, we need to add a new service instance
}
return ActionNone
@@ -160,7 +174,7 @@ func (p *Processor) addService(ctx context.Context, svc *v1.Service) error {
}
}
serviceIPs := instance.FetchServiceAddresses(svc)
serviceIPs, _ := instance.FetchServiceAddresses(svc)
// Check if we need to flush any conntrack connections (due to some dangling conntrack connections)
if svc.Annotations[kubevip.FlushContrack] == "true" {
@@ -202,7 +216,7 @@ func (p *Processor) addService(ctx context.Context, svc *v1.Service) error {
// Does the service have an active IPv6 endpoint
if svc.Annotations[kubevip.ActiveEndpointIPv6] != "" {
for _, serviceIP := range serviceIPs {
if !p.config.EnableEndpoints && vip.IsIPv6(serviceIP) {
if !p.config.EnableEndpoints && utils.IsIPv6(serviceIP) {
podIP = svc.Annotations[kubevip.ActiveEndpointIPv6]
@@ -217,7 +231,7 @@ func (p *Processor) addService(ctx context.Context, svc *v1.Service) error {
} else if svc.Annotations[kubevip.ActiveEndpoint] != "" { // Not expected to be IPv6, so should be an IPv4 address
for _, serviceIP := range serviceIPs {
podIPs := svc.Annotations[kubevip.ActiveEndpoint]
if !p.config.EnableEndpoints && vip.IsIPv6(serviceIP) {
if !p.config.EnableEndpoints && utils.IsIPv6(serviceIP) {
podIPs = svc.Annotations[kubevip.ActiveEndpointIPv6]
}
err = p.configureEgress(serviceIP, podIPs, svc.Namespace, string(svc.UID), svc.Annotations)
@@ -266,10 +280,12 @@ func (p *Processor) deleteService(uid types.UID) error {
serviceInstance = p.ServiceInstances[x]
}
}
// If we've been through all services and not found the correct one then error
if !found {
// TODO: - fix UX
// return fmt.Errorf("unable to find/stop service [%s]", uid)
log.Error("unable to find/stop service", "uid", uid)
return nil
}
@@ -283,11 +299,13 @@ func (p *Processor) deleteService(uid types.UID) error {
shared := false
vipSet := make(map[string]interface{})
for x := range updatedInstances {
for _, vip := range instance.FetchServiceAddresses(updatedInstances[x].ServiceSnapshot) { //updatedInstances[x].ServiceSnapshot.Spec.LoadBalancerIP {
vips, _ := instance.FetchServiceAddresses(updatedInstances[x].ServiceSnapshot)
for _, vip := range vips { //updatedInstances[x].ServiceSnapshot.Spec.LoadBalancerIP {
vipSet[vip] = nil
}
}
for _, vip := range instance.FetchServiceAddresses(serviceInstance.ServiceSnapshot) {
vips, _ := instance.FetchServiceAddresses(serviceInstance.ServiceSnapshot)
for _, vip := range vips {
if _, found := vipSet[vip]; found {
shared = true
}
@@ -308,9 +326,15 @@ func (p *Processor) deleteService(uid types.UID) error {
return fmt.Errorf("[service] error deleting DHCP Link : %v", err)
}
}
for i := range serviceInstance.VIPConfigs {
if serviceInstance.VIPConfigs[i].EnableBGP {
endpoints.ClearBGPHostsByInstance(serviceInstance, p.bgpServer)
if p.config.EnableBGP {
endpoints.ClearBGPHostsByInstance(serviceInstance, p.bgpServer)
}
if p.config.EnableRoutingTable && (p.config.EnableLeaderElection || p.config.EnableServicesElection) {
if errs := endpoints.ClearRoutesByInstance(serviceInstance.ServiceSnapshot, serviceInstance, &p.ServiceInstances); len(errs) > 0 {
for _, err := range errs {
log.Error("unable to clear routes", "err", err)
}
}
}
@@ -329,7 +353,7 @@ func (p *Processor) deleteService(uid types.UID) error {
// Update the service array
p.ServiceInstances = updatedInstances
log.Info("Removed instance from manager", "uid", uid, "remaining advertised services", len(p.ServiceInstances))
log.Info("Removed instance from manager", "uid", uid, "name", serviceInstance.ServiceSnapshot.Name, "remaining advertised services", len(p.ServiceInstances))
return nil
}
@@ -353,7 +377,8 @@ func (p *Processor) upnpMap(ctx context.Context, s *instance.Instance) {
// Reset Gateway IPs to remove stale addresses
s.UPNPGatewayIPs = make([]string, 0)
for _, vip := range instance.FetchServiceAddresses(s.ServiceSnapshot) {
vips, _ := instance.FetchServiceAddresses(s.ServiceSnapshot)
for _, vip := range vips {
for _, port := range s.ServiceSnapshot.Spec.Ports {
for _, gw := range gateways {
log.Info("[UPNP] Adding map", "vip", vip, "port", port.Port, "service", s.ServiceSnapshot.Name, "gateway", gw.WANIPv6FirewallControlClient.Location)
@@ -452,8 +477,8 @@ func (p *Processor) updateStatus(i *instance.Instance) error {
ingresses := []v1.LoadBalancerIngress{}
for _, c := range i.VIPConfigs {
if !vip.IsIP(c.VIP) {
ips, err := vip.LookupHost(c.VIP, p.config.DNSMode)
if !utils.IsIP(c.VIP) {
ips, err := utils.LookupHost(c.VIP, p.config.DNSMode)
if err != nil {
return err
}
@@ -484,7 +509,7 @@ func (p *Processor) updateStatus(i *instance.Instance) error {
if !cmp.Equal(currentService.Status.LoadBalancer.Ingress, ingresses) {
currentService.Status.LoadBalancer.Ingress = ingresses
_, err = p.clientSet.CoreV1().Services(currentService.Namespace).UpdateStatus(context.TODO(), currentService, metav1.UpdateOptions{})
if err != nil {
if err != nil && !apierrors.IsInvalid(err) {
log.Error("updating Service", "namespace", i.ServiceSnapshot.Namespace, "name", i.ServiceSnapshot.Name, "err", err)
return err
}

65
pkg/utils/dns.go Normal file
View File

@@ -0,0 +1,65 @@
package utils
import (
"fmt"
"net"
"github.com/pkg/errors"
)
// LookupHost resolves dnsName and return an IP or an error
func LookupHost(dnsName, dnsMode string) ([]string, error) {
result, err := net.LookupHost(dnsName)
if err != nil {
return nil, err
}
if len(result) == 0 {
return nil, errors.Errorf("empty address for %s", dnsName)
}
addrs := []string{}
switch dnsMode {
case "ipv4", "ipv6", "dual":
a, err := getIPbyFamily(result, dnsMode)
if err != nil {
return nil, err
}
addrs = append(addrs, a...)
default:
addrs = append(addrs, result[0])
}
return addrs, nil
}
func getIPbyFamily(addresses []string, family string) ([]string, error) {
var checkers []func(string) bool
families := []string{}
if family == "dual" || family == "ipv4" {
checkers = append(checkers, IsIPv4)
families = append(families, "IPv4")
}
if family == "dual" || family == "ipv6" {
checkers = append(checkers, IsIPv6)
families = append(families, "IPv6")
}
addrs := []string{}
for i, c := range checkers {
addr, err := getIPbyChecker(addresses, c)
if err != nil {
return nil, fmt.Errorf("error getting %s address: %w", families[i], err)
}
addrs = append(addrs, addr)
}
return addrs, nil
}
func getIPbyChecker(addresses []string, checker func(string) bool) (string, error) {
for _, addr := range addresses {
if checker(addr) {
return addr, nil
}
}
return "", fmt.Errorf("address not found")
}

58
pkg/utils/ip.go Normal file
View File

@@ -0,0 +1,58 @@
package utils
import (
"fmt"
"net"
)
// FormatIPWithSubnetMask takes a raw IP address and a subnet mask, and returns a formatted string in CIDR notation.
func FormatIPWithSubnetMask(rawIP string, subnetMask string) (string, error) {
addr := fmt.Sprintf("%s/%s", rawIP, subnetMask)
// Check if the input is valid
_, _, err := net.ParseCIDR(addr)
if err != nil {
return "", fmt.Errorf("invalid CIDR: %q, %w", addr, err)
}
return addr, nil
}
// IsIP returns if address is an IP or not
func IsIP(address string) bool {
ip := net.ParseIP(address)
return ip != nil
}
// IsIPv4 returns true only if address is a valid IPv4 address
func IsIPv4(address string) bool {
ip := net.ParseIP(address)
if ip == nil {
return false
}
return ip.To4() != nil
}
// IsIPv6 returns true only if address is a valid IPv6 address
func IsIPv6(address string) bool {
ip := net.ParseIP(address)
if ip == nil {
return false
}
return ip.To4() == nil
}
func IsIPv4CIDR(cidr string) bool {
ip, _, _ := net.ParseCIDR(cidr)
if ip == nil {
return false
}
return ip.To4() != nil
}
func IsIPv6CIDR(cidr string) bool {
ip, _, _ := net.ParseCIDR(cidr)
if ip == nil {
return false
}
return ip.To4() == nil
}

55
pkg/utils/netlink.go Normal file
View File

@@ -0,0 +1,55 @@
package utils
import (
"fmt"
"net"
"github.com/vishvananda/netlink"
)
// GetInterfaceByIP returns the network interface that has the specified IP address assigned.
func GetInterfaceByIP(ipAddr string) (*netlink.Link, error) {
ip := net.ParseIP(ipAddr)
if ip == nil {
return nil, fmt.Errorf("invalid IP address: %s", ipAddr)
}
links, err := netlink.LinkList()
if err != nil {
return nil, fmt.Errorf("failed to list network interfaces: %v", err)
}
for i := range links {
addrs, err := netlink.AddrList(links[i], netlink.FAMILY_ALL)
if err != nil {
return nil, fmt.Errorf("failed to list addresses for interface %s: %v", links[i].Attrs().Name, err)
}
for _, addr := range addrs {
if addr.IP.Equal(ip) {
return &links[i], nil
}
}
}
return nil, fmt.Errorf("no interface found with IP address: %s", ipAddr)
}
// GetNonLinkLocalIP returns the first non link-local IPv4/IPv6 address on the given interface.
func GetNonLinkLocalIP(iface *netlink.Link, family int) (string, error) {
a, err := netlink.AddrList(*iface, family)
if err != nil {
return "", fmt.Errorf("failed to list addresses for interface %s: %v", (*iface).Attrs().Name, err)
}
for _, addr := range a {
if addr.IPNet != nil {
ip := addr.IPNet.IP
if !ip.IsLinkLocalUnicast() {
return ip.String(), nil
}
}
}
return "", fmt.Errorf("failed to find non-local IP on interface: %s", (*iface).Attrs().Name)
}

View File

@@ -1,8 +1,6 @@
package utils
import (
"fmt"
"net"
"os"
)
@@ -13,15 +11,3 @@ func FileExists(filename string) bool {
}
return !info.IsDir()
}
// FormatIPWithSubnetMask takes a raw IP address and a subnet mask, and returns a formatted string in CIDR notation.
func FormatIPWithSubnetMask(rawIP string, subnetMask string) (string, error) {
addr := fmt.Sprintf("%s/%s", rawIP, subnetMask)
// Check if the input is valid
_, _, err := net.ParseCIDR(addr)
if err != nil {
return "", fmt.Errorf("invalid CIDR: %q, %w", addr, err)
}
return addr, nil
}

View File

@@ -18,15 +18,16 @@ import (
v1 "k8s.io/api/core/v1"
"github.com/kube-vip/kube-vip/pkg/iptables"
"github.com/kube-vip/kube-vip/pkg/networkinterface"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/pkg/utils"
"github.com/kube-vip/kube-vip/pkg/networkinterface"
)
const (
defaultValidLft = 60
iptablesComment = "%s kube-vip load balancer IP"
iptablesCommentMarkRule = "kube-vip load balancer IP set mark for masquerade"
ignoreServiceSecurityAnnotation = "kube-vip.io/ignore-service-security"
defaultValidLft = 60
iptablesComment = "%s kube-vip load balancer IP"
iptablesCommentMarkRule = "kube-vip load balancer IP set mark for masquerade"
)
// Network is an interface that enable managing operations for a given IP
@@ -94,7 +95,7 @@ func NewConfig(address string, iface string, loGlobalScope bool, subnet string,
networkLink := intfMgr.Get(link)
if IsIP(address) {
if utils.IsIP(address) {
result := &network{
link: networkLink,
routeTable: tableID,
@@ -134,7 +135,7 @@ func NewConfig(address string, iface string, loGlobalScope bool, subnet string,
networks = append(networks, result)
} else {
// try to resolve the address
ips, err := LookupHost(address, dnsMode)
ips, err := utils.LookupHost(address, dnsMode)
if err != nil {
// return early for ddns if no IP is allocated for the domain
// when leader starts, should do get IP from DHCP for the domain
@@ -580,7 +581,7 @@ func (configurator *network) IsDADFAIL() bool {
configurator.link.Lock.Lock()
defer configurator.link.Lock.Unlock()
if configurator.address == nil || !IsIPv6(configurator.address.IP.String()) {
if configurator.address == nil || !utils.IsIPv6(configurator.address.IP.String()) {
return false
}
@@ -669,7 +670,7 @@ func (configurator *network) SetServicePorts(service *v1.Service) {
configurator.ports = service.Spec.Ports
configurator.serviceName = service.Namespace + "/" + service.Name
configurator.ignoreSecurity = service.Annotations[ignoreServiceSecurityAnnotation] == "true"
configurator.ignoreSecurity = service.Annotations[kubevip.ServiceSecurityIgnore] == "true"
}
// IP - return the IP Address
@@ -770,7 +771,7 @@ func (configurator *network) SetMask(mask string) error {
size := 32
family := "IPv4"
if IsIPv6(configurator.IP()) {
if utils.IsIPv6(configurator.IP()) {
size = 128
family = "IPv6"
}

View File

@@ -5,6 +5,8 @@ import (
"time"
log "log/slog"
"github.com/kube-vip/kube-vip/pkg/utils"
)
// IPUpdater is the interface to plug dns updaters
@@ -33,11 +35,11 @@ func (d *ipUpdater) Run(ctx context.Context) {
return
default:
mode := "ipv4"
if IsIPv6(d.vip.IP()) {
if utils.IsIPv6(d.vip.IP()) {
mode = "ipv6"
}
ip, err := LookupHost(d.vip.DNSName(), mode)
ip, err := utils.LookupHost(d.vip.DNSName(), mode)
if err != nil {
log.Warn("cannot lookup", "name", d.vip.DNSName(), "err", err)
// fallback to renewing the existing IP

View File

@@ -14,69 +14,6 @@ import (
"github.com/vishvananda/netlink"
)
// LookupHost resolves dnsName and return an IP or an error
func LookupHost(dnsName, dnsMode string) ([]string, error) {
result, err := net.LookupHost(dnsName)
if err != nil {
return nil, err
}
if len(result) == 0 {
return nil, errors.Errorf("empty address for %s", dnsName)
}
addrs := []string{}
switch dnsMode {
case "ipv4", "ipv6", "dual":
a, err := getIPbyFamily(result, dnsMode)
if err != nil {
return nil, err
}
addrs = append(addrs, a...)
default:
addrs = append(addrs, result[0])
}
return addrs, nil
}
func getIPbyFamily(addresses []string, family string) ([]string, error) {
var checkers []func(string) bool
families := []string{}
if family == "dual" || family == "ipv4" {
checkers = append(checkers, IsIPv4)
families = append(families, "IPv4")
}
if family == "dual" || family == "ipv6" {
checkers = append(checkers, IsIPv6)
families = append(families, "IPv6")
}
addrs := []string{}
for i, c := range checkers {
addr, err := getIPbyChecker(addresses, c)
if err != nil {
return nil, fmt.Errorf("error getting %s address: %w", families[i], err)
}
addrs = append(addrs, addr)
}
return addrs, nil
}
func getIPbyChecker(addresses []string, checker func(string) bool) (string, error) {
for _, addr := range addresses {
if checker(addr) {
return addr, nil
}
}
return "", fmt.Errorf("address not found")
}
// IsIP returns if address is an IP or not
func IsIP(address string) bool {
ip := net.ParseIP(address)
return ip != nil
}
// getHostName return the hostname from the fqdn
func getHostName(dnsName string) string {
if dnsName == "" {
@@ -87,40 +24,6 @@ func getHostName(dnsName string) string {
return fields[0]
}
// IsIPv4 returns true only if address is a valid IPv4 address
func IsIPv4(address string) bool {
ip := net.ParseIP(address)
if ip == nil {
return false
}
return ip.To4() != nil
}
// IsIPv6 returns true only if address is a valid IPv6 address
func IsIPv6(address string) bool {
ip := net.ParseIP(address)
if ip == nil {
return false
}
return ip.To4() == nil
}
func IsIPv4CIDR(cidr string) bool {
ip, _, _ := net.ParseCIDR(cidr)
if ip == nil {
return false
}
return ip.To4() != nil
}
func IsIPv6CIDR(cidr string) bool {
ip, _, _ := net.ParseCIDR(cidr)
if ip == nil {
return false
}
return ip.To4() == nil
}
// GetDefaultGatewayInterface return default gateway interface link
func GetDefaultGatewayInterface() (*net.Interface, error) {
routes, err := netlink.RouteList(nil, syscall.AF_INET)

13
test.sh
View File

@@ -1,13 +0,0 @@
#!/usr/bin/bash
rm -f logs.txt
for i in $(seq 1 100);
do
echo "RUN $i"
GOMAXPROCS=4 make e2e-tests129
if [[ "$?" -ne 0 ]]; then
echo "FAILED AT RUN $i"
break
fi
done

View File

@@ -56,12 +56,12 @@ var _ = Describe("kube-vip BGP mode", Ordered, func() {
configPath string
kubeVIPBGPManifestTemplate *template.Template
goBGPConfigTemplate *template.Template
tempDirPath string
v129 bool
localIPv4 string
localIPv6 string
curDir string
networkInterface string
tempDirPathRoot string
bgpKill chan any
)
@@ -89,7 +89,7 @@ var _ = Describe("kube-vip BGP mode", Ordered, func() {
kubeVIPBGPManifestTemplate, err = template.New("kube-vip-bgp.yaml.tmpl").ParseFiles(templateBGPPath)
Expect(err).NotTo(HaveOccurred())
tempDirPath, err = os.MkdirTemp("", "kube-vip-test")
tempDirPathRoot, err = os.MkdirTemp("", "kube-vip-test-bgp")
Expect(err).NotTo(HaveOccurred())
v4addr, _, err := deployment.GetLocalIPv4(networkInterface)
Expect(err).ToNot(HaveOccurred())
@@ -109,9 +109,9 @@ var _ = Describe("kube-vip BGP mode", Ordered, func() {
goBGPConfigTemplate, err = template.New("config.toml.tmpl").ParseFiles(goBGPConfigPath)
Expect(err).ToNot(HaveOccurred())
goBGPConfigPath = filepath.Join(tempDirPath, "config.toml")
goBGPConfigPath = filepath.Join(tempDirPathRoot, "config.toml")
f, err := os.OpenFile(goBGPConfigPath, os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0600)
f, err := os.OpenFile(goBGPConfigPath, os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0644)
Expect(err).ToNot(HaveOccurred())
defer f.Close()
@@ -123,6 +123,9 @@ var _ = Describe("kube-vip BGP mode", Ordered, func() {
AfterAll(func() {
close(bgpKill)
if os.Getenv("E2E_KEEP_LOGS") != "true" {
Expect(os.RemoveAll(tempDirPathRoot)).To(Succeed())
}
})
Describe("kube-vip IPv4 services BGP mode functionality", Ordered, func() {
@@ -133,29 +136,39 @@ var _ = Describe("kube-vip BGP mode", Ordered, func() {
manifestValues *e2e.KubevipManifestValues
gobgpClient api.GobgpApiClient
gobgpPeers []*e2e.BGPPeerValues
tempDirPath string
nodesNumber = 1
)
BeforeAll(func() {
setupEnv(&tempDirPath, &cpVIP, &clusterName, manifestValues, localIPv4, localIPv6, imagePath, configPath,
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
setupEnv(&cpVIP, &clusterName, tempDirPath, manifestValues, localIPv4, localIPv6, imagePath, configPath,
k8sImagePath, e2e.IPv4Family, e2e.IPv4Family, []string{e2e.IPv4Family}, &client, &gobgpPeers, v129,
kubeVIPBGPManifestTemplate, &gobgpClient, logger, nodesNumber, "", "bgp-ipv4")
kubeVIPBGPManifestTemplate, &gobgpClient, logger, nodesNumber, "", "bgp-ipv4", false)
})
AfterAll(func() {
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
for _, p := range gobgpPeers {
_, err := gobgpClient.DeletePeer(context.TODO(), &api.DeletePeerRequest{
Address: p.IP,
})
Expect(err).ToNot(HaveOccurred())
Eventually(func() error {
_, err := gobgpClient.DeletePeer(context.TODO(), &api.DeletePeerRequest{
Address: p.IP,
})
return err
}, "30s", "200ms").Should(Succeed())
}
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("advertise IPv4 routes for services",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv4Family, api.Family_AFI_IP, []corev1.IPFamily{corev1.IPv4Protocol}, svcName, trafficPolicy, client, 1, gobgpClient, "")
testBGP(offset, e2e.IPv4Family, api.Family_AFI_IP, []corev1.IPFamily{corev1.IPv4Protocol}, svcName,
trafficPolicy, client, 1, gobgpClient, "", "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
@@ -163,7 +176,8 @@ var _ = Describe("kube-vip BGP mode", Ordered, func() {
DescribeTable("only stops advertising route if it was referenced by multiple services and all of them were deleted",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv4Family, api.Family_AFI_IP, []corev1.IPFamily{corev1.IPv4Protocol}, svcName, trafficPolicy, client, 2, gobgpClient, "")
testBGP(offset, e2e.IPv4Family, api.Family_AFI_IP, []corev1.IPFamily{corev1.IPv4Protocol}, svcName,
trafficPolicy, client, 2, gobgpClient, "", "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
@@ -178,29 +192,37 @@ var _ = Describe("kube-vip BGP mode", Ordered, func() {
manifestValues *e2e.KubevipManifestValues
gobgpClient api.GobgpApiClient
gobgpPeers []*e2e.BGPPeerValues
tempDirPath string
nodesNumber = 1
)
BeforeAll(func() {
setupEnv(&tempDirPath, &cpVIP, &clusterName, manifestValues, localIPv4, localIPv6, imagePath, configPath,
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
setupEnv(&cpVIP, &clusterName, tempDirPath, manifestValues, localIPv4, localIPv6, imagePath, configPath,
k8sImagePath, e2e.IPv6Family, e2e.IPv6Family, []string{e2e.IPv6Family}, &client, &gobgpPeers, v129,
kubeVIPBGPManifestTemplate, &gobgpClient, logger, nodesNumber, "", "bgp-ipv4")
kubeVIPBGPManifestTemplate, &gobgpClient, logger, nodesNumber, "", "bgp-ipv4", false)
})
AfterAll(func() {
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
for _, p := range gobgpPeers {
_, err := gobgpClient.DeletePeer(context.TODO(), &api.DeletePeerRequest{
Address: p.IP,
})
Expect(err).ToNot(HaveOccurred())
}
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("advertise IPv6 routes for services",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv6Family, api.Family_AFI_IP6, []corev1.IPFamily{corev1.IPv6Protocol}, svcName, trafficPolicy, client, 1, gobgpClient, "")
testBGP(offset, e2e.IPv6Family, api.Family_AFI_IP6, []corev1.IPFamily{corev1.IPv6Protocol}, svcName,
trafficPolicy, client, 1, gobgpClient, "", "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
@@ -208,7 +230,8 @@ var _ = Describe("kube-vip BGP mode", Ordered, func() {
DescribeTable("only stops advertising route if it was referenced by multiple services and all of them were deleted",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv6Family, api.Family_AFI_IP6, []corev1.IPFamily{corev1.IPv6Protocol}, svcName, trafficPolicy, client, 2, gobgpClient, "")
testBGP(offset, e2e.IPv6Family, api.Family_AFI_IP6, []corev1.IPFamily{corev1.IPv6Protocol}, svcName,
trafficPolicy, client, 2, gobgpClient, "", "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
@@ -223,29 +246,37 @@ var _ = Describe("kube-vip BGP mode", Ordered, func() {
manifestValues *e2e.KubevipManifestValues
gobgpClient api.GobgpApiClient
gobgpPeers []*e2e.BGPPeerValues
tempDirPath string
nodesNumber = 1
)
BeforeAll(func() {
setupEnv(&tempDirPath, &cpVIP, &clusterName, manifestValues, localIPv4, localIPv6, imagePath, configPath, k8sImagePath,
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
setupEnv(&cpVIP, &clusterName, tempDirPath, manifestValues, localIPv4, localIPv6, imagePath, configPath, k8sImagePath,
e2e.DualstackFamily, e2e.IPv4Family, []string{e2e.IPv4Family}, &client, &gobgpPeers, v129, kubeVIPBGPManifestTemplate, &gobgpClient,
logger, nodesNumber, "fixed", "mpbgp-ipv4")
logger, nodesNumber, "fixed", "mpbgp-ipv4", false)
})
AfterAll(func() {
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
for _, p := range gobgpPeers {
_, err := gobgpClient.DeletePeer(context.TODO(), &api.DeletePeerRequest{
Address: p.IP,
})
Expect(err).ToNot(HaveOccurred())
}
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("advertise IPv6 routes over IPv4 session",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv6Family, api.Family_AFI_IP6, []corev1.IPFamily{corev1.IPv6Protocol}, svcName, trafficPolicy, client, 1, gobgpClient, defaultFixedNexthopv6)
testBGP(offset, e2e.IPv6Family, api.Family_AFI_IP6, []corev1.IPFamily{corev1.IPv6Protocol}, svcName,
trafficPolicy, client, 1, gobgpClient, defaultFixedNexthopv6, "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
@@ -253,7 +284,8 @@ var _ = Describe("kube-vip BGP mode", Ordered, func() {
DescribeTable("only stops advertising route if it was referenced by multiple services and all of them were deleted",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv6Family, api.Family_AFI_IP6, []corev1.IPFamily{corev1.IPv6Protocol}, svcName, trafficPolicy, client, 2, gobgpClient, defaultFixedNexthopv6)
testBGP(offset, e2e.IPv6Family, api.Family_AFI_IP6, []corev1.IPFamily{corev1.IPv6Protocol}, svcName,
trafficPolicy, client, 2, gobgpClient, defaultFixedNexthopv6, "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
@@ -268,29 +300,37 @@ var _ = Describe("kube-vip BGP mode", Ordered, func() {
manifestValues *e2e.KubevipManifestValues
gobgpClient api.GobgpApiClient
gobgpPeers []*e2e.BGPPeerValues
tempDirPath string
nodesNumber = 1
)
BeforeAll(func() {
setupEnv(&tempDirPath, &cpVIP, &clusterName, manifestValues, localIPv4, localIPv6, imagePath, configPath, k8sImagePath,
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
setupEnv(&cpVIP, &clusterName, tempDirPath, manifestValues, localIPv4, localIPv6, imagePath, configPath, k8sImagePath,
e2e.DualstackFamilyIPv6, e2e.IPv6Family, []string{e2e.IPv6Family}, &client, &gobgpPeers, v129, kubeVIPBGPManifestTemplate, &gobgpClient,
logger, nodesNumber, "fixed", "mpbgp-ipv6")
logger, nodesNumber, "fixed", "mpbgp-ipv6", false)
})
AfterAll(func() {
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
for _, n := range gobgpPeers {
_, err := gobgpClient.DeletePeer(context.TODO(), &api.DeletePeerRequest{
Address: n.IP,
})
Expect(err).ToNot(HaveOccurred())
}
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("advertise IPv4 routes over IPv6 session",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv4Family, api.Family_AFI_IP, []corev1.IPFamily{corev1.IPv4Protocol}, svcName, trafficPolicy, client, 1, gobgpClient, defaultFixedNexthopv4)
testBGP(offset, e2e.IPv4Family, api.Family_AFI_IP, []corev1.IPFamily{corev1.IPv4Protocol}, svcName,
trafficPolicy, client, 1, gobgpClient, defaultFixedNexthopv4, "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
@@ -298,7 +338,8 @@ var _ = Describe("kube-vip BGP mode", Ordered, func() {
DescribeTable("only stops advertising route if it was referenced by multiple services and all of them were deleted",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv4Family, api.Family_AFI_IP, []corev1.IPFamily{corev1.IPv4Protocol}, svcName, trafficPolicy, client, 2, gobgpClient, defaultFixedNexthopv4)
testBGP(offset, e2e.IPv4Family, api.Family_AFI_IP, []corev1.IPFamily{corev1.IPv4Protocol}, svcName,
trafficPolicy, client, 2, gobgpClient, defaultFixedNexthopv4, "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
@@ -314,29 +355,37 @@ var _ = Describe("kube-vip BGP mode", Ordered, func() {
gobgpClient api.GobgpApiClient
gobgpPeers []*e2e.BGPPeerValues
containerIP string
tempDirPath string
nodesNumber = 1
)
BeforeAll(func() {
_, containerIP = setupEnv(&tempDirPath, &cpVIP, &clusterName, manifestValues, localIPv4, localIPv6, imagePath, configPath, k8sImagePath,
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
_, containerIP = setupEnv(&cpVIP, &clusterName, tempDirPath, manifestValues, localIPv4, localIPv6, imagePath, configPath, k8sImagePath,
e2e.DualstackFamily, e2e.IPv4Family, []string{e2e.IPv4Family}, &client, &gobgpPeers, v129, kubeVIPBGPManifestTemplate, &gobgpClient,
logger, nodesNumber, "auto_sourceif", "mpbgp-if-ipv4")
logger, nodesNumber, "auto_sourceif", "mpbgp-if-ipv4", false)
})
AfterAll(func() {
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
for _, p := range gobgpPeers {
_, err := gobgpClient.DeletePeer(context.TODO(), &api.DeletePeerRequest{
Address: p.IP,
})
Expect(err).ToNot(HaveOccurred())
}
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("advertise IPv6 routes over IPv4 session",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv6Family, api.Family_AFI_IP6, []corev1.IPFamily{corev1.IPv6Protocol}, svcName, trafficPolicy, client, 1, gobgpClient, containerIP)
testBGP(offset, e2e.IPv6Family, api.Family_AFI_IP6, []corev1.IPFamily{corev1.IPv6Protocol}, svcName,
trafficPolicy, client, 1, gobgpClient, containerIP, "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
@@ -344,7 +393,8 @@ var _ = Describe("kube-vip BGP mode", Ordered, func() {
DescribeTable("only stops advertising route if it was referenced by multiple services and all of them were deleted",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv6Family, api.Family_AFI_IP6, []corev1.IPFamily{corev1.IPv6Protocol}, svcName, trafficPolicy, client, 2, gobgpClient, containerIP)
testBGP(offset, e2e.IPv6Family, api.Family_AFI_IP6, []corev1.IPFamily{corev1.IPv6Protocol}, svcName,
trafficPolicy, client, 2, gobgpClient, containerIP, "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
@@ -360,29 +410,37 @@ var _ = Describe("kube-vip BGP mode", Ordered, func() {
gobgpClient api.GobgpApiClient
gobgpPeers []*e2e.BGPPeerValues
containerIP string
tempDirPath string
nodesNumber = 1
)
BeforeAll(func() {
containerIP, _ = setupEnv(&tempDirPath, &cpVIP, &clusterName, manifestValues, localIPv4, localIPv6, imagePath, configPath, k8sImagePath,
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
containerIP, _ = setupEnv(&cpVIP, &clusterName, tempDirPath, manifestValues, localIPv4, localIPv6, imagePath, configPath, k8sImagePath,
e2e.DualstackFamilyIPv6, e2e.IPv6Family, []string{e2e.IPv6Family}, &client, &gobgpPeers, v129, kubeVIPBGPManifestTemplate, &gobgpClient,
logger, nodesNumber, "auto_sourceif", "mpbgp-if-ipv6")
logger, nodesNumber, "auto_sourceif", "mpbgp-if-ipv6", false)
})
AfterAll(func() {
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
for _, n := range gobgpPeers {
_, err := gobgpClient.DeletePeer(context.TODO(), &api.DeletePeerRequest{
Address: n.IP,
})
Expect(err).ToNot(HaveOccurred())
}
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("advertise IPv4 routes over IPv6 session",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv4Family, api.Family_AFI_IP, []corev1.IPFamily{corev1.IPv4Protocol}, svcName, trafficPolicy, client, 1, gobgpClient, containerIP)
testBGP(offset, e2e.IPv4Family, api.Family_AFI_IP, []corev1.IPFamily{corev1.IPv4Protocol}, svcName,
trafficPolicy, client, 1, gobgpClient, containerIP, "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
@@ -390,32 +448,408 @@ var _ = Describe("kube-vip BGP mode", Ordered, func() {
DescribeTable("only stops advertising route if it was referenced by multiple services and all of them were deleted",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv4Family, api.Family_AFI_IP, []corev1.IPFamily{corev1.IPv4Protocol}, svcName, trafficPolicy, client, 2, gobgpClient, containerIP)
testBGP(offset, e2e.IPv4Family, api.Family_AFI_IP, []corev1.IPFamily{corev1.IPv4Protocol}, svcName,
trafficPolicy, client, 2, gobgpClient, containerIP, "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
})
Describe("kube-vip IPv4 services BGP mode functionality", Ordered, func() {
var (
cpVIP string
clusterName string
client kubernetes.Interface
manifestValues *e2e.KubevipManifestValues
gobgpClient api.GobgpApiClient
gobgpPeers []*e2e.BGPPeerValues
tempDirPath string
nodesNumber = 1
)
BeforeAll(func() {
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
setupEnv(&cpVIP, &clusterName, tempDirPath, manifestValues, localIPv4, localIPv6, imagePath, configPath,
k8sImagePath, e2e.IPv4Family, e2e.IPv4Family, []string{e2e.IPv4Family}, &client, &gobgpPeers, v129,
kubeVIPBGPManifestTemplate, &gobgpClient, logger, nodesNumber, "", "bgp-ipv4", true)
})
AfterAll(func() {
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
for _, p := range gobgpPeers {
Eventually(func() error {
_, err := gobgpClient.DeletePeer(context.TODO(), &api.DeletePeerRequest{
Address: p.IP,
})
return err
}, "30s", "200ms").Should(Succeed())
}
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("advertise IPv4 routes for services",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv4Family, api.Family_AFI_IP, []corev1.IPFamily{corev1.IPv4Protocol}, svcName,
trafficPolicy, client, 1, gobgpClient, "", "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
DescribeTable("only stops advertising route if it was referenced by multiple services and all of them were deleted",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv4Family, api.Family_AFI_IP, []corev1.IPFamily{corev1.IPv4Protocol}, svcName,
trafficPolicy, client, 2, gobgpClient, "", "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
DescribeTable("only stops advertising route if it was referenced by multiple services and all of them were deleted while using common lease",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv4Family, api.Family_AFI_IP, []corev1.IPFamily{corev1.IPv4Protocol}, svcName,
trafficPolicy, client, 2, gobgpClient, "", "common-lease")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
)
})
Describe("kube-vip IPv6 services BGP mode functionality", Ordered, func() {
var (
cpVIP string
clusterName string
client kubernetes.Interface
manifestValues *e2e.KubevipManifestValues
gobgpClient api.GobgpApiClient
gobgpPeers []*e2e.BGPPeerValues
tempDirPath string
nodesNumber = 1
)
BeforeAll(func() {
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
setupEnv(&cpVIP, &clusterName, tempDirPath, manifestValues, localIPv4, localIPv6, imagePath, configPath,
k8sImagePath, e2e.IPv6Family, e2e.IPv6Family, []string{e2e.IPv6Family}, &client, &gobgpPeers, v129,
kubeVIPBGPManifestTemplate, &gobgpClient, logger, nodesNumber, "", "bgp-ipv4", true)
})
AfterAll(func() {
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
for _, p := range gobgpPeers {
_, err := gobgpClient.DeletePeer(context.TODO(), &api.DeletePeerRequest{
Address: p.IP,
})
Expect(err).ToNot(HaveOccurred())
}
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("advertise IPv6 routes for services",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv6Family, api.Family_AFI_IP6, []corev1.IPFamily{corev1.IPv6Protocol}, svcName,
trafficPolicy, client, 1, gobgpClient, "", "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
DescribeTable("only stops advertising route if it was referenced by multiple services and all of them were deleted",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv6Family, api.Family_AFI_IP6, []corev1.IPFamily{corev1.IPv6Protocol}, svcName,
trafficPolicy, client, 2, gobgpClient, "", "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
DescribeTable("only stops advertising route if it was referenced by multiple services and all of them were deleted while using common lease",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv6Family, api.Family_AFI_IP6, []corev1.IPFamily{corev1.IPv6Protocol}, svcName,
trafficPolicy, client, 2, gobgpClient, "", "common-lease")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
)
})
Describe("kube-vip DualStack services BGP mode functionality with MP-BGP IPv6 over IPv4 - fixed nexthop", Ordered, func() {
var (
cpVIP string
clusterName string
client kubernetes.Interface
manifestValues *e2e.KubevipManifestValues
gobgpClient api.GobgpApiClient
gobgpPeers []*e2e.BGPPeerValues
tempDirPath string
nodesNumber = 1
)
BeforeAll(func() {
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
setupEnv(&cpVIP, &clusterName, tempDirPath, manifestValues, localIPv4, localIPv6, imagePath, configPath, k8sImagePath,
e2e.DualstackFamily, e2e.IPv4Family, []string{e2e.IPv4Family}, &client, &gobgpPeers, v129, kubeVIPBGPManifestTemplate, &gobgpClient,
logger, nodesNumber, "fixed", "mpbgp-ipv4", true)
})
AfterAll(func() {
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
for _, p := range gobgpPeers {
_, err := gobgpClient.DeletePeer(context.TODO(), &api.DeletePeerRequest{
Address: p.IP,
})
Expect(err).ToNot(HaveOccurred())
}
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("advertise IPv6 routes over IPv4 session",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv6Family, api.Family_AFI_IP6, []corev1.IPFamily{corev1.IPv6Protocol}, svcName,
trafficPolicy, client, 1, gobgpClient, defaultFixedNexthopv6, "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
DescribeTable("only stops advertising route if it was referenced by multiple services and all of them were deleted",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv6Family, api.Family_AFI_IP6, []corev1.IPFamily{corev1.IPv6Protocol}, svcName,
trafficPolicy, client, 2, gobgpClient, defaultFixedNexthopv6, "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
DescribeTable("only stops advertising route if it was referenced by multiple services and all of them were delete while using common lease",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv6Family, api.Family_AFI_IP6, []corev1.IPFamily{corev1.IPv6Protocol}, svcName,
trafficPolicy, client, 2, gobgpClient, defaultFixedNexthopv6, "common-lease")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
)
})
Describe("kube-vip DualStack services BGP mode functionality with MP-BGP IPv4 over IPv6 - fixed nexthop", Ordered, func() {
var (
cpVIP string
clusterName string
client kubernetes.Interface
manifestValues *e2e.KubevipManifestValues
gobgpClient api.GobgpApiClient
gobgpPeers []*e2e.BGPPeerValues
tempDirPath string
nodesNumber = 1
)
BeforeAll(func() {
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
setupEnv(&cpVIP, &clusterName, tempDirPath, manifestValues, localIPv4, localIPv6, imagePath, configPath, k8sImagePath,
e2e.DualstackFamilyIPv6, e2e.IPv6Family, []string{e2e.IPv6Family}, &client, &gobgpPeers, v129, kubeVIPBGPManifestTemplate, &gobgpClient,
logger, nodesNumber, "fixed", "mpbgp-ipv6", true)
})
AfterAll(func() {
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
for _, n := range gobgpPeers {
_, err := gobgpClient.DeletePeer(context.TODO(), &api.DeletePeerRequest{
Address: n.IP,
})
Expect(err).ToNot(HaveOccurred())
}
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("advertise IPv4 routes over IPv6 session",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv4Family, api.Family_AFI_IP, []corev1.IPFamily{corev1.IPv4Protocol}, svcName,
trafficPolicy, client, 1, gobgpClient, defaultFixedNexthopv4, "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
DescribeTable("only stops advertising route if it was referenced by multiple services and all of them were deleted",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv4Family, api.Family_AFI_IP, []corev1.IPFamily{corev1.IPv4Protocol}, svcName,
trafficPolicy, client, 2, gobgpClient, defaultFixedNexthopv4, "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
DescribeTable("only stops advertising route if it was referenced by multiple services and all of them were deleted while using common lease",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv4Family, api.Family_AFI_IP, []corev1.IPFamily{corev1.IPv4Protocol}, svcName,
trafficPolicy, client, 2, gobgpClient, defaultFixedNexthopv4, "common-lease")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
)
})
Describe("kube-vip DualStack services BGP mode functionality with MP-BGP IPv6 over IPv4 - auto_sourceif nexthop", Ordered, func() {
var (
cpVIP string
clusterName string
client kubernetes.Interface
manifestValues *e2e.KubevipManifestValues
gobgpClient api.GobgpApiClient
gobgpPeers []*e2e.BGPPeerValues
containerIP string
tempDirPath string
nodesNumber = 1
)
BeforeAll(func() {
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
_, containerIP = setupEnv(&cpVIP, &clusterName, tempDirPath, manifestValues, localIPv4, localIPv6, imagePath, configPath, k8sImagePath,
e2e.DualstackFamily, e2e.IPv4Family, []string{e2e.IPv4Family}, &client, &gobgpPeers, v129, kubeVIPBGPManifestTemplate, &gobgpClient,
logger, nodesNumber, "auto_sourceif", "mpbgp-if-ipv4", true)
})
AfterAll(func() {
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
for _, p := range gobgpPeers {
_, err := gobgpClient.DeletePeer(context.TODO(), &api.DeletePeerRequest{
Address: p.IP,
})
Expect(err).ToNot(HaveOccurred())
}
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("advertise IPv6 routes over IPv4 session",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv6Family, api.Family_AFI_IP6, []corev1.IPFamily{corev1.IPv6Protocol}, svcName,
trafficPolicy, client, 1, gobgpClient, containerIP, "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
DescribeTable("only stops advertising route if it was referenced by multiple services and all of them were deleted",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv6Family, api.Family_AFI_IP6, []corev1.IPFamily{corev1.IPv6Protocol}, svcName,
trafficPolicy, client, 2, gobgpClient, containerIP, "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
DescribeTable("only stops advertising route if it was referenced by multiple services and all of them were deleted while using common lease",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv6Family, api.Family_AFI_IP6, []corev1.IPFamily{corev1.IPv6Protocol}, svcName,
trafficPolicy, client, 2, gobgpClient, containerIP, "common-lease")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
)
})
Describe("kube-vip DualStack services BGP mode functionality with MP-BGP IPv4 over IPv6 - fixed nexthop", Ordered, func() {
var (
cpVIP string
clusterName string
client kubernetes.Interface
manifestValues *e2e.KubevipManifestValues
gobgpClient api.GobgpApiClient
gobgpPeers []*e2e.BGPPeerValues
containerIP string
tempDirPath string
nodesNumber = 1
)
BeforeAll(func() {
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
containerIP, _ = setupEnv(&cpVIP, &clusterName, tempDirPath, manifestValues, localIPv4, localIPv6, imagePath, configPath, k8sImagePath,
e2e.DualstackFamilyIPv6, e2e.IPv6Family, []string{e2e.IPv6Family}, &client, &gobgpPeers, v129, kubeVIPBGPManifestTemplate, &gobgpClient,
logger, nodesNumber, "auto_sourceif", "mpbgp-if-ipv6", true)
})
AfterAll(func() {
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
for _, n := range gobgpPeers {
_, err := gobgpClient.DeletePeer(context.TODO(), &api.DeletePeerRequest{
Address: n.IP,
})
Expect(err).ToNot(HaveOccurred())
}
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("advertise IPv4 routes over IPv6 session",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv4Family, api.Family_AFI_IP, []corev1.IPFamily{corev1.IPv4Protocol}, svcName,
trafficPolicy, client, 1, gobgpClient, containerIP, "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
DescribeTable("only stops advertising route if it was referenced by multiple services and all of them were deleted",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv4Family, api.Family_AFI_IP, []corev1.IPFamily{corev1.IPv4Protocol}, svcName,
trafficPolicy, client, 2, gobgpClient, containerIP, "")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
DescribeTable("only stops advertising route if it was referenced by multiple services and all of them were deleted while using common lease",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
testBGP(offset, e2e.IPv4Family, api.Family_AFI_IP, []corev1.IPFamily{corev1.IPv4Protocol}, svcName,
trafficPolicy, client, 2, gobgpClient, containerIP, "common-lease")
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
)
})
}
})
func testBGP(offset uint, lbFamily string, afiFamily api.Family_Afi, svcFamily []corev1.IPFamily, svcName string,
trafficPolicy corev1.ServiceExternalTrafficPolicy, client kubernetes.Interface, numberOfServices int, gobgpClient api.GobgpApiClient, expectedNexthop string) {
trafficPolicy corev1.ServiceExternalTrafficPolicy, client kubernetes.Interface, numberOfServices int,
gobgpClient api.GobgpApiClient, expectedNexthop string, serviceLease string) {
lbAddress := e2e.GenerateVIP(lbFamily, offset)
routeCheckFamily := &api.Family{
Afi: afiFamily,
Safi: api.Family_SAFI_UNICAST,
}
testServiceBGP(svcName, lbAddress, trafficPolicy, client, svcFamily, numberOfServices, gobgpClient, routeCheckFamily, expectedNexthop)
testServiceBGP(svcName, lbAddress, trafficPolicy, client, svcFamily, numberOfServices, gobgpClient, routeCheckFamily, expectedNexthop, serviceLease)
}
func setupEnv(tempDirPath, cpVIP, clusterName *string, manifestValues *e2e.KubevipManifestValues,
func setupEnv(cpVIP, clusterName *string, tempDirPath string, manifestValues *e2e.KubevipManifestValues,
localIPv4, localIPv6, imagePath, configPath, k8sImagePath, clusterAddrFamily, bgpClientAddrFamily string, peerAddrFamily []string, client *kubernetes.Interface,
gobgpPeers *[]*e2e.BGPPeerValues, v129 bool, kubeVIPBGPManifestTemplate *template.Template, gobgpClient *api.GobgpApiClient,
logger log.Logger, nodesNumber int, mpbgpnexthop, clusterNameSuffix string) (string, string) {
logger log.Logger, nodesNumber int, mpbgpnexthop, clusterNameSuffix string, serviceElection bool) (string, string) {
var err error
*tempDirPath, err = os.MkdirTemp("", "kube-vip-test")
Expect(err).ToNot(HaveOccurred())
*cpVIP = e2e.GenerateVIP(clusterAddrFamily, SOffset.Get())
@@ -471,7 +905,8 @@ func setupEnv(tempDirPath, cpVIP, clusterName *string, manifestValues *e2e.Kubev
ConfigPath: configPath,
ControlPlaneEnable: "false",
SvcEnable: "true",
SvcElectionEnable: "false",
SvcElectionEnable: fmt.Sprintf("%t", serviceElection),
EnableNodeLabeling: "false",
BGPAS: kubevipAS,
BGPPeers: strings.Join(kvPeersStr, ","),
MPBGPNexthop: mpbgpnexthop,
@@ -481,7 +916,8 @@ func setupEnv(tempDirPath, cpVIP, clusterName *string, manifestValues *e2e.Kubev
By(manifestValues.BGPPeers)
*clusterName, *client = prepareCluster(*tempDirPath, clusterNameSuffix, k8sImagePath, v129, kubeVIPBGPManifestTemplate, logger, manifestValues, networking, nodesNumber, nil)
*clusterName, *client, _ = prepareCluster(tempDirPath, clusterNameSuffix, k8sImagePath, v129,
kubeVIPBGPManifestTemplate, logger, manifestValues, networking, nodesNumber, nil, 1)
container := fmt.Sprintf("%s-control-plane", *clusterName)
@@ -553,7 +989,8 @@ func setupEnv(tempDirPath, cpVIP, clusterName *string, manifestValues *e2e.Kubev
func testServiceBGP(svcName, lbAddress string, trafficPolicy corev1.ServiceExternalTrafficPolicy,
client kubernetes.Interface, serviceAddrFamily []corev1.IPFamily, numberOfServices int,
gobgpClient api.GobgpApiClient, gobgpFamily *api.Family, expectedNexthop string) {
gobgpClient api.GobgpApiClient, gobgpFamily *api.Family, expectedNexthop string,
serviceLease string) {
lbAddresses := vip.Split(lbAddress)
services := []string{}
@@ -563,11 +1000,15 @@ func testServiceBGP(svcName, lbAddress string, trafficPolicy corev1.ServiceExter
for _, svc := range services {
createTestService(svc, dsNamespace, dsName, lbAddress,
client, corev1.IPFamilyPolicyPreferDualStack, serviceAddrFamily, trafficPolicy)
client, corev1.IPFamilyPolicyPreferDualStack, serviceAddrFamily, trafficPolicy, serviceLease, 80)
time.Sleep(time.Second)
}
for _, addr := range lbAddresses {
By(withTimestamp(fmt.Sprintf("checking bgp route for address %q", addr)))
paths := checkGoBGPPaths(context.Background(), gobgpClient, gobgpFamily, []*api.TableLookupPrefix{{Prefix: addr}}, 1)
Expect(paths).ToNot(BeNil())
Expect(paths).ToNot(BeEmpty())
Expect(strings.Contains(paths[0].Prefix, lbAddress)).To(BeTrue())
if expectedNexthop != "" {
Expect(strings.Contains(paths[0].String(), fmt.Sprintf("next_hop:\"%s\"", expectedNexthop)) || strings.Contains(paths[0].String(), fmt.Sprintf("next_hops:\"%s\"", expectedNexthop))).To(BeTrue())
@@ -575,11 +1016,16 @@ func testServiceBGP(svcName, lbAddress string, trafficPolicy corev1.ServiceExter
}
for i := range numberOfServices {
By(withTimestamp(fmt.Sprintf("deleting service '%s/%s'", dsNamespace, services[i])))
err := client.CoreV1().Services(dsNamespace).Delete(context.TODO(), services[i], metav1.DeleteOptions{})
Expect(err).ToNot(HaveOccurred())
time.Sleep(time.Second)
if i < numberOfServices-1 {
for _, addr := range lbAddresses {
By(withTimestamp(fmt.Sprintf("checking bgp route for address %q", addr)))
paths := checkGoBGPPaths(context.Background(), gobgpClient, gobgpFamily, []*api.TableLookupPrefix{{Prefix: addr}}, 1)
Expect(paths).ToNot(BeNil())
Expect(paths).ToNot(BeEmpty())
Expect(strings.Contains(paths[0].Prefix, lbAddress)).To(BeTrue())
if expectedNexthop != "" {
Expect(strings.Contains(paths[0].String(), fmt.Sprintf("next_hop:\"%s\"", expectedNexthop)) || strings.Contains(paths[0].String(), fmt.Sprintf("next_hops:\"%s\"", expectedNexthop))).To(BeTrue())
@@ -589,6 +1035,7 @@ func testServiceBGP(svcName, lbAddress string, trafficPolicy corev1.ServiceExter
}
for _, addr := range lbAddresses {
By(withTimestamp(fmt.Sprintf("checking bgp route for address %q - should be deleted", addr)))
checkGoBGPPaths(context.Background(), gobgpClient, gobgpFamily, []*api.TableLookupPrefix{{Prefix: addr}}, 0)
}
}
@@ -629,6 +1076,25 @@ func newGoBGPClient(address string, port uint32) (api.GobgpApiClient, error) {
}
func checkGoBGPPaths(ctx context.Context, client api.GobgpApiClient, family *api.Family, prefixes []*api.TableLookupPrefix, expectedPaths int) []*api.Destination {
// ticker := time.NewTicker(time.Second)
// defer ticker.Stop()
// to := time.NewTimer(time.Second * 180)
// defer to.Stop()
// for {
// select {
// case <-to.C:
// return nil
// case <-ticker.C:
// paths, err := getGoBGPPaths(ctx, client, family, prefixes)
// if err != nil {
// return nil
// }
// if len(paths) == expectedPaths {
// return paths
// }
// }
// }
var paths []*api.Destination
Eventually(func() error {
var err error
@@ -640,7 +1106,7 @@ func checkGoBGPPaths(ctx context.Context, client api.GobgpApiClient, family *api
return fmt.Errorf("expected %d paths, but found %d", expectedPaths, len(paths))
}
return nil
}, "120s").ShouldNot(HaveOccurred())
}, "360s", "1s").ShouldNot(HaveOccurred())
return paths
}

View File

@@ -28,7 +28,7 @@ import (
"github.com/kube-vip/kube-vip/testing/services/pkg/deployment"
)
var _ = Describe("kube-vip routing table mode", func() {
var _ = Describe("kube-vip routing table mode", Ordered, func() {
if Mode == ModeRT {
var (
logger log.Logger
@@ -36,10 +36,16 @@ var _ = Describe("kube-vip routing table mode", func() {
k8sImagePath string
configPath string
kubeVIPRoutingTableManifestTemplate *template.Template
tempDirPath string
tempDirPathRoot string
v129 bool
)
BeforeAll(func() {
var err error
tempDirPathRoot, err = os.MkdirTemp("", "kube-vip-test-rt")
Expect(err).NotTo(HaveOccurred())
})
BeforeEach(func() {
klog.SetOutput(GinkgoWriter)
logger = e2e.TestLogger{}
@@ -57,15 +63,20 @@ var _ = Describe("kube-vip routing table mode", func() {
templateRoutingTablePath := filepath.Join(curDir, "kube-vip-routing-table.yaml.tmpl")
kubeVIPRoutingTableManifestTemplate, err = template.New("kube-vip-routing-table.yaml.tmpl").ParseFiles(templateRoutingTablePath)
Expect(err).NotTo(HaveOccurred())
})
tempDirPath, err = os.MkdirTemp("", "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
AfterAll(func() {
if os.Getenv("E2E_KEEP_LOGS") != "true" {
Expect(os.RemoveAll(tempDirPathRoot)).To(Succeed())
}
})
Describe("kube-vip IPv4 control-plane routing table mode functionality", Ordered, func() {
var (
cpVIP string
clusterName string
client kubernetes.Interface
tempDirPath string
nodesNumber = 3
)
@@ -84,13 +95,22 @@ var _ = Describe("kube-vip routing table mode", func() {
ControlPlaneEnable: "true",
SvcEnable: "false",
SvcElectionEnable: "false",
EnableNodeLabeling: "false",
}
clusterName, _ = prepareCluster(tempDirPath, "rt-ipv4", k8sImagePath, v129, kubeVIPRoutingTableManifestTemplate, logger, manifestValues, networking, nodesNumber, nil)
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "rt-ipv4", k8sImagePath, v129,
kubeVIPRoutingTableManifestTemplate, logger, manifestValues, networking, nodesNumber, nil, 1)
})
AfterAll(func() {
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
cleanupCluster(clusterName, ConfigMtx, logger)
})
It("setups IPv4 address and route on control-plane node", func() {
@@ -105,7 +125,7 @@ var _ = Describe("kube-vip routing table mode", func() {
container = fmt.Sprintf("%s-control-plane", clusterName)
}
checkIPAddress(cpVIP, container, true)
Expect(checkIPAddress(cpVIP, container, true)).To(BeTrue())
e2e.CheckRoutePresence(cpVIP, container, true)
}
})
@@ -115,6 +135,8 @@ var _ = Describe("kube-vip routing table mode", func() {
var (
cpVIP string
clusterName string
client kubernetes.Interface
tempDirPath string
nodesNumber = 3
)
@@ -134,13 +156,22 @@ var _ = Describe("kube-vip routing table mode", func() {
ControlPlaneEnable: "true",
SvcEnable: "false",
SvcElectionEnable: "false",
EnableNodeLabeling: "false",
}
clusterName, _ = prepareCluster(tempDirPath, "rt-ipv6", k8sImagePath, v129, kubeVIPRoutingTableManifestTemplate, logger, manifestValues, networking, nodesNumber, nil)
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "rt-ipv6", k8sImagePath, v129,
kubeVIPRoutingTableManifestTemplate, logger, manifestValues, networking, nodesNumber, nil, 1)
})
AfterAll(func() {
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
cleanupCluster(clusterName, ConfigMtx, logger)
})
It("setups IPv6 address and route on control-plane node", func() {
@@ -155,7 +186,7 @@ var _ = Describe("kube-vip routing table mode", func() {
container = fmt.Sprintf("%s-control-plane", clusterName)
}
checkIPAddress(cpVIP, container, true)
Expect(checkIPAddress(cpVIP, container, true)).To(BeTrue())
e2e.CheckRoutePresence(cpVIP, container, true)
}
})
@@ -165,6 +196,8 @@ var _ = Describe("kube-vip routing table mode", func() {
var (
cpVIP string
clusterName string
client kubernetes.Interface
tempDirPath string
nodesNumber = 3
)
@@ -198,11 +231,18 @@ var _ = Describe("kube-vip routing table mode", func() {
ipnet: localIPv6Net,
}
clusterName, _ = prepareCluster(tempDirPath, "rt-ds-ipv4", k8sImagePath, v129, kubeVIPRoutingTableManifestTemplate, logger, manifestValues, networking, nodesNumber, addSAN)
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "rt-ds-ipv4", k8sImagePath, v129,
kubeVIPRoutingTableManifestTemplate, logger, manifestValues, networking, nodesNumber, addSAN, 1)
})
AfterAll(func() {
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
cleanupCluster(clusterName, ConfigMtx, logger)
})
It("setups DualStack addresses and routes on control-plane nodes", func() {
@@ -220,7 +260,7 @@ var _ = Describe("kube-vip routing table mode", func() {
addresses := vip.Split(cpVIP)
for _, addr := range addresses {
checkIPAddress(addr, container, true)
Expect(checkIPAddress(addr, container, true)).To(BeTrue())
e2e.CheckRoutePresence(addr, container, true)
}
}
@@ -231,6 +271,8 @@ var _ = Describe("kube-vip routing table mode", func() {
var (
cpVIP string
clusterName string
client kubernetes.Interface
tempDirPath string
nodesNumber = 3
)
@@ -266,11 +308,18 @@ var _ = Describe("kube-vip routing table mode", func() {
ipnet: localIPv4Net,
}
clusterName, _ = prepareCluster(tempDirPath, "rt-ds-ipv6", k8sImagePath, v129, kubeVIPRoutingTableManifestTemplate, logger, manifestValues, networking, nodesNumber, addSAN)
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "rt-ds-ipv6", k8sImagePath, v129,
kubeVIPRoutingTableManifestTemplate, logger, manifestValues, networking, nodesNumber, addSAN, 1)
})
AfterAll(func() {
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
cleanupCluster(clusterName, ConfigMtx, logger)
})
It("setups DualStack addresses and routes on control-plane nodes", func() {
@@ -288,7 +337,7 @@ var _ = Describe("kube-vip routing table mode", func() {
addresses := vip.Split(cpVIP)
for _, addr := range addresses {
checkIPAddress(addr, container, true)
Expect(checkIPAddress(addr, container, true)).To(BeTrue())
e2e.CheckRoutePresence(addr, container, true)
}
}
@@ -303,6 +352,7 @@ var _ = Describe("kube-vip routing table mode", func() {
manifestValues *e2e.KubevipManifestValues
svcElection bool
ipFamily []corev1.IPFamily
tempDirPath string
nodesNumber = 1
)
@@ -329,17 +379,25 @@ var _ = Describe("kube-vip routing table mode", func() {
ipFamily = []corev1.IPFamily{corev1.IPv4Protocol}
clusterName, client = prepareCluster(tempDirPath, "rt-svc-ipv4", k8sImagePath, v129, kubeVIPRoutingTableManifestTemplate, logger, manifestValues, networking, nodesNumber, nil)
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "rt-svc-ipv4", k8sImagePath, v129,
kubeVIPRoutingTableManifestTemplate, logger, manifestValues, networking, nodesNumber, nil, 1)
})
AfterAll(func() {
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("configures an IPv4 routes for services",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateVIP(e2e.IPv4Family, offset)
testServiceRT(svcName, lbAddress, fmt.Sprintf("kubevip-%s", svcName), dsNamespace, clusterName, trafficPolicy, client, svcElection, ipFamily, 1)
testServiceRT(svcName, lbAddress, "plndr-svcs-lock", "kube-system", clusterName,
trafficPolicy, client, svcElection, ipFamily, 1, false)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
@@ -348,7 +406,8 @@ var _ = Describe("kube-vip routing table mode", func() {
DescribeTable("only removes route if it was referenced by multiple services and all of them were deleted",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateVIP(e2e.IPv4Family, offset)
testServiceRT(svcName, lbAddress, "plndr-svcs-lock", "kube-system", clusterName, trafficPolicy, client, svcElection, ipFamily, 2)
testServiceRT(svcName, lbAddress, "plndr-svcs-lock", "kube-system", clusterName,
trafficPolicy, client, svcElection, ipFamily, 2, false)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
@@ -363,6 +422,7 @@ var _ = Describe("kube-vip routing table mode", func() {
manifestValues *e2e.KubevipManifestValues
svcElection bool
ipFamily []corev1.IPFamily
tempDirPath string
nodesNumber = 1
)
@@ -389,17 +449,25 @@ var _ = Describe("kube-vip routing table mode", func() {
ipFamily = []corev1.IPFamily{corev1.IPv6Protocol}
clusterName, client = prepareCluster(tempDirPath, "rt-svc-ipv6", k8sImagePath, v129, kubeVIPRoutingTableManifestTemplate, logger, manifestValues, networking, nodesNumber, nil)
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "rt-svc-ipv6", k8sImagePath, v129,
kubeVIPRoutingTableManifestTemplate, logger, manifestValues, networking, nodesNumber, nil, 1)
})
AfterAll(func() {
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("configures an IPv6 routes for services",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateVIP(e2e.IPv6Family, offset)
testServiceRT(svcName, lbAddress, fmt.Sprintf("kubevip-%s", svcName), dsNamespace, clusterName, trafficPolicy, client, svcElection, ipFamily, 1)
testServiceRT(svcName, lbAddress, "plndr-svcs-lock", "kube-system", clusterName,
trafficPolicy, client, svcElection, ipFamily, 1, false)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
@@ -408,7 +476,8 @@ var _ = Describe("kube-vip routing table mode", func() {
DescribeTable("only removes route if it was referenced by multiple services and all of them were deleted",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateVIP(e2e.IPv6Family, offset)
testServiceRT(svcName, lbAddress, "plndr-svcs-lock", "kube-system", clusterName, trafficPolicy, client, svcElection, ipFamily, 2)
testServiceRT(svcName, lbAddress, "plndr-svcs-lock", "kube-system", clusterName,
trafficPolicy, client, svcElection, ipFamily, 2, false)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
@@ -423,6 +492,7 @@ var _ = Describe("kube-vip routing table mode", func() {
manifestValues *e2e.KubevipManifestValues
svcElection bool
ipFamily []corev1.IPFamily
tempDirPath string
nodesNumber = 1
)
@@ -450,17 +520,25 @@ var _ = Describe("kube-vip routing table mode", func() {
ipFamily = []corev1.IPFamily{corev1.IPv4Protocol, corev1.IPv6Protocol}
clusterName, client = prepareCluster(tempDirPath, "rt-ds-svc-ipv4", k8sImagePath, v129, kubeVIPRoutingTableManifestTemplate, logger, manifestValues, networking, nodesNumber, nil)
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "rt-ds-svc-ipv4", k8sImagePath, v129,
kubeVIPRoutingTableManifestTemplate, logger, manifestValues, networking, nodesNumber, nil, 1)
})
AfterAll(func() {
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("configures an IPv4 and IPv6 routes for services",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateDualStackVIP(offset)
testServiceRT(svcName, lbAddress, fmt.Sprintf("kubevip-%s", svcName), dsNamespace, clusterName, trafficPolicy, client, svcElection, ipFamily, 1)
testServiceRT(svcName, lbAddress, "plndr-svcs-lock", "kube-system", clusterName,
trafficPolicy, client, svcElection, ipFamily, 1, false)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
@@ -469,7 +547,8 @@ var _ = Describe("kube-vip routing table mode", func() {
DescribeTable("only removes route if it was referenced by multiple services and all of them were deleted",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateDualStackVIP(offset)
testServiceRT(svcName, lbAddress, "plndr-svcs-lock", "kube-system", clusterName, trafficPolicy, client, svcElection, ipFamily, 2)
testServiceRT(svcName, lbAddress, "plndr-svcs-lock", "kube-system", clusterName,
trafficPolicy, client, svcElection, ipFamily, 2, false)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
@@ -484,6 +563,7 @@ var _ = Describe("kube-vip routing table mode", func() {
manifestValues *e2e.KubevipManifestValues
svcElection bool
ipFamily []corev1.IPFamily
tempDirPath string
nodesNumber = 1
)
@@ -513,17 +593,25 @@ var _ = Describe("kube-vip routing table mode", func() {
ipFamily = []corev1.IPFamily{corev1.IPv4Protocol, corev1.IPv6Protocol}
clusterName, client = prepareCluster(tempDirPath, "rt-ds-svc-ipv6", k8sImagePath, v129, kubeVIPRoutingTableManifestTemplate, logger, manifestValues, networking, nodesNumber, nil)
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "rt-ds-svc-ipv6", k8sImagePath, v129,
kubeVIPRoutingTableManifestTemplate, logger, manifestValues, networking, nodesNumber, nil, 1)
})
AfterAll(func() {
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("configures an IPv4 and IPv6 routes for services",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateDualStackVIP(offset)
testServiceRT(svcName, lbAddress, fmt.Sprintf("kubevip-%s", svcName), dsNamespace, clusterName, trafficPolicy, client, svcElection, ipFamily, 1)
testServiceRT(svcName, lbAddress, "plndr-svcs-lock", "kube-system", clusterName,
trafficPolicy, client, svcElection, ipFamily, 1, false)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
@@ -532,17 +620,338 @@ var _ = Describe("kube-vip routing table mode", func() {
DescribeTable("only removes route if it was referenced by multiple services and all of them were deleted",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateDualStackVIP(offset)
testServiceRT(svcName, lbAddress, "plndr-svcs-lock", "kube-system", clusterName, trafficPolicy, client, svcElection, ipFamily, 2)
testServiceRT(svcName, lbAddress, "plndr-svcs-lock", "kube-system", clusterName,
trafficPolicy, client, svcElection, ipFamily, 2, false)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
})
Describe("kube-vip IPv4 services routing table mode functionality with services election", Ordered, func() {
var (
cpVIP string
clusterName string
client kubernetes.Interface
manifestValues *e2e.KubevipManifestValues
svcElection bool
ipFamily []corev1.IPFamily
tempDirPath string
nodesNumber = 1
)
BeforeAll(func() {
cpVIP = e2e.GenerateVIP(e2e.IPv4Family, SOffset.Get())
networking := &kindconfigv1alpha4.Networking{
IPFamily: kindconfigv1alpha4.IPv4Family,
}
manifestValues = &e2e.KubevipManifestValues{
ControlPlaneVIP: cpVIP,
ImagePath: imagePath,
ConfigPath: configPath,
ControlPlaneEnable: "false",
SvcEnable: "true",
SvcElectionEnable: "true",
}
var err error
svcElection, err = strconv.ParseBool(manifestValues.SvcElectionEnable)
Expect(err).ToNot(HaveOccurred())
ipFamily = []corev1.IPFamily{corev1.IPv4Protocol}
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "rt-svc-ipv4", k8sImagePath, v129,
kubeVIPRoutingTableManifestTemplate, logger, manifestValues, networking, nodesNumber, nil, 1)
})
AfterAll(func() {
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("configures an IPv4 routes for services",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateVIP(e2e.IPv4Family, offset)
testServiceRT(svcName, lbAddress, fmt.Sprintf("kubevip-%s", svcName), dsNamespace, clusterName,
trafficPolicy, client, svcElection, ipFamily, 1, false)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
DescribeTable("only removes route if it was referenced by multiple services and all of them were deleted",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateVIP(e2e.IPv4Family, offset)
testServiceRT(svcName, lbAddress, fmt.Sprintf("kubevip-%s", svcName), dsNamespace, clusterName,
trafficPolicy, client, svcElection, ipFamily, 2, false)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
DescribeTable("only removes route if it was referenced by multiple services and all of them were deleted if common lease is used",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateVIP(e2e.IPv4Family, offset)
testServiceRT(svcName, lbAddress, fmt.Sprintf("kubevip-%s", svcName), dsNamespace, clusterName,
trafficPolicy, client, svcElection, ipFamily, 2, true)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
)
})
Describe("kube-vip IPv6 services routing table mode functionality with services election", Ordered, func() {
var (
cpVIP string
clusterName string
client kubernetes.Interface
manifestValues *e2e.KubevipManifestValues
svcElection bool
ipFamily []corev1.IPFamily
tempDirPath string
nodesNumber = 1
)
BeforeAll(func() {
cpVIP = e2e.GenerateVIP(e2e.IPv6Family, SOffset.Get())
networking := &kindconfigv1alpha4.Networking{
IPFamily: kindconfigv1alpha4.IPv6Family,
}
manifestValues = &e2e.KubevipManifestValues{
ControlPlaneVIP: cpVIP,
ImagePath: imagePath,
ConfigPath: configPath,
ControlPlaneEnable: "false",
SvcEnable: "true",
SvcElectionEnable: "true",
}
var err error
svcElection, err = strconv.ParseBool(manifestValues.SvcElectionEnable)
Expect(err).ToNot(HaveOccurred())
ipFamily = []corev1.IPFamily{corev1.IPv6Protocol}
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "rt-svc-ipv6", k8sImagePath, v129,
kubeVIPRoutingTableManifestTemplate, logger, manifestValues, networking, nodesNumber, nil, 1)
})
AfterAll(func() {
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("configures an IPv6 routes for services",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateVIP(e2e.IPv6Family, offset)
testServiceRT(svcName, lbAddress, fmt.Sprintf("kubevip-%s", svcName), dsNamespace, clusterName,
trafficPolicy, client, svcElection, ipFamily, 1, false)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
DescribeTable("only removes route if it was referenced by multiple services and all of them were deleted",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateVIP(e2e.IPv6Family, offset)
testServiceRT(svcName, lbAddress, fmt.Sprintf("kubevip-%s", svcName), dsNamespace, clusterName,
trafficPolicy, client, svcElection, ipFamily, 2, false)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
DescribeTable("only removes route if it was referenced by multiple services and all of them were deleted when common lease is used",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateVIP(e2e.IPv6Family, offset)
testServiceRT(svcName, lbAddress, fmt.Sprintf("kubevip-%s", svcName), dsNamespace, clusterName,
trafficPolicy, client, svcElection, ipFamily, 2, true)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
)
})
Describe("kube-vip DualStack services routing table mode functionality - IPv4 primary with services election", Ordered, func() {
var (
cpVIP string
clusterName string
client kubernetes.Interface
manifestValues *e2e.KubevipManifestValues
svcElection bool
ipFamily []corev1.IPFamily
tempDirPath string
nodesNumber = 1
)
BeforeAll(func() {
cpVIP = e2e.GenerateDualStackVIP(SOffset.Get())
networking := &kindconfigv1alpha4.Networking{
IPFamily: kindconfigv1alpha4.DualStackFamily,
}
manifestValues = &e2e.KubevipManifestValues{
ControlPlaneVIP: cpVIP,
ImagePath: imagePath,
ConfigPath: configPath,
ControlPlaneEnable: "false",
SvcEnable: "true",
SvcElectionEnable: "true",
EnableEndpointslices: "true",
}
var err error
svcElection, err = strconv.ParseBool(manifestValues.SvcElectionEnable)
Expect(err).ToNot(HaveOccurred())
ipFamily = []corev1.IPFamily{corev1.IPv4Protocol, corev1.IPv6Protocol}
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "rt-ds-svc-ipv4", k8sImagePath, v129,
kubeVIPRoutingTableManifestTemplate, logger, manifestValues, networking, nodesNumber, nil, 1)
})
AfterAll(func() {
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("configures an IPv4 and IPv6 routes for services",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateDualStackVIP(offset)
testServiceRT(svcName, lbAddress, fmt.Sprintf("kubevip-%s", svcName), dsNamespace, clusterName,
trafficPolicy, client, svcElection, ipFamily, 1, false)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
DescribeTable("only removes route if it was referenced by multiple services and all of them were deleted",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateDualStackVIP(offset)
testServiceRT(svcName, lbAddress, fmt.Sprintf("kubevip-%s", svcName), dsNamespace, clusterName,
trafficPolicy, client, svcElection, ipFamily, 2, false)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
DescribeTable("only removes route if it was referenced by multiple services and all of them were deleted when common lease is used",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateDualStackVIP(offset)
testServiceRT(svcName, lbAddress, fmt.Sprintf("kubevip-%s", svcName), dsNamespace, clusterName,
trafficPolicy, client, svcElection, ipFamily, 2, true)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
)
})
Describe("kube-vip DualStack services routing table mode functionality - IPv6 primary with services election", Ordered, func() {
var (
cpVIP string
clusterName string
client kubernetes.Interface
manifestValues *e2e.KubevipManifestValues
svcElection bool
ipFamily []corev1.IPFamily
tempDirPath string
nodesNumber = 1
)
BeforeAll(func() {
cpVIP = e2e.GenerateDualStackVIP(SOffset.Get())
networking := &kindconfigv1alpha4.Networking{
IPFamily: kindconfigv1alpha4.DualStackFamily,
PodSubnet: "fd00:10:244::/56,10.244.0.0/16",
ServiceSubnet: "fd00:10:96::/112,10.96.0.0/16",
}
manifestValues = &e2e.KubevipManifestValues{
ControlPlaneVIP: cpVIP,
ImagePath: imagePath,
ConfigPath: configPath,
ControlPlaneEnable: "false",
SvcEnable: "true",
SvcElectionEnable: "true",
EnableEndpointslices: "true",
}
var err error
svcElection, err = strconv.ParseBool(manifestValues.SvcElectionEnable)
Expect(err).ToNot(HaveOccurred())
ipFamily = []corev1.IPFamily{corev1.IPv4Protocol, corev1.IPv6Protocol}
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "rt-ds-svc-ipv6", k8sImagePath, v129,
kubeVIPRoutingTableManifestTemplate, logger, manifestValues, networking, nodesNumber, nil, 1)
})
AfterAll(func() {
By(fmt.Sprintf("saving logs to %q", tempDirPath))
err := e2e.GetLogs(context.Background(), client, tempDirPath)
Expect(err).ToNot(HaveOccurred())
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("configures an IPv4 and IPv6 routes for services",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateDualStackVIP(offset)
testServiceRT(svcName, lbAddress, fmt.Sprintf("kubevip-%s", svcName), dsNamespace, clusterName,
trafficPolicy, client, svcElection, ipFamily, 1, false)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
DescribeTable("only removes route if it was referenced by multiple services and all of them were deleted",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateDualStackVIP(offset)
testServiceRT(svcName, lbAddress, fmt.Sprintf("kubevip-%s", svcName), dsNamespace, clusterName,
trafficPolicy, client, svcElection, ipFamily, 2, false)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
Entry("with external traffic policy - local", "test-svc-local", SOffset.Get(), corev1.ServiceExternalTrafficPolicyLocal),
)
DescribeTable("only removes route if it was referenced by multiple services and all of them were deleted when common lease is used",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateDualStackVIP(offset)
testServiceRT(svcName, lbAddress, fmt.Sprintf("kubevip-%s", svcName), dsNamespace, clusterName,
trafficPolicy, client, svcElection, ipFamily, 2, true)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
)
})
}
})
func testServiceRT(svcName, lbAddress, leaseName, leaseNamespace, clusterName string, trafficPolicy corev1.ServiceExternalTrafficPolicy,
client kubernetes.Interface, serviceElection bool, ipFamily []corev1.IPFamily, numberOfServices int) {
client kubernetes.Interface, serviceElection bool, ipFamily []corev1.IPFamily, numberOfServices int, commonLease bool) {
lbAddresses := vip.Split(lbAddress)
services := []string{}
@@ -551,28 +960,51 @@ func testServiceRT(svcName, lbAddress, leaseName, leaseNamespace, clusterName st
}
for _, svc := range services {
svcLease := ""
if commonLease {
svcLease = leaseName
}
createTestService(svc, dsNamespace, dsName, lbAddress,
client, corev1.IPFamilyPolicyPreferDualStack, ipFamily, trafficPolicy)
client, corev1.IPFamilyPolicyPreferDualStack, ipFamily, trafficPolicy, svcLease, 1)
time.Sleep(time.Second)
}
var container string
if serviceElection {
container = e2e.GetLeaseHolder(leaseName, leaseNamespace, client)
for _, svc := range services {
lease := fmt.Sprintf("kubevip-%s", svc)
if commonLease {
lease = leaseName
}
By(withTimestamp(fmt.Sprintf("getting lease holder for lease '%s/%s'", leaseNamespace, lease)))
container = e2e.GetLeaseHolder(lease, leaseNamespace, client)
for _, addr := range lbAddresses {
By(withTimestamp(fmt.Sprintf("checking route presence for address %q on container %q", addr, container)))
e2e.CheckRoutePresence(addr, container, true)
}
}
} else {
container = fmt.Sprintf("%s-control-plane", clusterName)
}
for _, addr := range lbAddresses {
e2e.CheckRoutePresence(addr, container, true)
for _, addr := range lbAddresses {
By(withTimestamp(fmt.Sprintf("checking route presence for address %q on container %q", addr, container)))
e2e.CheckRoutePresence(addr, container, true)
}
}
for i := range numberOfServices {
By(withTimestamp(fmt.Sprintf("deleting service %s/%s\n", dsNamespace, services[i])))
expected := i < numberOfServices-1
err := client.CoreV1().Services(dsNamespace).Delete(context.TODO(), services[i], metav1.DeleteOptions{})
Expect(err).ToNot(HaveOccurred())
time.Sleep(time.Second)
for _, addr := range lbAddresses {
By(withTimestamp(fmt.Sprintf("checking route presence for address %q on container %q - expected: %t", addr, container, expected)))
e2e.CheckRoutePresence(addr, container, expected)
if commonLease {
By(withTimestamp(fmt.Sprintf("getting lease holder for lease '%s/%s' - expected: %t", leaseNamespace, leaseName, expected)))
e2e.CheckLeasePresence(leaseName, leaseNamespace, client, expected)
}
}
}
}

View File

@@ -12,6 +12,7 @@ import (
"os"
"os/exec"
"path/filepath"
"strconv"
"strings"
"sync"
"text/template"
@@ -21,6 +22,7 @@ import (
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/rest"
"k8s.io/client-go/tools/clientcmd"
"k8s.io/client-go/tools/clientcmd/api"
"k8s.io/klog/v2"
@@ -49,7 +51,7 @@ const testJSON = `
path: "/apiServer/certSANs/-"
value: `
var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
var _ = Describe("kube-vip ARP/NDP broadcast neighbor", Ordered, func() {
if Mode == ModeARP {
var (
logger log.Logger
@@ -58,8 +60,8 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
configPath string
kubeVIPManifestTemplate *template.Template
kubeVIPHostnameManifestTemplate *template.Template
tempDirPath string
v129 bool
tempDirPathRoot string
)
BeforeEach(func() {
@@ -83,15 +85,27 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
hostnameTemplatePath := filepath.Join(curDir, "kube-vip-hostname.yaml.tmpl")
kubeVIPHostnameManifestTemplate, err = template.New("kube-vip-hostname.yaml.tmpl").ParseFiles(hostnameTemplatePath)
Expect(err).NotTo(HaveOccurred())
})
tempDirPath, err = os.MkdirTemp("", "kube-vip-test")
BeforeAll(func() {
var err error
tempDirPathRoot, err = os.MkdirTemp("", "kube-vip-test-arp")
Expect(err).NotTo(HaveOccurred())
})
AfterAll(func() {
if os.Getenv("E2E_KEEP_LOGS") != "true" {
Expect(os.RemoveAll(tempDirPathRoot)).To(Succeed())
}
})
Describe("kube-vip IPv4 functionality, vip_leaderelection=true", Ordered, func() {
var (
cpVIP string
client kubernetes.Interface
clusterName string
tempDirPath string
)
BeforeAll(func() {
@@ -108,17 +122,26 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
SvcEnable: "false",
SvcElectionEnable: "false",
EnableEndpointslices: "false",
EnableNodeLabeling: "false",
}
clusterName, _ = prepareCluster(tempDirPath, "ipv4", k8sImagePath, v129, kubeVIPManifestTemplate, logger, manifestValues, networking, 3, nil)
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "ipv4", k8sImagePath, v129,
kubeVIPManifestTemplate, logger, manifestValues, networking, 3, nil, 1)
})
AfterAll(func() {
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
Eventually(func() error {
return e2e.GetLogs(context.Background(), client, tempDirPath)
}, "60s", "5s").Should(Succeed())
cleanupCluster(clusterName, ConfigMtx, logger)
})
It("provides an IPv4 VIP address for the Kubernetes control plane nodes", func() {
testControlPlaneVIPs([]string{cpVIP}, clusterName)
It(clusterName+" provides an IPv4 VIP address for the Kubernetes control plane nodes", func() {
testControlPlaneVIPs([]string{cpVIP}, clusterName, client)
})
})
@@ -127,6 +150,7 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
cpVIP string
client kubernetes.Interface
clusterName string
tempDirPath string
)
BeforeAll(func() {
@@ -143,13 +167,22 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
SvcEnable: "true",
SvcElectionEnable: "false",
EnableEndpointslices: "false",
EnableNodeLabeling: "false",
}
clusterName, client = prepareCluster(tempDirPath, "svc-ipv4", k8sImagePath, v129, kubeVIPManifestTemplate, logger, manifestValues, networking, 1, nil)
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "svc-ipv4", k8sImagePath, v129,
kubeVIPManifestTemplate, logger, manifestValues, networking, 1, nil, 1)
})
AfterAll(func() {
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
Eventually(func() error {
return e2e.GetLogs(context.Background(), client, tempDirPath)
}, "60s", "5s").Should(Succeed())
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("configures an IPv4 VIP address for service",
@@ -176,6 +209,7 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
cpVIP string
client kubernetes.Interface
clusterName string
tempDirPath string
)
BeforeAll(func() {
@@ -192,13 +226,22 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
SvcEnable: "true",
SvcElectionEnable: "true",
EnableEndpointslices: "false",
EnableNodeLabeling: "false",
}
clusterName, client = prepareCluster(tempDirPath, "svc-el-ipv4", k8sImagePath, v129, kubeVIPManifestTemplate, logger, manifestValues, networking, 1, nil)
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "svc-el-ipv4", k8sImagePath, v129,
kubeVIPManifestTemplate, logger, manifestValues, networking, 1, nil, 1)
})
AfterAll(func() {
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
Eventually(func() error {
return e2e.GetLogs(context.Background(), client, tempDirPath)
}, "60s", "5s").Should(Succeed())
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("configures an IPv4 VIP address for service",
@@ -214,7 +257,10 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
Describe("kube-vip IPv6 functionality, vip_leaderelection=true", Ordered, func() {
var (
cpVIP string
client kubernetes.Interface
clusterName string
clusterCfg *rest.Config
tempDirPath string
)
BeforeAll(func() {
@@ -231,17 +277,28 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
SvcEnable: "false",
SvcElectionEnable: "false",
EnableEndpointslices: "false",
EnableNodeLabeling: "false",
}
clusterName, _ = prepareCluster(tempDirPath, "ipv6", k8sImagePath, v129, kubeVIPManifestTemplate, logger, manifestValues, networking, 3, nil)
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, clusterCfg = prepareCluster(tempDirPath, "ipv6", k8sImagePath, v129,
kubeVIPManifestTemplate, logger, manifestValues, networking, 3, nil, 1)
})
AfterAll(func() {
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
c, err := kubernetes.NewForConfig(clusterCfg)
Expect(err).ToNot(HaveOccurred())
Eventually(func() error {
return e2e.GetLogs(context.Background(), c, tempDirPath)
}, "60s", "5s").Should(Succeed())
cleanupCluster(clusterName, ConfigMtx, logger)
})
It("provides an IPv6 VIP address for the Kubernetes control plane nodes", func() {
testControlPlaneVIPs([]string{cpVIP}, clusterName)
It(clusterName+"provides an IPv6 VIP address for the Kubernetes control plane nodes", func() {
testControlPlaneVIPs([]string{cpVIP}, clusterName, client)
})
})
@@ -250,6 +307,7 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
cpVIP string
client kubernetes.Interface
clusterName string
tempDirPath string
)
BeforeAll(func() {
@@ -266,13 +324,22 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
SvcEnable: "true",
SvcElectionEnable: "false",
EnableEndpointslices: "false",
EnableNodeLabeling: "false",
}
clusterName, client = prepareCluster(tempDirPath, "svc-ipv6", k8sImagePath, v129, kubeVIPManifestTemplate, logger, manifestValues, networking, 1, nil)
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "svc-ipv6", k8sImagePath, v129,
kubeVIPManifestTemplate, logger, manifestValues, networking, 1, nil, 1)
})
AfterAll(func() {
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
Eventually(func() error {
return e2e.GetLogs(context.Background(), client, tempDirPath)
}, "60s", "5s").Should(Succeed())
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("configures an IPv6 VIP address for service",
@@ -299,6 +366,7 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
cpVIP string
client kubernetes.Interface
clusterName string
tempDirPath string
)
BeforeAll(func() {
@@ -315,13 +383,22 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
SvcEnable: "true",
SvcElectionEnable: "true",
EnableEndpointslices: "false",
EnableNodeLabeling: "false",
}
clusterName, client = prepareCluster(tempDirPath, "svc-el-ipv6", k8sImagePath, v129, kubeVIPManifestTemplate, logger, manifestValues, networking, 1, nil)
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "svc-el-ipv6", k8sImagePath, v129,
kubeVIPManifestTemplate, logger, manifestValues, networking, 1, nil, 1)
})
AfterAll(func() {
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
Eventually(func() error {
return e2e.GetLogs(context.Background(), client, tempDirPath)
}, "60s", "5s").Should(Succeed())
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("configures an IPv6 VIP address for service",
@@ -337,7 +414,9 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
Describe("kube-vip DualStack functionality - IPv4 primary, vip_leaderelection=true", Ordered, func() {
var (
cpVIP string
client kubernetes.Interface
clusterName string
tempDirPath string
)
BeforeAll(func() {
@@ -354,25 +433,35 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
SvcEnable: "false",
SvcElectionEnable: "false",
EnableEndpointslices: "true",
EnableNodeLabeling: "false",
}
clusterName, _ = prepareCluster(tempDirPath, "ds-ipv4", k8sImagePath, v129, kubeVIPManifestTemplate, logger, manifestValues, networking, 3, nil)
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "ds-ipv4", k8sImagePath, v129,
kubeVIPManifestTemplate, logger, manifestValues, networking, 3, nil, 1)
})
AfterAll(func() {
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
Eventually(func() error {
return e2e.GetLogs(context.Background(), client, tempDirPath)
}, "60s", "5s").Should(Succeed())
cleanupCluster(clusterName, ConfigMtx, logger)
})
It("provides a DualStack VIP addresses for the Kubernetes control plane nodes", func() {
testControlPlaneVIPs(vip.Split(cpVIP), clusterName)
It(clusterName+" provides a DualStack VIP addresses for the Kubernetes control plane nodes", func() {
testControlPlaneVIPs(vip.Split(cpVIP), clusterName, client)
})
})
Describe("kube-vip DualStack functionality - IPv4 primary, svc_enable=true, svc_election=false", Ordered, func() {
var (
cpVIP string
clusterName string
client kubernetes.Interface
clusterName string
tempDirPath string
)
BeforeAll(func() {
@@ -389,13 +478,23 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
SvcEnable: "true",
SvcElectionEnable: "false",
EnableEndpointslices: "true",
EnableNodeLabeling: "false",
}
clusterName, client = prepareCluster(tempDirPath, "ds-svc-ipv4", k8sImagePath, v129, kubeVIPManifestTemplate, logger, manifestValues, networking, 1, nil)
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "ds-svc-ipv4", k8sImagePath, v129,
kubeVIPManifestTemplate, logger, manifestValues, networking, 1, nil, 1)
})
AfterAll(func() {
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
By(fmt.Sprintf("saving logs to %q", tempDirPath))
Eventually(func() error {
return e2e.GetLogs(context.Background(), client, tempDirPath)
}, "60s", "5s").Should(Succeed())
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("configures an IPv4 and IPv6 VIP addresses for service",
@@ -420,8 +519,9 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
Describe("kube-vip DualStack functionality - IPv4 primary, svc_enable=true, svc_election=true", Ordered, func() {
var (
cpVIP string
clusterName string
client kubernetes.Interface
clusterName string
tempDirPath string
)
BeforeAll(func() {
@@ -438,13 +538,23 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
SvcEnable: "true",
SvcElectionEnable: "true",
EnableEndpointslices: "true",
EnableNodeLabeling: "false",
}
clusterName, client = prepareCluster(tempDirPath, "ds-svc-el-ipv4", k8sImagePath, v129, kubeVIPManifestTemplate, logger, manifestValues, networking, 1, nil)
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "ds-svc-el-ipv4", k8sImagePath, v129,
kubeVIPManifestTemplate, logger, manifestValues, networking, 1, nil, 1)
})
AfterAll(func() {
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
By(fmt.Sprintf("saving logs to %q", tempDirPath))
Eventually(func() error {
return e2e.GetLogs(context.Background(), client, tempDirPath)
}, "60s", "5s").Should(Succeed())
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("configures an IPv4 and IPv6 VIP addresses for service",
@@ -460,7 +570,9 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
Describe("kube-vip DualStack functionality - IPv6 primary, vip_leaderelection=true", Ordered, func() {
var (
cpVIP string
client kubernetes.Interface
clusterName string
tempDirPath string
)
BeforeAll(func() {
@@ -479,25 +591,36 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
SvcEnable: "true",
SvcElectionEnable: "false",
EnableEndpointslices: "true",
EnableNodeLabeling: "false",
}
clusterName, _ = prepareCluster(tempDirPath, "ds-ipv6", k8sImagePath, v129, kubeVIPManifestTemplate, logger, manifestValues, networking, 3, nil)
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "ds-ipv6", k8sImagePath, v129,
kubeVIPManifestTemplate, logger, manifestValues, networking, 3, nil, 1)
})
AfterAll(func() {
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
By(fmt.Sprintf("saving logs to %q", tempDirPath))
Eventually(func() error {
return e2e.GetLogs(context.Background(), client, tempDirPath)
}, "60s", "5s").Should(Succeed())
cleanupCluster(clusterName, ConfigMtx, logger)
})
It("provides a DualStack VIP addresses for the Kubernetes control plane nodes", func() {
testControlPlaneVIPs(vip.Split(cpVIP), clusterName)
It(clusterName+" provides a DualStack VIP addresses for the Kubernetes control plane nodes", func() {
testControlPlaneVIPs(vip.Split(cpVIP), clusterName, client)
})
})
Describe("kube-vip DualStack functionality - IPv6 primary, svc_enable=true, svc_election=false", Ordered, func() {
var (
cpVIP string
clusterName string
client kubernetes.Interface
clusterName string
tempDirPath string
)
BeforeAll(func() {
@@ -516,13 +639,29 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
SvcEnable: "true",
SvcElectionEnable: "false",
EnableEndpointslices: "true",
EnableNodeLabeling: "false",
}
clusterName, client = prepareCluster(tempDirPath, "ds-svc-ipv6", k8sImagePath, v129, kubeVIPManifestTemplate, logger, manifestValues, networking, 1, nil)
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "ds-svc-ipv6", k8sImagePath, v129,
kubeVIPManifestTemplate, logger, manifestValues, networking, 1, nil, 1)
})
AfterAll(func() {
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
By(fmt.Sprintf("saving logs to %q", tempDirPath))
Eventually(func() error {
return e2e.GetLogs(context.Background(), client, tempDirPath)
}, "60s", "5s").Should(Succeed())
cleanupCluster(clusterName, ConfigMtx, logger)
})
AfterEach(func() {
Eventually(func() error {
return e2e.GetLogs(context.Background(), client, tempDirPath)
}, "60s", "5s").Should(Succeed())
})
DescribeTable("configures an IPv4 and IPv6 VIP addresses for service",
@@ -547,8 +686,9 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
Describe("kube-vip DualStack functionality - IPv6 primary, svc_enable=true, svc_election=true", Ordered, func() {
var (
cpVIP string
clusterName string
client kubernetes.Interface
clusterName string
tempDirPath string
)
BeforeAll(func() {
@@ -567,13 +707,23 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
SvcEnable: "true",
SvcElectionEnable: "true",
EnableEndpointslices: "true",
EnableNodeLabeling: "false",
}
clusterName, client = prepareCluster(tempDirPath, "ds-svc-el-ipv6", k8sImagePath, v129, kubeVIPManifestTemplate, logger, manifestValues, networking, 1, nil)
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "ds-svc-el-ipv6", k8sImagePath, v129,
kubeVIPManifestTemplate, logger, manifestValues, networking, 1, nil, 1)
})
AfterAll(func() {
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
By(fmt.Sprintf("saving logs to %q", tempDirPath))
Eventually(func() error {
return e2e.GetLogs(context.Background(), client, tempDirPath)
}, "60s", "5s").Should(Succeed())
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("configures an IPv6 VIP address for service",
@@ -589,7 +739,9 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
Describe("kube-vip IPv4 functionality with legacy hostname", Ordered, func() {
var (
cpVIP string
client kubernetes.Interface
clusterName string
tempDirPath string
)
BeforeAll(func() {
@@ -606,23 +758,139 @@ var _ = Describe("kube-vip ARP/NDP broadcast neighbor", func() {
SvcEnable: "false",
SvcElectionEnable: "false",
EnableEndpointslices: "false",
EnableNodeLabeling: "false",
}
clusterName, _ = prepareCluster(tempDirPath, "ipv4-hostname", k8sImagePath, v129, kubeVIPHostnameManifestTemplate, logger, manifestValues, networking, 3, nil)
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "ipv4-hostname", k8sImagePath, v129,
kubeVIPHostnameManifestTemplate, logger, manifestValues, networking, 3, nil, 1)
})
AfterAll(func() {
cleanupCluster(clusterName, tempDirPath, ConfigMtx, logger)
By(fmt.Sprintf("saving logs to %q", tempDirPath))
Eventually(func() error {
return e2e.GetLogs(context.Background(), client, tempDirPath)
}, "60s", "5s").Should(Succeed())
cleanupCluster(clusterName, ConfigMtx, logger)
})
It("uses hostname fallback while providing an IPv4 VIP address for the Kubernetes control plane nodes", func() {
testControlPlaneVIPs([]string{cpVIP}, clusterName)
It(clusterName+" uses hostname fallback while providing an IPv4 VIP address for the Kubernetes control plane nodes", func() {
testControlPlaneVIPs([]string{cpVIP}, clusterName, client)
})
})
Describe("kube-vip IPv6 functionality with manually specified lease name, svc_enable=true, svc_election=true", Ordered, func() {
var (
cpVIP string
client kubernetes.Interface
clusterName string
tempDirPath string
)
BeforeAll(func() {
cpVIP = e2e.GenerateVIP(e2e.IPv6Family, SOffset.Get())
networking := &kindconfigv1alpha4.Networking{
IPFamily: kindconfigv1alpha4.IPv6Family,
}
manifestValues := &e2e.KubevipManifestValues{
ControlPlaneVIP: cpVIP,
ImagePath: imagePath,
ConfigPath: configPath,
SvcEnable: "true",
SvcElectionEnable: "true",
EnableEndpointslices: "false",
EnableNodeLabeling: "false",
}
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "svc-el-m-ipv6", k8sImagePath, v129,
kubeVIPManifestTemplate, logger, manifestValues, networking, 2, nil, 3)
})
AfterAll(func() {
By(fmt.Sprintf("saving logs to %q", tempDirPath))
Eventually(func() error {
return e2e.GetLogs(context.Background(), client, tempDirPath)
}, "60s", "5s").Should(Succeed())
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("configures an single IPv6 VIP address for multiple services",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateVIP(e2e.IPv6Family, offset)
testServiceCommonLease(svcName, lbAddress, dsNamespace,
trafficPolicy,
client, []corev1.IPFamily{corev1.IPv6Protocol}, 3)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
)
})
Describe("kube-vip DualStack functionality with manually specified lease name, svc_enable=true, svc_election=true", Ordered, func() {
var (
cpVIP string
client kubernetes.Interface
clusterName string
tempDirPath string
)
BeforeAll(func() {
cpVIP = e2e.GenerateDualStackVIP(SOffset.Get())
networking := &kindconfigv1alpha4.Networking{
IPFamily: kindconfigv1alpha4.DualStackFamily,
PodSubnet: "fd00:10:244::/56,10.244.0.0/16",
ServiceSubnet: "fd00:10:96::/112,10.96.0.0/16",
}
manifestValues := &e2e.KubevipManifestValues{
ControlPlaneVIP: cpVIP,
ImagePath: imagePath,
ConfigPath: configPath,
SvcEnable: "true",
SvcElectionEnable: "true",
EnableEndpointslices: "false",
EnableNodeLabeling: "false",
}
var err error
tempDirPath, err = os.MkdirTemp(tempDirPathRoot, "kube-vip-test")
Expect(err).NotTo(HaveOccurred())
clusterName, client, _ = prepareCluster(tempDirPath, "svc-el-m-ds", k8sImagePath, v129,
kubeVIPManifestTemplate, logger, manifestValues, networking, 2, nil, 3)
})
AfterAll(func() {
By(fmt.Sprintf("saving logs to %q", tempDirPath))
Eventually(func() error {
return e2e.GetLogs(context.Background(), client, tempDirPath)
}, "60s", "5s").Should(Succeed())
cleanupCluster(clusterName, ConfigMtx, logger)
})
DescribeTable("configures an single IPv4 and IPv6 VIP address for multiple services",
func(svcName string, offset uint, trafficPolicy corev1.ServiceExternalTrafficPolicy) {
lbAddress := e2e.GenerateDualStackVIP(offset)
testServiceCommonLease(svcName, lbAddress, dsNamespace,
trafficPolicy,
client, []corev1.IPFamily{corev1.IPv6Protocol, corev1.IPv4Protocol}, 3)
},
Entry("with external traffic policy - cluster", "test-svc-cluster", SOffset.Get(), corev1.ServiceExternalTrafficPolicyCluster),
)
})
}
})
func createKindCluster(logger log.Logger, config *v1alpha4.Cluster, clusterName string) kubernetes.Interface {
func createKindCluster(logger log.Logger, config *v1alpha4.Cluster, clusterName string) (kubernetes.Interface, *rest.Config) {
provider := cluster.NewProvider(
cluster.ProviderWithLogger(logger),
cluster.ProviderWithDocker(),
@@ -649,7 +917,7 @@ func createKindCluster(logger log.Logger, config *v1alpha4.Cluster, clusterName
client, err := kubernetes.NewForConfig(cfg)
Expect(err).ToNot(HaveOccurred())
return client
return client, cfg
}
// Assume the VIP is routable if status code is 200 or 500. Since etcd might glitch.
@@ -663,6 +931,8 @@ func assertConnection(protocol, ip, port, suffix string, transportTimeout, event
ip = fmt.Sprintf("[%s]", ip)
}
By(withTimestamp(fmt.Sprintf("checking connection to %s://%s:%s/%s", protocol, ip, port, suffix)))
transport := &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true}, // nolint
}
@@ -675,15 +945,19 @@ func assertConnection(protocol, ip, port, suffix string, transportTimeout, event
}
defer resp.Body.Close()
return resp.StatusCode
}, eventuallyTimeout).Should(BeElementOf([]int{http.StatusOK, http.StatusInternalServerError}), fmt.Sprintf("Failed to connect to %s", ip))
}, eventuallyTimeout, transportTimeout).Should(BeElementOf([]int{http.StatusOK, http.StatusInternalServerError}), fmt.Sprintf("Failed to connect to %s", ip))
By(withTimestamp(fmt.Sprintf("estabilished connection to %s://%s:%s/%s", protocol, ip, port, suffix)))
}
// Assume connection to the provided address is possible
func assertConnectionError(protocol, ip, port, suffix string, transportTimeout time.Duration) {
func assertConnectionError(protocol, ip, port, suffix string, transportTimeout time.Duration, eventuallyTimeout time.Duration) {
if strings.Contains(ip, ":") {
ip = fmt.Sprintf("[%s]", ip)
}
By(withTimestamp(fmt.Sprintf("checking connection error to %s://%s:%s/%s", protocol, ip, port, suffix)))
transport := &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true}, // nolint
}
@@ -692,16 +966,12 @@ func assertConnectionError(protocol, ip, port, suffix string, transportTimeout t
Eventually(func() error {
_, err := client.Get(fmt.Sprintf("%s://%s:%s/%s", protocol, ip, port, suffix))
return err
}, time.Second*30).Should(HaveOccurred())
}, eventuallyTimeout, transportTimeout).Should(HaveOccurred())
By(withTimestamp(fmt.Sprintf("connection %s://%s:%s/%s error as expected", protocol, ip, port, suffix)))
}
func killLeader(leaderIPAddr string, clusterName string) {
var leaderName string
Eventually(func() string {
leaderName = findLeader(leaderIPAddr, clusterName)
return leaderName
}, "600s").ShouldNot(BeEmpty())
func killLeader(leaderName string) {
cmd := exec.Command(
"docker", "kill", leaderName,
)
@@ -738,7 +1008,7 @@ func withTimestamp(text string) string {
return fmt.Sprintf("%s: %s", time.Now(), text)
}
func createTestDS(name, namespace string, client kubernetes.Interface) {
func createTestDS(name, namespace string, client kubernetes.Interface, port int) {
labels := make(map[string]string)
labels["app"] = name
d := v1.DaemonSet{
@@ -774,13 +1044,13 @@ func createTestDS(name, namespace string, client kubernetes.Interface) {
Image: "ghcr.io/traefik/whoami:v1.11",
Ports: []corev1.ContainerPort{
{
ContainerPort: 80,
ContainerPort: int32(port),
},
},
Env: []corev1.EnvVar{
{
Name: "PORT",
Value: "80",
Name: "WHOAMI_PORT_NUMBER",
Value: strconv.Itoa(port),
},
},
},
@@ -794,13 +1064,19 @@ func createTestDS(name, namespace string, client kubernetes.Interface) {
Expect(err).ToNot(HaveOccurred())
}
func createTestService(name, namespace, target, lbAddress string, client kubernetes.Interface, ipfPolicy corev1.IPFamilyPolicy, ipFamiles []corev1.IPFamily, externalPolicy corev1.ServiceExternalTrafficPolicy) {
func createTestService(name, namespace, target, lbAddress string, client kubernetes.Interface, ipfPolicy corev1.IPFamilyPolicy,
ipFamiles []corev1.IPFamily, externalPolicy corev1.ServiceExternalTrafficPolicy, leaseName string, port int) {
svcAnnotations := make(map[string]string)
svcAnnotations[kubevip.LoadbalancerIPAnnotation] = lbAddress
if leaseName != "" {
svcAnnotations[kubevip.ServiceLease] = leaseName
}
labels := make(map[string]string)
labels["app"] = target
By(withTimestamp(fmt.Sprintf("creating service %s/%s", namespace, name)))
s := corev1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: name,
@@ -816,7 +1092,7 @@ func createTestService(name, namespace, target, lbAddress string, client kuberne
Ports: []corev1.ServicePort{
{
Protocol: corev1.ProtocolTCP,
Port: 80,
Port: int32(port),
},
},
Selector: labels,
@@ -827,20 +1103,55 @@ func createTestService(name, namespace, target, lbAddress string, client kuberne
_, err := client.CoreV1().Services(namespace).Create(context.TODO(), &s, metav1.CreateOptions{})
return err
}, time.Second*60, time.Second).Should(Succeed())
By(withTimestamp(fmt.Sprintf("service %s/%s created", namespace, name)))
Eventually(func() error {
By(withTimestamp(fmt.Sprintf("getting service %s/%s\n", namespace, name)))
_, err := client.CoreV1().Services(namespace).Get(context.Background(), name, metav1.GetOptions{})
return err
}, time.Second*60, time.Second).Should(Succeed())
}
func checkIPAddress(lbAddress, container string, expected bool) {
Eventually(e2e.CheckIPAddressPresence(lbAddress, container, expected), "120s", "200ms").Should(BeTrue())
func checkIPAddress(lbAddress, container string, expected bool) bool {
By(withTimestamp(fmt.Sprintf("checking LB %q, should exist: %t", lbAddress, expected)))
ticker := time.NewTicker(time.Second)
defer ticker.Stop()
to := time.NewTimer(time.Second * 180)
defer to.Stop()
for {
select {
case <-to.C:
return false
case <-ticker.C:
if e2e.CheckIPAddressPresence(lbAddress, container, expected) {
return true
}
}
}
}
func checkIPAddressByLease(name, namespace, lbAddress string, expected bool, client kubernetes.Interface) {
Eventually(e2e.CheckIPAddressPresenceByLease(name, namespace, lbAddress, client, expected), "120s", "200ms").Should(BeTrue())
func checkIPAddressByLease(name, namespace, lbAddress string, expected bool, client kubernetes.Interface) bool {
By(withTimestamp(fmt.Sprintf("checking LB %q by lease %s/%s, should exist: %t", lbAddress, namespace, name, expected)))
ticker := time.NewTicker(time.Second)
defer ticker.Stop()
to := time.NewTimer(time.Second * 360)
defer to.Stop()
for {
select {
case <-to.C:
return false
case <-ticker.C:
if e2e.CheckIPAddressPresenceByLease(name, namespace, lbAddress, client, expected) {
return true
}
}
}
}
func prepareCluster(tempDirPath, clusterNameSuffix, k8sImagePath string,
v129 bool, kubeVIPManifestTemplate *template.Template, logger log.Logger,
manifestValues *e2e.KubevipManifestValues, networking *kindconfigv1alpha4.Networking, nodesNum int,
addSAN *san) (string, kubernetes.Interface) {
addSAN *san, dsNumber int) (string, kubernetes.Interface, *rest.Config) {
manifestPath := filepath.Join(tempDirPath, fmt.Sprintf("kube-vip-%s.yaml", clusterNameSuffix))
@@ -911,18 +1222,24 @@ func prepareCluster(tempDirPath, clusterNameSuffix, k8sImagePath string,
clusterName := fmt.Sprintf("%s-%s", filepath.Base(tempDirPath), clusterNameSuffix)
By(withTimestamp("creating a kind cluster with multiple control plane nodes"))
client := createKindCluster(logger, &clusterConfig, clusterName)
client, cfg := createKindCluster(logger, &clusterConfig, clusterName)
By(withTimestamp("creating test daemonset"))
createTestDS(dsName, dsNamespace, client)
for i := range dsNumber {
tmpDsName := dsName
if i > 0 {
tmpDsName = fmt.Sprintf("%s-%d", dsName, i)
}
createTestDS(tmpDsName, dsNamespace, client, 80+i)
}
By(withTimestamp("loading local docker image to kind cluster"))
e2e.LoadDockerImageToKind(logger, manifestValues.ImagePath, clusterName)
return clusterName, client
return clusterName, client, cfg
}
func cleanupCluster(clusterName, tempDirPath string, configMtx *sync.Mutex, logger log.Logger) {
func cleanupCluster(clusterName string, configMtx *sync.Mutex, logger log.Logger) {
if os.Getenv("E2E_PRESERVE_CLUSTER") == "true" {
return
}
@@ -939,11 +1256,9 @@ func cleanupCluster(clusterName, tempDirPath string, configMtx *sync.Mutex, logg
defer configMtx.Unlock()
return provider.Delete(clusterName, "")
}, "60s", "200ms").Should(Succeed())
Expect(os.RemoveAll(tempDirPath)).To(Succeed())
}
func testControlPlaneVIPs(cpVIPs []string, clusterName string) {
func testControlPlaneVIPs(cpVIPs []string, clusterName string, client kubernetes.Interface) {
Expect(cpVIPs).ToNot(BeEmpty())
By(withTimestamp("checking that the Kubernetes control plane nodes are accessible via the assigned VIP"))
@@ -954,8 +1269,16 @@ func testControlPlaneVIPs(cpVIPs []string, clusterName string) {
assertControlPlaneIsRoutable(cpVIP, time.Duration(0), 20*time.Second)
}
var leaderName string
Eventually(func() string {
leaderName = findLeader(cpVIPs[0], clusterName)
return leaderName
}, "600s").ShouldNot(BeEmpty())
Eventually(client.CoreV1().Nodes().Delete(context.Background(), leaderName, metav1.DeleteOptions{}), "60s", "1s").Should(Succeed())
By(withTimestamp("killing the leader Kubernetes control plane node to trigger a fail-over scenario"))
killLeader(cpVIPs[0], clusterName)
killLeader(leaderName)
By(withTimestamp("checking that the Kubernetes control plane nodes are still accessible via the assigned VIP with little downtime"))
for _, cpVIP := range cpVIPs {
@@ -983,11 +1306,12 @@ func testService(svcName, lbAddress, leaseName, leaseNamespace string, trafficPo
for _, svc := range services {
createTestService(svc, dsNamespace, dsName, lbAddress,
client, corev1.IPFamilyPolicyPreferDualStack, ipFamily, trafficPolicy)
client, corev1.IPFamilyPolicyPreferDualStack, ipFamily, trafficPolicy, "", 80)
time.Sleep(time.Second)
}
for _, addr := range lbAddresses {
checkIPAddressByLease(leases[0], leaseNamespace, addr, true, client)
Expect(checkIPAddressByLease(leases[0], leaseNamespace, addr, true, client)).To(BeTrue())
assertConnection("http", addr, "80", "", 3*time.Second, 60*time.Second)
}
@@ -998,18 +1322,83 @@ func testService(svcName, lbAddress, leaseName, leaseNamespace string, trafficPo
err := client.CoreV1().Services(dsNamespace).Delete(context.TODO(), services[i], metav1.DeleteOptions{})
Expect(err).ToNot(HaveOccurred())
time.Sleep(time.Second)
for _, addr := range lbAddresses {
if serviceElection {
checkIPAddress(addr, container, expected)
Expect(checkIPAddress(addr, container, expected)).To(BeTrue())
} else {
checkIPAddressByLease(leases[i], leaseNamespace, addr, expected, client)
Expect(checkIPAddressByLease(leases[i], leaseNamespace, addr, expected, client)).To(BeTrue())
}
if expected {
assertConnection("http", addr, "80", "", 3*time.Second, 60*time.Second)
} else {
assertConnectionError("http", addr, "80", "", 3*time.Second)
assertConnectionError("http", addr, "80", "", 3*time.Second, 30*time.Second)
}
}
}
}
func testServiceCommonLease(svcName, lbAddress, leaseNamespace string, trafficPolicy corev1.ServiceExternalTrafficPolicy,
client kubernetes.Interface, ipFamily []corev1.IPFamily, numberOfServices int) {
lbAddresses := vip.Split(lbAddress)
services := []string{}
for i := range numberOfServices {
services = append(services, fmt.Sprintf("%s-%d", svcName, i))
}
lease := "common-lease"
for i, svc := range services {
tmpDsName := dsName
if i > 0 {
tmpDsName = fmt.Sprintf("%s-%d", tmpDsName, i)
}
createTestService(svc, dsNamespace, tmpDsName, lbAddress,
client, corev1.IPFamilyPolicyPreferDualStack, ipFamily, trafficPolicy, lease, 80+i)
time.Sleep(time.Second)
}
for i, addr := range lbAddresses {
checkIPAddressByLease(lease, leaseNamespace, addr, true, client)
assertConnection("http", addr, strconv.Itoa(80+i), "", 3*time.Second, 60*time.Second)
}
container := e2e.GetLeaseHolder(lease, leaseNamespace, client)
nodes, err := client.CoreV1().Nodes().List(context.Background(), metav1.ListOptions{})
Expect(err).ToNot(HaveOccurred())
for _, node := range nodes.Items {
expected := node.Name == container
for _, addr := range lbAddresses {
checkIPAddress(addr, node.Name, expected)
}
}
for i := range numberOfServices {
expected := i < numberOfServices-1
By(fmt.Sprintf("deleting service %q", services[i]))
err := client.CoreV1().Services(dsNamespace).Delete(context.TODO(), services[i], metav1.DeleteOptions{})
Expect(err).ToNot(HaveOccurred())
time.Sleep(time.Second)
Eventually(func() error {
_, err := client.CoreV1().Services(dsNamespace).Get(context.Background(), services[i], metav1.GetOptions{})
return err
}).ShouldNot(Succeed())
for _, addr := range lbAddresses {
for _, node := range nodes.Items {
if node.Name == container {
checkIPAddress(addr, node.Name, expected)
} else {
checkIPAddress(addr, node.Name, false)
}
}
}
}

View File

@@ -16,7 +16,7 @@ import (
"sync"
"time"
"github.com/kube-vip/kube-vip/pkg/vip"
"github.com/kube-vip/kube-vip/pkg/utils"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/onsi/gomega/gexec"
@@ -141,7 +141,7 @@ func getKindNetworkSubnetCIDRs() []string {
func CheckIPAddressPresence(ip string, container string, expected bool) bool {
family := "-4"
if vip.IsIPv6(ip) {
if utils.IsIPv6(ip) {
family = "-6"
}
@@ -154,14 +154,18 @@ func CheckIPAddressPresence(ip string, container string, expected bool) bool {
cmd.Stdout = cmdOut
cmd.Stderr = cmdErr
if err := cmd.Run(); err != nil {
By(fmt.Sprintf("contianer %q interfaces error: %s", container, cmdErr.String()))
return false
}
By(fmt.Sprintf("contianer %q interfaces:\n %s", container, cmdOut.String()))
return strings.Contains(cmdOut.String(), ip) == expected
}
func CheckIPAddressPresenceByLease(name, namespace, ip string, client kubernetes.Interface, expected bool) bool {
container := GetLeaseHolder(name, namespace, client)
By("Lease holder: " + container)
if container == "" {
return false
}
@@ -170,7 +174,7 @@ func CheckIPAddressPresenceByLease(name, namespace, ip string, client kubernetes
func CheckRoutePresence(ip string, container string, expected bool) bool {
family := "-4"
if vip.IsIPv6(ip) {
if utils.IsIPv6(ip) {
family = "-6"
}
@@ -187,23 +191,48 @@ func CheckRoutePresence(ip string, container string, expected bool) bool {
cmd.Stderr = cmdErr
cmd.Run()
By("Routes: " + cmdOut.String())
result = strings.Contains(cmdOut.String(), ip) == expected
return result
}, "60s", "1s").Should(BeTrue())
}, "120s", "1s").Should(BeTrue())
return result
}
func GetLeaseHolder(name, namespace string, client kubernetes.Interface) string {
func CheckLeasePresence(name, namespace string, client kubernetes.Interface, expected bool) *coordinationv1.Lease {
var lease *coordinationv1.Lease
Eventually(func() error {
var err error
lease, err = client.CoordinationV1().Leases(namespace).Get(context.TODO(), name, metav1.GetOptions{})
return err
}, "300s").ShouldNot(HaveOccurred())
}, "360s", "1s").ShouldNot(HaveOccurred())
Expect(lease).ToNot(BeNil())
if expected {
Expect(lease.Spec.HolderIdentity).ToNot(BeNil())
} else {
if lease.Spec.HolderIdentity != nil {
if expected {
Expect(*lease.Spec.HolderIdentity).ToNot(BeEmpty())
} else {
Expect(*lease.Spec.HolderIdentity).To(BeEmpty())
}
}
}
return lease
}
func GetLeaseHolder(name, namespace string, client kubernetes.Interface) string {
var lease *coordinationv1.Lease
Eventually(func() string {
lease = CheckLeasePresence(name, namespace, client, true)
return *lease.Spec.HolderIdentity
}, "360s", "1s").ShouldNot(BeEmpty())
return *lease.Spec.HolderIdentity
}

View File

@@ -4,6 +4,8 @@ kind: Pod
metadata:
name: kube-vip
namespace: kube-system
labels:
app: kube-vip
spec:
containers:
- name: kube-vip
@@ -59,6 +61,8 @@ spec:
value: "{{ .SvcEnable }}"
- name: svc_election
value: "{{ .SvcElectionEnable }}"
- name: enable_node_labeling
value: "{{ .EnableNodeLabeling }}"
- name: enable_endpointslices
value: "{{ .EnableEndpointslices }}"
image: "{{ .ImagePath }}"

View File

@@ -4,6 +4,8 @@ kind: Pod
metadata:
name: kube-vip
namespace: kube-system
labels:
app: kube-vip
spec:
containers:
- name: kube-vip
@@ -28,6 +30,8 @@ spec:
value: "1"
- name: cp_enable
value: "true"
- name: enable_node_labeling
value: "{{ .EnableNodeLabeling }}"
image: "{{ .ImagePath }}"
imagePullPolicy: Never
securityContext:

View File

@@ -4,6 +4,8 @@ kind: Pod
metadata:
name: kube-vip
namespace: kube-system
labels:
app: kube-vip
spec:
containers:
- name: kube-vip
@@ -44,6 +46,8 @@ spec:
value: "{{ .SvcEnable }}"
- name: svc_election
value: "{{ .SvcElectionEnable }}"
- name: enable_node_labeling
value: "{{ .EnableNodeLabeling }}"
- name: enable_endpointslices
value: "{{ .EnableEndpointslices }}"
image: "{{ .ImagePath }}"

View File

@@ -4,6 +4,8 @@ kind: Pod
metadata:
name: kube-vip
namespace: kube-system
labels:
app: kube-vip
spec:
containers:
- name: kube-vip
@@ -38,6 +40,8 @@ spec:
value: "{{ .SvcEnable }}"
- name: svc_election
value: "{{ .SvcElectionEnable }}"
- name: enable_node_labeling
value: "{{ .EnableNodeLabeling }}"
- name: enable_endpointslices
value: "{{ .EnableEndpointslices }}"
image: "{{ .ImagePath }}"

129
testing/e2e/logs.go Normal file
View File

@@ -0,0 +1,129 @@
package e2e
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"os"
"path/filepath"
"time"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
apierrors "k8s.io/apimachinery/pkg/api/errors"
)
func GetLogs(ctx context.Context, client kubernetes.Interface, tempDirPath string) error {
if os.Getenv("E2E_KEEP_LOGS") != "true" {
if err := os.RemoveAll(tempDirPath); err != nil {
return fmt.Errorf("failed to remove temporary directory %q: %w", tempDirPath, err)
}
return nil
}
intCtx, cancel := context.WithTimeout(ctx, time.Second*10)
defer cancel()
path := filepath.Join(tempDirPath, "pods.json")
fpods, err := os.Create(path)
if err != nil {
return fmt.Errorf("failed to create file %q: %w", path, err)
}
defer fpods.Close()
// list pods
pods, err := client.CoreV1().Pods("").List(intCtx, metav1.ListOptions{})
if err != nil {
return fmt.Errorf("failed to get pods: %w", err)
}
data, err := json.Marshal(pods)
if err != nil {
return fmt.Errorf("failed to marshal pods data: %w", err)
}
if _, err := fpods.Write(data); err != nil {
return fmt.Errorf("failed to save pods: %w", err)
}
// list services
path = filepath.Join(tempDirPath, "services.json")
fsvcs, err := os.Create(path)
if err != nil {
return fmt.Errorf("failed to create file %q: %w", path, err)
}
defer fpods.Close()
svcs, err := client.CoreV1().Services("").List(intCtx, metav1.ListOptions{})
if err != nil {
return fmt.Errorf("failed to get pods: %w", err)
}
data, err = json.Marshal(svcs)
if err != nil {
return fmt.Errorf("failed to marshal pods data: %w", err)
}
if _, err := fsvcs.Write(data); err != nil {
return fmt.Errorf("failed to save pods: %w", err)
}
// list leases
path = filepath.Join(tempDirPath, "leases.json")
fleases, err := os.Create(path)
if err != nil {
return fmt.Errorf("failed to create file %q: %w", path, err)
}
defer fpods.Close()
leases, err := client.CoordinationV1().Leases("").List(intCtx, metav1.ListOptions{})
if err != nil {
return fmt.Errorf("failed to get leases: %w", err)
}
data, err = json.Marshal(leases)
if err != nil {
return fmt.Errorf("failed to marshal leases data: %w", err)
}
if _, err := fleases.Write(data); err != nil {
return fmt.Errorf("failed to save leases: %w", err)
}
// get kube-vip pods
listOptions := metav1.ListOptions{
LabelSelector: "app=kube-vip",
}
kvpods, err := client.CoreV1().Pods("").List(intCtx, listOptions)
if err != nil {
return fmt.Errorf("failed to list pods: %w", err)
}
for _, pod := range kvpods.Items {
if pod.Status.Phase != corev1.PodRunning {
continue
}
req := client.CoreV1().Pods(pod.Namespace).GetLogs(pod.Name, &corev1.PodLogOptions{})
podLogs, err := req.Stream(intCtx)
if err != nil {
if apierrors.IsNotFound(err) {
continue
}
return fmt.Errorf("reading request stream: %w", err)
}
defer podLogs.Close()
buf := new(bytes.Buffer)
_, err = io.Copy(buf, podLogs)
if err != nil {
return fmt.Errorf("failed to copy logs to buffer: %w", err)
}
path := filepath.Join(tempDirPath, fmt.Sprintf("%s-%s.log", pod.Namespace, pod.Name))
err = os.WriteFile(path, buf.Bytes(), 0600)
if err != nil {
return fmt.Errorf("failed to write the log file %q: %w", path, err)
}
}
return nil
}

View File

@@ -15,6 +15,7 @@ type KubevipManifestValues struct {
SvcEnable string
SvcElectionEnable string
EnableEndpointslices string
EnableNodeLabeling string
ControlPlaneEnable string
BGPAS uint32
BGPPeers string

View File

@@ -59,6 +59,12 @@ func main() {
t.ServiceName = "kube-vip-service"
t.LeaderName = "kube-vip-deploy-leader"
var err error
t.TempDirPath, err = os.MkdirTemp("", "kube-vip-services")
if err != nil {
slog.Fatal(err)
}
if t.ControlPlane {
err := t.CreateKind()
if !t.RetainCluster {

View File

@@ -6,6 +6,7 @@ import (
"time"
"github.com/gookit/slog"
"github.com/kube-vip/kube-vip/pkg/kubevip"
appsv1 "k8s.io/api/apps/v1"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/cache"
@@ -44,18 +45,21 @@ func (d *Deployment) CreateKVDs(ctx context.Context, clientset *kubernetes.Clien
Namespace: "kube-system",
Labels: map[string]string{
"app.kubernetes.io/name": "kube-vip-ds",
"app": "kube-vip",
},
},
Spec: appsv1.DaemonSetSpec{
Selector: &metav1.LabelSelector{
MatchLabels: map[string]string{
"app.kubernetes.io/name": "kube-vip-ds",
"app": "kube-vip",
},
},
Template: v1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: map[string]string{
"app.kubernetes.io/name": "kube-vip-ds",
"app": "kube-vip",
},
},
Spec: v1.PodSpec{
@@ -73,7 +77,7 @@ func (d *Deployment) CreateKVDs(ctx context.Context, clientset *kubernetes.Clien
},
{
Name: "vip_subnet",
Value: "32",
Value: "auto,auto",
},
{
Name: "svc_enable",
@@ -235,23 +239,23 @@ func (s *Service) CreateService(ctx context.Context, clientset *kubernetes.Clien
}
if s.egress {
svc.Annotations = map[string]string{ //kube-vip.io/egress: "true"
"kube-vip.io/egress": "true",
svc.Annotations = map[string]string{
kubevip.Egress: "true",
}
}
if s.egressInternal {
svc.Annotations["kube-vip.io/egress-internal"] = "true"
svc.Annotations[kubevip.EgressInternal] = "true"
}
if s.egressIPv6 {
svc.Annotations["kube-vip.io/egress-ipv6"] = "true"
svc.Annotations[kubevip.EgressIPv6] = "true"
}
//svc.Annotations["kube-vip.io/egress-denied-networks"] = "172.18.0.0/24"
//svc.Annotations["kube-vip.io/egress-allowed-networks"] = "172.18.0.0/24"
//svc.Annotations[kubevip.EgressDeniedNetworks] = "172.18.0.0/24"
//svc.Annotations[kubevip.EgressAllowedNetworks] = "172.18.0.0/24"
//svc.Annotations["kube-vip.io/egress-allowed-networks"] = "192.168.0.0/24, 172.18.0.0/24"
//svc.Annotations[kubevip.EgressAllowedNetworks] = "192.168.0.0/24, 172.18.0.0/24"
if s.policyLocal {
svc.Spec.ExternalTrafficPolicy = v1.ServiceExternalTrafficPolicyTypeLocal
@@ -269,7 +273,7 @@ func (s *Service) CreateService(ctx context.Context, clientset *kubernetes.Clien
if err != nil {
slog.Fatal(err)
}
svc.Annotations["kube-vip.io/loadbalancerIPs"] = fmt.Sprintf("%s,%s", ipv4VIP, ipv6VIP)
svc.Annotations[kubevip.LoadbalancerIPAnnotation] = fmt.Sprintf("%s,%s", ipv4VIP, ipv6VIP)
svc.Labels["implementation"] = "kube-vip"
svc.Spec.IPFamilies = []v1.IPFamily{v1.IPv4Protocol, v1.IPv6Protocol}
ipfPolicy := v1.IPFamilyPolicyRequireDualStack
@@ -312,9 +316,9 @@ func (s *Service) CreateService(ctx context.Context, clientset *kubernetes.Clien
for _, ingress := range svc.Status.LoadBalancer.Ingress {
loadBalancerAddresses = append(loadBalancerAddresses, ingress.IP)
}
slog.Infof("🔎 found load balancer addresses [%s] on node [%s]", loadBalancerAddresses, svc.Annotations["kube-vip.io/vipHost"])
slog.Infof("🔎 found load balancer addresses [%s] on node [%s]", loadBalancerAddresses, svc.Annotations[kubevip.VipHost])
ready = true
currentLeader = svc.Annotations["kube-vip.io/vipHost"]
currentLeader = svc.Annotations[kubevip.VipHost]
}
}
default:

View File

@@ -7,10 +7,13 @@ import (
"fmt"
"net"
"net/http"
"os"
"strings"
"time"
"github.com/gookit/slog"
"github.com/kube-vip/kube-vip/pkg/kubevip"
"github.com/kube-vip/kube-vip/testing/e2e"
"github.com/vishvananda/netlink"
"k8s.io/client-go/kubernetes"
@@ -29,12 +32,35 @@ import (
func (config *TestConfig) StartServiceTest(ctx context.Context, clientset *kubernetes.Clientset) []error {
var err error
var errs []error
globalTempDirPath, err := os.MkdirTemp("/tmp", "kube-vip-service-tests")
if err != nil {
slog.Error(err)
return []error{fmt.Errorf("failed to create temporary directory: %w", err)}
}
defer func() {
if os.Getenv("E2E_KEEP_LOGS") != "true" {
if err := os.RemoveAll(globalTempDirPath); err != nil {
slog.Error(fmt.Errorf("failed to remove temporary directory %q: %w", globalTempDirPath, err))
}
}
}()
if config.Simple {
err = config.SimpleDeployment(ctx, clientset)
if err != nil {
slog.Error(err)
errs = append(errs, err)
}
tempDirPath, err := os.MkdirTemp(globalTempDirPath, "Simple")
if err != nil {
slog.Error(err)
return []error{fmt.Errorf("failed to create temporary directory: %w", err)}
}
slog.Infof("saving logs to %q", tempDirPath)
if err := e2e.GetLogs(ctx, clientset, tempDirPath); err != nil {
slog.Error(err)
}
}
if config.Deployments {
@@ -43,6 +69,15 @@ func (config *TestConfig) StartServiceTest(ctx context.Context, clientset *kuber
slog.Error(err)
errs = append(errs, err)
}
tempDirPath, err := os.MkdirTemp(globalTempDirPath, "Deployments")
if err != nil {
slog.Error(err)
return []error{fmt.Errorf("failed to create temporary directory: %w", err)}
}
slog.Infof("saving logs to %q", tempDirPath)
if err := e2e.GetLogs(ctx, clientset, tempDirPath); err != nil {
slog.Error(err)
}
}
if config.LeaderFailover {
@@ -52,6 +87,15 @@ func (config *TestConfig) StartServiceTest(ctx context.Context, clientset *kuber
slog.Error(err)
errs = append(errs, err)
}
tempDirPath, err := os.MkdirTemp(globalTempDirPath, "LeaderFailover")
if err != nil {
slog.Error(err)
return []error{fmt.Errorf("failed to create temporary directory: %w", err)}
}
slog.Infof("saving logs to %q", tempDirPath)
if err := e2e.GetLogs(ctx, clientset, tempDirPath); err != nil {
slog.Error(err)
}
}
if config.LeaderActive {
@@ -61,6 +105,15 @@ func (config *TestConfig) StartServiceTest(ctx context.Context, clientset *kuber
slog.Error(err)
errs = append(errs, err)
}
tempDirPath, err := os.MkdirTemp(globalTempDirPath, "LeaderActive")
if err != nil {
slog.Error(err)
return []error{fmt.Errorf("failed to create temporary directory: %w", err)}
}
slog.Infof("saving logs to %q", tempDirPath)
if err := e2e.GetLogs(ctx, clientset, tempDirPath); err != nil {
slog.Error(err)
}
}
if config.LocalDeploy {
@@ -70,6 +123,15 @@ func (config *TestConfig) StartServiceTest(ctx context.Context, clientset *kuber
slog.Error(err)
errs = append(errs, err)
}
tempDirPath, err := os.MkdirTemp(globalTempDirPath, "LocalDeploy")
if err != nil {
slog.Error(err)
return []error{fmt.Errorf("failed to create temporary directory: %w", err)}
}
slog.Infof("saving logs to %q", tempDirPath)
if err := e2e.GetLogs(ctx, clientset, tempDirPath); err != nil {
slog.Error(err)
}
}
if config.Egress {
@@ -79,6 +141,15 @@ func (config *TestConfig) StartServiceTest(ctx context.Context, clientset *kuber
slog.Error(err)
errs = append(errs, err)
}
tempDirPath, err := os.MkdirTemp(globalTempDirPath, "EgressDeployment")
if err != nil {
slog.Error(err)
return []error{fmt.Errorf("failed to create temporary directory: %w", err)}
}
slog.Infof("saving logs to %q", tempDirPath)
if err := e2e.GetLogs(ctx, clientset, tempDirPath); err != nil {
slog.Error(err)
}
}
if config.Egress && config.EgressInternal {
@@ -88,6 +159,15 @@ func (config *TestConfig) StartServiceTest(ctx context.Context, clientset *kuber
slog.Error(err)
errs = append(errs, err)
}
tempDirPath, err := os.MkdirTemp(globalTempDirPath, "EgressDeploymentInternal")
if err != nil {
slog.Error(err)
return []error{fmt.Errorf("failed to create temporary directory: %w", err)}
}
slog.Infof("saving logs to %q", tempDirPath)
if err := e2e.GetLogs(ctx, clientset, tempDirPath); err != nil {
slog.Error(err)
}
}
if config.EgressIPv6 {
@@ -97,6 +177,15 @@ func (config *TestConfig) StartServiceTest(ctx context.Context, clientset *kuber
slog.Error(err)
errs = append(errs, err)
}
tempDirPath, err := os.MkdirTemp(globalTempDirPath, "EgressIPv6")
if err != nil {
slog.Error(err)
return []error{fmt.Errorf("failed to create temporary directory: %w", err)}
}
slog.Infof("saving logs to %q", tempDirPath)
if err := e2e.GetLogs(ctx, clientset, tempDirPath); err != nil {
slog.Error(err)
}
}
if config.EgressIPv6 && config.EgressInternal {
@@ -106,6 +195,15 @@ func (config *TestConfig) StartServiceTest(ctx context.Context, clientset *kuber
slog.Error(err)
errs = append(errs, err)
}
tempDirPath, err := os.MkdirTemp(globalTempDirPath, "EgressIPv6Internal")
if err != nil {
slog.Error(err)
return []error{fmt.Errorf("failed to create temporary directory: %w", err)}
}
slog.Infof("saving logs to %q", tempDirPath)
if err := e2e.GetLogs(ctx, clientset, tempDirPath); err != nil {
slog.Error(err)
}
}
if config.DualStack {
@@ -115,6 +213,15 @@ func (config *TestConfig) StartServiceTest(ctx context.Context, clientset *kuber
slog.Error(err)
errs = append(errs, err)
}
tempDirPath, err := os.MkdirTemp(globalTempDirPath, "DualStack")
if err != nil {
slog.Error(err)
return []error{fmt.Errorf("failed to create temporary directory: %w", err)}
}
slog.Infof("saving logs to %q", tempDirPath)
if err := e2e.GetLogs(ctx, clientset, tempDirPath); err != nil {
slog.Error(err)
}
}
const testComplete = "🏆 Testing Complete [%d] passed / [%d] failed"
@@ -211,7 +318,7 @@ func leaderFailover(ctx context.Context, name, leaderNode *string, clientset *ku
}
if svc.Name == *name {
if len(svc.Status.LoadBalancer.Ingress) != 0 {
slog.Infof("🔎 found load balancer address [%s] on node [%s]", svc.Status.LoadBalancer.Ingress[0].IP, svc.Annotations["kube-vip.io/vipHost"])
slog.Infof("🔎 found load balancer address [%s] on node [%s]", svc.Status.LoadBalancer.Ingress[0].IP, svc.Annotations[kubevip.VipHost])
}
}
case watch.Modified:
@@ -221,12 +328,12 @@ func leaderFailover(ctx context.Context, name, leaderNode *string, clientset *ku
}
if svc.Name == *name {
if len(svc.Status.LoadBalancer.Ingress) != 0 {
slog.Infof("🔍 updated with address [%s] on node [%s]", svc.Status.LoadBalancer.Ingress[0].IP, svc.Annotations["kube-vip.io/vipHost"])
slog.Infof("🔍 updated with address [%s] on node [%s]", svc.Status.LoadBalancer.Ingress[0].IP, svc.Annotations[kubevip.VipHost])
err = httpTest(svc.Status.LoadBalancer.Ingress[0].IP)
if err != nil {
return err
}
*leaderNode = svc.Annotations["kube-vip.io/vipHost"]
*leaderNode = svc.Annotations[kubevip.VipHost]
}
}
default:
@@ -294,7 +401,7 @@ func podFailover(ctx context.Context, name, leaderNode *string, clientset *kuber
}
if svc.Name == *name {
if len(svc.Status.LoadBalancer.Ingress) != 0 {
slog.Infof("🔎 found load balancer address [%s] on node [%s]", svc.Status.LoadBalancer.Ingress[0].IP, svc.Annotations["kube-vip.io/vipHost"])
slog.Infof("🔎 found load balancer address [%s] on node [%s]", svc.Status.LoadBalancer.Ingress[0].IP, svc.Annotations[kubevip.VipHost])
}
}
case watch.Modified:
@@ -304,12 +411,12 @@ func podFailover(ctx context.Context, name, leaderNode *string, clientset *kuber
}
if svc.Name == *name {
if len(svc.Status.LoadBalancer.Ingress) != 0 {
slog.Infof("🔍 updated with address [%s] on node [%s]", svc.Status.LoadBalancer.Ingress[0].IP, svc.Annotations["kube-vip.io/vipHost"])
slog.Infof("🔍 updated with address [%s] on node [%s]", svc.Status.LoadBalancer.Ingress[0].IP, svc.Annotations[kubevip.VipHost])
err = httpTest(svc.Status.LoadBalancer.Ingress[0].IP)
if err != nil {
slog.Fatal(err)
}
*leaderNode = svc.Annotations["kube-vip.io/vipHost"]
*leaderNode = svc.Annotations[kubevip.VipHost]
}
}
default:

View File

@@ -4,9 +4,13 @@ import (
"context"
"fmt"
"net"
"os"
"time"
"github.com/gookit/slog"
"github.com/kube-vip/kube-vip/testing/e2e"
v1 "k8s.io/api/core/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
)
@@ -48,25 +52,40 @@ type TestConfig struct {
// Docker config
DockerNIC string
// temp dir root
TempDirPath string
}
func (config *TestConfig) SimpleDeployment(ctx context.Context, clientset *kubernetes.Clientset) error {
// Simple Deployment test
var err error
defer func() error { //nolint
slog.Infof("🧪 ---> simple deployment defer <---")
tempDirPath, err := os.MkdirTemp(config.TempDirPath, "simple-deployment")
if err != nil {
slog.Infof("🧪 ---> simple deployment mkdir err <---: %w", err)
return err
}
if err = e2e.GetLogs(ctx, clientset, tempDirPath); err != nil {
slog.Infof("🧪 ---> simple deployment getlogs err <---: %w", err)
return err
}
slog.Infof("🧹 deleting Service [%s], deployment [%s]", config.ServiceName, config.DeploymentName)
err = clientset.CoreV1().Services(v1.NamespaceDefault).Delete(ctx, config.ServiceName, metav1.DeleteOptions{})
if err != nil {
slog.Fatal(err)
}
err = clientset.AppsV1().Deployments(v1.NamespaceDefault).Delete(ctx, config.DeploymentName, metav1.DeleteOptions{})
if err != nil {
if err = deleteDeployment(ctx, clientset, config.DeploymentName); err != nil {
slog.Fatal(err)
}
return nil
}() //nolint
var err error
slog.Infof("🧪 ---> simple deployment <---")
deploy := Deployment{
name: config.DeploymentName,
@@ -90,6 +109,8 @@ func (config *TestConfig) SimpleDeployment(ctx context.Context, clientset *kuber
config.SuccessCounter++
}
slog.Infof("🧪 ---> simple deployment end <---")
return err
}
@@ -98,6 +119,16 @@ func (config *TestConfig) MultipleDeployments(ctx context.Context, clientset *ku
var err error
defer func() error { //nolint
tempDirPath, err := os.MkdirTemp(config.TempDirPath, "multiple-deployments")
if err != nil {
return err
}
if err = e2e.GetLogs(ctx, clientset, tempDirPath); err != nil {
slog.Infof("🧪 ---> simple deployment getlogs err <---: %w", err)
return err
}
for i := 1; i < 5; i++ {
slog.Infof("🧹 deleting service [%s]", fmt.Sprintf("%s-%d", config.ServiceName, i))
err = clientset.CoreV1().Services(v1.NamespaceDefault).Delete(ctx, fmt.Sprintf("%s-%d", config.ServiceName, i), metav1.DeleteOptions{})
@@ -105,9 +136,8 @@ func (config *TestConfig) MultipleDeployments(ctx context.Context, clientset *ku
slog.Fatal(err)
}
}
slog.Infof("🧹 deleting deployment [%s]", config.DeploymentName)
err = clientset.AppsV1().Deployments(v1.NamespaceDefault).Delete(ctx, config.LeaderName, metav1.DeleteOptions{})
if err != nil {
if err = deleteDeployment(ctx, clientset, config.LeaderName); err != nil {
slog.Fatal(err)
}
return nil
@@ -146,15 +176,24 @@ func (config *TestConfig) Failover(ctx context.Context, clientset *kubernetes.Cl
var err error
defer func() error {
slog.Infof("🧹 deleting Service [%s], deployment [%s]", config.ServiceName, config.DeploymentName)
err = clientset.CoreV1().Services(v1.NamespaceDefault).Delete(ctx, config.ServiceName, metav1.DeleteOptions{})
tempDirPath, err := os.MkdirTemp(config.TempDirPath, "failover")
if err != nil {
slog.Fatal(err)
}
if err = e2e.GetLogs(ctx, clientset, tempDirPath); err != nil {
slog.Infof("🧪 ---> simple deployment getlogs err <---: %w", err)
return err
}
err = clientset.AppsV1().Deployments(v1.NamespaceDefault).Delete(ctx, config.DeploymentName, metav1.DeleteOptions{})
slog.Infof("🧹 deleting Service [%s], deployment [%s]", config.ServiceName, config.DeploymentName)
err = clientset.CoreV1().Services(v1.NamespaceDefault).Delete(ctx, config.ServiceName, metav1.DeleteOptions{})
if err != nil {
return err
slog.Fatal(err)
}
if err = deleteDeployment(ctx, clientset, config.DeploymentName); err != nil {
slog.Fatal(err)
}
return nil
}() //nolint
@@ -208,14 +247,24 @@ func (config *TestConfig) ActiveFailover(ctx context.Context, clientset *kuberne
var err error
defer func() error {
tempDirPath, err := os.MkdirTemp(config.TempDirPath, "active-failover")
if err != nil {
slog.Fatal(err)
}
if err = e2e.GetLogs(ctx, clientset, tempDirPath); err != nil {
slog.Infof("🧪 ---> simple deployment getlogs err <---: %w", err)
return err
}
slog.Infof("🧹 deleting Service [%s], deployment [%s]", config.ServiceName, config.DeploymentName)
err = clientset.CoreV1().Services(v1.NamespaceDefault).Delete(ctx, config.ServiceName, metav1.DeleteOptions{})
if err != nil {
return err
slog.Fatal(err)
}
err = clientset.AppsV1().Deployments(v1.NamespaceDefault).Delete(ctx, config.DeploymentName, metav1.DeleteOptions{})
if err != nil {
return err
if err = deleteDeployment(ctx, clientset, config.DeploymentName); err != nil {
slog.Fatal(err)
}
return nil
}() //nolint
@@ -254,17 +303,26 @@ func (config *TestConfig) LocalDeployment(ctx context.Context, clientset *kubern
// Multiple deployment tests
var err error
defer func() error {
tempDirPath, err := os.MkdirTemp(config.TempDirPath, "local-deployment")
if err != nil {
slog.Fatal(err)
}
if err = e2e.GetLogs(ctx, clientset, tempDirPath); err != nil {
slog.Infof("🧪 ---> simple deployment getlogs err <---: %w", err)
return err
}
for i := 1; i < 5; i++ {
slog.Infof("🧹 deleting service [%s]", fmt.Sprintf("%s-%d", config.ServiceName, i))
err = clientset.CoreV1().Services(v1.NamespaceDefault).Delete(ctx, fmt.Sprintf("%s-%d", config.ServiceName, i), metav1.DeleteOptions{})
if err != nil {
return err
slog.Fatal(err)
}
}
slog.Infof("🧹 deleting deployment [%s]", config.DeploymentName)
err := clientset.AppsV1().Deployments(v1.NamespaceDefault).Delete(ctx, config.LeaderName, metav1.DeleteOptions{})
if err != nil {
return err
if err = deleteDeployment(ctx, clientset, config.LeaderName); err != nil {
slog.Fatal(err)
}
return nil
}() //nolint
@@ -310,17 +368,26 @@ func (config *TestConfig) LocalDeployment(ctx context.Context, clientset *kubern
func (config *TestConfig) EgressDeployment(ctx context.Context, clientset *kubernetes.Clientset, internal bool) error {
// egress test
var err error
defer func() error {
tempDirPath, err := os.MkdirTemp(config.TempDirPath, "egress-deployment")
if err != nil {
slog.Fatal(err)
}
if err = e2e.GetLogs(ctx, clientset, tempDirPath); err != nil {
slog.Infof("🧪 ---> simple deployment getlogs err <---: %w", err)
return err
}
slog.Infof("🧹 deleting Service [%s], deployment [%s]", config.ServiceName, config.DeploymentName)
err = clientset.CoreV1().Services(v1.NamespaceDefault).Delete(ctx, config.ServiceName, metav1.DeleteOptions{})
if err != nil {
return err
slog.Fatal(err)
}
err = clientset.AppsV1().Deployments(v1.NamespaceDefault).Delete(ctx, config.DeploymentName, metav1.DeleteOptions{})
if err != nil {
return err
if err = deleteDeployment(ctx, clientset, config.DeploymentName); err != nil {
slog.Fatal(err)
}
return nil
}() //nolint
@@ -392,14 +459,24 @@ func (config *TestConfig) Egressv6Deployment(ctx context.Context, clientset *kub
var err error
defer func() error {
tempDirPath, err := os.MkdirTemp(config.TempDirPath, "egress-v6-deployment")
if err != nil {
slog.Fatal(err)
}
if err = e2e.GetLogs(ctx, clientset, tempDirPath); err != nil {
slog.Infof("🧪 ---> simple deployment getlogs err <---: %w", err)
return err
}
slog.Infof("🧹 deleting Service [%s], deployment [%s]", config.ServiceName, config.DeploymentName)
err = clientset.CoreV1().Services(v1.NamespaceDefault).Delete(ctx, config.ServiceName, metav1.DeleteOptions{})
if err != nil {
return err
slog.Fatal(err)
}
err = clientset.AppsV1().Deployments(v1.NamespaceDefault).Delete(ctx, config.DeploymentName, metav1.DeleteOptions{})
if err != nil {
return err
if err = deleteDeployment(ctx, clientset, config.DeploymentName); err != nil {
slog.Fatal(err)
}
return nil
}() //nolint
@@ -462,7 +539,7 @@ func (config *TestConfig) Egressv6Deployment(ctx context.Context, clientset *kub
}
if egress == "" {
return fmt.Errorf("no loadbalancer address found")
return fmt.Errorf("no loadbalancer egress address found")
}
found = tcpServer(&egress, timeout, "tcp6")
@@ -503,14 +580,59 @@ func (config *TestConfig) DualStackDeployment(ctx context.Context, clientset *ku
config.SuccessCounter++
tempDirPath, err := os.MkdirTemp(config.TempDirPath, "dualstack-deployment")
if err != nil {
slog.Fatal(err)
}
if err = e2e.GetLogs(ctx, clientset, tempDirPath); err != nil {
slog.Infof("🧪 ---> simple deployment getlogs err <---: %w", err)
return err
}
slog.Infof("🧹 deleting Service [%s], deployment [%s]", config.ServiceName, config.DeploymentName)
err = clientset.CoreV1().Services(v1.NamespaceDefault).Delete(ctx, config.ServiceName, metav1.DeleteOptions{})
if err != nil {
return err
slog.Fatal(err)
}
err = clientset.AppsV1().Deployments(v1.NamespaceDefault).Delete(ctx, config.DeploymentName, metav1.DeleteOptions{})
if err != nil {
return err
if err = deleteDeployment(ctx, clientset, config.DeploymentName); err != nil {
slog.Fatal(err)
}
return err
}
func deleteDeployment(ctx context.Context, clientset *kubernetes.Clientset, name string) error {
fgPropagation := metav1.DeletePropagationForeground
delOpts := metav1.DeleteOptions{
PropagationPolicy: &fgPropagation,
}
slog.Infof("🧹 deleting deployment [%s]", name)
if err := clientset.AppsV1().Deployments(v1.NamespaceDefault).Delete(ctx, name, delOpts); err != nil {
return fmt.Errorf("failed to delete deployment %q: %w", name, err)
}
slog.Infof("🧹 waiting for the deployment [%s] deletion", name)
t := time.NewTicker(time.Millisecond * 200)
checkCtx, checkCancel := context.WithTimeout(ctx, time.Second*30)
defer checkCancel()
for {
select {
case <-checkCtx.Done():
t.Stop()
return fmt.Errorf("failed to check deployment's %q deletion: %w", name, ctx.Err())
case <-t.C:
_, err := clientset.AppsV1().Deployments(v1.NamespaceDefault).Get(checkCtx, name, metav1.GetOptions{})
if err != nil && apierrors.IsNotFound(err) {
slog.Infof("🧹 deployment [%s] was deleted", name)
return nil
} else if err != nil {
return fmt.Errorf("failed to wait for the deployment %q to be deleted: %w", name, err)
}
}
}
}