homelab/traefik
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Native Kubernetes service load-balancing

Browse source
This commit is contained in:
Romain 2023-03-20 16:46:05 +01:00 • committed by GitHub
parent 7af9d16208
commit 6e460cd652
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
37 changed files with 1013 additions and 67 deletions
Download patch file Download diff file Expand all files Collapse all files

246
docs/content/routing/providers/kubernetes-crd.md
View file

@ -349,15 +349,16 @@ Register the `IngressRoute` [kind](../../reference/dynamic-configuration/kuberne
sameSite: none
strategy: RoundRobin
weight: 10
tls: # [11]
secretName: supersecret # [12]
options: # [13]
name: opt # [14]
namespace: default # [15]
certResolver: foo # [16]
domains: # [17]
- main: example.net # [18]
sans: # [19]
nativeLB: true # [11]
tls: # [12]
secretName: supersecret # [13]
options: # [14]
name: opt # [15]
namespace: default # [16]
certResolver: foo # [17]
domains: # [18]
- main: example.net # [19]
sans: # [20]
- a.example.net
- b.example.net
```
@ -374,15 +375,16 @@ Register the `IngressRoute` [kind](../../reference/dynamic-configuration/kuberne
| [8] | `routes[n].services` | List of any combination of [TraefikService](#kind-traefikservice) and reference to a [Kubernetes service](https://kubernetes.io/docs/concepts/services-networking/service/) (See below for `ExternalName Service` setup) |
| [9] | `services[n].port` | Defines the port of a [Kubernetes service](https://kubernetes.io/docs/concepts/services-networking/service/). This can be a reference to a named port. |
| [10] | `services[n].serversTransport` | Defines the reference to a [ServersTransport](#kind-serverstransport). The ServersTransport namespace is assumed to be the [Kubernetes service](https://kubernetes.io/docs/concepts/services-networking/service/) namespace (see [ServersTransport reference](#serverstransport-reference)). |
| [11] | `tls` | Defines [TLS](../routers/index.md#tls) certificate configuration |
| [12] | `tls.secretName` | Defines the [secret](https://kubernetes.io/docs/concepts/configuration/secret/) name used to store the certificate (in the `IngressRoute` namespace) |
| [13] | `tls.options` | Defines the reference to a [TLSOption](#kind-tlsoption) |
| [14] | `options.name` | Defines the [TLSOption](#kind-tlsoption) name |
| [15] | `options.namespace` | Defines the [TLSOption](#kind-tlsoption) namespace |
| [16] | `tls.certResolver` | Defines the reference to a [CertResolver](../routers/index.md#certresolver) |
| [17] | `tls.domains` | List of [domains](../routers/index.md#domains) |
| [18] | `domains[n].main` | Defines the main domain name |
| [19] | `domains[n].sans` | List of SANs (alternative domains) |
| [11] | `services[n].nativeLB` | Controls, when creating the load-balancer, whether the LB's children are directly the pods IPs or if the only child is the Kubernetes Service clusterIP. |
| [12] | `tls` | Defines [TLS](../routers/index.md#tls) certificate configuration |
| [13] | `tls.secretName` | Defines the [secret](https://kubernetes.io/docs/concepts/configuration/secret/) name used to store the certificate (in the `IngressRoute` namespace) |
| [14] | `tls.options` | Defines the reference to a [TLSOption](#kind-tlsoption) |
| [15] | `options.name` | Defines the [TLSOption](#kind-tlsoption) name |
| [16] | `options.namespace` | Defines the [TLSOption](#kind-tlsoption) namespace |
| [17] | `tls.certResolver` | Defines the reference to a [CertResolver](../routers/index.md#certresolver) |
| [18] | `tls.domains` | List of [domains](../routers/index.md#domains) |
| [19] | `domains[n].main` | Defines the main domain name |
| [20] | `domains[n].sans` | List of SANs (alternative domains) |
??? example "Declaring an IngressRoute"
@ -639,6 +641,47 @@ More information in the dedicated server [load balancing](../services/index.md#l
task: app2
```
!!! important "Kubernetes Service Native Load-Balancing"
To avoid creating the server load-balancer with the pods IPs and use Kubernetes Service clusterIP directly,
one should set the service `NativeLB` option to true.
Please note that, by default, Traefik reuses the established connections to the backends for performance purposes. This can prevent the requests load balancing between the replicas from behaving as one would expect when the option is set.
By default, `NativeLB` is false.
??? example "Example"
```yaml
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: test.route
namespace: default
spec:
entryPoints:
- foo
routes:
- match: Host(`example.net`)
kind: Rule
services:
- name: svc
port: 80
# Here, nativeLB instructs to build the servers load balancer with the Kubernetes Service clusterIP only.
nativeLB: true
---
apiVersion: v1
kind: Service
metadata:
name: svc
namespace: default
spec:
type: ClusterIP
...
```
### Kind: `Middleware`
`Middleware` is the CRD implementation of a [Traefik middleware](../../middlewares/http/overview.md).
@ -1103,45 +1146,47 @@ Register the `IngressRouteTCP` [kind](../../reference/dynamic-configuration/kube
terminationDelay: 400 # [11]
proxyProtocol: # [12]
version: 1 # [13]
tls: # [14]
secretName: supersecret # [15]
options: # [16]
name: opt # [17]
namespace: default # [18]
certResolver: foo # [19]
domains: # [20]
- main: example.net # [21]
sans: # [22]
nativeLB: true # [14]
tls: # [15]
secretName: supersecret # [16]
options: # [17]
name: opt # [18]
namespace: default # [19]
certResolver: foo # [20]
domains: # [21]
- main: example.net # [22]
sans: # [23]
- a.example.net
- b.example.net
passthrough: false # [23]
passthrough: false # [24]
```
| Ref | Attribute | Purpose |
|------|--------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [1] | `entryPoints` | List of [entrypoints](../routers/index.md#entrypoints_1) names |
| [2] | `routes` | List of routes |
| [3] | `routes[n].match` | Defines the [rule](../routers/index.md#rule_1) of the underlying router |
| [4] | `routes[n].priority` | Defines the [priority](../routers/index.md#priority_1) to disambiguate rules of the same length, for route matching |
| [5] | `middlewares[n].name` | Defines the [MiddlewareTCP](#kind-middlewaretcp) name |
| [6] | `middlewares[n].namespace` | Defines the [MiddlewareTCP](#kind-middlewaretcp) namespace |
| [7] | `routes[n].services` | List of [Kubernetes service](https://kubernetes.io/docs/concepts/services-networking/service/) definitions (See below for `ExternalName Service` setup) |
| [8] | `services[n].name` | Defines the name of a [Kubernetes service](https://kubernetes.io/docs/concepts/services-networking/service/) |
| [9] | `services[n].port` | Defines the port of a [Kubernetes service](https://kubernetes.io/docs/concepts/services-networking/service/). This can be a reference to a named port. |
| [10] | `services[n].weight` | Defines the weight to apply to the server load balancing |
| [11] | `services[n].terminationDelay` | corresponds to the deadline that the proxy sets, after one of its connected peers indicates it has closed the writing capability of its connection, to close the reading capability as well, hence fully terminating the connection. It is a duration in milliseconds, defaulting to 100. A negative value means an infinite deadline (i.e. the reading capability is never closed). |
| [12] | `proxyProtocol` | Defines the [PROXY protocol](../services/index.md#proxy-protocol) configuration |
| [13] | `version` | Defines the [PROXY protocol](../services/index.md#proxy-protocol) version |
| [14] | `tls` | Defines [TLS](../routers/index.md#tls_1) certificate configuration |
| [15] | `tls.secretName` | Defines the [secret](https://kubernetes.io/docs/concepts/configuration/secret/) name used to store the certificate (in the `IngressRoute` namespace) |
| [16] | `tls.options` | Defines the reference to a [TLSOption](#kind-tlsoption) |
| [17] | `options.name` | Defines the [TLSOption](#kind-tlsoption) name |
| [18] | `options.namespace` | Defines the [TLSOption](#kind-tlsoption) namespace |
| [19] | `tls.certResolver` | Defines the reference to a [CertResolver](../routers/index.md#certresolver_1) |
| [20] | `tls.domains` | List of [domains](../routers/index.md#domains_1) |
| [21] | `domains[n].main` | Defines the main domain name |
| [22] | `domains[n].sans` | List of SANs (alternative domains) |
| [23] | `tls.passthrough` | If `true`, delegates the TLS termination to the backend |
| Ref | Attribute | Purpose |
|------|-------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [1] | `entryPoints` | List of [entrypoints](../routers/index.md#entrypoints_1) names |
| [2] | `routes` | List of routes |
| [3] | `routes[n].match` | Defines the [rule](../routers/index.md#rule_1) of the underlying router |
| [4] | `routes[n].priority` | Defines the [priority](../routers/index.md#priority_1) to disambiguate rules of the same length, for route matching |
| [5] | `middlewares[n].name` | Defines the [MiddlewareTCP](#kind-middlewaretcp) name |
| [6] | `middlewares[n].namespace` | Defines the [MiddlewareTCP](#kind-middlewaretcp) namespace |
| [7] | `routes[n].services` | List of [Kubernetes service](https://kubernetes.io/docs/concepts/services-networking/service/) definitions (See below for `ExternalName Service` setup) |
| [8] | `services[n].name` | Defines the name of a [Kubernetes service](https://kubernetes.io/docs/concepts/services-networking/service/) |
| [9] | `services[n].port` | Defines the port of a [Kubernetes service](https://kubernetes.io/docs/concepts/services-networking/service/). This can be a reference to a named port. |
| [10] | `services[n].weight` | Defines the weight to apply to the server load balancing |
| [11] | `services[n].terminationDelay` | corresponds to the deadline that the proxy sets, after one of its connected peers indicates it has closed the writing capability of its connection, to close the reading capability as well, hence fully terminating the connection. It is a duration in milliseconds, defaulting to 100. A negative value means an infinite deadline (i.e. the reading capability is never closed). |
| [12] | `services[n].proxyProtocol` | Defines the [PROXY protocol](../services/index.md#proxy-protocol) configuration |
| [13] | `services[n].proxyProtocol.version` | Defines the [PROXY protocol](../services/index.md#proxy-protocol) version |
| [14] | `services[n].nativeLB` | Controls, when creating the load-balancer, whether the LB's children are directly the pods IPs or if the only child is the Kubernetes Service clusterIP. |
| [15] | `tls` | Defines [TLS](../routers/index.md#tls_1) certificate configuration |
| [16] | `tls.secretName` | Defines the [secret](https://kubernetes.io/docs/concepts/configuration/secret/) name used to store the certificate (in the `IngressRoute` namespace) |
| [17] | `tls.options` | Defines the reference to a [TLSOption](#kind-tlsoption) |
| [18] | `options.name` | Defines the [TLSOption](#kind-tlsoption) name |
| [19] | `options.namespace` | Defines the [TLSOption](#kind-tlsoption) namespace |
| [20] | `tls.certResolver` | Defines the reference to a [CertResolver](../routers/index.md#certresolver_1) |
| [21] | `tls.domains` | List of [domains](../routers/index.md#domains_1) |
| [22] | `domains[n].main` | Defines the main domain name |
| [23] | `domains[n].sans` | List of SANs (alternative domains) |
| [24] | `tls.passthrough` | If `true`, delegates the TLS termination to the backend |
??? example "Declaring an IngressRouteTCP"
@ -1275,6 +1320,45 @@ Register the `IngressRouteTCP` [kind](../../reference/dynamic-configuration/kube
- port: 80
```
!!! important "Kubernetes Service Native Load-Balancing"
To avoid creating the server load-balancer with the pods IPs and use Kubernetes Service clusterIP directly,
one should set the TCP service `NativeLB` option to true.
By default, `NativeLB` is false.
??? example "Examples"
```yaml
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
name: test.route
namespace: default
spec:
entryPoints:
- foo
routes:
- match: HostSNI(`*`)
services:
- name: svc
port: 80
# Here, nativeLB instructs to build the servers load balancer with the Kubernetes Service clusterIP only.
nativeLB: true
---
apiVersion: v1
kind: Service
metadata:
name: svc
namespace: default
spec:
type: ClusterIP
...
```
### Kind: `MiddlewareTCP`
`MiddlewareTCP` is the CRD implementation of a [Traefik TCP middleware](../../middlewares/tcp/overview.md).
@ -1348,16 +1432,18 @@ Register the `IngressRouteUDP` [kind](../../reference/dynamic-configuration/kube
- name: foo # [4]
port: 8080 # [5]
weight: 10 # [6]
nativeLB: true # [7]
```
| Ref | Attribute | Purpose |
|------|--------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [1] | `entryPoints` | List of [entrypoints](../routers/index.md#entrypoints_1) names |
| [2] | `routes` | List of routes |
| [3] | `routes[n].services` | List of [Kubernetes service](https://kubernetes.io/docs/concepts/services-networking/service/) definitions (See below for `ExternalName Service` setup) |
| [4] | `services[n].name` | Defines the name of a [Kubernetes service](https://kubernetes.io/docs/concepts/services-networking/service/) |
| [6] | `services[n].port` | Defines the port of a [Kubernetes service](https://kubernetes.io/docs/concepts/services-networking/service/). This can be a reference to a named port. |
| [7] | `services[n].weight` | Defines the weight to apply to the server load balancing |
| Ref | Attribute | Purpose |
|-----|-------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------|
| [1] | `entryPoints` | List of [entrypoints](../routers/index.md#entrypoints_1) names |
| [2] | `routes` | List of routes |
| [3] | `routes[n].services` | List of [Kubernetes service](https://kubernetes.io/docs/concepts/services-networking/service/) definitions (See below for `ExternalName Service` setup) |
| [4] | `services[n].name` | Defines the name of a [Kubernetes service](https://kubernetes.io/docs/concepts/services-networking/service/) |
| [5] | `services[n].port` | Defines the port of a [Kubernetes service](https://kubernetes.io/docs/concepts/services-networking/service/). This can be a reference to a named port. |
| [6] | `services[n].weight` | Defines the weight to apply to the server load balancing |
| [7] | `services[n].nativeLB` | Controls, when creating the load-balancer, whether the LB's children are directly the pods IPs or if the only child is the Kubernetes Service clusterIP. |
??? example "Declaring an IngressRouteUDP"
@ -1479,6 +1565,44 @@ Register the `IngressRouteUDP` [kind](../../reference/dynamic-configuration/kube
- port: 80
```
!!! important "Kubernetes Service Native Load-Balancing"
To avoid creating the server load-balancer with the pods IPs and use Kubernetes Service clusterIP directly,
one should set the UDP service `NativeLB` option to true.
By default, `NativeLB` is false.
??? example "Example"
```yaml
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteUDP
metadata:
name: test.route
namespace: default
spec:
entryPoints:
- foo
routes:
- services:
- name: svc
port: 80
# Here, nativeLB instructs to build the servers load balancer with the Kubernetes Service clusterIP only.
nativeLB: true
---
apiVersion: v1
kind: Service
metadata:
name: svc
namespace: default
spec:
type: ClusterIP
...
```
### Kind: `TLSOption`
`TLSOption` is the CRD implementation of a [Traefik "TLS Option"](../../https/tls.md#tls-options).

21
docs/content/routing/providers/kubernetes-ingress.md
View file

@ -299,6 +299,17 @@ which in turn will create the resulting routers, services, handlers, etc.
#### On Service
??? info "`traefik.ingress.kubernetes.io/service.nativelb`"
Controls, when creating the load-balancer, whether the LB's children are directly the pods IPs or if the only child is the Kubernetes Service clusterIP.
The Kubernetes Service itself does load-balance to the pods.
Please note that, by default, Traefik reuses the established connections to the backends for performance purposes. This can prevent the requests load balancing between the replicas from behaving as one would expect when the option is set.
By default, NativeLB is false.
```yaml
traefik.ingress.kubernetes.io/service.nativelb: "true"
```
??? info "`traefik.ingress.kubernetes.io/service.serversscheme`"
Overrides the default scheme.
@ -888,11 +899,15 @@ TLS certificates can be managed in Secrets objects.
### Communication Between Traefik and Pods
!!! info "It is not possible to route requests directly to [Kubernetes services](https://kubernetes.io/docs/concepts/services-networking/service/ "Link to Kubernetes service docs")"
!!! info "Routing directly to [Kubernetes services](https://kubernetes.io/docs/concepts/services-networking/service/ "Link to Kubernetes service docs")"
You can use an `ExternalName` service to forward requests to the Kubernetes service through DNS.
To route directly to the Kubernetes service,
one can use the `traefik.ingress.kubernetes.io/service.nativelb` annotation on the Kubernetes service.
It controls, when creating the load-balancer,
whether the LB's children are directly the pods IPs or if the only child is the Kubernetes Service clusterIP.
For doing so, you have to [allow external name services](https://doc.traefik.io/traefik/providers/kubernetes-ingress/#allowexternalnameservices "Link to docs about allowing external name services").
One alternative is to use an `ExternalName` service to forward requests to the Kubernetes service through DNS.
To do so, one must [allow external name services](https://doc.traefik.io/traefik/providers/kubernetes-ingress/#allowexternalnameservices "Link to docs about allowing external name services").
Traefik automatically requests endpoint information based on the service provided in the ingress spec.
Although Traefik will connect directly to the endpoints (pods),