Support multiple namespaces for Consul and ConsulCatalog providers

Co-authored-by: Kevin Pollet <pollet.kevin@gmail.com>
2022-06-03 12:00:09 +02:00 · 2022-06-03 12:00:09 +02:00 · f90e3817e8
commit f90e3817e8
parent f352c34136
28 changed files with 531 additions and 152 deletions
--- a/docs/content/deprecation/features.md
+++ b/docs/content/deprecation/features.md
@ -2,10 +2,11 @@

 This page is maintained and updated periodically to reflect our roadmap and any decisions around feature deprecation.

-| Feature                                               | Deprecated | End of Support | Removal |
-|-------------------------------------------------------|------------|----------------|---------|
-| [Pilot Dashboard (Metrics)](#pilot-dashboard-metrics) | 2.7        | 2.8            | 2.9     |
-| [Pilot Plugins](#pilot-plugins)                       | 2.7        | 2.8            | 2.9     |
+| Feature                                                       | Deprecated | End of Support | Removal |
+|---------------------------------------------------------------|------------|----------------|---------|
+| [Pilot Dashboard (Metrics)](#pilot-dashboard-metrics)         | 2.7        | 2.8            | 2.9     |
+| [Pilot Plugins](#pilot-plugins)                               | 2.7        | 2.8            | 2.9     |
+| [Consul Enterprise Namespaces](#consul-enterprise-namespaces) | 2.8        | TBD            | TBD     |

 ## Impact

@ -18,3 +19,8 @@ In 2.9, the Pilot platform and all Traefik integration code will be permanently

 Starting on 2.7 the pilot token will not be a requirement anymore.  
 At 2.9, a new plugin catalog home should be available, decoupled from pilot.
+
+### Consul Enterprise Namespaces
+
+Starting on 2.8 the `namespace` option of Consul and Consul Catalog providers is deprecated, 
+please use the `namespaces` options instead.  
--- a/docs/content/migration/v2.md
+++ b/docs/content/migration/v2.md
@ -460,7 +460,11 @@ In `v2.6.1`, the Datadog tags added to a span changed from `service.name` to `tr

 ## v2.8

-### TLS client authentication 
+### TLS client authentication

 In `v2.8`, the `caOptional` option is deprecated as TLS client authentication is a server side option.
 This option available in the ForwardAuth middleware, as well as in the HTTP, Consul, Etcd, Redis, ZooKeeper, Marathon, Consul Catalog, and Docker providers has no effect and must not be used anymore.
+
+### Consul Enterprise Namespaces
+
+In `v2.8`, the `namespace` option of Consul and Consul Catalog providers is deprecated, please use the `namespaces` options instead.
--- a/docs/content/providers/consul-catalog.md
+++ b/docs/content/providers/consul-catalog.md
@ -525,7 +525,7 @@ providers:
 ```

 ```bash tab="CLI"
--providers.consulcatalog.defaultRule="Host(`{{ .Name }}.{{ index .Labels \"customLabel\"}}`)"
+--providers.consulcatalog.defaultRule=Host(`{{ .Name }}.{{ index .Labels \"customLabel\"}}`)
 # ...
 ```

@ -669,30 +669,76 @@ For additional information, refer to [Restrict the Scope of Service Discovery](.

 ### `namespace`

+??? warning "Deprecated in favor of the [`namespaces`](#namespaces) option."
+
+    _Optional, Default=""_
+    
+    The `namespace` option defines the namespace in which the consul catalog services will be discovered.
+    
+    !!! warning
+    
+        The namespace option only works with [Consul Enterprise](https://www.consul.io/docs/enterprise),
+        which provides the [Namespaces](https://www.consul.io/docs/enterprise/namespaces) feature.
+    
+    !!! warning
+    
+        One should only define either the `namespaces` option or the `namespace` option.
+    
+    ```yaml tab="File (YAML)"
+    providers:
+      consulCatalog:
+        namespace: "production" 
+        # ...
+    ```
+    
+    ```toml tab="File (TOML)"
+    [providers.consulCatalog]
+      namespace = "production"
+      # ...
+    ```
+    
+    ```bash tab="CLI"
+    --providers.consulcatalog.namespace=production
+    # ...
+    ```
+
+### `namespaces`
+
 _Optional, Default=""_

-The `namespace` option defines the namespace in which the consul catalog services will be discovered.
+The `namespaces` option defines the namespaces in which the consul catalog services will be discovered.
+When using the `namespaces` option, the discovered configuration object names will be suffixed as shown below:
+
+```text
+<resource-name>@consulcatalog-<namespace>
+```

 !!! warning

-    The namespace option only works with [Consul Enterprise](https://www.consul.io/docs/enterprise),
+    The namespaces option only works with [Consul Enterprise](https://www.consul.io/docs/enterprise),
    which provides the [Namespaces](https://www.consul.io/docs/enterprise/namespaces) feature.

+!!! warning
+
+    One should only define either the `namespaces` option or the `namespace` option.
+
 ```yaml tab="File (YAML)"
 providers:
  consulCatalog:
-    namespace: "production" 
+    namespaces: 
+      - "ns1"
+      - "ns2"
    # ...
 ```

 ```toml tab="File (TOML)"
 [providers.consulCatalog]
-  namespace = "production"
+  namespaces = ["ns1", "ns2"]
  # ...
 ```

 ```bash tab="CLI"
--providers.consulcatalog.namespace=production
+--providers.consulcatalog.namespaces=ns1,ns2
 # ...
 ```

--- a/docs/content/providers/consul.md
+++ b/docs/content/providers/consul.md
@ -61,30 +61,76 @@ providers:

 ### `namespace`

+??? warning "Deprecated in favor of the [`namespaces`](#namespaces) option."
+
+    _Optional, Default=""_
+    
+    The `namespace` option defines the namespace to query.
+    
+    !!! warning
+    
+        The namespace option only works with [Consul Enterprise](https://www.consul.io/docs/enterprise),
+        which provides the [Namespaces](https://www.consul.io/docs/enterprise/namespaces) feature.
+    
+    !!! warning
+    
+        One should only define either the `namespaces` option or the `namespace` option.
+    
+    ```yaml tab="File (YAML)"
+    providers:
+      consul:
+        # ...
+        namespace: "production"
+    ```
+    
+    ```toml tab="File (TOML)"
+    [providers.consul]
+      # ...
+      namespace = "production"
+    ```
+    
+    ```bash tab="CLI"
+    --providers.consul.namespace=production
+    ```
+
+### `namespaces`
+
 _Optional, Default=""_

-The `namespace` option defines the namespace to query.
+The `namespaces` option defines the namespaces to query.
+When using the `namespaces` option, the discovered configuration object names will be suffixed as shown below:
+
+```text
+<resource-name>@consul-<namespace>
+```

 !!! warning

-    The namespace option only works with [Consul Enterprise](https://www.consul.io/docs/enterprise),
+    The namespaces option only works with [Consul Enterprise](https://www.consul.io/docs/enterprise),
    which provides the [Namespaces](https://www.consul.io/docs/enterprise/namespaces) feature.

+!!! warning
+
+    One should only define either the `namespaces` option or the `namespace` option.
+
 ```yaml tab="File (YAML)"
 providers:
  consul:
+    namespaces: 
+      - "ns1"
+      - "ns2"
    # ...
-    namespace: "production"
 ```

 ```toml tab="File (TOML)"
 [providers.consul]
+  namespaces = ["ns1", "ns2"]
  # ...
-  namespace = "production"
 ```

 ```bash tab="CLI"
--providers.consul.namespace=production
+--providers.consul.namespaces=ns1,ns2
+# ...
 ```

 ### `username`
--- a/docs/content/reference/static-configuration/cli-ref.md
+++ b/docs/content/reference/static-configuration/cli-ref.md
@ -403,7 +403,10 @@ Enable Consul backend with default settings. (Default: ```false```)
 KV store endpoints (Default: ```127.0.0.1:8500```)

 `--providers.consul.namespace`:  
-KV Namespace
+Sets the namespace used to discover the configuration (Consul Enterprise only).
+
+`--providers.consul.namespaces`:  
+Sets the namespaces used to discover the configuration (Consul Enterprise only).

 `--providers.consul.password`:  
 KV Password
@ -492,11 +495,14 @@ Expose containers by default. (Default: ```true```)
 `--providers.consulcatalog.namespace`:  
 Sets the namespace used to discover services (Consul Enterprise only).

+`--providers.consulcatalog.namespaces`:  
+Sets the namespaces used to discover services (Consul Enterprise only).
+
 `--providers.consulcatalog.prefix`:  
-Prefix for consul service tags. Default 'traefik' (Default: ```traefik```)
+Prefix for consul service tags. (Default: ```traefik```)

 `--providers.consulcatalog.refreshinterval`:  
-Interval for check Consul API. Default 15s (Default: ```15```)
+Interval for check Consul API. (Default: ```15```)

 `--providers.consulcatalog.requireconsistent`:  
 Forces the read to be fully consistent. (Default: ```false```)
@ -594,9 +600,6 @@ Enable Etcd backend with default settings. (Default: ```false```)
 `--providers.etcd.endpoints`:  
 KV store endpoints (Default: ```127.0.0.1:2379```)

-`--providers.etcd.namespace`:  
-KV Namespace
-
 `--providers.etcd.password`:  
 KV Password

@ -858,9 +861,6 @@ Enable Redis backend with default settings. (Default: ```false```)
 `--providers.redis.endpoints`:  
 KV store endpoints (Default: ```127.0.0.1:6379```)

-`--providers.redis.namespace`:  
-KV Namespace
-
 `--providers.redis.password`:  
 KV Password

@ -900,9 +900,6 @@ Enable ZooKeeper backend with default settings. (Default: ```false```)
 `--providers.zookeeper.endpoints`:  
 KV store endpoints (Default: ```127.0.0.1:2181```)

-`--providers.zookeeper.namespace`:  
-KV Namespace
-
 `--providers.zookeeper.password`:  
 KV Password

--- a/docs/content/reference/static-configuration/env-ref.md
+++ b/docs/content/reference/static-configuration/env-ref.md
@ -459,11 +459,14 @@ Expose containers by default. (Default: ```true```)
 `TRAEFIK_PROVIDERS_CONSULCATALOG_NAMESPACE`:  
 Sets the namespace used to discover services (Consul Enterprise only).

+`TRAEFIK_PROVIDERS_CONSULCATALOG_NAMESPACES`:  
+Sets the namespaces used to discover services (Consul Enterprise only).
+
 `TRAEFIK_PROVIDERS_CONSULCATALOG_PREFIX`:  
-Prefix for consul service tags. Default 'traefik' (Default: ```traefik```)
+Prefix for consul service tags. (Default: ```traefik```)

 `TRAEFIK_PROVIDERS_CONSULCATALOG_REFRESHINTERVAL`:  
-Interval for check Consul API. Default 15s (Default: ```15```)
+Interval for check Consul API. (Default: ```15```)

 `TRAEFIK_PROVIDERS_CONSULCATALOG_REQUIRECONSISTENT`:  
 Forces the read to be fully consistent. (Default: ```false```)
@ -481,7 +484,10 @@ Watch Consul API events. (Default: ```false```)
 KV store endpoints (Default: ```127.0.0.1:8500```)

 `TRAEFIK_PROVIDERS_CONSUL_NAMESPACE`:  
-KV Namespace
+Sets the namespace used to discover the configuration (Consul Enterprise only).
+
+`TRAEFIK_PROVIDERS_CONSUL_NAMESPACES`:  
+Sets the namespaces used to discover the configuration (Consul Enterprise only).

 `TRAEFIK_PROVIDERS_CONSUL_PASSWORD`:  
 KV Password
@ -594,9 +600,6 @@ Enable Etcd backend with default settings. (Default: ```false```)
 `TRAEFIK_PROVIDERS_ETCD_ENDPOINTS`:  
 KV store endpoints (Default: ```127.0.0.1:2379```)

-`TRAEFIK_PROVIDERS_ETCD_NAMESPACE`:  
-KV Namespace
-
 `TRAEFIK_PROVIDERS_ETCD_PASSWORD`:  
 KV Password

@ -858,9 +861,6 @@ Enable Redis backend with default settings. (Default: ```false```)
 `TRAEFIK_PROVIDERS_REDIS_ENDPOINTS`:  
 KV store endpoints (Default: ```127.0.0.1:6379```)

-`TRAEFIK_PROVIDERS_REDIS_NAMESPACE`:  
-KV Namespace
-
 `TRAEFIK_PROVIDERS_REDIS_PASSWORD`:  
 KV Password

@ -900,9 +900,6 @@ Enable ZooKeeper backend with default settings. (Default: ```false```)
 `TRAEFIK_PROVIDERS_ZOOKEEPER_ENDPOINTS`:  
 KV store endpoints (Default: ```127.0.0.1:2181```)

-`TRAEFIK_PROVIDERS_ZOOKEEPER_NAMESPACE`:  
-KV Namespace
-
 `TRAEFIK_PROVIDERS_ZOOKEEPER_PASSWORD`:  
 KV Password

--- a/docs/content/reference/static-configuration/file.toml
+++ b/docs/content/reference/static-configuration/file.toml
@ -28,12 +28,6 @@
    [entryPoints.EntryPoint0.forwardedHeaders]
      insecure = true
      trustedIPs = ["foobar", "foobar"]
-    [entryPoints.EntryPoint0.udp]
-      timeout = 42
-    [entryPoints.EntryPoint0.http2]
-      maxConcurrentStreams = 42
-    [entryPoints.EntryPoint0.http3]
-      advertisedPort = 42
    [entryPoints.EntryPoint0.http]
      middlewares = ["foobar", "foobar"]
      [entryPoints.EntryPoint0.http.redirections]
@ -53,6 +47,8 @@
        [[entryPoints.EntryPoint0.http.tls.domains]]
          main = "foobar"
          sans = ["foobar", "foobar"]
+    [entryPoints.EntryPoint0.http2]
+      maxConcurrentStreams = 42
    [entryPoints.EntryPoint0.http3]
      advertisedPort = 42
    [entryPoints.EntryPoint0.udp]
@ -161,6 +157,7 @@
    connectByDefault = true
    serviceName = "foobar"
    namespace = "foobar"
+    namespaces = ["foobar", "foobar"]
    watch = true
    [providers.consulCatalog.endpoint]
      address = "foobar"
@ -194,6 +191,7 @@
    password = "foobar"
    token = "foobar"
    namespace = "foobar"
+    namespaces = ["foobar", "foobar"]
    [providers.consul.tls]
      ca = "foobar"
      caOptional = true
@ -206,7 +204,6 @@
    username = "foobar"
    password = "foobar"
    token = "foobar"
-    namespace = "foobar"
    [providers.etcd.tls]
      ca = "foobar"
      caOptional = true
@ -219,7 +216,6 @@
    username = "foobar"
    password = "foobar"
    token = "foobar"
-    namespace = "foobar"
    [providers.zooKeeper.tls]
      ca = "foobar"
      caOptional = true
@ -232,7 +228,6 @@
    username = "foobar"
    password = "foobar"
    token = "foobar"
-    namespace = "foobar"
    [providers.redis.tls]
      ca = "foobar"
      caOptional = true
--- a/docs/content/reference/static-configuration/file.yaml
+++ b/docs/content/reference/static-configuration/file.yaml
@ -32,12 +32,6 @@ entryPoints:
      trustedIPs:
      - foobar
      - foobar
-    http2:
-      maxConcurrentStreams: 42
-    http3:
-      advertisedPort: 42
-    udp:
-      timeout: 42
    http:
      redirections:
        entryPoint:
@ -60,6 +54,8 @@ entryPoints:
            sans:
              - foobar
              - foobar
+    http2:
+      maxConcurrentStreams: 42
    http3:
      advertisedPort: 42
    udp:
@ -173,6 +169,9 @@ providers:
    connectByDefault: true
    serviceName: foobar
    namespace: foobar
+    namespaces:
+      - foobar
+      - foobar
    watch: true
    endpoint:
      address: foobar
@ -210,6 +209,9 @@ providers:
    password: foobar
    token: foobar
    namespace: foobar
+    namespaces:
+      - foobar
+      - foobar
    tls:
      ca: foobar
      caOptional: true
@ -224,7 +226,6 @@ providers:
    username: foobar
    password: foobar
    token: foobar
-    namespace: foobar
    tls:
      ca: foobar
      caOptional: true
@ -239,7 +240,6 @@ providers:
    username: foobar
    password: foobar
    token: foobar
-    namespace: foobar
    tls:
      ca: foobar
      caOptional: true
@ -254,7 +254,6 @@ providers:
    username: foobar
    password: foobar
    token: foobar
-    namespace: foobar
    tls:
      ca: foobar
      caOptional: true