From 278ac080872ba73acdcfbcc173bb78cb9607bc83 Mon Sep 17 00:00:00 2001 From: Bogdan Dobrelya <bdobreli@redhat.com> Date: Tue, 9 Jan 2018 14:01:50 +0100 Subject: [PATCH] Fix HA docs API access endpoints explained (#2126) * Fix HA docs API access endpoints explained Follow-up commit 81347298a3ef7932cbeb55e877644ca22d1625f1 and fix the endpoint value provided in HA docs. Signed-off-by: Bogdan Dobrelya <bogdando@mail.ru> * Clarify internal LB with external LB use case * Clarify how to use both internal and external, non-cluster aware and not managed with Kubespray, LB solutions. * Clarify the requirements, like TLS/SSL termination, for such an external LB. Unlike to the 'cluster-aware' external LB config, endpoints' security must be managed by that non-cluster aware external LB. * Note that masters always contact their local apiservers via https://bip:sp. It's highly unlikely to go down and it reduces latency that might be introduced when going host->lb->host. Only computes go that path. Signed-off-by: Bogdan Dobrelya <bogdando@mail.ru> * Add a note for supplementary_addresses_in_ssl_keys Explain how to benefit from supplementary_addresses_in_ssl_keys Signed-off-by: Bogdan Dobrelya <bogdando@mail.ru> --- docs/ha-mode.md | 51 ++++++++++++++++++++++++++++++++++--------------- 1 file changed, 36 insertions(+), 15 deletions(-) diff --git a/docs/ha-mode.md b/docs/ha-mode.md index 3ee2063c3..f3bc97e1c 100644 --- a/docs/ha-mode.md +++ b/docs/ha-mode.md @@ -68,7 +68,10 @@ listen kubernetes-apiserver-https balance roundrobin ``` -And the corresponding example global vars config: + Note: That's an example config managed elsewhere outside of Kubespray. + +And the corresponding example global vars for such a "cluster-aware" +external LB with the cluster API access modes configured in Kubespray: ``` apiserver_loadbalancer_domain_name: "my-apiserver-lb.example.com" loadbalancer_apiserver: @@ -78,40 +81,58 @@ loadbalancer_apiserver: Note: The default kubernetes apiserver configuration binds to all interfaces, so you will need to use a different port for the vip from that the API is - listening on, or set the kube_apiserver_bind_address so that the API only + listening on, or set the `kube_apiserver_bind_address` so that the API only listens on a specific interface (to avoid conflict with haproxy binding the port on the VIP adddress) This domain name, or default "lb-apiserver.kubernetes.local", will be inserted -into the `/etc/hosts` file of all servers in the `k8s-cluster` group. Note that +into the `/etc/hosts` file of all servers in the `k8s-cluster` group and wired +into the generated self-signed TLS/SSL certificates as well. Note that the HAProxy service should as well be HA and requires a VIP management, which -is out of scope of this doc. Specifying an external LB overrides any internal -localhost LB configuration. +is out of scope of this doc. + +There is a special case for an internal and an externally configured (not with +Kubespray) LB used simultaneously. Keep in mind that the cluster is not aware +of such an external LB and you need no to specify any configuration variables +for it. + + Note: TLS/SSL termination for externally accessed API endpoints' will **not** + be covered by Kubespray for that case. Make sure your external LB provides it. + Alternatively you may specify an externally load balanced VIPs in the + `supplementary_addresses_in_ssl_keys` list. Then, kubespray will add them into + the generated cluster certifactes as well. - Note: In order to achieve HA for HAProxy instances, those must be running on - the each node in the `k8s-cluster` group as well, but require no VIP, thus - no VIP management. +Aside of that specific case, the `loadbalancer_apiserver` considered mutually +exclusive to `loadbalancer_apiserver_localhost`. Access API endpoints are evaluated automagically, as the following: -| Endpoint type | kube-master | non-master | -|------------------------------|----------------|---------------------| -| Local LB (default) | https://lc:sp | https://lc:nsp | -| External LB, no internal | https://lb:lp | https://lb:lp | -| No ext/int LB, bind 0.0.0.0 | https://lc:sp | https://m[0].aip:sp | -| No ext/int LB, a custom bind | https://bip:sp | https://m[0].aip:sp | +| Endpoint type | kube-master | non-master | external | +|------------------------------|----------------|---------------------|---------------------| +| Local LB (default) | https://bip:sp | https://lc:nsp | https://m[0].aip:sp | +| Local LB + Unmanaged here LB | https://bip:sp | https://lc:nsp | https://ext | +| External LB, no internal | https://bip:sp | https://lb:lp | https://lb:lp | +| No ext/int LB | https://bip:sp | https://m[0].aip:sp | https://m[0].aip:sp | Where: * `m[0]` - the first node in the `kube-master` group; * `lb` - LB FQDN, `apiserver_loadbalancer_domain_name`; +* `ext` - Externally load balanced VIP:port and FQDN, not managed by Kubespray; * `lc` - localhost; -* `bip` - a custom bind IP value (defaults to '0.0.0.0'); +* `bip` - a custom bind IP or localhost for the default bind IP '0.0.0.0'; * `nsp` - nginx secure port, `nginx_kube_apiserver_port`, defers to `sp`; * `sp` - secure port, `kube_apiserver_port`; * `lp` - LB port, `loadbalancer_apiserver.port`, defers to the secure port; * `ip` - the node IP, defers to the ansible IP; * `aip` - `access_ip`, defers to the ip. +A second and a third column represent internal cluster access modes. The last +column illustrates an example URI to access the cluster APIs externally. +Kubespray has nothing to do with it, this is informational only. + +As you can see, the masters' internal API endpoints are always +contacted via the local bind IP, which is `https://bip:sp`. + **Note** that for some cases, like healthchecks of applications deployed by Kubespray, the masters' APIs are accessed via the insecure endpoint, which consists of the local `kube_apiserver_insecure_bind_address` and -- GitLab