an 'until' clause is required or 'retries' is ignored

(see note @ https://docs.ansible.com/ansible/latest/user_guide/playbooks_loops.html#do-until-loops)

* robust handling of API server SANs

* use apiserver_loadbalancer_domain_name if it is defined, according to PR 3977

* enable kubelet client certificate rotation

* change to variable kubelet_rotate_certificates

Both kubedns and dnsmasq modes are long not maintained.
We should run dns_late steps at the end because sshd
makes DNS lookups during Ansible run and has 2s timeouts
for each failed lookup trying to connect to coredns before
it is ready.

* Added configurable local apiserver proxy liveness probe

* Enable API LB healthcheck by default

* Fix template spacing and moved healthz location to nginx http section

* Fix healthcheck listen address to allow kubelet request healthcheck

This reverts commit ea7a6f1c.

This change modified the certs dir for Kubernetes, but did not move the directories for existing clusters.

Pods on down/unresponsive nodes can't be deleted without
--force --grace-period=0.

Fixes #4314

* Remove hard dependence on facts for all nodes

* Update main.yaml

* Update main.yaml

* fact gathering should run only once per node
* eliminate ansible version check, it is at the beginning of each
  playbook

This was already approved in #4106 but there are CI issues
with that PR due to references to kubernetes incubator.

After upgrading to Kubespray 2.8.1 with Kubeadm enabled Rook
Ceph volume provision failed due to the flexvolume plugin dir not
being correct. Adding the var fixed the issue

because of etc-kubernetes-manifests not empty.

* Adding ability to maintain existing Encryption Secrets at Rest.

If secrets_encryption.yaml is present it will not be overriten with a new kube_encrypt_token.

This should allow for it to be set ahead of a playbook running or maintain it if cluster.yml is ran on the same cluster and the ansible host does not have access to the secrets.

* Setting existing kube_encrypt_token across all master nodes in case it was missing in one or more nodes.

Fix issue where `kubeadm join` could wait forever for joining.

Fix issue where `kubeadm join` were not reaching the user, making
impossible to find the cause of the failure.

New behaviour is to first attempt to join without bypassing the
verifications checks and to display them if needed.

If this fails it still attempts to join by ignoring the check in
order to make previous behavior.

A timeout of 60 seconds is allocated for a joining.

Related-bug: #3973

check kube_pods_subnet and kube_service_addresses to valid ip network range, not single ip address (#4188)

This PR ensures that the e2fsprogs and xfsprogs packages are
installed on all Kubernetes nodes and that the packages are
the latest versions. It also ensures that the nodes can
create XFS filesystems when necessary, since not all distros
install xfsprogs by default.

e2fsprogs - ext2/ext3/ext4 file system utilities
xfsprogs - Utilities for managing the XFS filesystem

Avoid waiting forever for this task that should be very quick.

Fixes: #4148

* use not deprecated ansible_play_hosts variable

* Using tests as filters is deprecated

* Fix deprecation warning about pkg list

* Set cluster DNS correctly in case of nodelocal dns cache

* Pass in cluster_ip based on dns mode

* Disable nodelocaldns by default

* Fix syntax error

* Fix syntax issue

* Add nodelocadns ip to vars of node installation

* Change location of nodelocaldns_ip

* Try to remove newlines from jinja template

* Add debug for config file

* Move parameter logic outside of template

* Adapt templates after feedback

* Remove debugging

* Suport calico 3.4.0

Signed-off-by: wangxf1987 <xiaofeix.wang@gmail.com>

* Remove symlink + cni conflist template when 3.3.0+, handle Canal, addition of install-cni: sidecar(3.3.0) or initontainer(3.4.0), KUBECONFIG_FILEPATH, calico_cert_dir, advertise cluster ips

* scheduler.alpha.kubernetes.io/critical-pod deprecated since 1.12

Addressing the discussion started in #4064, this PR moves kubeadm and
hyperkube binaries to /usr/local/bin before running them on the master
nodes.

It is to address the case where local_release_dir points to /tmp
(kubespray default) and /tmp is mounted with noexec mode, preventing
any binaries to be run in that partition.

In role "node", we still move kubeadm to bin_dir only on the worker
nodes.