* Fix: adding new ips with inventory builder (#7577)

* moved conflig loading logic
to after checking whether the config
should be loaded, and added check for
whether the config should be loaded

* added check for removing nodes from config
if the user wants to remove a node, we
need to load the config

* Fix tox errors

The main functions are wrapped by a sys.exit function which expects and
argument. The curent implementation isn't returning values in all cases.
This change ensures main functions return a value in all cases.

Previously IDs of container images were gotten from tar files of container
images but that way was wrong. If multiple json files are contained in a
tar file, the script got multiple IDs and tried to pass these IDs on
`docker tag` command. Then the command was failed.

This updates the script to get image IDs from `docker image inspect` command
to fix this issue.
In addition, this adds a check a registry container exists already or not
before deploying registry container to avoid a container conflict failure.

* fix(misc): terraform/aws

- handles deployment with a single availability zone
- handles deployment with more than two availability zone
- handles etcd collocation with control-plane nodes (`aws_etcd_num=0`)
- allows to set a bastion instances count (`aws_bastion_num`)
- allows to set bastion/etcd/control-plane/workers rootfs volume size
- removes variables from terraform.tfvars that were not re-used
- adds .terraform.lock.hcl to .gitignore
- changes/updates base image from ubuntu-18.03 to debian-10

tested by a few coworkers of mine, and myself: thanks for the outstanding
work, on both those terraform samples and kubespray playbooks.
I did not test ubuntu deployments, I could still swap from buster to
focal. LMK.

* fix(gitlab-ci)

AFAIU, terraform.tfvars indentation should be fixed for / no diff
returned running `terraform fmt -check -diff`

https://gitlab.com/kargo-ci/kubernetes-sigs-kubespray/-/jobs/1445622114

When running the script, I faced the following error but it was
difficult to know the root problem due to lack of error handling.

  docker tag" requires exactly 2 arguments.
  See 'docker tag --help'.

  Usage:  docker tag SOURCE_IMAGE[:TAG] TARGET_IMAGE[:TAG]

  Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE

To investigate such errors easily, this adds an error handling.

Update README for control_plane's external volume type variable

* Ansible: move to Ansible 3.4.0 which uses ansible-base 2.10.10

* Docs: add a note about ansible upgrade post 2.9.x

* CI: ensure ansible is removed before ansible 3.x is installed to avoid pip failures

* Ansible: use newer ansible-lint

* Fix ansible-lint 5.0.11 found issues

* syntax issues
* risky-file-permissions
* var-naming
* role-name
* molecule tests

* Mitogen: use 0.3.0rc1 which adds support for ansible 2.10+

* Pin ansible-base to 2.10.11 to get package fix on RHEL8

Clean up extra spaces, although these errors are not important, they affect the code specification.

https://github.com/kubernetes-incubator/kubespray is an old link,
this updates the link.

I recently reviewed the code, although these errors are not important, they affect the code specification.

* terraform/openstack: Use path.root for ansible_bastion_template.txt

The path.root variable points to the root module path. Using this
instead of a relative path makes less assumptions about the current
working directory.

* terraform/openstack: Add group_vars_path variable

Previously, the group_vars path was assumed to be in CWD. The
default value for the group_vars_path variable is still relative
to CWD and thus should be backwards compatible if unset.

* Packet->Equinix Metal rename #6901 

Updates throughout to reflect #6901 renaming for Packet to Equinix Metal.

* Rename Packet to Equinix Metal throughout the project #6901

Packet is renamed to Equinix Metal in more contexts including
documentation links. The Terraform provider used is still the Packet
provider. The environment variables and configuration options still
refer to the Packet name.

Signed-off-by: Marques Johansson <mjohansson@equinix.com>

Co-authored-by: Edward Vielmetti <ed@packet.net>

Basically we need to make necessary TCP/UDP ports open.
However the necessary ports are so many, and sometimes it is difficult
to figure out that is due to firewall issues or not if facing deployment
issues.
To distinguish a root problem on such situation, this adds contrib
playbook to disable the service firewall for Kubespray development
and test.

* Replace map in Terraform scripts with tomap (#7576)

* Fix Terraform linter warnings (#7576)

Fix coredns image repo and tag typo for #7570

* rename ansible groups to use _ instead of -

k8s-cluster -> k8s_cluster
k8s-node -> k8s_node
calico-rr -> calico_rr
no-floating -> no_floating

Note: kube-node,k8s-cluster groups in upgrade CI
      need clean-up after v2.16 is tagged

* ensure old groups are mapped to the new ones

ref: https://opendev.org/openstack/hacking/commit/b921c4de513c9cc624d6ecf68e4f4493e6e72c0d

Context: Load-balancing in Exoscale is performed by associating many
workers with the same EIP. This works, however, the workers cannot access
themselves via the EIP, which is needed at least for cert-managers
"self-test".

Problem: The old iptables based workaround felt fragile and disappointed
me at least once.

New solution: Add the EIP to a loopback interface on each worker.

* Remove contrib/vault

This is marked as broken since 2018 / 3dcb9146
This still reference apiserver.pem, not used since ddffdb63

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>

* Finish nuking vault from the codebase

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>

This replaces kube-master with kube_control_plane because of [1]:

  The Kubernetes project is moving away from wording that is
  considered offensive. A new working group WG Naming was created
  to track this work, and the word "master" was declared as offensive.
  A proposal was formalized for replacing the word "master" with
  "control plane". This means it should be removed from source code,
  documentation, and user-facing configuration from Kubernetes and
  its sub-projects.

NOTE: The reason why this changes it to kube_control_plane not
      kube-control-plane is for valid group names on ansible.

[1]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-cluster-lifecycle/kubeadm/2067-rename-master-label-taint/README.md#motivation

* terraform support for UpCloud

* terraform support for UpCloud

* terraform support for UpCloud

* terraform support for UpCloud

* terraform support for UpCloud

* terraform support for UpCloud

* terraform support for UpCloud

* Updates to README.md and main.tf files

* formatting and updating readme

* added a .terraform_validate CI job

* fixed format issue

* added sample inventory

* added symbolic link to group_vars

* added missing tf variables and minor fixes

* added text formatting

* minor formatting fixes

The default master size for exoscale is 2cpu and 2GB ram.
I have found this to be too low, so this increases it to
2cpu and 4GB ram.

* Add terraform scripts for vSphere

* Fixup: Add terraform scripts for vSphere

* Add inventory generation

* Use machines var to provide IPs

* Add README file

* Add default.tfvars file

* Fix newlines at the end of files

* Remove master.count and worker.count variables

* Fixup cloud-init formatting

* Fixes after initial review

* Add warning about disabled DHCP

* Fixes after second review

* Add sample-inventory

This replaces KUBE_MASTERS with KUBE_CONTROL_HOSTS because of [1]:

```
  The Kubernetes project is moving away from wording that is
  considered offensive. A new working group WG Naming was created
  to track this work, and the word "master" was declared as offensive.
  A proposal was formalized for replacing the word "master" with
  "control plane". This means it should be removed from source code,
  documentation, and user-facing configuration from Kubernetes and
  its sub-projects.
```

[1]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-cluster-lifecycle/kubeadm/2067-rename-master-label-taint/README.md#motivation

allow users to set image_uuid instead of name, this allows the use of openstack community images (#7283)

* contrib/terraform/exoscale: Rework SSH public keys

Exoscale has a few limitations with `exoscale_ssh_keypair` resources.
Creating several clusters with these scripts may lead to an error like:

```
Error: API error ParamError 431 (InvalidParameterValueException 4350): The key pair "lj-sc-ssh-key" already has this fingerprint
```

This patch reworks handling of SSH public keys. Specifically, we rely on
the more cloud-agnostic way of configuring SSH public keys via
`cloud-init`.

* contrib/terraform/exoscale: terraform fmt

* contrib/terraform/exoscale: Add terraform validate

* contrib/terraform/exoscale: Inline public SSH keys

The Terraform scripts need to install some SSH key, so that Kubespray
(i.e., the "Ansible part") can take over. Initially, we pointed the
Terraform scripts to `~/.ssh/id_rsa.pub`. This proved to be suboptimal:
Operators sharing responbility for a cluster risk unnecessarily replacing resources.

Therefore, it has been determined that it's best to inline the public
SSH keys. The chosen variable `ssh_public_keys` provides some uniformity
with `contrib/azurerm`.

* Fix Terraform Exoscale test

* Fix Terraform 0.14 test

* Added terraform support for Exoscale

* Fixed markdown lint error on exoscale terraform

Implemented in #6547

This variable was added as KUBE_MASTERS_MASTERS. That's probably a typo.
Remove the redundant `_MASTERS` suffix. Also, document the variable in the
help message.

This fixes the following failures:

./contrib/offline/README.md:14:1 MD014/commands-show-output Dollar signs used before commands without showing output [Context: "$ ./manage-offline-container-i..."]
./contrib/offline/README.md:20:1 MD014/commands-show-output Dollar signs used before commands without showing output [Context: "$ ./manage-offline-container-i..."]

One challenge of offline deployment was how to collect necessary
container images as a preparation. This adds a script to solve it.

This fixes markdownlint failures under roles/

This fixes markdown failures on contrib/terraform.