Reproduce upstream CI groups in developer's machine

upstream CI: Build containers in parallel jobs

Update ansible-lint and pylint versions

ipahost: Remove dangling dns records during test setup

idoverride{user,group}: Fix delete_continue with state absent

All tasks for idoverrideuser and idoverridegroup with state absent
failed with "'continue' is required" when delete_continue was not set.

This happended as delete_continue was internally None and continue: None
was provided to the API.

The fix is simply to use '"continue": delete_continue or False' so that
continue is set to False in this case.

When testing ipahost through the test playbooks, if there are previous
DNS A/AAAA records, the test fails due to a false positive idempotence
issue.

This patch ensures that all DNS records for the test hosts are absent
before test execution.

This issue could be seen in the 2023-11-06 Azure Nightly pipeline
execution.

When running ansible-freeipa's Azure pipelines for nightly and weekly
tests, due to the amount of tests to execute, tests are grouped and
executed in parallel jobs.

Due to a still unkonwn issue, depending on the order the tests are
executed, some random failures may occur and debugging them is hard due
to current implementation of the tests.

This patch adds support for replicating the tests of a specific Azure
test group once the seed used to create groups and the group number are
provided, allowing the test failures to be replicated on the developer's
workstation where it can be more easily debugged.

A new option is added to 'utils/run-tests.sh', '-A SEED.G' that is used
to define the seed and group to replicate the tests. The seed is a date,
with the format "YYYYMMDD", so, for example '-A 20230611.2' would
execute the same tests, in the same order as the second group of tests
for date 2023-06-11. To aid in usability 'YYYY-MM-DD' may also be used.

When using '-A' neither '-s' (test suites) or specific tests (positional
arguments) can be used.

Also, to help fixing tests, an option to stop the tests on the first
test failure ('-x') was added to the script.

ansible-lint 6.21+ and pylint 3.0+ will be required for Ansible
collections to be approved on Ansible Galaxy.

This patch updates pre-commit and upstream linters to use the required
versions.

ipaidview: Fail to apply unknown (invalid) hosts

hbacsvcgroup: Remove obsolete result_handler

hbacrule: Fix use of builtin sudo hbacsvcgroup

The task to apply an unknown (invalid) host to an idview was not failing
as expected and only reported no change.

A new host verification step has been added to fail before trying to
apply invalid hosts. unapplying an invalid host is not failing as the
invalid host is indeed not applied.

The result_handler is not needed anymore as the idempotency issues with
members have been fixed already for this module.

Related: #685 hbacsvcgroup: Fix member management idempotence issues.

hbacsvcgroup names are converted to lower case while creation with
hbacsvcgroup_add.

The hbacsvcgroup for sudo is builtin with the name "Sudo" though. This
breaks the lower case comparison. Therefore all memberservice_hbacsvcgroup
items are converted to lower case if "Sudo" is in the list.

In the current build container pipeline, all steps are serialized in a
single job, and if one of the jobs fail to build, due to broken
dependent image, or some Azure glitch, like slow connection, the only
way to rebuild the failed container is to rebuild all containers.

By building containers in parallel jobs, if a container fails to build
it is possible to restart only the failed job.

upstream CI: Fix test selection for CheckPR pipeline.

The usual scenario for PR checks is to execute only a few tests, and
searching for the results in several jobs makes it harder to find
issues.

By using a single job run the tests would take some more time to
complete, although not much, as only a small subset is executed, and
test verification would be easier and less error prone.

Dependind on how long it took for the jobs to start, a different seed
would be used to group tests and tests could either repeat or not be
selected at all.

By using a seed based on the day the test run reduces the chance of
using different random seeds, and still allow for the tests to be
executed in a different order.

The execution in different order is important to identify tests that
work or fail only if executed after other tests.

Due to an error on processing Ansible key 'import_tasks' the script that
creates a list of modules to test is broken making some modules to be
not tested.

By fixing the handling of 'import_tasks' and module import, the list is
correct again and the list of modules to be tested now include the ones
which depend on the modified module.

upstream CI: Pin ansible-lint version to 6.20 series

The release version 6.21.0 of ansible-lint introduced a bug that breaks
the reporting of 'warning' messages. [1]

This patch pins ansible-lint version to the latest one in the 6.20
series, so that it can still be used to check pull requests.

[1]: https://github.com/ansible/ansible-lint/issues/3853

upstream CI: Pin Python version to 3.11 

utils/ansible-freeipa.spec.in: Add ref for idoverridegroup management

Azure Ubuntu images have Python 3.12 available, and as we did not pin
the requested Python version, the latest available one was used, causing
image preparation and tests to fail.

This patch pins Python version to 3.11 until test can be executed with
Python 3.12 and later.

New idp management module

The idoverridegroup management reference has been added to the
description.

There is a new idp management module placed in the plugins folder:

    plugins/modules/ipaidp.py

The idp module allows to ensure presence or absence of external Identity
Providers.

Here is the documentation for the module:

    README-idp.md

New idp example playbooks:

    playbooks/idp/idp-present.yml
    playbooks/idp/idp-absent.yml

New tests for the module:

    tests/idp/test_idp.yml
    tests/idp/test_idp_client_context.yml

Revert "upstream ci: Run nightly tests against Ansible 2.9"

Most of our usptream CI test imagens do not handle Ansible 2.9 so, this
cange is being reverted.

This reverts commit 34654d10.

Ensure CI runs against the oldest supported Ansible versions.

Bump linter versions.

Recently it was announced that Ansible 2.9 will be supported for some
time, and this patch ensures that we run the nightly tests against this
version of Ansible.

Recently, a change in the deployment roles forced the change to the
minimum version of ansible-core, and the change was unnoticed until
reported.

With this patch, we ensure all PRs checks are executed against the
minimun supported ansible-core version, so we can ensure that both
documentation and role metadata are correct and still valid.

New idoverridegroup management module.

There is a new idoverridegroup management module placed in the plugins
folder:

    plugins/modules/ipaidoverridegroup.py

The idoverridegroup module allows to ensure presence and absence of
idoverrides for groups.

Here is the documentation for the module:

    README-idoverridegroup.md

New example playbooks have been added:

    playbooks/idoverridegroup/idoverridegroup-absent.yml
    playbooks/idoverridegroup/idoverridegroup-present.yml

New tests for the module can be found at:

    tests/idoverridegroup/test_idoverridegroup.yml
    tests/idoverridegroup/test_idoverridegroup_client_context.yml

New idoverrideuser management module.

There is a new idoverrideuser management module placed in the plugins
folder:

    plugins/modules/ipaidoverrideuser.py

The idoverrideuser module allows to ensure presence and absence of
idoverrides for users and certificate members.

Here is the documentation for the module:

    README-idoverrideuser.md

New example playbooks have been added:

    playbooks/idoverrideuser/idoverrideuser-absent.yml
    playbooks/idoverrideuser/idoverrideuser-certificate-absent.yml
    playbooks/idoverrideuser/idoverrideuser-certificate-present.yml
    playbooks/idoverrideuser/idoverrideuser-present.yml

New tests for the module can be found at:

    tests/idoverrideuser/test_idoverrideuser.yml
    tests/idoverrideuser/test_idoverrideuser_client_context.yml

Do not use "del os.environ" as the variable might not exist

The use of del os.environ assumes that the environment variable exists.
If the variable does not exist, this call will result in a traceback.
The solution is to use os.environ.pop(VARIABLE, None) instead.

This is the ansible-freeipa fix for https://pagure.io/freeipa/issue/9446
(Nightly test failure for replica installation with --setup-ca)