The new generate_test_data.yml is creating the test data with
ansible.builtin.shell and is not calling shell scripts any more. The
generation in the yml file and also the set_fact calls make sure that
the test data is used.

Most of the content has been moved to the new function _run_playbook to
reduce the traceback output in the case of a test failure.

This patch removes 'molecule' as a dependency for tests, by using the
scripts under `utils` to setup the environment.

By not using molecule, we have more flexibility on using either docker
or podman as the container engine, and makes it easy to reproduce the
environment on different distros, allowing for a more consistent error
reproduction off Azure.

On some systems it is required or desired to run Ansible with a specific
Python interpreter. This patch allows the selection of the Python binary
to use for the pytest playbook tests by setting the environment variable
IPA_PYTHON_PATH. Set it the the full path of the Python interpreter.

As we do not use molecule features, using a Dockerfile and the
ansible-freeipa deployment roles is enough to create the container
testing images.

This patch removes the usage of molecule in favor of the custom
ansible-freeipa image building script, which allow us to have a similar
process for creating images both on the ustream CI, or on a developer's
environment.

Also, CentOS 7 is removed from the build script, as it in not possible
to run CentOS 7 containers with current versions of systemd.

The new infra/image/build.sh script is used instead of molecule.

The tests test_users_present.yml and test_users_absent.yml has been
merged into test_users_present_absent.yml to fix the independent test
order execution.

The test test_users_present_slice.yml has been renamed to
test_users_present_absent_slice.yml.

Both tests are cleaning up possible exiting entries before running the tests
and are verifying the present and absent task result.

The cleanup of the test services and hosts have been missing, which
could lead to a test failure.

Truncate stdout and stderr in the way that it hopefully shows all
important information. At least 15 lines of stdout (Ansible tasks) and
remaining from stderr to fill up to maxlen size.

The test log mostly only showed lines from stderr as the log was a
concatenation of stdout and stderr and only the last 2500 chars of this
have been used. This was hiding the test result from stdout and failures
have not beeen visible.

setuptools might not be installed before importing and using
galaxy_importer. This could result in a backtrace by disabling
ANSIBLE_TEST_LOCAL_IMAGE in galaxy-importer.cfg to run latest tests.

The parameters

- subtree (ipapermlocation)
- target (ipapermtarget)
- targetto (ipapermtargetto)
- targetfrom (ipapermtargetfrom)

have not been idempotent as the result returned from permission_show was
a DN and not a string.

The find_permission function has been exetended to convert the values
for these parameters to strings.

Fixes: #1257

The generated certificates have been X.509 v1. This is not supported any
more. Only X.509 v3 is supported.

A new certificates/extensions.conf file has been added to make v3
certificates.

The existing certificates/pkinit/extensions.conf has been renamed to
certificates/pkinit-extensions.conf with additional changes. For example
"[kdc_cert]" had to be removed for v3.

The extensions config files are using environment variables, which are
set by the generate-certificates.sh script before calling openssl.

The script generate-certificates.sh has been reworked for a simpler
structure, also new options have been added: "ca" and "cleanup".

This patch force processing of permission, attribute and group
attributes in lower case, to match behavior of IPA CLI, transforming
all of them into lowercase characters.

The new behavior fixes idempotence issues when mixing different
capitalization in different tasks for the same attribute.

A new test playbook is avaiable at:

    tests/delegation/test_delegation_member_case_insensitive.yml

Some attributes for ipagroup objects are stored using lower case letters
and should be converted upon retrieving parameter data.

This patch adds the missing conversion and provides a new test playbook:

    tests/group/test_group_case_insensitive.yml

ipahostgroup parameters 'host', 'hostgroup', 'membermanager_user' and
'membermanager_group' must be compared in a case insensitive manner
and stored as lower case strings.

This patch fixes the comparison and storage of this parameters, and
change the handling of members to use the same structure as in newer
modules.

Two new tests files were added:

    tests/hostgroup/test_hostgroup_case_insensitive.yml
    tests/hostgroup/test_hostgroup_membermanager_case_insensitive.yml

The config tests are currently setting the pac type to empty or without
MS-PAC type. This results in failed authorization for IPA API.

An issue has been opened for FreeIPA to address this:
https://pagure.io/freeipa/issue/9527

FreeIPA provides a default hbacsvcgroup named "Sudo", with capital 'S',
that is different from every other hbacsvcgroup, which are all
represented by lower case letters.

As data from IPA API was not modified, this causes an idempotence error
when using different capitalization with the 'hbacsvcgroup' parameter.

This patch fixes the issue by using the CaseInsensitive comparator to
create the hbacsvcgroup list.

Tests were update to make sure a regression is not included in the
future.

On IPA CLI sudorule-add/del-runasuser accept 'group' as a parameter,
and this option was missing in ansible-freeipa ipasudorule module.

This patch adds a new parameter 'runasuser_group' to allow setting
Groups of RunAs Users, as allowed by CLI and WebUI.

New example playboks can be found at:

    playbooks/sudorule/ensure-sudorule-runasusesr-group-is-absent.yml
    playbooks/sudorule/ensure-sudorule-runasusesr-group-is-present.yml

FreeIPA suports renaming groupobjects with the CLI parameter "rename",
and this parameter was missing in ansible-freeipa ipagroup module.

This patch adds support for a new state 'renamed' and the 'rename'
parameter.

Tests were updated to cope with the changes.

- Replace outdated noqa 503 with noqa no-handler
- Drop outdated and not needed noqa 505 for include_vars
- Drop outdated noqa deprecated-command-syntax for
  ansible.builtin.shell using cmd tag

These warnings have been reported by utils/lint_check.sh using
ansible-lint 6.22.1.

Use module_defaults to improve reading test cases.

FreeIPA suports renaming user objects with the CLI parameter "rename",
and this parameter was missing in ansible-freeipa ipauser module.

This patch adds support for a new state 'renamed' and the 'rename'
parameter.

Tests were updated to cope with the changes.

Related to RHBZ#2234379, RHBZ#2234380

Fixes #1103

IPA DNS Zones management can be delegated by adding a "Manage DNS zone"
permission. The CLI commands that manage these permissions are
dnszone-add-permission and dnszone-remove-permission.

The ansible-freeipa module ipadnszone did not have this capability, and
it now support dnszone per-zone management delegation by setting the
module parameter 'permission'. If set to 'true' the permission will be
assigned to the zone, if set to false the permission will be removed.

This reverts commit de3c6c0a.

After the change for a single job to run PR tests, and if there is any
change to ansible_module_utils, all the playbook tests are executed,
and the result is a failure due to timeout.

This PR increases the timeout so that a PR with changes to
ansible_module_utils can have the tests executed.

With ansible-core 2.14.12 using jinja2 templating in conditional
statements with ansible_facts['fqdn'] is marked as unsafe and results in
a failure.

The issues with using jinja2 templating in conditional statements for
asserts have been solved and a new server_fqdn fact has been added for
ansible_facts['fqdn'].

The uri parameters auth_uri, dev_auth_uri, token_uri, userinfo_uri and
keys_uri have not been validated before. Also the base_url was not
normalized. The auth_uri, dev_auth_uri, token_uri and userinfo_uri need
to be set for new entries, but might be empty or empty string for reset
or updates.

The ipaidpclientsecret needs to be decoded from binary string in
find_idp result to not trigger no change ipd_mod calls.

The code for validate_uri and base_url normalization has been copied
from the ipaserver idp plugin.

ansible_freeipa_module:
urlparse from urllib.parse with a fallback to six.moves.urllib.parse is
imported and also exported. urlparse is needed for validate_uri in ipaidp
module.

Resolves: RHEL-17954, RHEL-17955, RHEL-17957 and RHEL-17958

If a playbook to ensure the existence of a user contains 'random:false'
and 'update_password: always' is executed twice, the second execution
will raise an exception due to "No modifications to perform", as there
is actually nothing to modify.

The fix for the issue is to remove 'random' if it is not set to true, as
setting it to 'false' would have no effect on the user object.

Related: https://issues.redhat.com/browse/RHEL-4934

The reset of the minlength parameter failed with an internal error so
far. This has been fixed in IPA and therefore requires to fix the test
in ansible-freeipa also.

Related: https://pagure.io/freeipa/issue/9297

All tasks for idoverrideuser and idoverridegroup with state absent
failed with "'continue' is required" when delete_continue was not set.

This happended as delete_continue was internally None and continue: None
was provided to the API.

The fix is simply to use '"continue": delete_continue or False' so that
continue is set to False in this case.

When testing ipahost through the test playbooks, if there are previous
DNS A/AAAA records, the test fails due to a false positive idempotence
issue.

This patch ensures that all DNS records for the test hosts are absent
before test execution.

This issue could be seen in the 2023-11-06 Azure Nightly pipeline
execution.

The task to apply an unknown (invalid) host to an idview was not failing
as expected and only reported no change.

A new host verification step has been added to fail before trying to
apply invalid hosts. unapplying an invalid host is not failing as the
invalid host is indeed not applied.

hbacsvcgroup names are converted to lower case while creation with
hbacsvcgroup_add.

The hbacsvcgroup for sudo is builtin with the name "Sudo" though. This
breaks the lower case comparison. Therefore all memberservice_hbacsvcgroup
items are converted to lower case if "Sudo" is in the list.

In the current build container pipeline, all steps are serialized in a
single job, and if one of the jobs fail to build, due to broken
dependent image, or some Azure glitch, like slow connection, the only
way to rebuild the failed container is to rebuild all containers.

By building containers in parallel jobs, if a container fails to build
it is possible to restart only the failed job.

The usual scenario for PR checks is to execute only a few tests, and
searching for the results in several jobs makes it harder to find
issues.

By using a single job run the tests would take some more time to
complete, although not much, as only a small subset is executed, and
test verification would be easier and less error prone.

Dependind on how long it took for the jobs to start, a different seed
would be used to group tests and tests could either repeat or not be
selected at all.

By using a seed based on the day the test run reduces the chance of
using different random seeds, and still allow for the tests to be
executed in a different order.

The execution in different order is important to identify tests that
work or fail only if executed after other tests.

Azure Ubuntu images have Python 3.12 available, and as we did not pin
the requested Python version, the latest available one was used, causing
image preparation and tests to fail.

This patch pins Python version to 3.11 until test can be executed with
Python 3.12 and later.