Due to Ansible filtering out values in the output that might be match
values in sensible attributes that have `no_log` set, if a module need
to return data to the controller, it cannot rely on
`ansible_module.exit_json` if there is a chance that a partial match
may occur.

See: https://github.com/ansible/ansible/issues/71789

The change provided here uses the same implementation that is used on
Ansible's `AnsibleModule.exit_json`, without the data filtering layer,
so every attribute with be printed and, therefore, logged by Ansible.

This is needed for the Vault module, as we need to return values that
are explicit requested by the user and that might, at least partially,
match the values in attributes with `no_log` set.

Tests that reproduced the issue, and show it was fixed were provided
for all Vault types.

Modified support for processing result of IPA API commands so that
client code can define its own processing and add return values to
self.exit_args based on command result.

If a subclass need to process the result of IPA API commands it should
override the method `process_command_result`. The default implementation
will simply evaluate if `changed` should be true.

This PR allow ipadnszone module to ensure that multiple dns zones
are absent at once, to be consistent with other ansible-freeipa
modules.

To fix this issue, it was required that custom arguents must be
passed using keyword arguments so that `get_ipa_command_args()`
is kept generic.

When a fail_json is called a SystemExit exeception is raised.
Since the FreeIPABaseModule has an internal context manager to deal
with exceptions this ContextManager captures the SystemExit. After
dealing destroying the kinit session the SystemExit must be raised again
to allow the fail_json to work properly.

This function can be used to check if a command is available in the API.

This is used in ipauser module to check if user_add_certmapdata is available
in the API.

This function is needed to properly convert issuer and subject from a
certificate or the issuer and subject parameters in ipauser for certmapdata
to the data representation where the items in DN are reversed.

The function additionally provides a fallback solution for IPA < 4.5.
Certmapdata is not supported for IPA < 4.5, but the conversion is done
before the API version can be checked.

For certmapdata processing in ipauser it is needed to be able to load a cert
from a string given in the task to be able to get the issuer and subject of
the certificate. The format of the certifiacte here is lacking the markers
for the begin and end of the certificate. Therefore load_pem_x509_certificate
can not be used directly. Also in IPA < 4.5 it is needed to load the
certificate with load_certificate instead of load_pem_x509_certificate. The
function is implementing this properly.

Because of a missing check member attributes (for use with action: member)
are cleared when a non-member attribute is changed. The fix simply adds a
check for None (parameter not set) to gen_add_del_lists in
ansible_freeipa_module to make sure that the parameter is only changed if
it should be changed.

All places where the add and removal lists have been generated manually
have been changed to also use gen_add_del_lists.

Resolves: #252 (The "Manager" attribute is removed when updating any user
                attribute)

In the case that the admin password has been set and become was not set
the call to backend.connect in api_connect failed. The solution is simply
to set os.environ["KRB5CCNAME"] in temp_kinit after kinit_password has
been called using the temporary ccache. os.environ["KRB5CCNAME"] is not
used automatically by api.Backend.[ldap2,rpcclient].connect. Afterwards
os.environ["KRB5CCNAME"] is unset in temp_kdestroy if ccache_name is not
None.

Fixes: #249 (Kerberos errors while using the modules with a non-sudoer user)

When accessing an instance of AnsibleFreeIPAParams with .get the obj was
by-passing the call to _afm_convert which was the primaty reason why it
was created.

Also the class now extends Mapping instead of dict.

For beeing able to catch ipalib.errors.NotFound errors in ipahost it is
needed to import ipalib.errors. ipalib.errors is now imported as
ipalib_errors to not have name conflicts with the errors list used in some
of the modules.

Related: https://bugzilla.redhat.com/show_bug.cgi?id=1804838

This change is needed to properly handle base64 encoding of certificates
stored as bytes, under Python 3, as used by IPA service. It does not
affect Python 2.7 as bytes are identical to str in this version of the
language.

When retireving certificates stored by FreeIPA service data is returned
as bytes, under Python 3, and encoding then breaks, as there is no
bytes.public_bytes method. In Python 3, encoding with base64 will be the
same for strings and bytes.

ipahost was so far ignoring IP addresses when the host already existed.
This happened because host_mod is not providing functionality to do this.
Now ipaddress is a list and it is possible to ensure a host with several
IP addresses (these can be IPv4 and IPv6). Also it is possible to ensure
presence and absence of IP addresses for an exising host using action
member.

There are no IP address conclict checks as this would lead into issues with
updating an existing host that already is using a duplicate IP address for
example for round-robin (RR). Also this might lead into issues with ensuring
a new host with several IP addresses in this case. Also to ensure a list of
hosts with changing the IP address of one host to another in the list would
result in issues here.

New example playbooks have been added:

    playbooks/host/host-present-with-several-ip-addresses.yml
    playbooks/host/host-member-ipaddresses-absent.yml
    playbooks/host/host-member-ipaddresses-present.yml

A new test has been added for verification:

    tests/host/test_host_ipaddresses.yml

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1783976
       https://bugzilla.redhat.com/show_bug.cgi?id=1783979

Bool types are not iterable. Therefore the comparison using sets was failing
with a TypeError. This prevented to change the bool parameters for hosts.

A test for the host module has been added to verify that the bool parameters
can be modified.

New test:

  tests/host/test_host_bool_params.yml

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1784514

There are some commands in the IPA API that do not require
arguments, and current implementation does not allow these
commands to be execute.

This patch adds api_command_no_name to allow the execution
of such commands, which is required, for example, to create
a vaultcontainer management module.

The ipahost management module was not able to add several hosts at once.
Addtionally there have been settings missing.

ansible_freeipa_module has been extended to provide two additional functions
that are needed to simplify the extension of the ipahost module:

    gen_add_del_lists(user_list, res_list)
    encode_certificate(cert)

gen_add_del_lists will generate the lists for the addition and removal of
members using the provided user and ipa settings.

encode_certificate will encode a certificate using base64 with also taking
FreeIPA and Python versions into account.

The missing settings in ipahost have been:

    certificate
    managedby_host
    principal
    create_keytab_[user,group,host,hostgroup]
    retrieve_keytab_[user,group,host,hostgroup]
    sshpubkey
    userclass
    auth_ind
    requires_pre_auth
    ok_as_delegate
    ok_to_auth_as_delegate

The README-host.md file has been updated to provide information about the
new settings and also the members. Also examples for the new things have
been added.

New example playbooks have been added:

    playbooks/host/add-host.yml
    playbooks/host/host-member-allow_create_keytab-absent.yml
    playbooks/host/host-member-allow_create_keytab-present.yml
    playbooks/host/host-member-allow_retrieve_keytab-absent.yml
    playbooks/host/host-member-allow_retrieve_keytab-present.yml
    playbooks/host/host-member-certificate-absent.yml
    playbooks/host/host-member-certificate-present.yml
    playbooks/host/host-member-managedby_host-absent.yml
    playbooks/host/host-member-managedby_host-present.yml
    playbooks/host/host-member-principal-absent.yml
    playbooks/host/host-member-principal-present.yml
    playbooks/host/host-present-with-allow_create_keytab.yml
    playbooks/host/host-present-with-allow_retrieve_keytab.yml
    playbooks/host/host-present-with-certificate.yml
    playbooks/host/host-present-with-managedby_host.yml
    playbooks/host/host-present-with-principal.yml
    playbooks/host/host-present-with-randompassword.yml
    playbooks/host/host-present.yml
    playbooks/host/hosts-member-certificate-absent.yml
    playbooks/host/hosts-member-certificate-present.yml
    playbooks/host/hosts-member-managedby_host-absent.yml
    playbooks/host/hosts-member-managedby_host-present.yml
    playbooks/host/hosts-member-principal-absent.yml
    playbooks/host/hosts-member-principal-present.yml
    playbooks/host/hosts-present-with-certificate.yml
    playbooks/host/hosts-present-with-managedby_host.yml
    playbooks/host/hosts-present-with-randompasswords.yml

New tests have been added for the module:

    tests/host/certificate/cert1.der
    tests/host/certificate/cert1.pem
    tests/host/certificate/cert2.der
    tests/host/certificate/cert2.pem
    tests/host/certificate/cert3.der
    tests/host/certificate/cert3.pem
    tests/host/certificate/private1.key
    tests/host/certificate/private2.key
    tests/host/certificate/private3.key
    tests/host/certificate/test_host_certificate.yml
    tests/host/certificate/test_hosts_certificate.yml
    tests/host/test_host.yml
    tests/host/test_host_allow_create_keytab.yml
    tests/host/test_host_allow_retrieve_keytab.yml
    tests/host/test_host_managedby_host.yml
    tests/host/test_host_principal.yml
    tests/host/test_host_random.yml
    tests/host/test_hosts.yml
    tests/host/test_hosts_managedby_host.yml
    tests/host/test_hosts_principal.yml

The use of gssapi.creds.Credentials is not good if krb5 ticket forwarding
is used. It will fail. gssapi.Credentials with usage and store is the proper
way to do this.

This patch allows the connection to the ipa api to use a context different
than 'server', if needed.

With IPA 4.5 integers for examle in pwpolicy_find are returned as
integer values. The internally generated value will be converted from
integer to string (using to_text) if the value from find call result
is a string (or unicode for Python2).

The function api_get_realm is returning the realm of a connected FreeIPA
api. This is needed for proper principal checks in the extended ipauser
module that supports principals now.

The conversion is needed because older FreeIPA versions are returning
tuples in some cases instead of lists. To be able to compare them the
conversion to a list is needed.

to_text has not been imported from ansible.module_utils._text but it was
used in _afm_convert. The import has been added.

The module_params_get function can and should be used as a replacement of
ansible_module.params.get. For Python2 it is needed to convert parameters
to unicode. Otherwise there will be an error in the FreeIPA API command.

The private function _afm_convert has been added to do the conversion
recursively.

api_check_param can be used to verify if params are available for a command
in the used FreeIPA version. The function has been added as api is normally
not imported into modules.

The order of lists returned by find commands is not guaranteed. Therefore
lists are now converted to sets to compare them properly.

api_command is always used within try clause, therefore it is not needed
to have an extra try clause within api_command. Additionally it is needed
to get the dofferent errors in the next level.

The GSSAPI can be enabled in the management modules with either the
KRB5CCNAME or the KRB5_CLIENT_KTNAME environment variable.

For KRB5CCNAME it is needed to create a ccache file

  kinit admin@TEST.LOCAL -c /root/admin.ccache

that is transferred to the nodes (here into /root) and activated in the
playbook with

  environment:
    KRB5CCNAME: /root/admin.ccache

For KRB5_CLIENT_KTNAME a admin keytab has to be generated

  ipa-getkeytab -s ipaserver.test.local -p admin@TEST.LOCAL -k \
  /root/admin.keytab

that is transferred to the nodes (here into /root) and activated in the
playbook with

  environment:
    KRB5_CLIENT_KTNAME: /root/admin.keytab

It will be needed to set ipaadmin_principal if the admin principal is not
admin.

The management modules can be used without a password in this case.

For debug and error reporting it is needed to have the ansible module also
in the valid_creds function.

date_format parses the supported date formats and creates a datetime object.

compare_args_ipa compares generated args with args returned by IPA command
find_ functions.

There are now two topology management modules placed in the plugins folder:

  plugins/modules/ipatopologysegment.py
  plugins/modules/ipatopologysuffix.py

Topology segments can be added, removed and reinitialized with the
ipatopologysegment module. Also it is possible to verify topology suffixes
with the ipatopologysuffix module.

A new module_utils for plugins has been added:

  plugins/module_utils/ansible_freeipa_module.py

And documentation for the modules:

  README-topology.md

New sample playbooks are available in playbooks/topology:

  playbooks/topology/add-topologysegment.yml
  playbooks/topology/delete-topologysegment.yml
  playbooks/topology/reinitialize-topologysegment.yml
  playbooks/topology/verify-topologysuffix.yml

The plugins folder can be used with the new Ansible Collections supported
by Ansible 2.8 and Ansible galaxy 3.2.