Skip to content
Commit 3109e9d1 authored by Rafael Guterres Jeffman's avatar Rafael Guterres Jeffman
Browse files

utils/run-tests.sh: Replicate Azure's test grouping

When running ansible-freeipa's Azure pipelines for nightly and weekly
tests, due to the amount of tests to execute, tests are grouped and
executed in parallel jobs.

Due to a still unkonwn issue, depending on the order the tests are
executed, some random failures may occur and debugging them is hard due
to current implementation of the tests.

This patch adds support for replicating the tests of a specific Azure
test group once the seed used to create groups and the group number are
provided, allowing the test failures to be replicated on the developer's
workstation where it can be more easily debugged.

A new option is added to 'utils/run-tests.sh', '-A SEED.G' that is used
to define the seed and group to replicate the tests. The seed is a date,
with the format "YYYYMMDD", so, for example '-A 20230611.2' would
execute the same tests, in the same order as the second group of tests
for date 2023-06-11. To aid in usability 'YYYY-MM-DD' may also be used.

When using '-A' neither '-s' (test suites) or specific tests (positional
arguments) can be used.

Also, to help fixing tests, an option to stop the tests on the first
test failure ('-x') was added to the script.
parent f1a6f444
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment