JUnit XML Reporting Toolkit

A command line tool for helping manage JUnit XML formatted reports.

Installation

To get the toolkit locally just use Cargo:

cargo install juxr

NOTE: Unsigned binaries built by GitHub Actions are provided for convenience and are available in Releases. The recommended installation path is to build from source using cargo install juxr.

Extract reports over Standard I/O

As a developer, I have some tests running in a temporary Kubernetes pod and I need to exract the test results and any associated attachments

The toolkit provides three sub-commands to assist with this.

Firstly there is the export subcommand. You run this command in your pod and it will pipe the reports (and any attachments) to standard out.

Then there is the import subcommand. You run this command on the receiving end and pipe the Kubernetes logs through it.

Finally, there is a specialized variant of export in the exec subcommand. You run this again in your pod mut use it to wrap the test launch command.

You will want to include the toolkit in your test image, e.g.:

If you are using the export you will change your entry point to run something like juxr export -r **/TEST-*.xml or whatever the test report pattern you need.

As you will likely want to have the pod terminate with a success/failure exit code, a simpler option is to just change the command that runs your tests from some_command arg arg to juxr exec -r **/TEST-*.xml -- some_command arg arg as this will take care of launching the command and propagating the error code after exporting the test reports.

Have a look at the juxr help export and juxr help exec for details of the other export options such as secret redaction and renaming of tests / suites.

If you do not want to use the juxr toolkit inside your test container you can achieve the same result with base64 encoding, however this will not provide for

needle="[[juxr::stream::$RANDOM::junit-test-report::TEST-custom.xml]]"
# NOTE: there must be a new line before the needle
echo "" 
echo $needle
base64 < TEST-custom.xml
echo $needle

Then on the receiving end you just pipe the logs through juxr import, e.g. if using helm test to run your test container:

helm test --logs | juxr import -o helm-test-results/

The import command will output all non-needle bookended content to standard out and write the files to the specified output directory

Convert TAP formatted reports to JUnit XML format

As a developer I have a testing tool that outputs TAP formatted test reports but I need to consume JUnit XML formatted reports

The tap subcommand will convert TAP version 12 or 13 output into JUnit XML format.

You can either pipe the output through or have the test command run by the toolkit, e.g.

some_command arg arg | juxr tap --name "some_command.tests" -o test-results/

or

juxr tap --name "some_command.tests" -o test-results/ -- some_command arg arg

The later form will attempt to infer test durations and will propagate the exit code

Generate a JUnit XML report from executing a single command

As a developer I have a single command which I would like to turn into a JUnit XML report

The test sub command will run a single command and produce a JUnit XML report that includes the commands output, e.g.

juxr test --name "some.command" --test "arg1 arg2" -o test-results/ -- some_command arg1 arg2

This will produce a TEST-some.command.xml file in test-results.

Look at juxr help test for details on how to control the exit code mapping to differentiate the test status.

Generate a JUnit XML report from running a suite of simple command

As a developer, I have a series of different commands which represent independent tests I would like to run and record.

For this use case we recommend that you use something like the excellent bats test framework. You can generate JUnit XML Foratted reports with this framework and you have control over the sequencing of tests.

However, sometimes you just need something quick and dirty... enter the run subcommand.

This takes a flexible YAML formatted description of a suite of tests and runs them in alphabetical order. An example suite of tests could look like:

"001 echo greeting using the current shell": >
  if [[ $(($(($RANDOM%10))%2)) -eq 1 ]] ;
  then
    echo hello world ;
  else
    echo hi world ;
  fi
"002 echo a greeting using the echo binary":
  - echo
  - "hello world"
"003 the false command when executed by the current shell should exit with non-zero":
  cmd: "false" # need to quote values that YAML might perform type conversions on
  success: 1
  failure: 0
"004 skip this test":
  cmd:
    - "false"
  skipped: 1
  failure: 2 # need to move it off it's default of 1
"005 exit code 1 or 2 is success 3 or 4 means skip and 5 or 6 is failure otherwise an error":
  cmd:
    - sh
    - "-c"
    - "exit 2"
  success:
    - 1
    - 2
  skipped:
    - 3
    - 4
  failure:
    - 5
    - 6

The top level keys are the test names.