How to Add Data to Pytest JUnit XML Output

Learn how to add custom data to pytest JUnit XML output. When running tests with pytest, one of the most common ways to view and analyze test results is through the JUnit XML output format.

This format is widely used because it integrates well with various Continuous Integration (CI) tools like Jenkins, GitLab CI, Travis CI, and others.

However, sometimes the default JUnit XML output does not provide all the data you might need for your CI pipeline or reporting purposes.

In this guide, we’ll explore how to add custom data to pytest JUnit XML output to enhance your reports and provide more detailed information.

By default, pytest’s JUnit XML output looks something like this:

<testsuites>
    <testsuite name="pytest" tests="2" failures="1" skipped="1" time="0.002">
        <testcase name="test_example_1" time="0.001"/>
        <testcase name="test_example_2" time="0.001">
            <failure message="AssertionError">...</failure>
        </testcase>
    </testsuite>
</testsuites>

Adding Custom Data to the JUnit XML Output

To enhance the default JUnit XML output, you can add custom data, such as test metadata, extra information about the test environment, or other details that might be relevant to your CI pipeline or reporting.

There are two primary ways to add data to the pytest JUnit XML output:

1. Using pytest Hooks
2. Using pytest Markers

Let’s dive into these two methods.

1. Using Pytest Hooks to Add Custom Data

What Are Pytest Hooks?

Pytest hooks are special functions that allow you to modify or extend pytest’s behavior during test collection, execution, or reporting. By using hooks, you can easily add extra data to the JUnit XML output.

Pytest provides the pytest_junit_xml hook, which can be used to modify the content of the JUnit XML output.

Example: Adding Custom Test Data

You can modify the JUnit XML file using the pytest_junit_xml hook to add additional data to individual tests. Here’s an example:

Create a conftest.py file in your project’s test directory (if you don’t have one already).

# conftest.py
import pytest

# Hook to modify the JUnit XML output
def pytest_junit_xml(testreport, xml):
    # Add custom data to a test case
    testcase = xml.find(".//testcase[@name='%s']" % testreport.nodeid)
    if testcase is not None:
        # Example: Adding custom information (e.g., custom tag or extra data)
        extra_info = pytest.config.getoption("--custom-data")
        if extra_info:
            extra_element = xml.Element("custom_data")
            extra_element.text = extra_info
            testcase.append(extra_element)

In this example:

• The pytest_junit_xml hook allows you to access the test report and XML.
• The hook searches for a <testcase> element that matches the test name (testreport.nodeid).
• If a match is found, we add custom data to the <testcase> element, such as a new <custom_data> tag containing the value passed with the --custom-data command-line option.

Running with Custom Data

Now, when running pytest, you can pass a custom option to include extra data:

pytest --junitxml=report.xml --custom-data="My custom test data"

This command will generate the report.xml with the custom data added to each test case in the report.

2. Using Pytest Markers to Add Custom Data

What Are Pytest Markers?

Pytest markers are a way to add metadata to tests. Markers can be used to categorize tests or attach custom attributes. You can use pytest markers to associate custom data with your tests, which can later be added to the JUnit XML output.

Example: Adding Custom Data with Markers

You can define a custom marker to associate extra data with a specific test, then retrieve this data when generating the JUnit XML output.

Here’s how you can do it:

1. Mark your test with custom metadata:

import pytest

@pytest.mark.custom_data(info="Special data for this test")
def test_example():
    assert 1 == 1

2. Modify the JUnit XML output using pytest_junit_xml:

In your conftest.py file, use the pytest_junit_xml hook to read the marker data and add it to the JUnit XML:

# conftest.py
import pytest

def pytest_junit_xml(testreport, xml):
    # Check if the test has the 'custom_data' marker
    custom_data_marker = testreport.get_closest_marker("custom_data")
    if custom_data_marker:
        # Add custom data to the XML output
        testcase = xml.find(".//testcase[@name='%s']" % testreport.nodeid)
        if testcase is not None:
            extra_element = xml.Element("custom_data")
            extra_element.text = custom_data_marker.kwargs.get("info", "No custom data")
            testcase.append(extra_element)

This code will check for the custom_data marker and add the info value to the <testcase> element in the JUnit XML file.

Running Tests with Markers

Once you have marked your test with the @pytest.mark.custom_data decorator, you can run pytest normally, and the custom data will appear in the JUnit XML output.

pytest --junitxml=report.xml

Example JUnit XML Output with Custom Data

After running your tests with the custom data added, the JUnit XML output might look like this:

<testsuites>
    <testsuite name="pytest" tests="1" failures="0" time="0.003">
        <testcase name="test_example" time="0.001">
            <custom_data>Special data for this test</custom_data>
        </testcase>
    </testsuite>
</testsuites>

The <custom_data> element has been added to the relevant test case, allowing you to include custom information about the test.

Conclusion

Adding custom data to pytest JUnit XML output can be incredibly helpful for enhancing your test reports and integrating with CI tools. Whether you want to include environment variables, additional test information, or any other custom data, using pytest hooks or markers provides a flexible way to achieve this.

• Use the pytest_junit_xml hook if you need to modify the JUnit XML format directly.
• Use pytest markers to add metadata to specific tests, which can be included in the XML output.

By following these methods, you can create more informative test reports and improve your testing workflows.

Let me know if you need more clarification or run into any issues!