Testing without test results is not optimal. What good is it if you manually test and do not write down the test results? What's the benefit of running tests by runners and not storing the output? In both cases, only a person doing or running the tests can see the results and verify that they pass. We lose test visibility in the team, test content, and if tests pass or fail. To avoid that, we want to store results and keep the test history at hand. Not only that, but we should be able to search, process, and make reports. I will focus on two aspects today, test results formats and delivery methods.
Result format
All testing tools produce a result artifact. It can be standardized unit test formats like TRX, JUnit, or xUnit. It can be a CSV, JSON, or XML file, easily integrating with custom applications or systems. Additionally, results can also be more visual, like HTML or PDF. Every format has its purpose, so let's categorize them.
1. Store in a format that is compatible with CI/CD systems
This is probably most common integration nowadays. All modern development and test teams use some sort of CI/CD system like Azure DevOps, GitLab, Github, Jenkins to mention a few. Those system have many plugins and support many test formats, so your test suite needs to generate the file format you want for your CI/CD pipeline to ingest.
2. Format to report for management & customers
Your management or customers are asking for Excel or PDF reports every month to check progress. Depending on their need, you could send a detailed report of 1.000 tests or just a summary report.
3. Custom formats
You could generate JSON or XML files in a format that your internal software for processing or gathering test results supports. Of course, you can generate other file formats, but those are the most common in the last years.
Result delivery method
Once you have test results in a format of your choice, you need to deliver them to a tester, developer, customer, or machine. I categorize delivery methods below.
1. Upload to file storage
One way to deliver your test results is by uploading them to file storage. Examples are FTP server, cloud storage provider, on-prem NAS/SAN, or local hard drive.
2. Send by e-mail
This is the most basic delivery of your tests results. Attach a file and send it! It is important to keep in mind that e-mails are not secure and sensitive data should be delivered by other means.
3. Integration with 3rd party systems
Integrations with CI/CD tools is the most known candidate in this category. But you could fit any other integrations here, like storing results to a database, sending results as JSON to a REST API, or XML to Web service. Sending test results by messaging systems is also an option.
Conclusion
At Beekn, we aim to give you a flexible platform to format and deliver test results in your favorite flavor. Maybe you want to send PDF to your customer, and you want to use e-mail as a delivery medium. Or your DevOps engineers want to integrate the test results into the CI/CD system by using a webhook.
In the end, the goal is to store your test results in a format and delivery method that makes the most sense to you.