Merging coverage data from multiple test runs

One of the questions we are asked most frequently is how we merge coverage data from multiple test runs. This seems to be an unnecessary bottleneck that we have solved for many of our customers.

For DO-178B/C certification, you must show that you have covered 100% of your code. This is typically done by a combination of system testing, unit testing, and marking relevant code as covered by manual analysis. To check your code coverage at various stages in the testing process, and when you submit your certification evidence, you may need to merge coverage data obtained from running:

1. Multiple system tests 
2. Both system and unit tests
3. Tests using different tools 

Doing this manually would take significant effort and can be a source of errors, so the process should be automated by a tool. According to DO-178B/C, this tool would need to be qualified. Qualifying a tool is a costly process that takes much less effort if the tool vendor has already developed a qualification argument for the tool, and provides this with the tool. 

Using our RVS tools, it is easy to merge data from multiple test runs. This is because of the way we store output data. When you use RapiCover to determine code coverage, the output file generated by the tool includes a copy of the instrumented source code. The file references each code element in this copy, and whether or not it was tested. Storing the data in this way means that it takes minimal effort to merge coverage data from multiple test runs (Figure 1), addressing points 1 and 2 above. This also works when you have applied different instrumentation to each run of your code. What's more, we have already developed evidence for you to supply to qualify using this feature in DO-178B/C projects.  

Figure 1. Merging coverage data using RapiCover

Addressing point 3 is a little more difficult. Different tools handle data differently, so merging data from them requires developing a custom solution. When you are using RapiCover along with third-party tools, we can develop an importer that lets you merge coverage results from RapiCover and these tools into a single report. Or you could use RapiCover together with RapiTest, our RVS tool that automates the generation of test code and executes this code on-target. Because RapiCover and RapiTest are built using the same technology, it is easy to use the two tools together in your testing environment, and both tools can be qualified for use in DO-178B/C projects. 

Merging coverage data from multiple test runs shouldn't be difficult, and it isn't. You definitely shouldn't have to do it manually. If you would like to see how easy it is to do this using RapiCover and RapiTest, you can request trial versions from our website.