RVS assurance issue policy

This page describes Rapita Systems’ policy for analysis, resolution and notification of assurance issues within RVS software.

What is an assurance issue?

An assurance issue is any issue or bug that affects the assurance of the tool outputs from one of our qualified products.

Assurance issues can fall into one of the following categories:

• Over-claim: the tool made a stronger claim than it should, e.g. claiming code coverage that was not achieved.
• Missed construct: the tool omitted some information from the analysis, e.g. only reporting a subset of test results.
• Functional change: the tool modified the functional behavior during the test, e.g. due to source level instrumentation.

Product	Behaviors assessed
RapiTest	Reporting a test pass when the test actually failed (over-claim) Failure to report a test result entirely (missed construct) Functional change from the test harness (functional change)
RapiCover	Failure to report a coverage shortfall/coverage hole (over-claim) Failure to record a required coverage item (missed construct) Functional change due to instrumentation (functional change)
RapiTime	Reporting an optimistic execution time result (over-claim) Functional change due to instrumentation (functional change)

Analysis of assurance issues

Potential assurance issues may be identified during the use of RVS by Rapita staff or by our customers. When we become aware of a potential assurance issue in RVS by any means, we analyze it to identify whether it is an issue (whether it can cause a functional change in the software or lead to a false positive result). While analyzing an issue, we analyze its scope, identifying the version(s) of RVS that it affects. When checking for affected versions, we check Current, Superseded and Deprecated versions of RVS that have been released within the last 5 years, but do not by default check versions that were released more than 5 years ago, or versions that have reached their End of Life (unless Frozen Version Support has been arranged for those versions). For more information on our Product life cycle policy, see our Product life cycle policy webpage.

After our analysis is complete, we publish a bulletin for the issue and send a notification about it to assurance issue contacts (see Notification of assurance issues).

At your request, we will check whether an assurance issue exists in specific qualifiable versions that are more than 5 years old but have not yet reached their End of Life.

Notification of assurance issues

For each assurance issue, we publish a bulletin, which you can view at any time from our Customer portal as a registered RVS user, and which we send by email to assurance issue contacts (see below). Bulletins identify the RVS version(s) the issue affects, describe the issue and the situations in which it can be encountered, and describe workarounds and strategies to mitigate against the issue. We also identify the status of the issue, which can be one of:

• Open – The issue is still active and its resolution has not yet been scheduled.
• Planned – The issue is active but its resolution has been planned for a specific RVS version.
• Resolved – The issue has been resolved in a version of RVS.

By default, we add all registered RVS users to our assurance issue email notification list. We regularly keep users on this list updated by email with updates to assurance issues, including:

• New issues that have been identified
• Changes in assurance issue status at each RVS release

We send Assurance Issue update notifications by email after each scheduled RVS release.

If you have questions about our notification process or would like us to add or update assurance issue contacts, please contact us at support@rapitasystems.com.

Resolution of assurance issues

We aim to provide workarounds or fixes for assurance issues as soon as possible. When we schedule resolutions for assurance issues, we schedule them for a specific major, minor or incremental RVS version. Fixes can include a range of updates including, but not limited to, clarifications in documentation, fixes to the off-the-shelf tools, or fixes to the generic integration support libraries that we provide.

When we notify users of new assurance issues, we identify if we have already scheduled a resolution for the issue.

After an assurance issue has been resolved, we update the status of that issue on our Customer portal.