RVS assurance issue policy

This page describes Rapita Systems’ policy for analysis, resolution and notification of assurance issues within RVS software.

What is an RVS assurance issue?

An RVS assurance issue is any issue or bug that affects the assurance of the tool outputs from one of our qualified RVS products.

RVS Assurance issues can fall into one of the following categories:

• Over-claim: the tool made a stronger claim than it should, e.g. claiming code coverage that was not achieved.
• Missed construct: the tool omitted some information from the analysis, e.g. only reporting a subset of test results.
• Functional change: the tool modified the functional behavior during the test, e.g. due to source level instrumentation.

Product	Behaviors assessed
RapiTest	Reporting a test pass when the test actually failed (over-claim) Failure to report a test result entirely (missed construct) Functional change from the test harness (functional change)
RapiCover	Failure to report a coverage shortfall/coverage hole (over-claim) Failure to record a required coverage item (missed construct) Functional change due to instrumentation (functional change)
RapiTime	Reporting an optimistic execution time result (over-claim) Functional change due to instrumentation (functional change)

Analysis of RVS assurance issues

Potential assurance issues may be identified during the use of RVS by Rapita staff or by our customers.

When we become aware of a potential assurance issue in RVS by any means, we analyze it to identify whether it is an issue (whether it can cause a functional change in the software or lead to a false positive result).

When analyzing an issue, we analyze its scope, identifying the version(s) of RVS that it affects. When checking for affected versions, we check Current, Superseded and Deprecated versions of RVS that have been released within the last 5 years, but do not by default check versions that were released more than 5 years ago, or versions that have reached their End of Life (unless Frozen Version Support has been arranged for those versions). For more information on our Product life cycle policy, see our Product life cycle policy webpage.

After our analysis is complete, we publish a bulletin for the issue and send a notification about it to assurance issue contacts (see Notification of RVS assurance issues).

At your request, we will check whether an assurance issue exists in specific qualifiable versions that are more than 5 years old but have not yet reached their End of Life.

Notification of RVS assurance issues

When we deliver RVS, we add registered RVS users as contacts to receive notifications about RVS assurance issues. We endeavor to ensure that at least one individual from an organization remains on this list of contacts on a best effort basis.

For each assurance issue we have analyzed, we publish a bulletin which we send by email to RVS assurance issue contacts. Bulletins identify the RVS version(s) the issue affects, describe the issue and the situations in which it can be encountered, and describe workarounds and strategies to mitigate against the issue. We also identify the status of the issue, which can be one of:

• Open – The issue is still active and its resolution has not yet been scheduled.
• Planned – The issue is active but its resolution has been planned for a specific RVS version.
• Resolved – The issue has been resolved in a version of RVS.

We send RVS assurance issue update notifications by email after each scheduled RVS release. Each update identifies:

• New assurance issues that have been announced since the previous update
• Any issues for which the status has changed as of the corresponding RVS release
• Remaining Open assurance issues

Registered RVS users can request an update on the status of or more information about RVS assurance issues at any time by emailing us at support@rapitasystems.com.

If you have questions about our notification process or would like us to add or update RVS assurance issue contacts, please contact us at support@rapitasystems.com.

Resolution of RVS assurance issues

We aim to provide workarounds or fixes for assurance issues as soon as possible. When we schedule resolutions for assurance issues, we schedule them for a specific major, minor or incremental RVS version. Fixes can include a range of updates including, but not limited to, clarifications in documentation, fixes to the off-the-shelf tools, or fixes to the generic integration support libraries that we provide.

When we notify users of new assurance issues, we identify if we have already scheduled a resolution for the issue.

We notify users of resolved issues in an assurance issue update email, which we send to RVS assurance issue contacts shortly after each scheduled release of RVS (see Notification of RVS assurance issues).