Race condition testing

Daniel Wright

2019-11-20

Race conditions occur in multithreaded software when multiple threads attempt to modify a piece of shared data at the same time. This can become a potentially serious problem if the output of one thread’s execution affects another thread’s execution, in a way that causes unexpected or incorrect results.

Exposing potential race conditions is crucial because it shows you where you need to re-design your code to eliminate unexpected results. Certification authorities in the critical software industry are increasingly requesting that software developers demonstrate adequate testing for race conditions in their code.

The problem with race conditions

Whether or not a race condition occurs – and whether or not it results in errors – depends on the relative timing behaviors of the competing threads. In a non-deterministic system, these behaviors can be different for each run of the software.

In the example below, Threads 1 and 2 are accessing and modifying a global variable, ‘Y’. The behavior of the threads changes from one run to the next, but because each thread is able to complete without interruption, a correct (if different) value results each time.

During the third run, however, the behavior changes again. Here, Thread 1 is interrupted before it completes, and Thread 2 modifies the global variable that Thread 1 used as its input. Now, when Thread 1 completes, it writes a value that is unexpected.

Using RapiTest to test potential race conditions

RapiTest, and our powerful Spreadsheet Test Format, include features specifically designed to tackle the challenges that arise with testing non-deterministic systems. We’ve also designed customized integration libraries to optimize your integration for multithreaded testing, and a unique visualization tool that that helps you understand how your threads interact.

RapiTest helps you test potential race conditions in your code. Using RapiTest, you can:

Design and run a multithreaded test.
Identify potential race conditions.
Visualize the interactions between threads.

Design and run a multithreaded test

Our Spreadsheet Test Format is the perfect tool for writing multithreaded tests, as it lets you define complex behavior in an easily-readable way. It has several useful features including a Grouping Checks feature, and specific operators (e.g. ‘wait’, ‘kill’) that give you control over your threads’ behavior and execution.

These features allow you to define exactly when and how your threads run during testing, giving you the ’reproducibility’ you need to test them effectively.

Identify and analyze potential race conditions

When you run a multithreaded test with RapiTest, potential race conditions are highlighted in the results as failing asserts. You can drill down through each function to explore where and how these failures have occurred.

Once you know where to look, you can investigate and re-design your code to prevent race conditions from occurring, i.e. by adding mutual exclusions.

Visualize the interactions between threads

Being able to visualize the behavior of your system is valuable, not only in terms of enhancing your own understanding of your code, but also being able to share that understanding, to collaborate with other developers and solve problems collectively.

Our products include powerful visualization features; for example, RapiTask includes an inbuilt viewer that shows high-level graphical views of your system’s runtime behavior, including across multiple cores.

Working with our customers, we’ve gone a step further and designed a unique tool for visualizing the interactions between threads in a multithreaded system.

The tool renders the call tree that occurs in each test run, with the threads displayed as different colours. Details of the function calls are recorded in each line, giving you a concise yet detailed overview of the progress of each thread, and the effects of context switching.

Implicit context switches are shown in grey, while forced context switches – i.e. those that you have defined in your test design – are shown in bright red.

Viewing your tests in this way highlights expected and unexpected behavior, according to your understanding of the code. It’s also a simple way to identify potential issues and share these with colleagues.

The tool also allows you to take a step back, and view the behavior of different test runs side by side. From here, you can recognize problematic patterns, drill into the details, and work to address these in your code.