When developing safety-critical applications to DO-178C, AC 20-193, AMC 20-193 and CAST-32A guidelines or ISO 26262 standards, there are special requirements for using multicore processors. Evidence must be produced to demonstrate that software operates within its timing deadlines.

The goal of multicore timing analysis is to produce execution time evidence for these complex systems. In multicore processors, multiple cores compete for the same shared resources, resulting in potential interference channels that can affect execution time. MACH¹⁷⁸ and Rapita's Multicore Timing Solution account for interference to produce robust execution time evidence in multicore systems.

In AMC 20-193, the official guidance document for multicore aspects of certification for ED-12C projects, an interference channel is defined as "a platform property that may cause interference between software applications or tasks". This definition can be applied to a range of ‘platform properties’, including thermal factors etc.

Of these interference channels, interference caused by the sharing of certain resources in multicore systems is one of the most significant in terms of execution times. Interference based on shared resources may occur in multicore systems when multiple cores simultaneously compete for use of shared resources such as buses, caches and main memory.

Rapita’s solutions for multicore timing analysis analyze the effects of this type of interference channel.

A very simple example of a shared resource interference channel is shown below:

In this simplified example, tasks running independently on the two cores may need to access main memory simultaneously via the memory controller. These accesses can interfere with each other, potentially degrading system performance.

Rapita have been providing execution time analysis services and tooling since 2004.

RapiTime, part of the Rapita Verification Suite (RVS), is the timing analysis component of MACH¹⁷⁸ and the Multicore Timing Solution. Our customers have qualified RapiTime on several DO178C DAL A projects where it has been successfully used to generate certification evidence by some of the most well-known aerospace companies in the world. See our Case Studies.

Learn more about our tool qualification support for RapiTime in projects requiring DO-178B/C certification.

As well as providing a mature tool chain, we support you in ensuring that your test data is good enough, so that the timing information you generate from the target is reliable.

Our RapiDaemons are configured and tested (see the FAQ: ‘configuring and porting’) to ensure that they behave as expected on your multicore platform.

We also assess available observability channels as part of Platform Analysis. This primarily applies to the use of Hardware Event Monitors, where we assess their accuracy and usefulness for obtaining meaningful insights into the system under observation.

It is possible for companies to perform multicore timing analysis internally, but it is a highly complex undertaking which is very costly in terms of budget and effort. Anecdotally, one of our customers reported that it took them five years and a budget in the millions of dollars to analyze one specific platform.

MACH¹⁷⁸ and our Multicore Timing Solution are typically delivered as a turn-key solution, from initial system analysis and configuration all the way through to providing evidence for certification.

RTOS vendors may provide partitioning mechanisms for their multicore processors, but these do not guarantee the complete elimination of multicore interference. Instead, they are designed to provide an upper limit on interference, sometimes at the expense of average-case performance.

In aerospace, these partitioning mechanisms may be referred to as ‘robust partitioning’. The multicore guidance in AC 20-193, AMC 20-193 and CAST-32A identify allowances for some of the objectives if you have robust partitioning in place, but it is still necessary to verify that the partitioning is as robust as it is claimed to be.

From a certification standpoint, regardless of the methodology behind the RTOS vendor’s approach to eliminating interference, the effectiveness of the technology needs to be verified.

Yes – our approach can be used to get an in-depth understanding of how sensitive software can be to other software. For example:

• Task 1 executes acceptably in isolation and with most other tasks, but if it executes simultaneously with Task 127, its function X takes 10 times as long to return.
• This intelligence can feed into system integration activities to ensure that function X can never execute at the same time as Task 127.

The information from this type of analysis can also provide insights into potential improvements to the implementation of the two tasks. Sensitive tasks are not always the guilty party: other tasks can be overly aggressive and cause delays in the rest of the system.

For safety reasons, WCET will always be somewhat pessimistic. However, techniques that work well for single-core systems risk generating a WCET that is unreasonably large when applied to multicore systems, because the effects of contention can become disproportionate. The objective, therefore, is to calculate a value that is plausible and useful, without being optimistic. Optimism in relation to WCET is inherently unsafe.

It is not enough to identify how sensitive an application’s tasks are to different types and levels of interference; it is also necessary to understand what degree of interference a task may suffer in reality. It is possible to lessen the pessimism in WCET analysis by viewing the processor under observation through this paradigm.

The degree to which we can reduce pessimism is dependent on how effectively we can analyze the system. Factors influencing this include:

• The overhead of the tracing mechanism (which affects depth of instrumentation)
• The availability and reliability of hardware event monitors
• The availability of information regarding other tasks executing on the system
• The quality of tests that exercise the code

Cache partitioning is all about predictability, not performance. Your code may execute faster on average without cache partitioning, but it probably wouldn't be as predictable and could be quite sensitive to other software components executing in parallel.

Cache partitioning aims to remove all the sensitivity to other tasks sharing the caches, thus making your task more predictable – but potentially at the expense of overall performance. In critical systems, predictability is of far greater importance than performance.

Rapita’s solution for multicore timing analysis can be used to exercise cache partitioning mechanisms by analyzing any shared – and usually undocumented – structures internal to the caches.

To analyze how a specific task is affected by contention on a specific resource, we need to be able to synchronize the execution of the task with the execution of RapiDaemons (the applications that generate contention on the resource).

Usually, it is highly desirable to have RTOS/HV support for enabling user-level access to hardware event monitors. Context switch information is also very valuable when performing multicore timing analysis.

Yes. Our solution makes it easy to specify the core on which you run your tests, and the level of resource contention to apply from each other core in the system.

We can also analyze systems that use non-synchronized clocks such as those often present in AMP platforms by using the RTBx to timestamp data.

The maximum number of hardware event monitors we can collect values from depends on the performance monitoring unit(s) (or equivalent) on the hardware. An ARM A53, for example, lets us collect at least 30 metrics, but only access 6 in a single test. By running tests multiple times, however, we could collect all 30 metrics.

Developing a one-button tool solution for multicore timing analysis would be impossible. This is because interference, which can have a huge impact on a task’s execution time, must be taken into account when analyzing multicore timing behavior.

Analyzing interference effects is a difficult challenge that cannot be automatically solved through a software-only solution. Using approaches developed for timing analysis of single-core systems would result in a high level of pessimism, as it would assume that the highest level of interference possible is feasible, while this is almost never the case.

It is possible to collect a range of metrics by instrumenting your source code with the Rapita Verification Suite (RVS), including a range of execution time metrics:

• RapiTime: high-water mark and maximum execution times
• RapiTask: scheduling metrics such as periodicity, separation, fragmentation and core migration

It is also possible to collect information on events in your hardware using hardware event monitors. The information we can collect depends on the performance monitoring unit(s) (or equivalent) of your system, but typically includes events such as L2 cache accesses, bus accesses, memory accesses and instructions executed. We can also collect information about operating system activity such as task switching and interrupt handling via event tracing or hooks.

Yes, we formally test and assess the accuracy of hardware event monitors to ensure the validity of results we collect for the software under analysis.

By following a V-model process, our engineers investigate multicore systems and produce evidence about multicore timing behavior. Our approach has been designed to support projects within the DO-178C (AC 20-193, AMC 20-193 and CAST-32A) and ISO 26262 contexts.

You can see an example workflow of how Rapita approaches multicore timing analysis in our White Paper.

We can analyze almost all hardware architectures. Our engineers work with you to determine the optimal strategy for integrating our RVS tools with your target, including hardware characterization and design considerations to best fit the hardware you're using.

To work with an architecture that is new to us, we first identify which metrics we can collect from the hardware, then adapt RapiDaemons for the architecture and implement a strategy to collect data from it.

For a list of multicore architectures, RTOSs, boards and middleware we've worked with, see our compatibility page.

MACH¹⁷⁸ is a workflow and solution for certifying multicore aerospace projects in accordance with DO-178C, AC 20-193, AMC 20-193 and CAST-32A guidance.

The MACH¹⁷⁸ workflow is supported by a range of products and services, which help to address AC 20-193, AMC 20-193 and AA-22-01 objectives.

MACH_¹⁷⁸ can help you on your multicore DO-178C (AC 20-193 and AMC 20-193) compliance journey in a few ways:

• Template plans and procedures for compliance and verification activities can help you write your PSAC and SVP, helping you meet A(M)C 20-193’s MCP_Planning_1 and MCP_planning_2 objectives. These are available in MACH¹⁷⁸ Foundations, along with checklists and white papers on specific multicore topics.
• Software tools and qualification can help you set up a tool automation environment that you can use to produce verification evidence, helping you meet A(M)C 20-193’s MCP_Resource_Usage_4 and MCP_Software_1 objectives.
• Services including Platform Analysis and Characterization and Software Analysis and Characterization can help you produce verification evidence for your platform, helping you meet A(M)C 20-193’s MCP_Resource_Usage_1, 3 and 4, and MCP_Software_1 objectives.
• The MACH¹⁷⁸ Blueprint provides a platform that you can use to learn more about A(M)C 20-193 compliance and run multicore research and development projects with the MACH¹⁷⁸ workflow. This includes results and project files from an example A(M)C 20-193 compliance project.

In addition to this support, training and consultancy is available to help answer any questions you may have.

Not yet, but we are currently working on projects that will go through certification with the FAA and EASA.

Rapita is a recognized leader in multicore timing analysis, with multiple professional publications on the subject:

• Steven H. VanderLeest and Samuel R. Thompson, “Measuring the Impact of Interference Channels on Multicore Avionics”  to appear in the Proceedings of the 39th Digital Avionics Systems Conference (DASC), San Antonio, TX, Oct 2020.
• VanderLeest, S.H. and Evripidou, C., “An Approach to Verification of Interference Concerns for Multicore Systems (CAST-32A),” best papers of 2020 SAE Aerotech, to appear in SAE International Journal of Advances and Current Practices in Mobility.
• VanderLeest, S.H. and Evripidou, C., “An Approach to Verification of Interference Concerns for Multicore Systems (CAST-32A),” SAE Technical Paper 2020-01-0016, 2020, doi:10.4271/2020-01-0016.
• Steven H. VanderLeest, Jesse Millwood, Christopher Guikema, “A Framework for Analyzing Shared Resource Interference in a Multicore System,” Proceedings of the 37th Digital Avionics Systems Conference (DASC), London, Sep 2018.

There is no standard list that fits all platforms. Some interference channels can be more common than others, such as the ones related to the memory hierarchy (i.e. caches and main memory). The identification of interference channels (for which we provide a service) is an important activity that identifies the interference channels whose impact on the system’s timing behavior must be assessed.

The length of time needed to comprehensively analyze interference channels is typically between 2 and 12 months, but this duration depends heavily on the scope of the analysis, the complexity of the system, and whether we have already analyzed a similar platform or not. Our solution includes an initial pilot phase in which we study the system and estimate the amount of time needed for subsequent phases, including analysis of interference channels.

Some of the multicore systems that we’ve worked with are listed in our FAQ “Which hardware architectures can you analyze?”.

If we have already worked on a similar multicore platform to yours, it may take less time to perform platform analysis for your platform.

Yes. We have run projects analyzing the Nvidia Xavier AGX (CUDA) and AMD’s Embedded Radeon E9171 GPU (featuring the CoreAVI Vulkan SC driver).

As per AC 20-193, AMC 20-193 and CAST-32A, each interference channel on the system must be analyzed to ensure that its effects on timing behavior have been mitigated. Some channels may be eliminated by architectural analysis or because they are deactivated for your system. For all channels that have not been eliminated, the guidance requires that multicore timing analysis be done to ensure that software operates within its timing deadlines. For each channel, one must provide analysis test reports and a summary to show that the set of tests is complete – Rapita provides this service to show completeness. 

Note that AC 20-193 and AMC 20-193 include objectives other than testing, such as planning objectives. We provide templates to help you complete these.

This depends on the performance requirements of the platform and the hosted software.

The primary benefit of cache partitioning is that it provides protection from one core/partition evicting another. There are two broad approaches to achieve this:

• Hardware: In hardware, the processor has built in support for partitioning the cache, allocating each core in the system its own area that it can use. This is supported on the T2080, for example (see e6500 TRM section 2.12.4).
• Software: In set-associative caches, the location in cache that each block of memory may be loaded to is known. Using techniques like cache colouring, the software is placed in specific memory blocks in such a way that it is ensured that there will be no two cores/partitions that can end up using the same section of the cache.

In terms of execution time, the prime benefit of cache partitioning is typically a significant reduction in the variability i.e. a comparison with and without cache partitioning would indicate that execution times have a greater spread when there is cross-core interference present. This can be a valuable contribution towards the claim for robust partitioning.

The downside of cache partitioning is that, depending on the nature of the hosted application, it can have a significant impact on performance on the average case and even on the worst-case execution time. The reason for this is that each core/partition now has a smaller section of the cache to work with; if it no longer fits into the cache, then it will see an increased cache miss rate which has a direct impact on execution time. Whether this is acceptable should be carefully tested and evaluated.

A common misconception for shared cache partitioning is that it eliminates the effects of interference from shared L2 caches. Depending on the hardware, the shared caches can have shared buffers/queues that are not part of the partitioning. Therefore, even though the interference due to one core or partition evicting another can approach zero, the increase in cache misses can cause slowdowns due to contention on these shared internal structures.

For an IMA platform, it is recommended that the effectiveness of cache partitioning is evaluated empirically. Specifically, perform experiments/tests where the cache partitioning is enabled where RapiDaemons targeting the L2 cache generate interference, and compare against equivalent interference scenarios where the partitioning is disabled. It is quite likely that there will be observed slowdowns in the average case, and potentially also in the worst-case. The results from this analysis could be converted into constraints for the partition developers and integrators. For example "hosted IMA partitions on any core may not exceed X number of accesses outside L1 cache over a time window of Y nanoseconds".

Yes, we provide template text that can either be incorporated into your Plan for Software Aspects of Certification and Software Verification Plan, or used standalone as multicore guidance and referenced from your existing plans.

These templates describe the procedures by which AC 20-193, AMC 20-193 and CAST-32A objectives can be met using the MACH¹⁷⁸ workflow.

Yes. MACH¹⁷⁸ can be used to produce timing evidence needed to satisfy DO-178C objectives in line with AC 20-193, AMC 20-193 and CAST-32A guidance.

Rapita Systems are uniquely positioned to offer the combination of expertise and tools required to effectively address AC 20-193, AMC 20-193 and CAST-32A objectives.

Whilst the challenge of certifying multicore systems for safety-critical applications is a relatively new one for the industry as a whole, we have been researching this area for over a decade. Rapita are working with key industry stakeholders, including major chip-manufacturers like NXP, to support them in refining the evidence required to satisfy certification authorities.

Rapita have extensive experience in providing software verification solutions for some of the best-known aerospace and automotive companies in the world. For example, BAE Systems used RapiTime (one of the tools in MACH¹⁷⁸) to identify worst-case execution time optimizations for the Mission Control Computer on their Hawk Trainer jet.

See more of our Case Studies.

Yes. With MACH¹⁷⁸, assurance evidence can be developed incrementally and independently for the multicore platform and each hosted application.

The solution is designed to meet use cases for each of the avionics roles identified in DO-297/ED-124, whether you're a Certification Applicant, System Integrator, Platform or Application Supplier. The solution supports the needs of Certification Applicants and System Integrators by defining a consistent strategy for generating certification evidence across all platforms and applications.

RVS, Rapita's verification toolsuite, supports multicore timing analysis by letting you create and run multicore timing tests, automatically capture verification results as these tests run on your multicore platform, efficiently analyze collected results, and generate compliance evidence when your analysis is complete.

RVS includes a range of features that support multicore timing analysis, including:  

• RVS supports the collection of a range of metrics during testing, including execution time results and values from hardware event monitors on your platform, such as the number of cache hits and cache misses.
• RVS makes it easy to view and analyze multicore timing results by letting you filter your results on the performance metrics and tests you want to see and letting you select a baseline against which to compare your results.
• RVS lets you generate custom exports to provide compliance evidence. Custom exports reduce your documentation effort by automatically pulling in results from your reports and automatically reporting pass/fail status based on your success criteria.
• RapiTime lets you automatically merge timing results collected during multicore timing analysis into a single report. This lets you collect and analyze results from multiple test runs, multiple builds and multiple time points, and supports running results on multiple test rigs.

RapiDaemons support multicore timing analysis by allowing the timing behavior of a multicore platform to be analyzed under different levels of resource contention. By stressing specific shared resources at known levels, they support precise analysis.

For more information on RapiDaemons, see the RapiDaemons web page.

Yes. While AMC 20-193 and CAST-32A do not specifically provide guidance on compliance for multicore platforms with different processors, multicore interference may be present in these systems, and as such the guidance applies to them. 

In general, statistical modeling approaches such as Queueing Theory are not applicable to timing analysis of multicore software as software timing behavior does not fit standard statistical assumptions. While the analysis of multicore timing results is based on a representation of the measured real distribution of execution times, it would not be correct to try to estimate averages and standard distributions from the data because we cannot assume that it follows any standard statistical distribution without experimental evidence to show this. 

In some cases, software timing behavior may fit standard statistical assumptions, but this is the exception rather than the rule and must be proven before relying on results from statistical modeling. 

MACH¹⁷⁸ Foundations is a library of plans, procedures, checklists and other resources that provide a solid foundation for engineering teams working on multicore DO-178C (AC 20-193 or AMC 20-193) projects, especially when following the MACH¹⁷⁸ workflow.

It includes the following documents to support your multicore compliance journey:

• Template plans for A(M)C 20-193 certification (additions to the PSAC and SVP)
• Procedures, templates and checklists to support using the MACH178 workflow for A(M)C 20-193 compliance
• Training resources to help you get started applying the MACH178 workflow to your project
• White papers to provide guidance on specific aspects of multicore certification

MACH¹⁷⁸ Foundations and the MACH¹⁷⁸ workflow are the culmination of tens of person years of work from specialist multicore hardware and verification engineers at Rapita Systems, making it an excellent stepping stone on your compliance journey.

The following software tools support multicore DO-178C (AC 20-193 and AMC 20-193) compliance using the MACH¹⁷⁸ workflow:

• RapiDaemons enable the targeted generation of traffic on shared hardware resources to generate interference during tests, so the impact of interference on platform and software behavior can be analyzed
• RapiTest supports writing and executing tests (including the application of RapiDaemons) to analyze and verify software performance
• RapiTime supports collecting performance metrics including execution time and values from Hardware Event Monitors during testing, and analyzing the results
• RapiTask supports visualizing and analyzing software scheduling

RapiTest, RapiTime and RapiDaemons, which are used to automate MACH¹⁷⁸ processes, are classified as Tool Qualification Level 5 tools as per DO-330. Qualification kits and services provide the evidence needed to qualify their use in MACH¹⁷⁸ projects.

The following services support multicore DO-178C (AC 20-193 and AMC 20-193) compliance using the MACH¹⁷⁸ workflow:

• Tool Integration – this service ensures a robust integration of MACH¹⁷⁸ software tools into your platform and development environment so verification activities can be performed efficiently
• Platform Analysis and Characterization – this service supports the analysis of interference channels and characterization of their potential impact on specific multicore platforms
• Software Analysis and Characterization – this consultancy service supports your approach to meeting A(M)C 20-193’s MCP_Software_1 and MCP_Software_2 objectives
• Training – we provide a range of training to support your A(M)C 20-193 compliance journey and application of the MACH¹⁷⁸ workflow
• Consultancy – we can answer any further questions you have about A(M)C 20-193 compliance with specialist consultancy services

The MACH¹⁷⁸ Blueprint is an off-the-shelf platform that you can use to learn more about multicore DO-178C (AC 20-193 and AMC 20-193) compliance, and run research and development activities.

The MACH¹⁷⁸ Blueprint is based on an example multicore project on which A(M)C 20-193 compliance evidence has been produced using the MACH¹⁷⁸ workflow. 

The Blueprint includes the following:

• Example DO-178C plans (PSAC, SVP) for the Blueprint platform to cover multicore compliance and verification activities
• Example results from using the MACH¹⁷⁸ workflow on the Blueprint project, including completed checklists for the compliance and verification activities performed
• Software tools and project files that can be used to run select parts of the MACH¹⁷⁸ workflow on the Blueprint project, including writing and editing tests, and analyzing and exporting results. 
• Training to support your use of the MACH¹⁷⁸ Blueprint

Example plans and results are instantiated versions of template documents available in MACH¹⁷⁸ Foundations.  

You can find a repository of resources to learn more about certifying multicore software on our multicore resources webpage.

We run public and private training courses on multicore certification. You can find a list of our upcoming public courses here, or get in touch to discuss private courses.