How are unexpected behaviors discovered on multicore platforms?

To identify interference channels on a multicore platform, it is necessary to analyze technical documentation that describes the functionality of platform components. This documentation will generally be accurate, descriptive and helpful, but will likely contain some errors, omissions, contradictions and ambiguities. This might include missing or vague information about a feature or definitions that are in contradiction with the actual functionality. In general, issues can often be resolved or clarified through support from the vendor to enable continued analysis.

In our experience, most unexpected behavior is on-target behavior which differs from that expected from an analysis of the documentation. As such, anomalies or errors are generally detected through on-target testing, and they are generally discovered when attempting to stress associated interference channels by targeted testing of a specific device or feature. A DDR interference channel, for example, will require tests to correctly miss L1, L2 and maybe even L3 cache. Depending on the architecture of the system, this may require using different datasets, strides through pages and access patterns.

Performance counters can be used to provide assurance for the correct execution of tests to characterize interference channels. Their use can highlight unexpected behaviors due to deviations from expected test outcomes, which should prompt further investigation and analysis. In an extreme example of this, we discovered and reverse-engineered an entire undocumented cache, which of course had associated interference channels.