Go beyond simple profiling

If you've measured execution time, eliminating the context switches and the variability they cause (see our last blog post), you may still have a wide variety of execution times that arise from the thread you're measuring. A natural question is why this should happen. To be able to answer such questions, you need to have the right level of information.

Some embedded development products, such as integrated development environments (IDEs) or debuggers feature simple performance measurement tools that measure minimum, maximum and average execution times.

A more informative approach is to collect execution time profiles (ETPs), which shows the actual distribution of execution times.

Execution Time Profile (ETP)

If the ETP shown was measured with a typical profiler that gives only minimum, maximum and average times, it would give the following results:

Minimum: 28,000 cycles

Maximum: 86,000 cycles

Average: 38,000 cycles

With only this information, it could lead to several incorrect assumptions about the timing of the software. It is considerably more informative to see the entire distribution, which can lead to a much greater understanding of the underlying behaviour of the system.

It is also helpful to be able to relate execution time measurements to specific test cases.

Execution time to test case mapping

With this kind of information, if a specific test case has an unexpected impact, it can be identified and it can be repeated. Unusual timings can be related back to specific test cases to identify their cause.

Next week: Minimize instrumentation overheads