Roofline model

Example of a naïve Roofline plot where two kernels are reported. The first (vertical dashed red line) has an arithmetic intensity ${\displaystyle O_{1}}$ that is underneath the peak bandwidth ceiling (diagonal solid black line), and is then memory-bound. Instead, the second (corresponding to the rightmost vertical dashed red line) has an arithmetic intensity ${\displaystyle O_{2}}$ that is underneath the peak performance ceiling (horizontal solid black line), and thus is compute-bound.

The naïve Roofline[3] is obtained by applying simple bound and bottleneck analysis.[8] In this formulation of the Roofline model, there are only two parameters, the peak performance and the peak bandwidth of the specific architecture, and one variable, the arithmetic intensity. The peak performance, in general expressed as GFLOPS, can be usually derived from architectural manuals, while the peak bandwidth, that references to peak DRAM bandwidth to be specific, is instead obtained via benchmarking.[1][3] The resulting plot, in general with both axes in logarithmic scale, is then derived by the following formula:[1]

$${\displaystyle P=\min {\begin{cases}\pi \\\beta \times I\end{cases}}}$$

where ${\displaystyle P}$ is the attainable performance, ${\displaystyle \pi }$ is the peak performance, ${\displaystyle \beta }$ is the peak bandwidth and ${\displaystyle I}$ is the arithmetic intensity. The point at which the performance saturates at the peak performance level ${\displaystyle \pi }$, that is where the diagonal and horizontal roof meet, is defined as ridge point.[4] The ridge point offers insight on the machine's overall performance, by providing the minimum arithmetic intensity required to be able to achieve peak performance, and by suggesting at a glance the amount of effort required by the programmer to achieve peak performance.[4]

A given kernel or application is then characterized by a point given by its arithmetic intensity ${\displaystyle I}$ (on the x-axis). The attainable performance ${\displaystyle P}$ is then computed by drawing a vertical line that hits the Roofline curve. Hence. the kernel or application is said to be memory-bound if ${\displaystyle I\leq \pi /\beta }$. Conversely, if ${\displaystyle I\geq \pi /\beta }$, the computation is said to be compute-bound.[1]

The naive Roofline provides just an upper bound (the theoretical maximum) to performance. Although it can still give useful insights on the attainable performance, it does not provide a complete picture of what is actually limiting it. If, for instance, the considered kernel or application performs far below the Roofline, it might be useful to capture other performance ceilings, other than simple peak bandwidth and performance, to better guide the programmer on which optimization to implement, or even to assess the suitability of the architecture used with respect to the analyzed kernel or application.[3] The added ceilings impose then a limit on the attainable performance that is below the actual Roofline, and indicate that the kernel or application cannot break through anyone of these ceilings without first performing the associated optimization.[3][4]

The Roofline plot can be expanded upon three different aspects: communication, adding the bandwidth ceilings; computation, adding the so-called in-core ceilings; and locality, adding the locality walls.

• 
  An example of a Roofline model with added bandwidth ceilings. In this model, the two additional ceilings represent the absence of software prefetching and NUMA organization of memory.
• 
  An example Roofline model with added in-core ceilings, where the two added ceilings represent the lack of instruction level parallelism and task level parallelism.
• 
  An example Roofline model with locality walls. The wall labeled as 3 C's denotes the presence all three types of cache misses: compulsory, capacity and conflict misses. The wall labeled as 2 C's represent the presence of either compulsory and capacity or compulsory and conflict misses. The last wall denotes the presence of just compulsory misses.

Since its introduction,[3][4] the model has been further extended to account for a broader set of metrics and hardware-related bottlenecks. Already available in literature there are extensions that take into account the impact of NUMA organization of memory,[6] of out-of-order execution,[9] of memory latencies,[9][10] and to model at a finer grain the cache hierarchy[5][9] in order to better understand what is actually limiting performance and drive the optimization process.

Also, the model has been extended to better suit specific architectures and the related characteristics, such as FPGAs.[11]

• Software performance testing
• Benchmark (computing)

• The Roofline Model: A Pedagogical Tool for Auto-tuning Kernels on Multicore Architectures
• Applying the Roofline model
• Extending the Roofline Model: Bottleneck Analysis with Microarchitectural Constraints