Prefetching

Prefetching works by predicting which memory addresses or resources will be accessing and load them into faster access storage, like caches.

Prefetching may be used:

• Hardware-level, such as CPU memory controllers
• Software-level, strategies in compilers, operating systems, logic in web browsers or file systems

Processors (CPU's) often include prefetching that attempts to reduce cache misses by loading data into cache before it is requested by the running program. This is for programs that access memory in predictable patterns, such as loops that iterate over arrays.

Hardware prefetching can be done without software involvement and can be found in most modern CPU's. For example, Intel CPU's feature a variety of prefetch that work across multiple cache levels.

• Stride prefetching detects constant-stride memory access patterns (fixed distance between consecutive memory accesses)
• Stream prefetching identifies long sequences of contiguous memory accesses (sequential access to a block of memory)
• Correlation prefetching learns patterns between cache misses and triggers prefetches based on those patterns

Prefetch instructions can be written into the code by the programmer or by the compiler. Prefetch instructions specify the memory addresses to be prefetched and the desired prefetch distance.

In software, there are instructions that can be written with:

• prefetch on x86 architecture
• __builtin_prefetch in the GCC compiler
• _mm_prefetch in the Intel Intrinsics Guide

Operating systems

Operating systems use prefetching to reduce file and memory access latency.

• Anticipatory Paging, attempts to prefetch memory pages based on pages in RAM and not in RAM

Web Browsers

Web browsers apply prefetching techniques to improve perceived performance. Common examples include:

• DNS prefetching: resolves domain names of links likely to be clicked
• Link prefetching: loads linked web pages or assets before the user clicks them
• Resource preloading: fetches critical resources (like fonts or scripts) in parallel with the current page load

Prefetching can significantly improve performance, but it can not always be beneficial if implemented wrong. If predictions are inaccurate, prefetching may waste bandwidth, processing time, or cause cache pollution. In systems with limited resources or highly unpredictable workloads, prefetching can degrade performance rather than improve it.

Implementing both software and hardware prefetching can also lead to degraded performance because of interactions that might occur between each other from how it was implemented.