What is a Cache?

When requesting data from any source, theres always some delay.

Access speed is regularly some form of bottleneck in performance.

One of the ways this can be addressed is with caching.

There are a huge range of implementations both in software and hardware.

Caches can act as read caches, write caches, or both.

The read cache that youre most likely to be familiar with is the web client cache.

Here the web app stores a local copy of requested resources.

RAM itself also acts as a read cache for data in the hard drive.

A common example of this would be the SLC cache in modern SSDs.

Using the flash memory in this way optimises it for both fast write speeds and high storage density.

Each of these methods handles write operations differently and have benefits and drawbacks.

The three options are write-around, write-through, and write-back.

Write-around can be useful if youre expecting a large volume of writes as it minimises cache churn.

It doesnt require write operations to write to disk to be considered complete though.

How to remove data from the cache?

One of the limiting factors of any cache is capacity.

Memory technologies used for caching also tend to be more expensive than the memory theyre caching from.

RAM has less capacity than storage and CPU cache has less capacity than RAM.

The SLC cache has less capacity than the TLC memory.

There are a range of different approaches to this.

Least frequently used, prefers to evict cache entries that have the lowest access count.

Least recently used prefers to evict cache entries that havent been used in a while.

The best approach is generally a combination of all three, informed by usage stats.

Its important to regularly verify that the live copy being served still matches the cached copy.

In websites specifically, its also extremely important to identify what data can and cant be cached.

For example, its perfectly fine for a large unchanging JavaScript file to be cached.

You cant cache session-specific data, though.

A cache is typically limited in capacity meaning it needs to evict entries once it is full.