Cache Memory Introduction and Cache Performance Issues

Cache Memory Introduction, Types and Cache Performance Issues

What is Cache Memory?
What are different types of cache memory?
What are the performance issues in cache memory?
How to improve cache performance?
What is the locality of reference in cache memory?
performance issues in cache memory
This article covers the basic concepts of cache memory such as what is cache memory in computer and  types of cache memory in computer architecture and  cache performance.

What is Cache Memory?

In order to execute an instruction CPU need to fetch that instruction from main memory. But data accessing cost from main memory is high and it takes more time. So we need a device or memory from which CPU can access the instruction in less time as compared to main memory.
A register may be this device which is the fastest memory but the problem with the register is that it can not hold the whole program, it can hold only a few instructions and it is more costly.
So there is another memory known as the cache memory which is faster from main memory but store less amount of data as compared to main memory and more data as compare to register.
Cache memory is a smaller and faster memory. This is the importance of cache memory. Cache generally holds those part of the program which is most frequently needed by the CPU.I hope that now it is clear that what is cache memory in computer ?

cache memory location in computer

Figure 1. Cache memory organization in a computer system

Types of cache memory in computer architecture or Levels of Cache Memory

In this section of cache memory tutorial, we will explain types of cache memory in computer architecture. Generally, there are 3 different levels or types of cache in the computer. Here level number refers to the distance of cache from the CPU. Each level of cache is as follow-

L1 Cache – Level 1 cache memory is also known as primary cache or internal cache. It is built directly into the processor. Its capacity is very small from 8 kb to 128 Kb. Level 1 Cache is closest to CPU.

L2 Cache –  Level 2 cache is slower as compared to level 1 cache. But its storage capacity is more up to 16 MB. Nowadays most of the processor have advance transfer cache which is a level 2 cache memory.

L3 Cache –  This type of cache is separate from the processor. It is slower as compared to the cache of level  L1 and L2.

types of cache memory

Figure 2: Different Levels of cache memory

What is Cache Performance

Now we come to the cache performance calculation. So as I explained in the previous section that  When the processor needs to read or write a location in main memory, it first checks for a corresponding entry in the cache. In this situation, there may be two cases.
Case 1: If the processor finds that the memory location is in the cache, then a cache hit occurs or sometimes we say that cache hit has occurred and CPU retrieved the data from the cache.
Case 2: If the processor does not find the memory location in the cache, then this is a cache miss or some time we say that cache miss has occurred. Now in this case for a cache miss, at first the cache allocates a new entry and copies in data from main memory, then the request is fulfilled from the contents of the cache.
In order to calculate the performance of the CPU, you need to understand these two-term cache hit and cache miss. We have already discussed it in the previous paragraph of this section.
Generally, the performance of cache memory is frequently measured in terms of a quantity called Hit ratio.
Hit ratio = hit / (hit + miss) =  no. of hits/total accesses

How to improve Cache performance?

If somehow we are able to reduce the miss rate then cache performance becomes much better. So how can we reduce the cache miss rate?

(1)   If we use the larger cache block size, then sometimes it helps in reducing the cache miss rate. Larger block size helps with spatial locality. But sometimes these methods are not suitable because the large size of cache block reduces a total number of blocks in the cache.  Which increase the miss penalty.
(2)   Another solution is Increase the Associativity.  Higher associativity can improve the reduce the miss rate.

The Locality of reference in Cache Memory

The locality of reference in cache memory some time also called the principle of locality of reference. This is the term which applied to a situation where the same value or related storage locations are frequently accessed.
The locality of reference may be of three types of temporal, spatial and sequential. Let us see one by one to each.

Temporal locality: In the temporal locality, a resource that is referenced at one point in time is referenced again soon afterward.

Spatial locality: In the spatial locality, the likelihood of referencing a storage location is greater if a storage location near it has been recently referenced.

Sequential locality: In sequential locality storage is accessed sequentially, in descending or ascending order.

The occurrence of locality depends on the manner in which computer programs are created. Generally, related data are stored in consecutive locations in storage.
One common way of computing is that processing is performed on a single item and then the next item. Interpret that if more processing is done, the single item will be accessed more than once, which leads to the temporal locality of reference.
When moves to the next item this implies that the next item will be read, hence it is the spatial locality of reference because memory locations are generally read in batches.

Also, read: Cache memory Mapping Techniques

Leave a Reply

Your email address will not be published. Required fields are marked *