Cache Memory

Memory Access Methods

These are 4 types of memory access methods:

In this method, the memory is accessed in a specific linear sequential manner, like accessing
in a single Linked List. The access time depends on the location of the data.
2. Random Access: In this method, any location of the memory can be accessed randomly
like accessing in Array. Physical locations are independent in this access method.
• Direct Access: In this method, individual blocks or records have a unique address based
on physical location. access is accomplished by direct access to reach a general vicinity
plus sequential searching, counting or waiting to reach the final destination. This method
is a combination of above two access methods. The access time depends on both the
memory organization and characteristics of storage technology. The access is semi-
random or direct.
4. Associate Access: In this memory, a word is accessed rather than its address. This
access method is a special type of random access method. Application of thus
Associate memory access is Cache memory.
Cache Performance

• When the processor needs to read or write a location in the main memory, it first
checks for a corresponding entry in the cache.
• If the processor finds that the memory location is in the cache, a Cache Hit has
occurred and data is read from the cache.
• If the processor does not find the memory location in the cache, a cache miss has
occurred. For a cache miss, the cache allocates a new entry and copies in data
from the main memory, then the request is fulfilled from the contents of the cache.
• The performance of cache memory is frequently measured in terms of a quantity
called Hit ratio.
Hit Ratio(H) = hit / (hit + miss) = no. of hits/total
accesses
Miss Ratio = miss / (hit + miss) = no. of miss/total
accesses = 1 - hit ratio(H)

We can improve Cache performance using higher cache block size, and higher
associativity, reduce miss rate, reduce miss penalty, and reduce the time to hit in the
cache.
Types of Cache Memory

• L1 or Level 1 Cache: It is the first level of cache memory that is present inside the
processor. It is present in a small amount inside every core of the processor separately.
The size of this memory ranges from 2KB to 64 KB.
• L2 or Level 2 Cache: It is the second level of cache memory that may present inside or
outside the CPU. If not present inside the core, It can be shared between two cores
depending upon the architecture and is connected to a processor with the high-speed bus.
The size of memory ranges from 256 KB to 512 KB.
• L3 or Level 3 Cache: It is the third level of cache memory that is present outside the
CPU and is shared by all the cores of the CPU. Some high processors may have this
cache. This cache is used to increase the performance of the L2 and L1 cache. The size of
this memory ranges from 1 MB to 8MB.
Cache Memory design

The key elements are concisely summarized here. we are going to see that similar style
problems should be self-addressed in addressing storage and cache style. They represent the
subsequent categories: Cache size, Block size, Mapping function, Replacement algorithm,
and Write policy. These are explained as following below.
Cache Memory Design

• Cache Size: It seems that moderately tiny caches will have a big impact on performance.
• Block Size: Block size is the unit of information changed between cache and main
memory. As the block size will increase from terribly tiny to larger sizes, the hit
magnitude relation can initially increase as a result of the principle of locality.
• As the block size increases, a lot of helpful knowledge square measure brought into the
cache. The hit magnitude relation can begin to decrease, however, because the block
becomes even larger and also the chance of victimization the new fetched knowledge
becomes but the chance of reusing the information that ought to be abstracted of the cache
to form area for the new block.
Mapping Function

• When a replacement block of data is scan into the cache, the mapping performs
determines that cache location the block will occupy.
• Two constraints have an effect on the planning of the mapping perform. First, once one
block is scan in, another could be replaced. We would wish to do that in such the simplest
way to minimize the chance that we are going to replace a block which will be required
within the close to future.
• A lot of versatile the mapping perform, a lot of scopes we’ve to style a replacement
algorithmic rule to maximize the hit magnitude relation. Second, a lot of versatile the
mapping perform, a lot of advanced is that the electronic equipment needed to look the
cache to see if a given block is within the cache.
Replacement Algorithm

The replacement algorithmic rule chooses, at intervals, the constraints of the mapping
perform, which block to interchange once a replacement block is to be loaded into the cache
and also the cache already has all slots full of alternative blocks. We would wish to replace
the block that’s least possible to be required once more within the close to future. Although
it’s impossible to spot such a block, a fairly effective strategy is to interchange the block
that has been within the cache longest with no relevance. This policy is spoken because of
the least-recently-used (LRU) algorithmic rule. Hardware mechanisms square measure
required to spot the least-recently-used block
Write Policy

• When a block in the cache gets changed, we need to update the corresponding data in the main
memory too. How and when we do this is decided by a policy called the write policy.
There are two main approaches:
• Write-Through: This means updating the main memory every time the block in the cache is
changed. It keeps the main memory up-to-date but can lead to a lot of write operations, which
might slow things down.
• Write-Back: Here, we only write the changes back to main memory when the block is replaced in
the cache. This reduces the number of write operations, but it means the main memory might not
have the latest data. This can cause issues in systems with multiple processors or when interacting
with input/output devices.
• In simple terms, it's about deciding when to sync up the changes made in the cache with the main
memory.
Advantages of Cache Memory Design

• Faster Access Time: Cache memory is designed to provide faster access to frequently
accessed data. It stores a copy of data that is frequently accessed from the main memory,
allowing the CPU to retrieve it quickly. This results in reduced access latency and
improved overall system performance.
• Reduced Memory Latency: Cache memory sits closer to the CPU compared to the main
memory. As a result, accessing data from the cache has lower latency compared to
accessing data from the main memory. This helps in reducing the memory access time and
improves the efficiency of the system.
• Improved System Performance: By reducing the memory access time and providing
faster access to frequently used data, cache memory significantly enhances the overall
performance of the system. It helps in reducing CPU idle time, improving instruction
execution speed, and increasing the throughput of the system.
Disadvantages of Cache Memory Design
• Limited Capacity: Cache memory has limited capacity compared to the main memory. It
is designed to store a subset of frequently used data. As a result, it may not be able to
accommodate all the data needed by the CPU. Cache capacity limitations can lead to
cache misses, where the required data is not found in the cache, resulting in slower
memory access from the main memory.
• Increased Complexity: Cache memory adds complexity to the overall system design. It
requires sophisticated algorithms and hardware mechanisms for cache management,
including cache replacement policies, coherence protocols, and cache consistency
maintenance. Managing cache coherence and maintaining data consistency between cache
and main memory can be challenging in multiprocessor systems.
• Cache Consistency Issues: In multiprocessor systems, cache coherence becomes a
critical issue. When multiple processors have their own caches, ensuring the consistency
of data across caches can be complex. Cache coherence protocols are required to ensure
that all processors observe a consistent view of memory. Implementing cache coherence
protocols adds complexity and can introduce additional overhead.
Cache Mapping

There are three different types of mapping used for the purpose of cache memory
which is as follows:
• Direct Mapping
• Associative Mapping
• Set-Associative Mapping
Direct Mapping

• The simplest technique, known as direct mapping, maps each block of main memory into
only one possible cache line.
• In Direct mapping, assign each memory block to a specific line in the cache. If a line is
previously taken up by a memory block when a new block needs to be loaded, the old
block is trashed.
• An address space is split into two parts index field and a tag field. The cache is used to
store the tag field whereas the rest is stored in the main memory. Direct mapping`s
performance is directly proportional to the Hit ratio.
i = j modulo m
where
i = cache line number
j = main memory block number
m = number of lines in the cache
Associative Mapping

In this type of mapping, associative memory is used to store the content and addresses of
the memory word. Any block can go into any line of the cache. This means that the word id
bits are used to identify which word in the block is needed, but the tag becomes all of the
remaining bits. This enables the placement of any word at any place in the cache memory. It
is considered to be the fastest and most flexible mapping form. In associative mapping, the
index bits are zero.
Set Associative Mapping

• This form of mapping is an enhanced form of direct mapping where the drawbacks of
direct mapping are removed.
• Set associative addresses the problem of possible thrashing in the direct mapping method.
It does this by saying that instead of having exactly one line that a block can map to in the
cache, we will group a few lines together creating a set.
• Then a block in memory can map to any one of the lines of a specific set. Set-associative
mapping allows each word that is present in the cache can have two or more words in the
main memory for the same index address.
• Set associative cache mapping combines the best of direct and associative cache mapping
techniques. In set associative mapping the index bits are given by the set offset bits. In
this case, the cache consists of a number of sets, each of which consists of a number of
lines.