**2.2 Initialization**

224 Emerging Informatics – Innovative Concepts and Applications

Multi Layer cache system. Processing node (P-Node) has L3 cache and it performs low latency of time response. L3 cache is dedicated block cache memory on P-Node. Because operating system managed memory doesn't hold specific block data of application services inside of the memory [10, 11], there have been a number of efforts to improve I/O performance however these are substantially different from our work. In [12], the author proposes Unified Buffer Cache (UBC). The focus is to unify the file system and virtual memory caches of file data to improve I/O transactions. However it is an unmanaged one level cache that is very much different from the L3 cache. Similarly [13], [14] provide solution for high I/O, based on RAM disk memory [18] and solid-state disk, and is altogether different from L3 block cache [16]. Other side of block cache is L4 cache. It is inside block device in the Content Node (C-Node). Pre-fetching read is performed by C-Node L4 cache. Thus, two different cache

nodes manage user pushing type services model with low latency time web service.

Section 3 is its evaluation and section 4 concludes our work.

**2.1 System architecture** 

The system has following features.

1. Dual Data Fields for Processing and Content. 2. Processing Data Field for Process trio Node group. 3. Content Data Field for Content Trio Node group.

**2. Autonomous multi-layer cache system architecture** 

push web service. The system architecture is shown as Figure 1.

Fig. 1. Autonomous Decentralized Multi-Layer Cache system architecture

4. P-Node and C-Node are configured Trio Node group on each Data Field.

5. Data availability is achieved by dual storage disk partitions.

Section 1 is introduction and section 2 is Autonomous multi-layer cache system architecture.

Autonomous decentralized multi-layer cache system is designed to achieve low latency user

Initialization process is shown in Figure-2.


Fig. 2. Initialization process

Autonomous Decentralized Multi-Layer

cache.Figure 4 shows L3 block cache P-Node.

Fig. 4. L3 Block cache P-Node

they achieve low latency write event manner.

Cache System to Low Latency User Push Web Services 227

L3 write back cache. L3 write back cache holds write data until the dirty flush timing comes. At the flush time L3 cache autonomously flush the dirty data. The summarization of P-Node features is following. 1) Three nodes are collaborating as trio node group. 2) Write event request executes by L3 cache write back policy in local memory space. 3) Each P-Node is mounted two storage partitions on C-Node to maintain its data availability. 4) Minimized write I/O latency. 5) Autonomous read cache node expansion. The uniqueness of P-Node is trio node configuration. Each node has two storage partitions and three nodes that have mirroring partitions as mirrored. Therefore, if one P-Node fails, other node can continue the system operation and maintain its availability. But, the main purpose of two partitions are mirrored by three nodes is write back cache with no dirty data flushing policy. Because P-Node achieves timeliness write I/O event without any write speed performance drop. P-Node receives write event request by its L3 write back cache. Creating dirty data flush is done by other P-Node that was mounted on same partition without L3 write-back

Other space is OS memory space. Other one of node service runs on this memory space. L3 block cache policy is always write-back cache and other one is no write-back policy. The P-Node performs the high I/O response from the request by L3 block cache and performs write eventually coherency process. Thus, P-Node activates by two nodes minimum and

There are three features of L3 cache. Firstly it converts local memory into I/O block device cache, and mounts this on C-Node target device. Secondly cache policy selection as per the application requirement, and lastly an appropriate technique to search and access the data efficiently in the L3 block cache. The selection of block device is very flexible, for instance if system needs more high I/O speed for I/O and network transactions then the local memory is the ultimate choice. The local memory is transformed to a block I/O device to behave as a cache layer. Consequently it performs high I/O transactions and significantly improves response by utilizing bandwidth equal to the local memory. L3 cache management software detects the block address, which was requested, and keeps this data on the cache device. So, most of the requests are fulfilled from the cache. If the requested block data is not available on the cache, then it is accessed from target storage device. The third function of the L3

10. C-Node (A/B), C-Node (B/C) and C-Node(C/A). Therefore, P-Node trio group also has same storage partition as P-Node (A/B), P-node (B/C) and P-Node (C/A).

After mounting of C-Node successfully, application program and service are loaded from Cnode. Now P-Node is ready to start web service for C-Node.

### **2.3 Processing Node (P-Node) autonomous failure sense and replacement process**

Processing node (P-Node) is timeliness high I/O response event execution node. To execute timeliness event, P-Node has L3 cache space in its local memory to do one hop write event. Each P-Node in trio group communicates with each other. When, one of P-node fails, other P-node in trio group detects the status. The failure replace process is shown in Figure 3. In this case, P-Node (A/B) is failed and P-Node (B/C) detected this status by the event of corelated storage partition as "B". P-Node (B/C) broadcasted the message for inquiry of available extra P-Node via P-DF (Step-1). There are three P-nodes available and the nearest position P-Node is to be the candidate of new member node. The fastest replier P-Node is selected by this policy (Step-2). P-Node (B/C) and P-Node (C/A) are creating new P-Node connection (Step-3). New P-Node (A/B) starts to mount C-Node (A/B) and load application program from C-Node (A/B). During the creation of new P-Node (A/B), the service for storage partition "A", "B" and "C" are sustaining by P-Node (B/C) and P-Node (C/A). P-Node doesn't have any application program and its data. These data and service program are stored in C-Node. Therefore, P-Node replacement event can be so easy and so quickly for change. The data in L3 cache on failure P-Node (A/B), the possibility of dirty data on storage partition "A". This dirty data "A" is flushed by P-Node (C/A) message. Therefore, write dirty data coherency is protected.

Fig. 3. P-Node (A/B) replacement process

#### **2.4 Processing Node (P-Node)**

Processing Node (P-Node) has L3 cache in its local memory space. Unlike OS local memory and UBC, this cache space is storage block address cache and is generated by cache control software with RAM disk driver. Therefore, L3 cache is not visible from OS. The main purpose of L3 cache is timeliness write I/O event execution by write back cache policy. Each P-Node is mounted two partitions on C-Node and one side of partition is mounted through

10. C-Node (A/B), C-Node (B/C) and C-Node(C/A). Therefore, P-Node trio group also has

After mounting of C-Node successfully, application program and service are loaded from C-

**2.3 Processing Node (P-Node) autonomous failure sense and replacement process**  Processing node (P-Node) is timeliness high I/O response event execution node. To execute timeliness event, P-Node has L3 cache space in its local memory to do one hop write event. Each P-Node in trio group communicates with each other. When, one of P-node fails, other P-node in trio group detects the status. The failure replace process is shown in Figure 3. In this case, P-Node (A/B) is failed and P-Node (B/C) detected this status by the event of corelated storage partition as "B". P-Node (B/C) broadcasted the message for inquiry of available extra P-Node via P-DF (Step-1). There are three P-nodes available and the nearest position P-Node is to be the candidate of new member node. The fastest replier P-Node is selected by this policy (Step-2). P-Node (B/C) and P-Node (C/A) are creating new P-Node connection (Step-3). New P-Node (A/B) starts to mount C-Node (A/B) and load application program from C-Node (A/B). During the creation of new P-Node (A/B), the service for storage partition "A", "B" and "C" are sustaining by P-Node (B/C) and P-Node (C/A). P-Node doesn't have any application program and its data. These data and service program are stored in C-Node. Therefore, P-Node replacement event can be so easy and so quickly for change. The data in L3 cache on failure P-Node (A/B), the possibility of dirty data on storage partition "A". This dirty data "A" is flushed by P-Node (C/A) message. Therefore,

Processing Node (P-Node) has L3 cache in its local memory space. Unlike OS local memory and UBC, this cache space is storage block address cache and is generated by cache control software with RAM disk driver. Therefore, L3 cache is not visible from OS. The main purpose of L3 cache is timeliness write I/O event execution by write back cache policy. Each P-Node is mounted two partitions on C-Node and one side of partition is mounted through

same storage partition as P-Node (A/B), P-node (B/C) and P-Node (C/A).

node. Now P-Node is ready to start web service for C-Node.

write dirty data coherency is protected.

Fig. 3. P-Node (A/B) replacement process

**2.4 Processing Node (P-Node)** 

L3 write back cache. L3 write back cache holds write data until the dirty flush timing comes. At the flush time L3 cache autonomously flush the dirty data. The summarization of P-Node features is following. 1) Three nodes are collaborating as trio node group. 2) Write event request executes by L3 cache write back policy in local memory space. 3) Each P-Node is mounted two storage partitions on C-Node to maintain its data availability. 4) Minimized write I/O latency. 5) Autonomous read cache node expansion. The uniqueness of P-Node is trio node configuration. Each node has two storage partitions and three nodes that have mirroring partitions as mirrored. Therefore, if one P-Node fails, other node can continue the system operation and maintain its availability. But, the main purpose of two partitions are mirrored by three nodes is write back cache with no dirty data flushing policy. Because P-Node achieves timeliness write I/O event without any write speed performance drop. P-Node receives write event request by its L3 write back cache. Creating dirty data flush is done by other P-Node that was mounted on same partition without L3 write-back cache.Figure 4 shows L3 block cache P-Node.

Fig. 4. L3 Block cache P-Node

Other space is OS memory space. Other one of node service runs on this memory space. L3 block cache policy is always write-back cache and other one is no write-back policy. The P-Node performs the high I/O response from the request by L3 block cache and performs write eventually coherency process. Thus, P-Node activates by two nodes minimum and they achieve low latency write event manner.

There are three features of L3 cache. Firstly it converts local memory into I/O block device cache, and mounts this on C-Node target device. Secondly cache policy selection as per the application requirement, and lastly an appropriate technique to search and access the data efficiently in the L3 block cache. The selection of block device is very flexible, for instance if system needs more high I/O speed for I/O and network transactions then the local memory is the ultimate choice. The local memory is transformed to a block I/O device to behave as a cache layer. Consequently it performs high I/O transactions and significantly improves response by utilizing bandwidth equal to the local memory. L3 cache management software detects the block address, which was requested, and keeps this data on the cache device. So, most of the requests are fulfilled from the cache. If the requested block data is not available on the cache, then it is accessed from target storage device. The third function of the L3

Autonomous Decentralized Multi-Layer

enhance the cache hit availability.

Fig. 5. L4 cache design diagram

Cache System to Low Latency User Push Web Services 229

Therefore, write event latency of time to be shorter than non L4 cache C-Node model. C-Node is Content Node that has dynamic data availability function for web application services. There is iSCSI base connection from P-Node to C-Node and it enables to connect P-Node for high I/O response execution. The capacity of storage is virtualized value that was shown and it is showing multi Terabyte storage capacity. C-Node has two partitions regularly and they are mirroring by the trio nodes as shown in Figure 5. The disk partitions inside of C-Node are main pool and sub pool such as "A" and "B", "B" and "C" and "C" and "A". Therefore, if "A" partition disk is failed; other duplicated partition can be used for maintaining the service. C-Node is connecting Data Filed and is waiting the event from upper P-Node. When P-Node receives the storage mount point from C-Node, P-Node mounts to C-Node and starts each service from each partition. Once each service is started, C-Node tries to maintain frequently accessed block address for each service on L4 cache to

Thus, L4 cache on C-Node holds the most used block data on the L4 cache and is utilized for P-Node expansion when service demand is increased. Generally, C-Node can connect other C-Node whenever P-Node in Data Field requests new extra storage capacity. The process of expansion event for C-Node executes by block data analysis monitor in P-Node. The block data analysis tool gets the block address total sum in the sampling time and if the total block address exceeds the threshold value, this P-Node broadcasts the expansion request to extra C-Node autonomously. The threshold value considers the balance L3<L4 to maintain timeliness I/O performance. C-Node has L4 cache that is located inside of each storage drive. The L4 cache implements pre-fetching technology to reduce the first read cache penalty. The accuracy of read cache hit ratio is determined by algorithm. There are many cache pre-fetching techniques available such as Least Recently Used (LRU), First In First Out (FIFO), Hidden Markov and NN+ CBR. In this case, LRU model is selected. C-Node autonomous storage node functions were implemented by following independent modules. 1) L4 cache layer management module. 2) Failure sense and event notice module. 3) Autonomous Local storage duplication module. 4) Pre-fetching read module. L4 cache enhances read / write cache hit ratio by dedicated L4 block cache space. Event the first read cache penalty, C-Node pre-fetches the most possible read data by its pre-fetching technique. Therefore, the almost read penalty is less than direct storage access. The basic features of L4 cache are 1) Loosely connection, transparency 1GB block cache.2) Write-back policy with

block cache technology as shown as Table –1 is search algorithm to look for data on the L3 block cache efficiently. A new cache search algorithm quickly searches required block of data for I/Os to further reduce the latency of time. The L3 block cache consisting of blocks is divided into groups where each group contains equal number of blocks. The jumping algorithm first points to the required group for which that sector corresponds and then search to the required block of data in the group. Jumping algorithm enhances much higher search time than standard sequential search technique for sequential requests. If the requests are randomly distributed, then more advantage is expected than standard search. Investigation for the value of number of column for block I/O data is carried out. In the current paper the value is 32, but If CPU performance is higher, then 64 or 128 may be selected as this tradeoff depends upon the CPU specifications.

Table 1. P-Node L3 block cache jumping search table

In order to elaborate this mechanism, following is its example. This simple code describes how to search desired block of data effectively on the L3 block cache:

```
IRP Offset =0; 
IRP length = 2048; 
NrBlocks = ((offset + NrSectors - 1)>> *(targetDisk->SectorPerBlock2)) +1; 
NrBlocks = ( ( 0 + 4 - 1)>> 3 ) +1; 
Index = (ULONG) ((blockSector.QuadPart) % (targetDisk->NrBlocks)); 
Index = 1 % 100 = 1; 
RowIndex = (Index / (targetDisk->index Columns)); 
ColumnIndex = 0%32= 0; 
Block = _SearchBlock (targetDisk, blockSector, RowIndex, Column Index); 
Block = _SearchBlock (targetDisk, 0, 0, 0);
```
Block of data will be searched in the list attached on this index that whether it exists on L3 block cache or not. Hashing technique is useful for large size data but it is weak for small set of data. Also when the data address has conflict, open address method is used to resolve the issue.

#### **2.5 Content Node (C-Node)**

Content node (C-Node) has all of service application program and data related to each storage partition. Each C-Node is configured trio group autonomously. The storage partition is encapsulated L4 block cache. The L4 cache is dedicated device block cache and it has write back policy. The request from P-node is entertained in write back cache manner.

block cache technology as shown as Table –1 is search algorithm to look for data on the L3 block cache efficiently. A new cache search algorithm quickly searches required block of data for I/Os to further reduce the latency of time. The L3 block cache consisting of blocks is divided into groups where each group contains equal number of blocks. The jumping algorithm first points to the required group for which that sector corresponds and then search to the required block of data in the group. Jumping algorithm enhances much higher search time than standard sequential search technique for sequential requests. If the requests are randomly distributed, then more advantage is expected than standard search. Investigation for the value of number of column for block I/O data is carried out. In the current paper the value is 32, but If CPU performance is higher, then 64 or 128 may be

In order to elaborate this mechanism, following is its example. This simple code describes

Block of data will be searched in the list attached on this index that whether it exists on L3 block cache or not. Hashing technique is useful for large size data but it is weak for small set of data. Also when the data address has conflict, open address method is used to resolve the

Content node (C-Node) has all of service application program and data related to each storage partition. Each C-Node is configured trio group autonomously. The storage partition is encapsulated L4 block cache. The L4 cache is dedicated device block cache and it has write back policy. The request from P-node is entertained in write back cache manner.

selected as this tradeoff depends upon the CPU specifications.

Table 1. P-Node L3 block cache jumping search table

RowIndex = (Index / (targetDisk->index Columns));

IRP Offset =0; IRP length = 2048;

issue.

Index = 1 % 100 = 1;

ColumnIndex = 0%32= 0;

**2.5 Content Node (C-Node)** 

NrBlocks = ( ( 0 + 4 - 1)>> 3 ) +1;

Block = \_SearchBlock (targetDisk, 0, 0, 0);

how to search desired block of data effectively on the L3 block cache:

NrBlocks = ((offset + NrSectors - 1)>> \*(targetDisk->SectorPerBlock2)) +1;

Block = \_SearchBlock (targetDisk, blockSector, RowIndex, Column Index);

Index = (ULONG) ((blockSector.QuadPart) % (targetDisk->NrBlocks));

Therefore, write event latency of time to be shorter than non L4 cache C-Node model. C-Node is Content Node that has dynamic data availability function for web application services. There is iSCSI base connection from P-Node to C-Node and it enables to connect P-Node for high I/O response execution. The capacity of storage is virtualized value that was shown and it is showing multi Terabyte storage capacity. C-Node has two partitions regularly and they are mirroring by the trio nodes as shown in Figure 5. The disk partitions inside of C-Node are main pool and sub pool such as "A" and "B", "B" and "C" and "C" and "A". Therefore, if "A" partition disk is failed; other duplicated partition can be used for maintaining the service. C-Node is connecting Data Filed and is waiting the event from upper P-Node. When P-Node receives the storage mount point from C-Node, P-Node mounts to C-Node and starts each service from each partition. Once each service is started, C-Node tries to maintain frequently accessed block address for each service on L4 cache to enhance the cache hit availability.

Fig. 5. L4 cache design diagram

Thus, L4 cache on C-Node holds the most used block data on the L4 cache and is utilized for P-Node expansion when service demand is increased. Generally, C-Node can connect other C-Node whenever P-Node in Data Field requests new extra storage capacity. The process of expansion event for C-Node executes by block data analysis monitor in P-Node. The block data analysis tool gets the block address total sum in the sampling time and if the total block address exceeds the threshold value, this P-Node broadcasts the expansion request to extra C-Node autonomously. The threshold value considers the balance L3<L4 to maintain timeliness I/O performance. C-Node has L4 cache that is located inside of each storage drive. The L4 cache implements pre-fetching technology to reduce the first read cache penalty. The accuracy of read cache hit ratio is determined by algorithm. There are many cache pre-fetching techniques available such as Least Recently Used (LRU), First In First Out (FIFO), Hidden Markov and NN+ CBR. In this case, LRU model is selected. C-Node autonomous storage node functions were implemented by following independent modules. 1) L4 cache layer management module. 2) Failure sense and event notice module. 3) Autonomous Local storage duplication module. 4) Pre-fetching read module. L4 cache enhances read / write cache hit ratio by dedicated L4 block cache space. Event the first read cache penalty, C-Node pre-fetches the most possible read data by its pre-fetching technique. Therefore, the almost read penalty is less than direct storage access. The basic features of L4 cache are 1) Loosely connection, transparency 1GB block cache.2) Write-back policy with

Autonomous Decentralized Multi-Layer

**2.7 Read event by L3/L4 cache** 

Cache System to Low Latency User Push Web Services 231

Node, P-node (C/A) signal cast to P-node (A/B) the message of write event was done. Then Pnode (A/B) flag off of write dirty and open the memory space for new write event entry. Therefore, always write-data response time is maintaining high I/O response by L3 write-back

L3 cache on P-Node effects read I/O event also. L3 cache is dedicated storage cache on P-Node and it maintains the data inside of L3 cache. The read event process is following. a) When received the read event "A" from requester, there are two P-Nodes that are related partition "A". b) One of P-Node has L3 block Cache space and other is no cache space of dedicated "A" partition. It just mounts "A" partition. c) When executes read event of "A" partition, the one of P-Node which has L3 cache for partition "A" executes the read event if L3 cache space holding the read event data and broadcast its acknowledge via data field as timeliness manner. d) Other one of P-Node executes by traditional read event execution if read data detects inside of memory space. e) The fastest read event executed node broadcasts the result of read request event to other nodes with event message. f) Once received the read event result from any P-Nodes, this node stops the read event execution. g) Eventually, the fastest P-Node result reaches the requester. P-Node read event is very flexible process. If one of node holding data for the event, anyone can be the fastest replier. Other node that has data for read request is stopped by the fastest node action. When the read event access size exceeds more than L3 cache, L4 cache on C-Node has possibility to maintain the next priority data. C-Node manages the size of L4 cache if P-Node L3 cache size exceeds L4 cache, C-Node expands the L4 cache capacity inside C-node disk

When the event of C-Node failure occurs, co-related P-Node detects C-Node status by its response behavior. For example, C-Node (A/B) is mounted by P-Node (A/B), therefore, P-Node (A/B) detects C-Node (A/B) failure by no response or reply timeout. P-Node (A/B) single casts the message of call extra C-Node via C-DF. If an unassigned C-node is existing, then this C-node broadcasts the status to related C-Node in trio group. The nearest C-Node becomes the new member of C-Node trio group and starts to communicate with three as new trio group. The application is stored another size of C-Node with latest data, C-Node (B/C) provides "Storage partition "B" and C-Node (C/A) supplies storage partition "A" Thus, data duplication is time consuming work but this event runs on only C-DF. Therefore, seeing from out band requester, it is hidden internal task and less impact for out band service. This is one of the advantages by dual data field design. After duplication event is finished, New C-node (A/B) is released to P-Node (A/B). Even before complete this duplication event; other two of C-Nodes with three P-Nodes can sustain the operation

Write I/O event requires two L3 cache nodes to manage write-data coherency by Eventually Consistent manner. This process is utilized by two L3 cache nodes with two L4 C-Nodes for

cache with non-L3 write-through cache without cache memory size limit.

configuration change and it maintains the ratio "L3<L4".

**2.8 Content Node (C-node) failure sense and replace** 

without any service interruption.

**3.1 Determination of geometrical architecture** 

**3. Evaluation** 

pre-read cache implementation and 3) Pre fetching boot up file, 4) Internal UPS that has automatic shutdown function acts whenever un-properly shutdown or upper system power fail re-securing dirty data on the cache table. 5) Can be expandable the total number of L4 cache size by multiple number of L4 cache drive.

This L4 cache space is completely individual autonomous memory space on the device. Today, there are many RAID storage subsystems available. But seeing from the individual device point of view, there are some difference points. RAID controller has buffer cache memory for RAID sets. But, this is the unified block cache space and it for block cache of RAID set and its volume. Also it is not safety idea when RAID subsystem electric power failure occurs. P-Node L4 cache is loosely connection individuality block cache. Only L4 cache device achieves no risk with high I/O speed with assurance benefit. Figure 6 is L4 cache block diagram in C-node. In this general case of design, 1GB DRAM for L4 cache memory with Flash SSD is target mounted device. Internally, 64bit based DRAM assigned and its I/O performance shows more than 270MB/s with 40KIOPS transaction performance under MLC SSD configuration.

Fig. 6. L4 cache design diagram

#### **2.6 Write data coherency process by P-Node**

When write data request received from out side requester, two P-Nodes that are mounted C-Node application program partition execute the write request event by L3 cache P-Node and non-L3 cache P-Node. For instance, If Write event required at storage partition "A", there are two P-Node related this event as P-Node (A/B) and P-Node (C/A). In this case, P-Node (A/B) has L3 cache for "A" and P-Node (C/A) hasn't. The ACK for write event requester replied by P-Node (A/B) using L3 write back cache is by one hop. Then P-Node (A/B) is holding dirty data. Another P-Node (C/A) executes write event to C-Node (C/A). It also does C-Node (A/B) by write event broadcast message via C-DF. Once received write event request from P-Node (C/A). Both of C-Node (A/B) and C-Node (C/A) have done the write event and ACK to P-Node (C/A) that the event is completed. When P-Node (C/A) received it from both of C-

Node, P-node (C/A) signal cast to P-node (A/B) the message of write event was done. Then Pnode (A/B) flag off of write dirty and open the memory space for new write event entry. Therefore, always write-data response time is maintaining high I/O response by L3 write-back cache with non-L3 write-through cache without cache memory size limit.

### **2.7 Read event by L3/L4 cache**

230 Emerging Informatics – Innovative Concepts and Applications

pre-read cache implementation and 3) Pre fetching boot up file, 4) Internal UPS that has automatic shutdown function acts whenever un-properly shutdown or upper system power fail re-securing dirty data on the cache table. 5) Can be expandable the total number of L4

This L4 cache space is completely individual autonomous memory space on the device. Today, there are many RAID storage subsystems available. But seeing from the individual device point of view, there are some difference points. RAID controller has buffer cache memory for RAID sets. But, this is the unified block cache space and it for block cache of RAID set and its volume. Also it is not safety idea when RAID subsystem electric power failure occurs. P-Node L4 cache is loosely connection individuality block cache. Only L4 cache device achieves no risk with high I/O speed with assurance benefit. Figure 6 is L4 cache block diagram in C-node. In this general case of design, 1GB DRAM for L4 cache memory with Flash SSD is target mounted device. Internally, 64bit based DRAM assigned and its I/O performance shows more than 270MB/s with 40KIOPS transaction performance

133MHz/72bit

When write data request received from out side requester, two P-Nodes that are mounted C-Node application program partition execute the write request event by L3 cache P-Node and non-L3 cache P-Node. For instance, If Write event required at storage partition "A", there are two P-Node related this event as P-Node (A/B) and P-Node (C/A). In this case, P-Node (A/B) has L3 cache for "A" and P-Node (C/A) hasn't. The ACK for write event requester replied by P-Node (A/B) using L3 write back cache is by one hop. Then P-Node (A/B) is holding dirty data. Another P-Node (C/A) executes write event to C-Node (C/A). It also does C-Node (A/B) by write event broadcast message via C-DF. Once received write event request from P-Node (C/A). Both of C-Node (A/B) and C-Node (C/A) have done the write event and ACK to P-Node (C/A) that the event is completed. When P-Node (C/A) received it from both of C-

Local Memory 128MB

Super Capacity for Uninteractive

> S-ATA Interface

NAND Flash

PC/RAID SATA interface

cache size by multiple number of L4 cache drive.

Control CPU

RT-OS

ChipSet

DTS Mcell-2 core FPGA/ASIC

L4 cache Memory 1Gb x 9 DDR-II ECC or 2Gb x 9 DDR-II ECC

133MHz/64bit

CPU/ Boot Flah ROM

under MLC SSD configuration.

Fig. 6. L4 cache design diagram

**2.6 Write data coherency process by P-Node** 

L3 cache on P-Node effects read I/O event also. L3 cache is dedicated storage cache on P-Node and it maintains the data inside of L3 cache. The read event process is following. a) When received the read event "A" from requester, there are two P-Nodes that are related partition "A". b) One of P-Node has L3 block Cache space and other is no cache space of dedicated "A" partition. It just mounts "A" partition. c) When executes read event of "A" partition, the one of P-Node which has L3 cache for partition "A" executes the read event if L3 cache space holding the read event data and broadcast its acknowledge via data field as timeliness manner. d) Other one of P-Node executes by traditional read event execution if read data detects inside of memory space. e) The fastest read event executed node broadcasts the result of read request event to other nodes with event message. f) Once received the read event result from any P-Nodes, this node stops the read event execution. g) Eventually, the fastest P-Node result reaches the requester. P-Node read event is very flexible process. If one of node holding data for the event, anyone can be the fastest replier. Other node that has data for read request is stopped by the fastest node action. When the read event access size exceeds more than L3 cache, L4 cache on C-Node has possibility to maintain the next priority data. C-Node manages the size of L4 cache if P-Node L3 cache size exceeds L4 cache, C-Node expands the L4 cache capacity inside C-node disk configuration change and it maintains the ratio "L3<L4".
