**1. Introduction**

22 Will-be-set-by-IN-TECH

134 Grid Computing – Technology and Applications, Widespread Coverage and New Horizons

Steere, D. C., Goel, A., Gruenberg, J., McNamee, D., Pu, C. & Walpole, J. (1999).

Strumpen, V. & Ramkumar, B. (1996). Portable checkpointing and recovery in heterogeneous environments, *Technical Report UI-ECE TR-96.6.1*, University of Iowa. Subramani, V., Kettimuthu, R., Srinivasan, S. & Sadayappan, P. (2002). Distributed job

van Renesse, R., Minsky, Y. & Hayden, M. (1998). A gossip-style failure detection service,

Weatherspoon, H. & Kubiatowicz, J. (2002). Erasure coding vs. replication: A quantitative

Wilkinson, B. (2009). *Grid Computing: Techniques and Applications*, Chapman & Hall/CRC. Zhu, Y. (2003). A survey on grid scheduling systems, *Technical report*, Computer Science

USENIX Association, New Orleans, LA, USA, pp. 145–158.

33(8): 103 – 111.

*Computing*, IEEE Computer Society, Washington, DC, USA, p. 359. Theodoridis, S. & Koutroumba, K. (2003). *Pattern Recognition*, Elsevier Academic Press. Valiant, L. G. (1990). A bridging model for parallel computation, *Communications of the ACM*

*Distributed Processing (Middleware 1998)*, Lake District, England.

*Peer-to-Peer Systems*, Springer-Verlag, London, UK, pp. 328–338.

Department, University of Science and Technology, Hong Kong.

A feedback-driven proportion allocator for real-rate scheduling, *OSDI âA˘Z99:*

*Proceedings of the Third Symposium on Operating Systems Design and Implementation*,

scheduling on computational grids using multiple simultaneous requests, *HPDC '02: Proceedings of the 11th IEEE International Symposium on High Performance Distributed*

*Proceedings of the IFIP International Conference on Distributed Systems Platforms and Open*

comparison, *IPTPS '01: Revised Papers from the First International Workshop on*

*´*

With the rapid development of the information techniques and the Internet popular applications, the demand for the high performance processing devices is becoming more and more vehement, such as the server construction of HDTV or IPTV for large scale VOD, and the parallel file system of grid computing. These applications need the ability to process the concurrent peak access. The parallel file system (J.Carretero, F. Perez, P. de Miguel& L.Alonso, 1996; P.F. Corbett & D. G, 1996; Craig S. Freedman, Josef Burger, & David J, 1996; Seogyun Kim, Jiseung Nam & Soon-ja Yeom, 2002) has become the very important research areas, and the parallel cache (Horst Eidenberger, 2005; J. Fernandez,J. Carretero, F. Garcia-Carballeira, A. Calderon & J. M. Perez-Menor, 2005; T. Kimbrel et. AI,1996; Pal Halvorsen, Carsten Griwodz, Vera Goebel, Ketil Lund, Thomas Plagemann & Jonathan Walpole,2006) is playing the key roles in these applications. At the same time, the numbers of the Intranet composed of computer clusters are quickly increasing, and a great deal of cheap personal computers are distributed everywhere, but the utilization of their resources is very low(E. P. Markatos & G. Dramitions,1996; Anurag Acharya & Sanjeev Setia,1998). Through mining and adopting these idle resources, we can get a lot of large-scale high performance computation, storage and communication resources which are not special. How to use these idle memories to construct the parallel cache for supporting the large scale applications, such as VOD for hot segments, is very interesting. It not only improves their performance, but also increases the utilization of these idle resources. However, the heterogeneous, dynamic and unstable characteristic of these resources brings a huge obstacle for us. The grid (I. Foster & C. Kesselman, 1999) techniques and multi-agents (E. Osawa, 1993; Wooldridge, M, 2002) become the main approaches to effectively use these resources, and the multi-agents have already become the feasible solutions for grid applications. There are many successful examples (O.F. Rana & D.W. Walker, 2000; J.O. kephart & Chess, D.M, 2003) of applications which are in conjunction with the automatic multi-agents system. But, the research of construction cache model for hot segment through grid idle memory is still not much. At the same time, the high performance cluster techniques (Rajkumar Buyya, 1999) are already mature and can be the foundation for supporting the parallel cache based on grid memory.

Research and Implementation of Parallel Cache Model Through Grid Memory 137

**Definition.2.** Computer cluster (CC) is defined as CC (Master, CS), where Master is the main computer of CC; CS= {CN1, CN2 … CNp} is the set of all cache nodes in computer

**Definition.3.** Logical computer cluster (LCC) is defined as LCC (id, LCS, B, CC), where id denotes the identifier of LCC; LCS is the set of cache nodes of LCC; CC is the computer

So, the dynamic network environment (DNE) can be defined as DNE (Master, CCS, SVS, N, R), where Master is the main computer of DNE; CSS is the set of all computer clusters in DNE; SVS is the set of file servers to storage the files; N is its network set; R is the connection rules. All the free memories of the idle cache nodes in DNE are called the grid memory.

**Definition. 4.** Basic memory unit (BMU) is the uniform size of basic memory block as the

**Definition. 5.** Grid memory capacity (Gmc) is the total numbers of BMU provided by all

The main functions of MAS are the computation resource management, the DNE monitoring and the task scheduler. MAS include four parts: The first part is the global control agent (GA) for managing PCMGM. The second part is the agent for managing one file server, and it is called as the server agent (SA). The third part is the agents for managing the cache nodes, and they are called as the node agents (NA), and each cache node has a NA. The last part is the application agent (AA) that is the connection manager between the

The main functions of GA are as follows: (1) Control and manage DNE; (2) Receive and dispatch the caches; (3) control and monitor the file access process; (4) load balance; (5) Control all the cache nodes in DNE; (6) Calculate the idle resources; (7) Monitor the states all

The main functions of SA are as follows: (1) File operations; (2) The hot segment 27

The main functions of NA are as follows: (1) Control the cache node to join or disjoin the DNE in dynamic; (2) Calculate the idle resources, and report them to GA; (3) Monitor the states and events in CN, and make the response adjustments; (4) Control the cache agents

The large file is always divided into a series of segments (SEG) in the file server and the access frequency of each segment is different, so the high access frequency segment is called as the hot segment. If the hot segment is duplicated into the grid memory, the access efficiency will be improved greatly. For describing this mechanism, we introduce some

cluster which comprises LCC. Network bandwidth of LCC denotes as B.

allotting unit in PCMGM, and its unit is MB.

**2.2 Management Agents System (MAS)** 

computing nodes and node agents;

(CA) to complete the file access task.

**2.3 Cache file and cache agent** 

conceptions.

management; (3) File transfer;

cache nodes in DNE. The Gmc value is dynamic changed.

users and PCMGM. The MAS structure is presented in figure 1.

cluster.

This book chapter introduced a Parallel Cache Model Based on Grid Memory (PCMGM) in the dynamic network environment (DNE). Through building an open grid environment, using the idle memory resources of DNE, adopting the multi-agents, the fuzzy theory and the self-learning methods, we designed the PCMGM model that can support the concurrent peak access for the hot segments in large scale internet applications, such as the large scale VOD system. The experimental results show that PCMGM can improve response time of the large scale application system for hot segments. It can be fit for the grid computing and the large scale VOD system in internet.
