**5. Conclusion**

Corporations and universities typically have hundreds or thousands of desktop machines, which are used by workers as their personal workstations or by students in instructional and research laboratories. When analyzing the usage of each of these machines one concludes that they sit idle for a significant amount of time. Even when the computer is in use, it normally has a large portion of idle resources. When we consider the night period, we conclude that most of the times they are not used at all. This situation lives in contradiction with the growing demand for highly intensive computational power required by several fields of research, including physics, chemistry, biology, and economics. Several corporations also rely on the intensive use of computing power to solve problems such as financial market simulations and studies for accurate oil well drilling. The movie industry makes intensive use of computers to render movies using an increasing number of special effects.

Opportunistic grid middleware enables the use of the existing computing infrastructure available in laboratories and offices in universities, research institutes, and companies to execute computationally intensive parallel applications. Nevertheless, executing this class of applications on such a dynamic and heterogeneous environment is a daunting task, especially when non-dedicated resources are used, as in the case of opportunistic computing. In an opportunistic grid middleware resources do not need to be dedicated for executing grid applications. The grid workload coexist with local applications executions, submitted by the nodes regular users. The middleware must take advantage of idle computing cycles that arise from unused time frames of the workstations that comprise the grid.

An opportunistic grid middleware must provide innovative solutions to circumvent problems arising from the heterogeneity, high scalability and dynamism of the execution environment. Among the several aspects that must be considered, central issues are related to the support for a variety of programming models, highly scalable distributed resource management, application scheduling and execution management, and fault tolerance, since opportunist grid environments are inherently prone of errors. In this chapter, we provided a comprehensive description of reputable solutions found in the literature to circumvent the above described problems, emphasizing the approaches adopted in the InteGrade middleware, an open-source multi-university effort to build a robust and flexible middleware for opportunistic grid computing available at http://www.integrade.org.br.

### **6. References**

Barbosa, R. M. & Goldman, A. (2004). Framework for mobile agents on computer grid environments, *First International Workshop on Mobility Aware Technologies and Applications (MATA 2004)*, Springer LNCS No 3284, Florianópolis, Brazil, pp. 147–157. 18 Will-be-set-by-IN-TECH

other on another cluster machine. During the recovery of failed parallel application, most of

The system stores most of the generated checkpoints using the ephemeral mode, since the usage of local networks generates a lower overhead. To prevent losing the entire computation due to a failure or disconnection of the cluster where the application was executing, periodically, the checkpoints are also stored in other clusters using the perennial storage mode, for instance, after every *k* generated checkpoints. With this strategy, InteGrade can obtain low overhead and high availability at the same time for checkpoint storage.

Corporations and universities typically have hundreds or thousands of desktop machines, which are used by workers as their personal workstations or by students in instructional and research laboratories. When analyzing the usage of each of these machines one concludes that they sit idle for a significant amount of time. Even when the computer is in use, it normally has a large portion of idle resources. When we consider the night period, we conclude that most of the times they are not used at all. This situation lives in contradiction with the growing demand for highly intensive computational power required by several fields of research, including physics, chemistry, biology, and economics. Several corporations also rely on the intensive use of computing power to solve problems such as financial market simulations and studies for accurate oil well drilling. The movie industry makes intensive use of computers to

Opportunistic grid middleware enables the use of the existing computing infrastructure available in laboratories and offices in universities, research institutes, and companies to execute computationally intensive parallel applications. Nevertheless, executing this class of applications on such a dynamic and heterogeneous environment is a daunting task, especially when non-dedicated resources are used, as in the case of opportunistic computing. In an opportunistic grid middleware resources do not need to be dedicated for executing grid applications. The grid workload coexist with local applications executions, submitted by the nodes regular users. The middleware must take advantage of idle computing cycles that arise

An opportunistic grid middleware must provide innovative solutions to circumvent problems arising from the heterogeneity, high scalability and dynamism of the execution environment. Among the several aspects that must be considered, central issues are related to the support for a variety of programming models, highly scalable distributed resource management, application scheduling and execution management, and fault tolerance, since opportunist grid environments are inherently prone of errors. In this chapter, we provided a comprehensive description of reputable solutions found in the literature to circumvent the above described problems, emphasizing the approaches adopted in the InteGrade middleware, an open-source multi-university effort to build a robust and flexible middleware for opportunistic grid

Barbosa, R. M. & Goldman, A. (2004). Framework for mobile agents on computer

grid environments, *First International Workshop on Mobility Aware Technologies and Applications (MATA 2004)*, Springer LNCS No 3284, Florianópolis, Brazil, pp. 147–157.

render movies using an increasing number of special effects.

from unused time frames of the workstations that comprise the grid.

computing available at http://www.integrade.org.br.

the processes will be able to recover their local checkpoints from the local machine.

**5. Conclusion**

**6. References**

Barlow, H. B. (1999). *Unsupervised learning: Foundations of Neural Computation*, MIT Press.


Liu, Y. (2004). Grid scheduling, *Technical report*, Department of Computer Science, University

Efficient Parallel Application Execution on Opportunistic Desktop Grids 133

Lopes, R. F., Silva, F. J. S. & Souza, B. B. (2005). Mag: A mobile agent based computational

Maheswaran, M., Ali, S., Siegel, H. J., Hensgen, D. & Freund, R. F. (1999). Dynamic matching

Maia, R., Cerqueira, R. & Cosme, R. (2006). Oil: an object request broker in the lua, *5th Tools*

Malluhi, Q. M. & Johnston, W. E. (1998). Coding for high availability of a distributed-parallel storage system, *IEEE Transactions Parallel Distributed Systems* 9(12): 1237–1252. Medeiros, R., Cirne, W., Brasileiro, F. V. & Sauvé, J. P. (2003). Faults in grids: why are they

OMG (2011). *Common Object Request Broker Architecture (CORBA) Specification, Version 3.1.1*,

Pham, V. A. & Karmouch, A. (1998). Mobile software agents: An overview, *Communications*

Pinheiro, V. G., Goldman, A. & Kon, F. (2011). Adaptive fault tolerance mechanisms for

Plank, J. S., Li, K. & Puening, M. A. (1998). Diskless checkpointing, *IEEE Transactions on Parallel*

Rabin, M. O. (1989). Efficient dispersal of information for security, load balancing, and fault

Ripeanu, M. & Foster, I. (2002). A decentralized, adaptive replica location mechanism,

Rowstron, A. I. T. & Druschel, P. (2001). Pastry: scalable, decentralized object location, and

Sobe, P. (2003). Stable checkpointing in distributed systems without shared disks, *IPDPS '03:*

*Distributed Computing*, IEEE Computer Society, Washington, DC, USA. Rodrigues, R. & Liskov, B. (2005). High availability in DHTs: Erasure coding vs. replication,

opportunistic environments: a mobile agent approach, *Concurrency and Computation:*

*HPDC '02: Proceedings of the 11 th IEEE International Symposium on High Performance*

*IPTPS '05: Revised Selected Papers from the Fourth International Workshop on Peer-to-Peer*

routing for large-scale peer-to-peer systems, *Middleware 2001: IFIP/ACM International Conference on Distributed Systems Platforms*, Heidelberg, Germany, pp. 329–350. Schopf, J. M. (2004). Ten actions when grid scheduling: the user as a grid scheduler, *Grid*

*Proceedings of the 17th International Symposium on Parallel and Distributed Processing*,

*Computing (GCC 2005)*, Springer LNCS Series, Beijing, China.

Computer Society, Washington, DC, USA, p. 30.

*Computing (GRID 2003)*, Phoenix, USA, pp. 18–24. MPI (1997). *MPI-2: extensions to the Message-Passing Interface*.

MPI (2009). *MPI: A Message-Passing Interface Standard Version 2.2*.

Object Management Group (OMG). URL: *http://www.omg.org/spec/CORBA/3.1.1/*

*and Distributed Systems* 9(10): 972–986.

tolerance, *Journal of the ACM* 36(2): 335–348.

*Systems*, Springer-Verlag, London, UK, pp. 226–239.

*resource management: state of the art and future trends* pp. 15–23.

IEEE Computer Society, Washington, DC, USA, p. 214.2.

*Magazine* 7(36): 26–37.

*Practice and Experience* .

URL: *http://www.mpi-forum.org/docs/mpi-20-html/mpi2-report.html*

URL: *http://www.mpi-forum.org/docs/mpi22-report/mpi22-report.htm*

22/01/2009.

of Iowa. http://www.cs.uiowa.edu/ yanliu/QE/QEreview.pdf, Acessado em:

grid platform, *Proceedings of the 4th International Conference on Grid and Cooperative*

and scheduling of a class of independent tasks onto heterogeneous computing systems, *HCW '99: Proceedings of the Eighth Heterogeneous Computing Workshop*, IEEE

*Session of the Brazilian Simposium on Computer Networks (SBRC2006)*, Curitiba, Brazil.

so bad and what can be done about it?, *4th IEEE International Workshop on Grid*


20 Will-be-set-by-IN-TECH

Elnozahy, M., Alvisi, L., Wang, Y.-M. & Johnson, D. B. (2002). A survey of rollback-recovery protocols in message-passing systems, *ACM Computing Surveys* 34(3): 375–408.

Filho, F. C., Castro, R., Marques, A., Neto, F. S., de Camargo, R. Y. & Kon, F. (2008). A group

Filho, F. C., Castro, R., Marques, A., Soares-Neto, F., de Camargo, R. Y. & Kon, F. (2009). A

Finger, M., Bezerra, G. C. & Conde, D. M. R. (2010). Resource use pattern analysis for

Ganesh, A. J., Kermarrec, A.-M. & Massoulie, L. (2003). Peer-to-peer membership

Goel, A., Steere, D. C., C, C. P. & Walpole, J. (1999). Adaptive resource management via

Goldchleger, A., Goldman, A., Hayashida, U. & Kon, F. (2005). The implementation of the

*Workshop on Middleware for Grid Computing*, ACM Press, Grenoble, France. Goldchleger, A., Kon, F., Goldman, A., Finger, M. & Bezerra, G. C. (2004). Integrade:

Hwang, S. & Kesselman, C. (2003). Gridworkflow: A flexible failure handling framework for

Ibarra, O. H. & Kim, C. E. (1977). Heuristic algorithms for scheduling independent tasks on

Ierusalimschy, R., de Figueiredo, L. H. & Filho, W. C. (1996). Lua - an extensible extension

Karablieh, F., Bazzi, R. A. & Hicks, M. (2001). Compiler-assisted heterogeneous checkpointing,

Karonis, N., Toonen, B. & Foster, I. (2003). Mpich-g2: a grid-enabled implementation of

Lee, C. & Talia, D. (2003). Grid programming models: Current tools, issues and directions,

Litzkow, M., Tannenbaum, T., Basney, J. & Livny, M. (1997). Checkpoint and migration

*SRDS '01: Proceedings of the 20th IEEE Symposium on Reliable Distributed Systems*, New

the message passing interface, *Journal of Parallel and Distributed Computing (JPDC)*

of UNIX processes in the Condor distributed processing system, *Technical Report UW-CS-TR-1346*, University of Wisconsin - Madison Computer Sciences Department.

nonidentical processors, *Journal of the ACM (JACM)* 24(2): 280–289.

language, *Software: Practice Experience* 26(6): 635–652.

*Grid Computing*, Wiley Online Library, pp. 555–578.

*SBRC'2009 Workshop on Grid Computing and Applications*, Recife, Brazil. Finger, M., Bezerra, G. C. & Conde, D. M. R. (2008). Resource use pattern analysis for

membership service for large-scale grids, *6th International Workshop on Middleware for*

robust and scalable group membership service for large-scale grids *(in portuguese)*,

opportunistic grids, *6th International Workshop on Middleware for Grid Computing*

predicting resource availability in opportunistic grids, *Concurrency and Computation:*

management for gossip-based protocols, *IEEE Transactions on Computers*

modular feedback control, *Technical report*, Oregon Graduate Institute, Department

bsp parallel computing model on the integrade grid middleware, *3rd International*

Object-oriented grid middleware leveraging idle computing power of desktop machines, *Concurrency and Computation: Practice and Experience* 16(5): 449–459. Grandinetti, L. (2005). *Grid Computing: The New Frontier of High Performance Computing*,

Everitt, B., Landau, S. & Leese, M. (2001). *Cluster Analysis*, 4th edn, MIT Press.

*Grid Computing*, Leuven, Belgium.

*(MGC 2008)*, Leuven, Belgium.

*Practice and Experience* 22(3).

of Computer Science and Engineering.

52(2): 139–149.

Elsevier.

the grid, *hpdc* 00: 126.

Orleans, USA, pp. 56–65.

63(3): 551–563.


**7** 

*China* 

**Research and Implementation of** 

Qingkui Chen, Lichun Na, He Jia, Song Lin Zhuang and Xiaodong Ding

*University of Shanghai for Science and Technology* 

*School of Computer Engineering,* 

**Parallel Cache Model Through Grid Memory** 

With the rapid development of the information techniques and the Internet popular applications, the demand for the high performance processing devices is becoming more and more vehement, such as the server construction of HDTV or IPTV for large scale VOD, and the parallel file system of grid computing. These applications need the ability to process the concurrent peak access. The parallel file system (J.Carretero, F. Perez, P. de Miguel& L.Alonso, 1996; P.F. Corbett & D. G, 1996; Craig S. Freedman, Josef Burger, & David J, 1996; Seogyun Kim, Jiseung Nam & Soon-ja Yeom, 2002) has become the very important research areas, and the parallel cache (Horst Eidenberger, 2005; J. Fernandez,J. Carretero, F. Garcia-Carballeira, A. Calderon & J. M. Perez-Menor, 2005; T. Kimbrel et. AI,1996; Pal Halvorsen, Carsten Griwodz, Vera Goebel, Ketil Lund, Thomas Plagemann & Jonathan Walpole,2006) is playing the key roles in these applications. At the same time, the numbers of the Intranet composed of computer clusters are quickly increasing, and a great deal of cheap personal computers are distributed everywhere, but the utilization of their resources is very low(E. P. Markatos & G. Dramitions,1996; Anurag Acharya & Sanjeev Setia,1998). Through mining and adopting these idle resources, we can get a lot of large-scale high performance computation, storage and communication resources which are not special. How to use these idle memories to construct the parallel cache for supporting the large scale applications, such as VOD for hot segments, is very interesting. It not only improves their performance, but also increases the utilization of these idle resources. However, the heterogeneous, dynamic and unstable characteristic of these resources brings a huge obstacle for us. The grid (I. Foster & C. Kesselman, 1999) techniques and multi-agents (E. Osawa, 1993; Wooldridge, M, 2002) become the main approaches to effectively use these resources, and the multi-agents have already become the feasible solutions for grid applications. There are many successful examples (O.F. Rana & D.W. Walker, 2000; J.O. kephart & Chess, D.M, 2003) of applications which are in conjunction with the automatic multi-agents system. But, the research of construction cache model for hot segment through grid idle memory is still not much. At the same time, the high performance cluster techniques (Rajkumar Buyya, 1999) are already mature and can be the foundation for supporting the parallel cache based

**1. Introduction** 

on grid memory.

