Agent-based hierarchical approach for executing bag-of-tasks in clouds

Włodzimierz Funika, Kamil Mazurek, Wojciech Kruczkowski

Abstract


Numerous unrelated, independent (no inter-task communication) tasks called “bag-oftasks”
(BoTs) compared with message passing applications can be highly parallelised and
executed in any acceptable order. A common practice when executing bag-of-tasks applications
(BoT) is to exploit the master-slave topology. Cloud environments offer some features
that facilitate executing BoT applications. One of the approaches to control cloud resources
is to use agents that can flexibly act in a dynamic environment. Given these assumptions we
designed a combination of these approaches, which can be classified as: a distributed, hierarchical
solution to the issue of scalable executing of bag-of-tasks. The concept of our system
relates to a project that is focused on processing huge quantities of data incoming from a
network of sensors by the Internet. Our aim is to create a mechanism for processing such data
as a system which executes jobs while exploiting load balancing for cloud resources using,
e.g., Eucalyptus. The idea is to create a hybrid architecture which takes advantage of some
centralized parts of the system and full distributedness in other parts. On the other hand
we balance dependencies between the system components using a hierarchic master-slave
structure.

Keywords


cloud computing;bag-of-tasks;load balancing; master-slave paradigm; scalability

Full Text:

PDF

References


Amazon EC2 website. http://aws.amazon.com/ec2. Access: 12.09.2013.

AWS SDK for Java. http://aws.amazon.com/sdkforjava. Access: 12.09.2013.

Cloud-Based Support for Distributed Multiscale Applications. https://portal.futuregrid.org/node/937/project-details. Access: 17.09.2013.

CXF website. http://cxf.apache.org. Access: 23.08.2013.

Euca2ools. http://www.eucalyptus.com/docs#euca2ools. Access: 17.09.2013.

Eucalyptus. http://www.eucalyptus.com. Access: 17.09.2013.

Eucalyptus and AWS. http://www.eucalyptus.com/aws-compatibility. Access: 17.09.2013.

FutureGrid website. https://portal.futuregrid.org. Access: 17.09.2013.

JSON website. http://www.json.org. Access: 23.08.2013.

Quartz Scheduler website. http://quartz-scheduler.org. Access: 23.08.2013.

Ambroszkiewicz S., Cetnarowicz K., Kozlak J., Nowak T., Penczek W.: Modeling Agent Organizations. In: Intelligent Information Systems, Advances in Soft Computing, vol. 4, pp. 135–144. Physica-Verlag HD, 2000. ISBN 978-3-7908-1309-8. http://dx.doi.org/10.1007/978-3-7908-1846-8_13.

Anglano C., Canonico M.: Scheduling algorithms for multiple Bag-of-Task applications on Desktop Grids: A knowledge-free approach. In: IPDPS, pp. 1–8. IEEE, 2008.

Antonis K., Garofalakis J., Mourtos I., Spirakis P.: A hierarchical adaptive distributed algorithm for load balancing. J. Parallel Distrib. Comput., 64(1): 151–162, 2004. ISSN 0743-7315. http://dx.doi.org/10.1016/j.jpdc.2003.07.002.

Balis B., Kasztelnik M., Bubak M., Bartynski T., Gubaa T., Nowakowski P., Broekhuijsen J.: The UrbanFlood Common Information Space for Early Warning Systems. Procedia Computer Science, vol. 4, pp. 96–105, 2011. ISSN 1877-0509. http://dx.doi.org/http://dx.doi.org/10.1016/j.procs.2011.04.011. In: Proceedings of the International Conference on Computational Science, ICCS 2011.

Beaumont O., Carter L., Ferrante J., Legrand A., Marchal L., Robert Y.: Centralized versus distributed schedulers for multiple bag-of-task applications. In:Proceedings of the 20th international conference on Parallel and distributed processing, IPDPS’06, pp. 24–24. IEEE Computer Society, Washington, DC, USA, 2006. ISBN 1-4244-0054-6.

Bruno J., Coffman Jr. E.G., Sethi R.: Scheduling independent tasks to reduce mean finishing time. Commun. ACM, 17(7): 382–387, 1974. ISSN 0001-0782. http://dx.doi.org/10.1145/361011.361064.

Cetnarowicz K.: From Algorithm to Agent. In: Proceedings of the 9th International Conference on Computational Science, ICCS 2009, pp. 825–834. Springer Verlag, Berlin, Heidelberg, 2009. ISBN 978-3-642-01972-2. http://dx.doi.org/10.1007/978-3-642-01973-9_92.

da Silva F. A. B., Senger H.: Scalability limits of Bag-of-Tasks applications running on hierarchical platforms. J. Parallel Distrib. Comput., 71(6): 788–801, 2011. ISSN 0743-7315. http://dx.doi.org/10.1016/j.jpdc.2011.01.002.

Gamma E., Helm R., Johnson R., Vlissides J. M.: Design patterns: elements of reusable object-oriented software. Addison-Wesley, Reading, Mass. [u.a.], 1994. ISBN 0-201-63361-2.

Geelan J.: Twenty one experts define cloud computing. Cloud Computing Journal, http://virtualization.sys-con.com/node/612375, 2009.

Gutierrez-Garcia J., Sim K. M.: Agent-Based Service Composition in Cloud Computing. In: Grid and Distributed Computing, Control and Automation, T. H. Kim, S. Yau, O. Gervasi, B. H. Kang, A. Stoica, D. lzak (Eds.), Communications in Computer and Information Science, vol. 121, pp. 1–10. Springer Berlin Heidelberg, 2010. ISBN 978-3-642-17624-1. http://dx.doi.org/10.1007/978-3-642-17625-8_1.

Gutierrez-Garcia J. O., Sim K. M.: A Family of Heuristics for Agent-Based Cloud Bag-of-Tasks Scheduling. In: Proceedings of the 2011 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery, CYBERC’11, pp. 416–423. IEEE Computer Society, Washington, DC, USA, 2011. ISBN 978-0-7695-4557-8. http://dx.doi.org/10.1109/CyberC.2011.74.

Ibarra O. H., Kim C. E.: Heuristic Algorithms for Scheduling Independent Tasks on Nonidentical Processors. J. ACM, 24(2): 280–289, 1977. ISSN 0004-5411. http://dx.doi.org/10.1145/322003.322011.

Kopacz T.: Programming in Windows Azure. Presentation from the 4th MIC conference ,,Nowoczesne technologie bli˙zej nas”, Poznan, Polandhttp://mic.psnc.pl/download/conf10/MIC_NTBN10_Azure.pdf, 2010.

Krol D., Wrzeszcz M., Kryza B., Dutka L., Kitowski J.: Massively Scalable Platform for Data Farming Supporting Heterogeneous Infrastructure. In: The Fourth International Conference on Cloud Computing, GRIDs, and Virtualization, IARIA Cloud Computing 2013, pp. 144–149. Valencia, Spain, 2013.

Marco J., Campos I., Coterillo I.e.a.: The interactive European grid: Project objectives and achievements. Computing and Informatics, 27(2): 161–171, 2008. ISSN 1335-9150.

Mell P., Grance T.: The NIST Definition of Cloud Computing - Recommendations of National Institute of Standards and Technology. Special Publication 800-145, NIST US DOC, Sept. 2011 http://csrc.nist.gov/publications/nistpubs/800-145/SP800-145.pdf.

Netto M. A., Buyya R.: Coordinated rescheduling of Bag-of-Tasks for executions on multiple resource providers. Concurr. Comput. : Pract. Exper., 24(12): 1362–1376, 2012. ISSN 1532-0626. http://dx.doi.org/10.1002/cpe.1841.

Oprescu A., Kielmann T.: Bag-of-Tasks Scheduling under Budget Constraints. In: Cloud Computing Technology and Science (CloudCom), 2010 IEEE Second International Conference on, pp. 351–359. 2010. http://dx.doi.org/10.1109/CloudCom.2010.32.

Pandey S., Karunamoorthy D., Buyya R.: Workflow Engine for Clouds. In: Cloud Computing: Principles and Paradigms, R. Buyya, J. Broberg, A. Goscinski (Eds.), pp. 321–344. John Wiley & Sons, Inc., New York, USA, 2011. ISBN 9780470940105. http://dx.doi.org/10.1002/9780470940105.ch12.

Schubert L., Assel M., Kipp A., Wesner S.: Resource Cloud Mashups. In: Cloud Computing: Principles and Paradigms, R. Buyya, J. Broberg, A. Goscinski (Eds.), pp. 533–548. John Wiley & Sons, Inc., New York, USA, 2011. ISBN 9780470940105. http://dx.doi.org/10.1002/9780470940105.ch21.

Senger H., da Silva F. A. B., Miranda Filho L. J.: Influence of Communication Models on the Scalability of Master-Slave Platforms Running Bag-of-Tasks Applications. In: Proceedings of the 2010 11th Symposium on Computing Systems, WSCAD-SCC ’10, pp. 25–32. IEEE Computer Society, Washington, DC, USA, 2010. ISBN 978-0-7695-4274-4. http://dx.doi.org/10.1109/WSCAD-SCC.2010.27.

Suchta A., Marsza lek K., Smoktunowicz U.: Little rain from great cloud (Z duzej chmury ma ly deszcz). Article on CRN website: http://www.crn.pl/artykuly/biznes/2011/09/z-duzej-chmury-maly-deszcz, 2011.

Voorsluys W., Broberg J., Buyya R.: Introduction to Cloud Computing. In: Cloud Computing: Principles and Paradigms, R. Buyya, J. Broberg, A. Goscinski (Eds.), pp. 1–41. John Wiley & Sons, Inc., New York, USA, 2011. ISBN 9780470940105. http://dx.doi.org/10.1002/9780470940105.ch1.

Wismuller R., Bubak M., Funika W.e.a.: Support for User-Defined Metrics in the On-line Performance Analysis Tool G-PM. In: Grid Computing – Second European AcrossGrids Conference AxGrids2004, M.D. Dikaiakos, ed., Lecture Notes in Computer Science, vol. 3165, pp. 159–168. Springer-Verlag, Nicosia, Cyprus, 2004.




DOI: https://doi.org/10.7494/csci.2014.15.1.35

Refbacks

  • There are currently no refbacks.