MapReduce, as a popular programming model for processing large data sets, has been widely applied. MapReduce 2.0 (MRV2) is a newly adopted one, which has a better performance. Those machines which have a lower performance in a cluster usually play a role who pull down the pace of job execution time. Speculative execution known as an approach for dealing with the above problems works by backing up those tasks running on a low performance machine to a higher one. Although multiple speculative execution strategies have been proposed, there are still a lot of pitfalls existing in the strategies. In this paper, Some pitfalls in proposed strategy have been modified and computer hardware has been taken into consideration (HWC-Speculation). In Hadoop-2.6, we have implemented it, called Hadoop-HWC. Experiment results show that our method can find a slow task correctly, also, the performance of MRV2 is improved.
목차
Abstract 1. Introduction 2. Related Work 3. The HWC-Speculation Strategy 3.1 Getting the Remaining Time of Current Task 3.2 Method of Selecting a Backup Node and Predicting Time 3.3 Selection of Backup Task 4. Experiment and Analysis 5. Conclusion Acknowledgments References[1]
Qi Liu [ School of Computer and Software, Nanjing University of Information Science and Technology, Nanjing, Jiangsu, 210044, China ]
Weidong Cai [ School of Computer and Software, Nanjing University of Information Science and Technology, Nanjing, Jiangsu, 210044, China ]
Zhangjie Fu [ Jiangsu Engineering Centre of Network Monitoring, Nanjing University of Information Science and Technology, Nanjing, Jiangsu, 210044, China ]
Jian Shen [ School of Computer and Software, Nanjing University of Information Science and Technology, Nanjing, Jiangsu, 210044, China ]
Corresponding author
Nigel Linge [ The University of Salford, Salford, Greater Manchester, M5 4WT, UK ]
보안공학연구지원센터(IJGDC) [Science & Engineering Research Support Center, Republic of Korea(IJGDC)]
설립연도
2006
분야
공학>컴퓨터학
소개
1. 보안공학에 대한 각종 조사 및 연구
2. 보안공학에 대한 응용기술 연구 및 발표
3. 보안공학에 관한 각종 학술 발표회 및 전시회 개최
4. 보안공학 기술의 상호 협조 및 정보교환
5. 보안공학에 관한 표준화 사업 및 규격의 제정
6. 보안공학에 관한 산학연 협동의 증진
7. 국제적 학술 교류 및 기술 협력
8. 보안공학에 관한 논문지 발간
9. 기타 본 회 목적 달성에 필요한 사업
간행물
간행물명
International Journal of Grid and Distributed Computing
간기
격월간
pISSN
2005-4262
수록기간
2008~2016
십진분류
KDC 505DDC 605
이 권호 내 다른 논문 / International Journal of Grid and Distributed Computing Vol.9 No.2