Earticle

초록

영어

Job scheduling is an important component of Hadoop. Hadoop default FIFO scheduling algorithm is simple and easy to achieve and widely used; but it lacks consideration in the characteristic of data locality, which will lead to heavy traffic during network transmission and task execution, then computing resources cannot be fully utilized and a series of other drawbacks. Meanwhile the static function of resource slots in Map and Reduce stages make such defects worse. This paper proposes a job scheduling algorithm (Interactive Scheduler Algorithm) IS based on interacting the master node and slave nodes from the data locality and tasks allocation. The algorithm improves FIFO and the usage of computing resource, realizes the dynamic conversion of map slots and reduces slots. In the end through the comparison of experiment it is proved that the IS has a great improvement in job scheduling for Hadoop.

목차

Abstract
 1. Introduction
 2. Related Definition of IS
  2.1. Transmission Time
  2.2. Waiting Time
  2.3. Effective Slot
  2.4. Effective Slot Rate
 3. IS Scheduling Algorithm
  3.1. The Goal of IS
  3.2. The Combination of HDFS and IS
  3.3. The Data Flow of IS
  3.4. The Implementation Process of IS
 4. Verification
 5. Conclusion
 Acknowledgments
 References

키워드

저자

• Xue Tao [ School of Computer Science, Xi’an Polytechnic University, Xi’an 710048, China ]
• Yan Minglei [ School of Computer Science, Xi’an Polytechnic University, Xi’an 710048, China ]

참고문헌

발행기관

간행물

표지보기 관심저널 등록

• 간행물명

  International Journal of Hybrid Information Technology

• 간기

  격월간

• pISSN

  1738-9968

• 수록기간

  2008~2016

• 십진분류

  KDC 505 DDC 605