현재 위치
Home
An Efficient Replicated System for the Metadata of HDFS
-
- 발행기관
- 보안공학연구지원센터(IJGDC) 바로가기
-
- 간행물
- International Journal of Grid and Distributed Computing SCOPUS 바로가기
-
- 통권
- Vol.9 No.5 (2016.05)바로가기
-
- 페이지
- pp.175-190
-
- 저자
-
- 언어
- 영어(ENG)
-
- URL
- https://www.earticle.net/Article/A275593
※ 원문제공기관과의 협약기간이 종료되어 열람이 제한될 수 있습니다.
원문정보
초록
- 영어
- Hadoop HDFS is an open source project from Apache Software Foundation for scalable, distributed computing and data storage. HDFS has become a critical component in today's cloud computing environment and a wide range of applications built on top of it. However, the initial design of HDFS has introduced a single-point-of-failure, since HDFS contains only one active namenode, if this namenode experiences software or hardware failures, the whole HDFS cluster is unusable, this is a reason why people are reluctant to deploy HDFS for an application whose requirement is high availability. In this paper, we present a solution to enable the high availability for HDFS's namenode through efficient metadata replication. Our solution has 3 major advantages than existing ones: We utilize multiple active namenodes, instead of one, to build a cluster to serve requests of metadata simultaneously; We implement a pub/sub system to handle the metadata replication process across these active namonodes efficiently; We also propose a novel replication algorithm to deal with the network delay when the namonodes are deployed in different areas. Based on the solution we build a prototype called NCluster and integrate it with HDFS. We evaluate NCluster to exhibit its feasibility and effectiveness. The experimental results show that our solution performs well with low replication cost, good throughput and scalability.
목차
Abstract
1. Introduction
2. Background and Related Work
2.1. Metadata of HDFS
2.2. Metadata Availability
2.3. Replication
3. System Design
3.1 Architecture
3.2 What to Replicate
3.3. How to Replicate
3.4 Consistency
3.5 Replication Optimization
3.6 Failover
4. Implemention
5. Evaluations
5.1. Experimental Setup
5.2. Replication Cost
5.3. NCluster vs Zookeeper
5.4 Throughput
5.5 Scalability
5.6 Geo-Replication
6. Conclusions and Future Work
References
키워드
HDFS namenode metadata availability replication NCluster저자
간행물 정보
발행기관
-
- 발행기관명
- 보안공학연구지원센터(IJGDC) [Science & Engineering Research Support Center, Republic of Korea(IJGDC)]
- 설립연도
- 2006
- 분야
- 공학>컴퓨터학
- 소개
- 1. 보안공학에 대한 각종 조사 및 연구 2. 보안공학에 대한 응용기술 연구 및 발표 3. 보안공학에 관한 각종 학술 발표회 및 전시회 개최 4. 보안공학 기술의 상호 협조 및 정보교환 5. 보안공학에 관한 표준화 사업 및 규격의 제정 6. 보안공학에 관한 산학연 협동의 증진 7. 국제적 학술 교류 및 기술 협력 8. 보안공학에 관한 논문지 발간 9. 기타 본 회 목적 달성에 필요한 사업
간행물
-
- 간행물명
- International Journal of Grid and Distributed Computing
- 간기
- 격월간
- pISSN
- 2005-4262
- 수록기간
- 2008~2016
- 십진분류
- KDC 505 DDC 605
이 권호 내 다른 논문 / International Journal of Grid and Distributed Computing Vol.9 No.5
피인용수 : 0건 (자료제공 : 네이버학술정보)
함께 이용한 논문 이 논문을 다운로드한 분들이 이용한 다른 논문입니다.
출처 : 네이버학술정보
0개의 논문이 장바구니에 담겼습니다.