Earticle

This study measured the stream data from the several sensors, and stores the database in MapReduce framework environment, and it aims to design system with the small performance and cluster analysis error rate through the KMSVM algorithm. Through the KM-SVM algorithm, the cluster analysis effective data was used for U-health system. In the results of experiment by using 2003 data sets obtained from 52 test subjects, the k-NN algorithm showed 79.29% cluster analysis accuracy, K-means algorithm showed 87.15 cluster analysis accuracy, and SVM algorithm showed 83.72%, KM-SVM showed 90.72%. As a result, the process speed and cluster analysis effective ratio of KM- SVM algorithm was better.

K-Means is a famous partition based clustering algorithm. Various extensions of K-Means have been proposed depending on the type of datasets being handled. Popular ones include K-Modes for categorical data and K-Prototype for mixed numerical and categorical data. The K-Means and its extensions suffer from one major limitation that is dependency on prior input of number of clusters K. Sometimes it becomes practically impossible to correctly estimate the optimum number of clusters in advance. Various ways have been suggested in literature to overcome this limitation for numerical data. But for categorical and mixed data work is still in progress. In this paper, we introduce a new algorithm based on the K-Means that takes mixed dataset as an input and generates appropriate number of clusters on the run using MapReduce programming style. The new algorithm not only overcomes the limitation of providing the value of K initially but also reduces the computation time using MapReduce framework.

Distributed computing accomplished broad appropriation because of consequently parallelizing and transparently executing tasks in distributed environments. Straggling tasks is an essential test confronted by all Big Data Processing Frameworks for example MapReduce, Dryad, and Spark. Stragglers are the assignments that run much slower than different tasks and since a job completes just when it’s last undertaking completions, stragglers postponement work fruition. Stragglers extraordinarily impact little occupations such that employments comprising of a couple of undertakings. Such occupations are fundamentally deferred regardless of the fact that a solitary undertaking is moderate .This paper survey stragglers recognition and rescheduling systems proposed so far and brings up their strengths and shortcomings. This paper additionally displays wise attributes and impediments of the existing state- of-the- craftsmanship calculations to take care of the issue of stragglers relief.

SnIClustering Algorithm is put forward to deal with the large number of intermediate values when processing MapReduce. SnIClustering Algorithm picks up a few representative data through cluster sampling, and then retains the useful data through filtration according to the distribution characteristics. By doing so, intermediate values of MapReduce can be reduced sharply, saving time and easing network load. The last step is to cluster the selected data and samples. Experimental results show that SnIClustering is suitable to process large-scale data, since it can both process large-scale data within a short time and maintain fine clustering effect.

In our previous work, a user similarity-based contents recommendation service using NFC was proposed for the same goal. This service used a small sample because it used only information of the user who had watched a museum. However, it has been shown that there are some limitations resulting from the difficulty of accurately predicting the user's preference. In order to lift this drawback, this paper introduces a user taste prediction service using big data for improving user-friendliness to a maximum. The proposed service predicts the user's taste using big data such as Twitter and blogs. It is possible to predict the exact user's preference and might recommend more suitable contents to the user's taste because it predicts the user`s taste using big data with a variety of user's social network information. So, it can recommend contents that match the user's taste. Our simulation results show the proposed big data-based approach can give each museum visiting user more accurate recommendation service appropriate to his or her taste compared with the previous one in terms of user preferences to exhibition-related contents.

MapReduce, a large-scale data processing paradigm, is gaining popularity. However, like other distributed computing frameworks, MapReduce suffers from the integrity assurance vulnerability: malicious workers in the MapReduce cluster could tamper with its computation result and thereby render the overall computation result inaccurate. Existing solutions are effective in defeating the malicious behavior of non-collusive workers, but are less effective in detecting collusive workers. In this paper, we propose the Verification-based Integrity Assurance Framework (VIAF). By using task replication and probabilistic result verification, VIAF can detect both non-collusive and collusive workers, even if the malicious workers dominate the environment. We have implemented VIAF on Hadoop, an open source MapReduce implementation. Our theoretical analysis and experimental result show that VIAF can achieve high job accuracy while imposing moderate performance overhead.

To meet the scalability and performance requirements of data analyses, which often involve voluminous data, efficient parallel or concurrent algorithms and frameworks are essential. We present a high-performance Korean morphological analyzer which employs the MapReduce framework on the graphics processing unit (GPU). MapReduce is a programming framework introduced by Google to aid the development of web search applications on a large number of central processing units (CPUs). GPUs are designed as a special-purpose co-processor. Their programming interfaces are typically formulated for graphics applications. Compared to CPUs, GPUs have greater computation power and memory bandwidth; however, GPUs are more difficult to program because of the design of their architectures. The performance of the Korean morphological analyzer using the MapReduce framework on the GPU is evaluated in comparison with the CPU-based model. The proposed Korean Morphological analyzer shows promising scalable performance on distributed computing with the GPU.

In the current era of data-intensive services, the handling of big data is a crucial issue that affects almost every discipline and industry. In this study, we propose a classification method for large volumes of numeric data, which is implemented in a distributed programming framework, i.e., MapReduce. The proposed method partitions the data space into a grid structure and it then models the probability distributions of classes for grid cells by collecting sufficient statistics using distributed MapReduce tasks. The class labeling of new data is achieved by k-nearest neighbor classification based on Bayesian inference.

MapReduce는 다양한 형식의 대용량 데이터를 병렬 처리하는데 있어 효과적인 도구로 인식되고 있다. 특히 MapReduce의 오픈 소스 구현인 Hadoop은 여러 분야에서 널리 이용되고 있으며, 가장 대표적인 빅데이터 솔루션으로 현재까지 많은 주목을 받아오고 있다. 하지만, MapReduce는 그 구조적 특정으로 인한 이점과 함께 여러 제약과 단점들을 가진다. 이에 따라 MapReduce의 개선을 위한 많은 연구와 시스템 개량이 학계와 산업계에서 동시에 수행되어 왔다. 본고에서는 대용량 데이터 분석을 위한 MapReduce 프레임워크의 특성과 이를 개선하기 위한 최근의 연구 내용들을 소개한다. 또한 향후의 대용량 데이터 처리는 어떠한 모습을 취하게 될 것인지를 예측해 본다.

빠른 웹 정보의 변화에 잘 대응하기 위해서는, 사실과 지식이 실제로 유효한 시간과 장소들도 함께 표현하고 그들 간의 관계도 추론할 수 있도록 웹 기술의 확장이 필요하다. 본 논문에서는 그동안 소규모 지식 베이스를 이용한 실험실 수준의 정성 시간 추론 연구들에서 벗어나, 웹 스케일의 대규모 지식 베이스를 추론할 수 있는 병렬 정성 시간 추론기인 MRQUTER의 설계와 구현을 소개한다. Hadoop 클러스터 시스템과 MapReduce 병렬 프로그래밍 프레임워크를 이용해 개발된 MRQUTER에서는 정성 시간 추론 과정을 인코딩 및 디코딩 작업, 역 관계 및 동일 관계 추론 작업, 이행 관계 추론 작업, 관계 정제 작업 등 몇 개의 MapReduce 작업으로 나누고, 맵 함수와 리듀스 함수로 구현되는 각각의 단위 추론 작업을 효율화하기 위한 최적화 기술들을 적용하였다. 대규모 벤치마킹 시간 지식 베이스를 이용한 실험을 통해, MRQUTER의 높은 추론 성능과 확장성을 확인하였다.

In order to meet rapid changes of Web information, it is necessary to extend the current Web technologies to represent both the valid time and location of each fact and knowledge, and reason their relationships. Until recently, many researches on qualitative temporal reasoning have been conducted in laboratory-scale, dealing with small knowledge bases. However, in this paper, we propose the design and implementation of a parallel qualitative temporal reasoner, MRQUTER, which can make reasoning over Web-scale large knowledge bases. This parallel temporal reasoner was built on a Hadoop cluster system using the MapReduce parallel programming framework. It decomposes the entire qualitative temporal reasoning process into several MapReduce jobs such as the encoding and decoding job, the inverse and equal reasoning job, the transitive reasoning job, the refining job, and applies some optimization techniques into each component reasoning job implemented with a pair of Map and Reduce functions. Through experiments using large benchmarking temporal knowledge bases, MRQUTER shows high reasoning performance and scalability.

미국의 Jeopardy! 퀴즈쇼와 같은 DeepQA 환경에서 인간을 대신해 컴퓨터가 효과적으로 답하기 위해서는, 광범위한 지식베이스와 빠른 시공간 추론 능력이 요구된다. 본 논문에서는 대표적인 병렬 분산 컴퓨팅 환경인 맵리듀스 프레임워크를 이용해, 새로운 방향 및 위상 관계를 효율적으로 추론할 수 있는 대용량 공간 추론 알고리즘을 제시한다. 이 추론 알고리즘은 CSD-9 방향 관계들과 RCC-8 위상 관계들을 포함한 대용량 공간 지식베이스를 입력으로 가정하며, 이로부터 새로운 방향 관계와 위상 관계들을 추론해내기 위해 지식베이스에 대한 경로 일관성 검사와 교차 일관성 검사를 수행한다. 맵리듀스 프레임워크의 원리에 따라 추론 계산의 병렬성을 극대화하기 위해, 맵 단계에서는 대용량의 지식베이스를 다수의 노드들에 효과적으로 분할하여 분산시키고, 리듀스 단계에서는 분산된 지식베이스들로부터 새로운 공간 지식을 유도하도록 공간 추론 알고리즘을 설계하였다. 본 연구에서는 맵리듀스 프레임워크로 구현한 대용량 공간 추론기와 샘플 공간 지식 베이스를 이용한 실험들을 수행하고, 이를 통해 본 논문에서 제안한 대용량 공간 추론기의 높은 성능을 확인할 수 있었다.

In order to answer the questions successfully on behalf of the human in DeepQA environments such as Jeopardy! of the American quiz show, the computer is required to have the capability of fast temporal and spatial reasoning on a large-scale commonsense knowledge base. In this paper, we present a scalable spatial reasoning algorithm for deriving efficiently new directional and topological relations using the MapReduce framework, one of well-known parallel distributed computing environments. The proposed reasoning algorithm assumes as input a large-scale spatial knowledge base including CSD-9 directional relations and RCC-8 topological relations. To infer new directional and topological relations from the given spatial knowledge base, it performs the cross-consistency checks as well as the path-consistency checks on the knowledge base. To maximize the parallelism of reasoning computations according to the principle of the MapReduce framework, we design the algorithm to partition effectively the large knowledge base into smaller ones and distribute them over multiple computing nodes at the map phase. And then, at the reduce phase, the algorithm infers the new knowledge from distributed spatial knowledge bases. Through experiments performed on the sample knowledge base with the MapReduce-based implementation of our algorithm, we proved the high performance of our large-scale spatial reasoner.