Earticle

There has been a snappy forward advancement in cloud. With the developing measures of affiliations, turning number of affiliations are being utilized as assets in the cloud. Issues arise regarding the certainties of various clients utilizing these assets like round measure of room associations which helps in keeping from the cost. Putting away these associations evades the high cost while posting information during the building of machine orders. This collection of information gives better execution whilst less “far point cost” and ready to make prepared modifications. Cloud can yield better possibilities through net where security weaknesses are restricted. Despite of “wellbeing” being one of the brimming with risk weakness, it adjusts unmistakable connection to go into assumed control handling general condition. This paper uses “MapReduce” library that parallelizes the figuring, and handles perplexed issues like data spread, stack modifying and adjustment to non-basic disappointment. Enormous information, spread across finished many machines, need to parallelize. Moves the data, and gives booking, adjustment to non-basic disappointment. In this paper, a graph of MapReduce programming model along with the applications are explored. The maker has delineated here the work procedure of MapReduce process. Some fundamental issues, like adjustment to non-basic disappointment, are considered in more detail. Without a doubt, even the outline of working of Map Reduce is given.

정보화 시대는 다양한 형태로 빠르게 축적되고 있는 빅데이터를 저장, 관리 및 처리하기 위한 새로운 연구 접근법을 요구하고 있다. 본 논문은 시계열 특성을 가지는 디지털운행기록계 빅데이터의 처리 분석을 대한 연구를 진행하고, 하둡분산파일시스템과 맵리듀스를 이용하여 패턴 기반의 분석을 위한 분산 처리의 성능 향상을 이루는 방법론을 제안한다. 전처리 과정, 필터링, 이차정렬을 포함하는 본 논문의 메커니즘을 이용하면, 여러 파일에 분할 저장된 시계열 빅데이터에서 시계열 특성의 패턴 을 이벤트 형태로 분석 및 생성할 수 있다. 본 논문은 빅데이터의 전처리 과정을 통한 성능 향상 효과 와, 제안한 시계열 분석 방법론인 시계열 맵리듀스 메커니즘을 적용한 실험과 분석 결과를 통하여 본 논문이 제안한 방법론의 타당성, 정확성 및 고성능 향상성을 보인다.

The information era requires another new research approach to store, manage and process the big data accumulated rapidly in a variety of forms. This paper pursues the research on handling and analyzing digital tachograph big data, which is recorded in a time series manner. It also proposes the methodology of performance improvement in distributed processing for patternbased analysis over Hadoop Distributed File System and MapReduce. By means of our proposing mechanism that includes preprocessing, filtering, and secondary sorting, it is possible that the time series featured pattern is captured, analyzed and generated in the form of an event on the top of time series big data, which would be scattered over distributed files. This paper also shows the effect of performance progress with big data preprocessing, and verifies properness, correctness, and high-performance improvement via experiments with our proposing time series MapReduce mechanism.

The database integration is being emphasized to one way of the companies collaboration. To database integration, companies are use like one database what their own, it can be provided more efficient service to customer. However, there exist some difficulty to database integration. that is the database security and database heterogeneity problems. In this paper, we proposed the MapReduce based p2p DBaaS hub system to solve database heterogeneity problem. The proposed system provides an environment for companies in the P2P cloud to integrate a database of each other. The proposed system uses DBaaS Hub for a collection of data in the P2P cloud, and use MapReduce for integrating the collected data.

The database integration is being emphasized to one way of the companies collaboration. To database integration, companies are use like one database what their own, it can be provided more efficient service to customer. However, there exist some difficulty to database integration. that is the database security and database heterogeneity problems. In this paper, we proposed the MapReduce based p2p DBaaS hub system to solve database heterogeneity problem. The proposed system provides an environment for companies in the P2P cloud to integrate a database of each other. The proposed system uses DBaaS Hub for a collection of data in the P2P cloud, and use MapReduce for integrating the collected data.

Support Vector Machine (SVM) is extremely powerful and widely accepted classifier in the field of machine learning due to its better generalization capability. However, SVM is not suiTable for large scale dataset due to its high computational complexity. The computation and storage requirement increases tremendously for large dataset. In this paper, we have proposed a MapReduce based SVM for large scale data. MapReduce is a distributed programming model which works on large scale dataset by dividing the huge datasets in smaller chunks. MapReduce distribution model works on several frame works like Hadoop Twister and so on. In this paper, we have analyzed the impact of penalty and kernel parameters on the performance of parallel SVM. The experimental result shows that the number of support vectors and predictive accuracy of SVM is affected by the choice of these parameters. From experimental results, it is also analyzed that the computation time taken by the SVM with multi-node cluster is less as compared to the single node cluster for large dataset.

As the web, social networking, and smartphone application have been popular, the data has grown drastically everyday. Thus, such data is called Big Data. Google met Big Data earlier than others and recognized the importance of the storage and computation of Big Data. Thus, Google implemented its parallel computing platform with Map/Reduce approach on Google Distributed File Systems (GFS) in order to compute Big Data. Map/Reduce motivates to redesign and convert the existing sequential algorithms to Map/Reduce algorithms for Big Data so that the paper presents Market Basket Analysis algorithm with Map/Reduce, one of popular data mining algorithms. The algorithm is to sort data set and to convert it to (key, value) pair to fit with Map/Reduce. Amazon Web Service (AWS) provides Apache Hadoop platform that provide Map/Reduce computing on Hadoop Distributed File Systems (HDFS) as one of many its services. In the paper, the proposed algorithm is executed on Amazon EC2 Map/Reduce platform with Hadoop. The experimental results show that the code with Map/Reduce increases the performance as adding more nodes but at a certain point, Map/Reduce has the limitation of exploring the parallelism with a bottle-neck that does not allow the performance gain. It is believed that the operations of distributing, aggregating, and reducing data in the nodes of Map/Reduce should cause the bottle-neck.

Data privacy preservation is one of the most disturbed issues on the current industry. Data privacy issues need to be addressed urgently before data sets are shared on cloud. Data anonymization refers to as hiding complex data for owners of data records. In this paper investigate the problem of big data anonymization for privacy preservation from the perspectives of scalability and time factor etc. At present, the scale of data in many cloud applications increases tremendously in accordance with the big data trend. Here propose a scalable Two Phase Top-Down Specialization (TPTDS) approach to anonymize large-scale data sets using the MapReduce framework on cloud. For the data anonymization-45,222 records of adults information with 15 attribute values was taken as the input big data. With the help of multidimensional anonymization on map reducing framework, here implemented the proposed Two-Phase Top-Down Specialization anonymization algorithm on hadoop will increases the efficiency of the big data processing system. In both phases of the approach, deliberately design multidientional MapReduce jobs to concretely accomplish the specialization computation in a highly scalable way. Data sets are generalized in a top-down manner and the better result was shown in multidmientional MapReduce framework by compairing the onedimentional MapReduce framework anonymization job. The anonymization was performed with specialization operation on the taxonomy tree. The experiment demonstrates that the solutions can significantly improve the scalability and efficiency of big data privacy preservation compared to existing approaches. This work has great applications to both public and private sectors that share information to the society.

DBSCAN is a density-based clustering algorithm. This algorithm clusters data of high density. The traditional DBSCAN clustering algorithm in finding the core object, will use this object as the center core, extends outwards continuously. At this point, the core objects growing, unprocessed objects are retained in memory, which will occupy a lot of memory and I/O overhead, algorithm efficiency is not high. In order to ensure the high efficiency of DBSCAN clustering algorithm, and reduce its memory footprint. In this paper, the original DBSCAN algorithm was improved, and the G-DBSCAN algorithm is proposed. G-DBSCAN algorithm reduces the number of query object as a starting point. Put the data into the grid, with the center point of the data in the grid to replace all the grid points as the algorithm input. The query object will be drastically reduced, thus improving the efficiency of the algorithm, reduces the memory footprint. In order to make the G-DBSCAN algorithm can adapt to large data processing, we will parallelize the G-DBSCAN algorithm, and combining it with Map Reduce framework. The results prove that G-DBSCAN and MRG-DBSCAN algorithm are feasible and effective.

Information retrieval is fast becoming the prevailing form of information access, surpassing traditional database style searching. Ontologies have become the tool of choice employed in many information retrieval systems and more prominently in semantic information retrieval. In order to overcome the disadvantages in key word based information retrieval systems, which transfer irrelevant information, ontology has been designed. A system with ontology mimics the real world, where every task is laced with certain meaning as this is basic idea behind knowledge processing. Hadoop, which is an open source frame work for storing and processing large datasets, is used for pre-processing the text documents. First, a set of text documents are considered. Pre-processing is performed on a large domain of data using Hadoop MapReduce. This includes the removal of the stop words along with stemming and excluding less frequency words. Despite this pre-processing, owing to the colossal number of index terms still floating in the considered domain data, the problem of high dimensionality is encountered. Therefore the dimensionality of such a group of terms is reduced by identifying it as a concept and those concepts can be viewed as a single dimension in a ontology based information retrieval system. Now ontology is constructed by assigning synonym set to each concept in this structure using tools like word net. Thus constructed ontology can be mapped on to the processed query which gives us the relevant information from the data pool considered.

Big Data is a term used to identify the datasets that due to their large size, is very difficult to manage with traditional techniques. This data may be in the order of magnitude of petabytes. It can be found easily on web, especially on social media in the form of customer blogs, reviews and comments. Generally it is unstructured data or semi-structured data. One can use this big data to generate values by calculating sentiment score. Map Reduce is one of the most popular algorithm in Hadoop environment to perform such task. The objective of present research is to automate the process of extracting sentiments expressed about specific features of a product. For this purpose three datasets generated by Amazon for different types of electronics product reviews has been used. The data sets used consists of reviews of the products Nikon Coolpix 4300 Camera, Nokia 6601 mobile and the Canon G3camera. Map Reduce algorithm on Hadoop environment that is considered faster, reliable and fault-tolerant for processing big amounts of data in-parallel on large clusters, has been used to extract sentiment score.

본 논문에서는, 맵-리듀스 모델 기반에서 나이브 베이스 알고리즘으로 학습과 추론을 수행하는 방안에 대해 소개하고자 한다. 이를 위해 Apache Mahout를 이용하여 분산 나이브 베이스 (Distributed Naive Bayes) 학습 알고리즘을 University of California, Irvine (UCI)의 벤치마크 데이터 집합에 적용하였다. 실험 결과, Apache Mahout의 분산 나이브 베이스 학습 알고리즘은 일반적인 WEKA의 나이브 베이스 학습 알고리즘과 그 성능면에서 큰 차이가 없음을 알 수 있었다. 이러한 결과는, 향후 빅 데이터 환경에서 Apache Mahout와 같은 맵-리듀스 모델 기반 시스템이 기계 학습에 큰 기여를 할 수 있음을 나타내는 것이다.

In this paper, we introduce a Naive Bayes learning algorithm for learning and reasoning in Map-Reduce model based environment. For this purpose, we use Apache Mahout to execute Distributed Naive Bayes on University of California, Irvine (UCI) benchmark data sets. From the experimental results, we see that Apache Mahout' s Distributed Naive Bayes algorithm is comparable to WEKA' s Naive Bayes algorithm in terms of performance. These results indicates that in the future Big Data environment, Map-Reduce model based systems such as Apache Mahout can be promising for machine learning usage.

본 연구는 기업의 전략 의사결정(Strategic Decision-Making) 실무에 빅데이터를 활용하기 위한 방안으로 하둡-맵 리듀스(Map Reduce)를 통해 처리한 데이터를 이용해 퍼지 인식도(Fuzzy Cognitive Map)의 인과 행렬을 작성하고, 작성된 퍼지 인식도를 활용하는 경영 의사결정 방법과 의사 결정 지원 시스템(DSS: Decision Support System)을 제안한다. 제안을 위해 관련 연구 및 개념, 퍼지 인식도를 기반으로 하는 의사결정 지원 시스템과 제안한 시스템이 갖는 장점, 그리고 퍼지 인식도 기반 의사결정 지원 시스템의 실제 활용 가능성에 대해서 실험을 통해 검증한 내용을 담고 있다.

빅 데이터의 데이터 수집 및 분석 기술에 대한 연구는 컴퓨터 과학 분야에서 각광 받고 있다. 또한 소셜 미디어로 인한 대량의 비정형 데이터 분석을 요구하는 다양한 분야에 접목되어 효용성을 인정받고 있다. 그러나 빅 데이터 개념을 기반으로 하는 하둡과 스파크는 유즈케이스에 따라 성능이 크게 달라진다는 문제점이 있다. 이러한 문제점을 해결하기 위해 본 논문에서는 하둡의 맵리듀스를 줄이고 아파치 스파크를 이용한 빅 데이터 분석을 위하여 머신러닝 알고리즘인 K-Means 알고리즘을 이용하여 프로세싱 모델의 성능을 비교한다.

본 논문에서는 하둡 플랫폼의 맵리듀스 모델에 기반하여 도서관 이용자들이 자주 대출하는 도서와 키워드 매칭을 통해 연관성이 높은 도서들을 추출하고 추천해 주는 도서 대출 추천 시스템을 구현 개발한다. 구현 개발된 시스템은 빅데이터의 특징을 갖는 도서관의 대출 로그 데이터로부터 타겟 도서와 유사한 키워드를 갖고 자주 대출되는 도서를 찾아 이용자에게 제공해 준다.