Earticle

아파치 스파크(Apache Spark)는 오픈 소스, 분산, 인메모리 컴퓨팅 프레임워크 및 아키텍처이다. 아파치 스파크를 통해 데이터 병렬 및 내결함성을 갖춘 분산 시스템 클러스터에서 실행되기 때문에 농업, 정보통신 산업 등 다양한 분야에 적용 및 활용할 수 있다. 한편, 빅데이터 시대를 맞아 많은 시공간 데이터가 발생하고 있다. 그러나, 아파치 스파크를 통한 분산 컴퓨팅 환경에서 많은 연산이 필요한 조인과 같은 연산을 제공하고 있지 않기 때문에 효율적으로 처리하지 못한다. 따라서, 본 논문에서는 대용량 시공간 데이터를 이용하여 그리드 분할 기법에 근거한 withindistance, contain 조인 질의처리 알고리즘을 제안한다. 성능평가 결과, 제안한 알고리즘이 기존 알고리즘보다 약 20%의 우수한 성능을 보인다.

Apache Spark is an open source, distributed, in-memory computing framework and architecture. Because it runs on a distributed system cluster with data parallelism and fault tolerance through Apache Spark, it can be applied and utilized in various fields such as agriculture, information and communication industries. Meanwhile, in the era of big data, large amount of spatio-temporal data is being generated. However, Apache Spark cannot efficiently process join operation because it does not support join operation that requires many computations in a distributed computing environment. Therefore, in this paper, we proposed an join query processing algorithm, i.e., withindistance and contain join, based on grid partitioning technique using large-scale spatio-temporal data. As a result of performance evaluation, our algorithm shows 20% better performance than the existing algorithm in terms of query processing time.

Blockchain technology changed the IoT data computing paradigm significantly with its secure communication protocol. It offers a considerable advantages compared to traditional IoT data computing systems with a simple and secure P2P network. Meanwhile, the integration process faces considerable challenges related to big data management. One of the main problems, broadcasting the big data through the blockchain nodes would cost significant space usage for all nodes. In this paper, we present a new method named HashBlock on sharing the hash of big data using blockchain network. Hash-based blockchain network guarantees the constant space usage for all types and sizes of data for each record. The preliminary experiments prove that the proposed method is expected to improve the big data transmission quality in terms of space efficiency.

클러스터링은 상호 의존적 구성에 필요한 구조적인 기술이다. 클러스터링은 가변적 업무부하를 처리하거나, 서비스 연속성을 저해하는 고장 발생 시 운영이 계속되도록 여러대의 컴퓨터시스템 기능을 서로 연결하는 메커니즘이다. 고 가 용성 클러스터링 기능은 가능한 오랜시간 서버 시스템이 작동하는데 중점을 둔다. 이 클러스터 는 멀티플 시스템 에서 실행되는 노드와 서비스를 중복하여 가지고 있어서 서로가 서로를 추적할 수 있다. Active-Standby 상태의 두 시스템 이 있을 경우 활성 서버에 장애가 발생했을 때 모든 서비스가 대기 서버에서 구동돼 서비스가 이루어진다. 이 기능을 절체 또는 스위치오버(switchover)라 한다. 고가용성 클러스터링 기능은 가능한 오랜시간 서버 시스템이 작동하는데 중 점을 둔다. 이 클러스터는 멀티플 시스템 에서 실행되는 노드와 서비스를 중복하여 가지고 있어서 서로가 서로를 추적 할 수 있다. 한 노드가 장애 발생 시 둘째 노드가 몇초 이내에 고장 난 노드 임무를 수행한다. 고가용성 클러스터링 구 조는 효율성 여부가 측정되어야 한다. 시스템 성능은 인프라시스템의 performance, latency, 응답시간(responseTime), CPU 부하율(CPU utilization), CPU상의 시스템 프로세스(system process)수로 대표된다.

Clustering is required to configure clustering interdependent structural technology. Clustering handles variable workloads or impede continuity of service to continue operating in the event of a failure. Long as high-availability clustering feature focuses on server operating systems.Active-standby state of two systems when the active server fails, all services are running on the standby server, it takes the service. This function switching or switchover is called failover. Long as high-availability clustering feature focuses on server operating systems. The cluster node that is running on multiple systems and services have to duplicate each other so you can keep track of. In the event of a node failure within a few seconds the second node, the node shall perform the duties broken. Structure for high-availability clustering efficiency should be measured. System performance of infrastructure systems performance, latency, response time, CPU load factor(CPU utilization), CPU processes on the system (system process) channels are represented.

본 논문은 클라우드 컴퓨팅 환경에서 다양한 분산 멀티미디어 서비스를 효율적으로 제공하기 위한 새로운 시맨틱 버 스 모델을 제안한다. 제안하는 시맨틱 버스 모델은 1) 계층적으로 함께 협력하는 애플리케이션과 전송 바인딩으로 구성되어 있으며, 2) 버스의 실시간 동적 적응을 지원하기 위해, 형상 제어 공간과 기능 제어 공간으로 구성된 메타 공간을 갖으며, 3) 프로파일 기반 서비스 및 자원 관리를 지원한다. 4) 또한, 제안하는 시맨틱 버스를 생성하기 위 한 인터페이스를 Backus-Naur Form 정의를 기반으로 제안하였다. 실험을 통해 제안된 시맨틱 버스 모델이 기존 의 균일 non-시맨틱 모델에 비해 향상된 처리량을 제공함을 검증하였으며, 제안된 버스 모델은 차세대 컴퓨팅 플랫 폼에서 서비스 제공 및 데이터 전송을 위한 핵심 인프라로 활용될 수 있다.

This paper proposes a new semantic bus model to efficiently provide multimedia services in a cloud computing environment. The proposed semantic bus model 1) consists of applications and transport bindings that cooperate together hierarchically; 2) has a metaspace consisting of a configuration control space and a function control space to support real-time dynamic adaptation of the bus; 3) supports profile-based service and resource management; 4) We also proposed Backus-Naur Form (BNF) defintion based interface specification that generates the proposed semantic bus models. Through experiments, it was verified that the proposed semantic bus model provides improved throughput compared to the existing non-semantic model. The proposed bus model can be used as a core infrastructure for service provision and data transmission in next-generation computing platforms.

위치기반 서비스란 휴대폰과 같은 모바일 단말기 속에 위성 항법장치(GPS)와 연결되는 칩을 부착하여 위치추적 서비스, 공공안전 서비스, 위치기반 정보 서비스 등 위치와 관련된 각종 정보를 제공하는 서비스를 일컫는다. 그리고 클라우드 컴퓨팅이란 개인용 컴퓨터 또는 기업의 서버에 개별적으로 저장해 두었던 자료와 소프트웨어들을 클라우드 클러스터로 구축하여 필요할 때 PC나 휴대폰 같은 각종 단말기를 이용하여 원격 작업을 수행할 수 있는 환경을 의미한다. 본 논문에서는 구글의 연구로부터 시작된 클라우드 컴퓨팅 기술에 주목하고 클라우드 컴퓨팅 환경을 위한 대용량 이동 객체의 정보를 저장하고 질의하는 시스템에 대하여 연구하였다. 그리고 위치 기반 서비스를 위한 대용량 이동객체에 대하여 효율적인 질의를 수행할 수 있는 저장 구조를 제안하였다. 본 논문에서 제안한 방법을 적용한 시스템의 효율성을 증명하기 위해 저장 및 질의 성능을 다양한 측면에서 실험하고, 실험 결과를 다른 분산처리 시스템과의 비교하여 성능을 입증하였다.

The location-based service is referred to as the service for providing various types of information regarding such as the location-tracing service, the public-safety service and the location-based information service by attaching chips connected with the GPS to the mobile phone. In addition, cloud computing means the environment which makes it possible to carry out remote operations by establishing cloud clusters with the data and software which have been individually stored in personal computer or corporate servers and using devices such as personal computers and cellular phones whenever it is necessary. Throughout this paper, the cloud computing technology which originates from the research by Google the system has been focused on and the system for storing a large volume of information for moving object and making query for cloud computing environment has been studied. Besides, the storing structure for executing efficient query regarding the high-capacity mobile devices for the location-based service has been suggested. In order to prove the efficiency of the system with the application of the method suggested in this paper, the storage and qualitative functions have been experimented in various aspects and compared with other distributed processing system's performance.

본 논문에서는 모바일 컴퓨팅 환경에서의 디바이스 연계 통합을 위한 스토리지 가상화 기술로서, PosCFS+ 분산파일 서비스를 소개한다. PosCFS+ 분산 파일 서비스는 사용자 영역 네트워크(PAN) 상의 장치별 스토리지 공간을 하나의 가상 스토리지 공간으로 자동 구성하며, 구성된 가상 스토리지 공간의 파일 입출력을 위하여 파일 메타데이터를 기준으로 가상 디렉토리를 동적으로 구성함으로써, 사용자의 데이터 접근성 개선 및 데이터 통합 관리를 지원한다.

In this paper, we present the design and implementation of a distributed file service, PosCFS+, for device ensembles. All the storages in the network could be integrated automatically into a virtual storage by the file service and then the service creates virtual directories to provide a semantic file addressing based on the file metadata on the virtual storage. Through the virtualization technique, the file service is able to enhance the data accessibility.

In recent years, distributed computing technology has been one of the cutting edge technologies for its low power and cost, which makes numerous IT organizations extend their hands in order to improve their financial ability. Because of these new features, grid computing, the original task scheduling mechanism, can’t work effectively in distributed computing environments, hence, we need a new task scheduling method to solve the problems. With considering the complex characters of the task in different distributed computing applications, firstly, we construct a more comprehensive task scheduling model, which has three sub objective functions. Secondly, we present an improved genetic algorithm to solve the multi- objective NP-hard problem. Finally, we implement some simulation experiments, and the evaluation results show us that the proposed model and improved GA are efficient and effective. The first part is the research status and related problems. The second part is the establishment of system architecture and task scheduling model. The last part is the experimental analysis and conclusion.

In recent years, with the rapid development of computer network, distributed computing system has a very vast application prospect and potential utility value, which opens up a wealth of opportunities for different applications. With the characteristics of dynamic, heterogeneity, distribution, openness, voluntariness, uncertainty and deception, how to obtain trustworthy computing resource becomes a key issue in large-scale distributed computing research. Therefore, with considering the complex characters of trust in distributed computing environment, firstly, we construct STE architecture to rank and observe trust, which includes STE Broker, Monitoring and STE Catalogue. Secondly, a more comprehensive dynamic trust evaluation model is constructed based on Bayesian network. Finally, we apply simulation platform to imitate trust evolution process and collect related data, and the proposed method has been serviced in complex simulation system, and the results have indicated that the model is unbiased and effective. The first part is the research status and related problems. The second part is the establishment of an evaluation model. The last part is the experimental analysis and conclusion.

Distributed computing guarantees to on a very basic level change the way we utilize PCs and get to and store our own specific and business data. With these new registering and correspondences models develop new data security challenges. Existing information security structures, for instance, encryption have fizzled in imagining information theft strikes, particularly those executed by an insider to the cloud supplier. We propose a substitute methodology for securing information in the cloud utilizing adversarial mimic improvement. We screen data access in the cloud and perceive unpredictable data access outlines. Right when unapproved access is suspected and after that confirmed using test questions, we dispatch a disinformation strike by giving back a considerable measure of fake information to the attacker. This secures against the misuse of the customer's real data. Trials coordinated in a neighbor-hood archive setting give confirmation this technique may give unprecedented levels of customer data security in a Cloud space.

Distributed computation follows the models of discrete structures in combinatorial forms. In higher-dimensions, the simplex structures of topological spaces as well as homology are employed to model and analyze distributed asynchronous computations. However, the monotone spaces are the general forms of topological spaces and can be effectively employed to analyze distributed computation. This paper proposes an analytical model of distributed computation in monotone spaces. It is illustrated that, the modeling of distributed computation in monotone spaces helps in determining consistent cuts under closure and convergence of computation. Furthermore, a connective mapping between the simplexes and monotone is constructed.

Distributed computing accomplished broad appropriation because of consequently parallelizing and transparently executing tasks in distributed environments. Straggling tasks is an essential test confronted by all Big Data Processing Frameworks for example MapReduce, Dryad, and Spark. Stragglers are the assignments that run much slower than different tasks and since a job completes just when it’s last undertaking completions, stragglers postponement work fruition. Stragglers extraordinarily impact little occupations such that employments comprising of a couple of undertakings. Such occupations are fundamentally deferred regardless of the fact that a solitary undertaking is moderate .This paper survey stragglers recognition and rescheduling systems proposed so far and brings up their strengths and shortcomings. This paper additionally displays wise attributes and impediments of the existing state- of-the- craftsmanship calculations to take care of the issue of stragglers relief.

Among a lot of rollback-recovery protocols developed for providing fault-tolerance for long-running distributed applications, sender-based message logging with checkpointing is one of the most lightweight fault-tolerance techniques to be capable of being applied in this field, significantly decreasing high failure-free overhead of synchronous logging by using message sender's volatile memory as storage for logging its corresponding message. However, attempting to apply this technique into large-scale and geographically distributed systems such as broker-based sensor networks, etc., the following problems should be addressed; reducing the number of control and data messages passing on core networks occurring during its fully message logging and recovery procedures. In this paper, we present a lightweight message logging protocol for distributed sensor networks to solve all of them by employing the current and future distributed systems' architectural features mentioned above.