Earticle

테이블에 의한 탐색 (Query-By-Table)은 사용자 질의로 입력되는 특정 테이블과 비교, 식별 대상이 되는 테이블 데이터를 검색하여 연관도에 따라 내림차순으로 정렬된 테이블 데이터 리스트를 반환하는 서비스이다. 이를 위해 방대한 테이블 데이터에 내재된 컬럼명, 컬럼 값, 메타데이터 등의 유용한 정보를 활용하여 각 테이블 데이터를 하나의 의미적 벡터로 변환하는 임베딩 (embedding) 기술을 활용할 수 있다. 본 연구는 사용자 질의로서 자연어 또는 키워드 수준이 아닌, 테이블 데이터 자체를 사용하여 그 질의와 연관된 순서대로 융합 가능한 테이블 데이터들을 탐색하는 기술을 제안한다. 또한, 질의 테이블과 비교되는 검색 대상 테이블 내부의 컬럼 간 상대적 중요도를 고려한 가중치를 적용하여 탐색 결과의 성능을 높이고자 한다. 사용자는 탐색된 테이블 데이터들에 대한 조인 융합을 수행하여 원천 테이블 데이터에서는 존재하지 않았던 유의미한 새로운 정보를 추출할 수 있다.

The development of the Web has made it easy to access large amounts of data. However, it is one of the big issues how to effectively store and improve query performance about these huge data. This paper proposes an RDF query optimization with hybrid storage structures to solve the issue. Hot-cold detection relocates data properly to improve the RDF query performance. After collecting data access information for a period of time, we identify the data with hot-cold detection. It reduces the write performance degradation of the SSD by moving all hot data at a time to the SSD.

아파치 스파크(Apache Spark)는 오픈 소스, 분산, 인메모리 컴퓨팅 프레임워크 및 아키텍처이다. 아파치 스파크를 통해 데이터 병렬 및 내결함성을 갖춘 분산 시스템 클러스터에서 실행되기 때문에 농업, 정보통신 산업 등 다양한 분야에 적용 및 활용할 수 있다. 한편, 빅데이터 시대를 맞아 많은 시공간 데이터가 발생하고 있다. 그러나, 아파치 스파크를 통한 분산 컴퓨팅 환경에서 많은 연산이 필요한 조인과 같은 연산을 제공하고 있지 않기 때문에 효율적으로 처리하지 못한다. 따라서, 본 논문에서는 대용량 시공간 데이터를 이용하여 그리드 분할 기법에 근거한 withindistance, contain 조인 질의처리 알고리즘을 제안한다. 성능평가 결과, 제안한 알고리즘이 기존 알고리즘보다 약 20%의 우수한 성능을 보인다.

Apache Spark is an open source, distributed, in-memory computing framework and architecture. Because it runs on a distributed system cluster with data parallelism and fault tolerance through Apache Spark, it can be applied and utilized in various fields such as agriculture, information and communication industries. Meanwhile, in the era of big data, large amount of spatio-temporal data is being generated. However, Apache Spark cannot efficiently process join operation because it does not support join operation that requires many computations in a distributed computing environment. Therefore, in this paper, we proposed an join query processing algorithm, i.e., withindistance and contain join, based on grid partitioning technique using large-scale spatio-temporal data. As a result of performance evaluation, our algorithm shows 20% better performance than the existing algorithm in terms of query processing time.

최근 페이스북, 트위터 등의 SNS(Social Networking Service)가 발전함에 따라, 사용자가 생성하는 데이터가 급격히 증가하고 있다. 사용자 데이터는 민감한 개인정보를 포함하기 때문에, 원본 데이터를 공격자로부터 보호하기 위해서는 데이터를 암호화하는 것이 필요하다. 따라서 암호화된 데이터의 복호화 없이 질의를 처리하는 암호화 질의 처리 기법이 제안되었다. 그러나 기존의 질의처리 기법은 암호화 데이터에 대한 색인 구조를 구축하고 이를 순차적 으로 탐색하기 때문에, 데이터의 크기가 증가함에 따라 질의탐색 비용이 증가하는 문제점이 존재한다. 이를 위해, P.B.Volk, et al.은 prefix 트리 기반 병렬 질의처리 알고리즘을 제안하였다. 제안하는 알고리즘은 암호화된 데이 터를 위해 prefix 트리 구조를 구축하고, 트리를 부분 트리로 분할하여 생성된 모든 부분 트리를 병렬적으로 탐색한 다. 그러나 이 알고리즘은 모든 부분 트리를 탐색하기 때문에, 트리 깊이에 따라 연산 비용이 급격히 증가하는 문제 점이 존재한다. 아울러, 이 알고리즘은 범위 질의나 부분 매칭 등의 다양한 질의를 지원하지 못하는 문제점이 존재 한다. 이러한 문제를 해결하기 위해, 본 논문에서는 prefix 트리 및 해시 테이블을 사용하는 GPU 기반 병렬 질의처 리 알고리즘을 제안한다. 제안하는 알고리즘은 prefix 트리 loop-up 테이블을 사용하여 범위 질의 및 부분매칭 질 의를 지원한다. 아울러 제안하는 알고리즘이 기존 P.B.Volk, et al. 의 알고리즘보다 검색 시간 측면에서 약 30% 우수한 성능을 나타냄을 보인다.

Recently, social networking services, such as Facebook and Twitter, have been widely used, so the amount of the data created by users has been dramatically increased. Because the user-created data can contain privacy information, it is required to encrypt the data for protecting the original data from adversaries. Thus, an encrypted query processing scheme has been proposed to process the query without the decryption of the encrypted data. The existing schemes construct an index for the encrypted data, so they can process the query by sequentially accessing the index. As a result, the query processing cost increases as the amount of the data is increased. For this, P.B.Volk, et al. proposed a prefix-tree based parallel query processing algorithm. The algorithm constructs a prefix-tree structure for the encrypted data and searches all sub-trees on GPU in parallel by dividing the tree into sub-trees. However, the algorithm has a problem that its computational cost is highly increased according to the depth of the tree because it searches all sub-trees. In addition, the algorithm does not support the various types of queries, such as a range query and a partial matching query. To solve these problems, we, in this paper, propose a GPU-based parallel query processing algorithm using both a prefix-tree and a hash table. By using the prefix-tree look-up table, the proposed algorithm can support both a range query and a partial matching query. In addition, we show that the proposed algorithm is about 30% better on retrieval performance than the existing algorithm by P.B.Volk, et al.

OLAP(On-Line Analytical Processing)은 데이터 큐브 또는 큐브라고 불리는 다차원 데이터 구조를 이용하여 복잡한 질의를 고속으로 처리하는 데이터 분석 기술이다. 전통적 방식의 OLAP은 디스크 기반 DBMS 환경으로 데 이터를 선 저장한 후 사용자의 질의에 응답하는 일회성 질의(One-Time Query) 수행 방식이었다. 하지만 지속적 으로 방대한 양의 데이터가 생성되는 데이터 스트림 환경에서 기존 처리 방식은 질의를 반복적으로 수행해야 하기 때문에 우수한 성능을 기대하기 어려우며, 동시적으로 다차원 계층 데이터에 질의를 수행하는데 한계가 존재한다. 본 연구에서는 이러한 문제점을 극복하기 위해 연속질의 기반 다차원 계층 큐브 처리 기법을 제안한다. 본 연구 모 델은 계층 데이터를 처리하는 하이퍼 데이터 큐브를 구축한다. 각 큐보이드들은 이전 집계된 데이터 큐보이드 중 가 장 작은 계산 비용 큐보이드를 집계하는 최소 비용 트리를 형성하여 성능적 향상을 기대한다. 본 연구 모델의 성능 을 검증하기 위해서 다양한 실험을 진행하였다.

OLAP(On-Line Analytical Processing) is one of the data analysis techniques that processes a complex query in a fast time using multi-dimensional data structure called the‘data cube’ or simply‘cube’. However, conventional OLAP system is not applicable to data streams because of these reasons: low performance; limitation of execution of a number of queries simultaneously. This paper proposes continuous query based evaluation of multi-dimensional hierarchical data cube. Minimal cost cube tree that computing a cuboid from the smallest cost, previously computed cuboid is constructed. Finally, the proposed method is verified by a series of experiments.

대용량 민감 데이터에 대한 아웃소싱이 각광받음에 따라, 이를 보호하기 위한 데이터 암호화 기법이 요구되고 있다. 이에 따라, 대용량 암호화 데이터 관리를 지원하는 분산 색인 구조 및 암호화된 데이터 상에서의 질의처리 알고리즘 이 요구되고 있다. 그러나 기존 분산 색인 구조 중 암호화 데이터의 특성을 고려한 연구는 존재하지 않는다. 또한, 기존 암호화 질의처리 알고리즘은 지원 가능한 질의 타입이 한정적이며, 상이한 방식으로 암호화된 컬럼 간 연산을 지원하지 못하는 문제점이 존재한다. 이를 해결하기 위해, 본 논문에서는 비트맵 기반 분산 색인 구조 및 암호화 질 의처리 기법을 제안한다. 제안하는 분산 암호화 색인 구조는 데이터 프라이버시를 보장하며, 다양한 종류의 질의에 대해 성능 향상을 제공한다. 아울러, 제안하는 암호화 질의처리 기법은 복호화를 수행하지 않고 질의처리를 수행함 으로써 데이터 보호 수준을 향상시키며, 높은 질의 처리 성능 및 정확도를 보장한다. 아울러 성능평가를 통해 제안 하는 색인구조 및 암호화 질의처리 기법이 대용량 민감 데이터 보호에 적합함을 보인다.

As the outsourcing of the large sensitive data has been highlighted, data encryption schemes to protect the sensitive data are required. Accordingly, it is necessary to develop not only a distributed index structure to manage the large amount of encrypted data, but also a query processing scheme over the encrypted data. However, there has been no index structure considering the encrypted data. Existing query processing schemes over the encrypted data can support limited types of queries. In addition, the schemes cannot support operations among data with different columns because they use different types of encryption schemes depending on their attribute type. To solve these problems, in this paper, we propose a bitmap-based distributed index structure and a query processing scheme for the encrypted data. The proposed distributed index structure guarantees data privacy preservation and performance improvement for the various types of queries. In addition, by processing a query over the encrypted data without data decryption, the proposed query processing scheme guarantees the high query performance and accuracy while preserving the data privacy. Finally, we show from our performance evaluation that our proposed index structure and query processing scheme are suitable for protecting the data privacy of the large sensitive data.

현업에서 사용하고 있는 정형 데이터는 데이터모델링에 대한 이해부족 및 적용의 미흡으로 정규화되지 않은 채로 테이블 형태로 관리되고 있는 현실이다. 데이터베이스 설계의 균형이 파괴되면 데이터 질의에 대한 응답속도에 영향을 미치며, 데이터 비만도가 높아지게 된다. 본 논문에서는 최적화된 데이터모델링을 통한 데이터베이스 설계를 통 하여 데이터 비만도가 어떻게 개선되었는지를 연구하였다. 데이터 비만도가 과다하게 나타나는 방사형 및 업무 중심의 고립형 설계에서 객체(데이터)와 객체간의 관계 중심의 데이터모델링을 통한 정방형 설계를 함으로 데이터 질의 경로 가 선명하게 가시화되었다. 데이터비만도 면에서도 기존의 연구지원 데이터베이스의 비만도는 57.2%였으나, 새로운 연 구지원 데이터베이스에서는 16.2%로 나타나 데이터 비만도가 40.5%가 개선되었으며, 데이터의 중복을 최소화함으로써 데이터의 정확성과 무결성이 보장되는 데이터베이스로 개선되었다.

The formal data used in the business is managed in a table form without normalization due to lack of understanding and application of data modeling. If the balance of the database design is destroyed, it affects the speed of response to the data query, and the data obesity becomes high. In this paper, it is investigated how data obesity improved through database design through optimized data modeling. The data query path was clearly visualized by square design through data modeling based on the relationship between object (data) and object, from the radial and task - oriented isolation design where data obesity is excessive. In terms of data obesity, the obesity degree of the current research support database was 57.2%, but it was 16.2% in the new research support database, and the data obesity degree was reducd by 40.5%. In addition, by minimizing redundancy of data, the database has been improved to ensure the accuracy and integrity of the data.

In recent years, there has been widespread concern about the problems of stream data query both academic and industrial communities. The problems obtained some results. At the same time, big data stream brings great benefits for information society. Information query about stream data form has also brought crucial challenges. However, it is seldom about the research of big data stream query in network space. This paper analyzes the characteristics of stream data query in massive data, discusses the challenges and research issues of data stream for big data query. Finally the works for the data stream query are surveyed.

With the development of science and technology as well as the advancement of national strategic deployment, study of hybrid heterogeneous system based on big data query has become a hot topic in internet industry both at home and abroad. Information technology has been widely applied in various fields and information has experienced explosive growth. However, different storage environments, collection systems and implementation platforms of information have hindered the communication and sharing of data between platforms and contributed a lot to deficient utilization of data. Thus the concept of heterogeneity comes into being. Heterogeneity in information system refers to difficulties in data utilization due to various data formats. This paper attempts to discuss the heterogeneous data integration methods based on big data and make an analysis. It explores advantages and convenience of design scheme on the basis of LDAP and offers detailed extracting rules for a better visual understanding of the corresponding model on its application.

As the increasing usage of data generating devices like cameras, mobile phones, Auto-ID technologies, and so on, huge amount of data are created. Data compression is very important. In order to use the suitable compression methods on reducing the data volume, this paper uses iceberg-cubes to compress the data based on the entropy query mechanism. Using the definition of iceberg-cubes, this paper uses the entropy query principle for getting the key value from the original datasets. The iceberg-cubes are then used for generating the compressed data which should be stored in the hardware devices. It is observed that these proposed algorithms could achieve 32.46% compression ratio averagely.

There has been a growing practical need for querying XML streaming data efficiently. Stream requires to be read sequentially and only once into memory, the query must be processed on the fly. QXSList technique is proposed for massive data processing, which takes the SAX events sequence as input, buffer the incoming elements for further processing, remove unnecessary elements from the buffer in time, and give the results on the fly. Data model and algorithm integrated framework are defined, the integrate methods of how to process predicate and wildcard are discussed respectively. Level value is used for determining the relationship of two elements and relational pointers are constructed for linking multi lists in this method. The experimental results show that our approach is effective and efficient on this problem, and outperforms the state-of-the-art algorithms and query engines especially for data size is very large. At the same time, memory usage is nearly constant.

Traditional data management is usually based on relational databases, which are capable of managing small amounts of data. But relational databases have some difficulty in inquiry, management, and analysis of large amounts of data and magnanimity data. The method of effective management of magnanimity data is a problem deserving of study. In this paper, traditional relational databases are moved to Hadoop, in order to implement query and analysis on Hadoop. This paper changes the amount of data record and number of nodes in clusters, and records the query time in different conditions. Advantages and disadvantages of query on Hadoop can be analyzed by comparing the statistics with the query time on relational database Oracle. The factors affecting the time of query on Hadoop can be found by analysis. Furthermore, the result is also a reference material of future research and data managements on cloud platforms.

Cloud computing provides virtualization of services, has enabled opportunities for integration and collaboration of many e-businesses around world. Also, semantic web technology has been evolving in parallel, which uses RDF and OWL as storage formats for business data. Integration of semantic web technology on cloud platform provides an efficient way of managing data for e-commerce based applications. Many databases storage approaches have been proposed for e-commerce applications. However, there is a need to model the data storage based on semantic information so that the access to data store based on semantic behavior of user context and past history. The proposed work models a mobile phone e-commerce application which uses RDF data store which is maintained in cloud environment. SPARQL queries are used for access and retrieval mechanism.

Wireless Sensor & Actuator networks take actuation decisions based on the data collected by the deployed set of sensor nodes. The method of data acquisition, leading to decision making, could be semi-automated or fully automated. In either case, the reliable delivery of information assumes critical importance since it has a direct impact on the decision making process for subsequent action by the actuator network. This paper presents in details a novel methodology “Layer Based Time Constrained Reliable Data Acquisition Mechanism” LTCRDM, which can be utilized for reliable delivery of information, sensed periodically or in response to a query, by the sensors deployed over a geographical area to a centralized sinkwhere the decision for eventual actuation is taken. Since the latency and reliability requirements in a WSAN are stringent, the mechanism detailed attempts delivery of maximum packets with minimum latency to ensure that estimation of the sensed event is accurate leading to correct decision making. The methodology ensures relatively low packet loss as compared to standard packet delivery mechanisms with latency time constraints. The algorithm for dissemination of query (LQDM) in the deployed nodes is also presented. Authors have provided detailed algorithm, results of simulation and observations using IEEE 802.15.4 PHY & MAC as underlying layers. Experimental results over a test-bed are also presented. A critical analysis of the results is presented for comparison against the standard methodologies in vogue.

This paper introduces three core modules of query reformulation engine, mapping document, query reformulation module, and statement conversion module. Mapping document is an XML document that keeps the mapping information between local data source and related data sources; using mapping document, applications could find data sources that have mapping relationship with its local data source. The query reformulation module reformulates the query statements submitted by users to local data resource to query statements to all data sources that have mapping relationship with local data resource. The statement conversion module converts XPath statements to OQL statements that are supported by OGSA-DQP; through OGSA-EDAI in the bottom layer, the access result to the data sources could be obtained. When a user submits an XPath statement to OGSA-DQP, it calls the query reformulation module, which first checks the mapping document to find information of other data sources, then expands and reformulates this XPath statement into query statements that are suitable for the mapped data sources. Afterward, the statement conversion module converts the reformulated XPath statements into OQL statements and returns to OGSA-DQP, which then performs the query operation.

Wireless Sensor & Actuator networks take actuation decisions based on the data collected by the deployed set of sensor nodes. The method of data acquisition, leading to decision making, could be semi-automated or fully automated. In either case, the reliable delivery of information assumes critical importance since it has a direct impact on the decision making process for subsequent action by the actuator network. This paper presents in details a novel methodology “Layer Based Time Constrained Reliable Data Acquisition Mechanism” LTCRDM, which can be utilized for reliable delivery of information, sensed periodically or in response to a query, by the sensors deployed over a geographical area to a centralized sink where the decision for eventual actuation is taken. Since the latency and reliability requirements in a WSAN are stringent, the mechanism detailed attempts delivery of maximum packets with minimum latency to ensure that estimation of the sensed event is accurate leading to correct decision making. The methodology ensures relatively low packet loss as compared to standard packet delivery mechanisms with latency time constraints. The algorithm for dissemination of query (LQDM) in the deployed nodes is also presented. Authors have provided detailed algorithm, results of simulation and observations using IEEE 802.15.4 PHY & MAC as underlying layers. Experimental results over a test-bed are also presented. A critical analysis of the results is presented for comparison against the standard methodologies in vogue.

Recently, we have witnessed the emergence of new data stream management systems that can scale up to a large number of machines for the real-time processing of big data. These systems typically provide a procedural programming interface with their own APIs. However, to enable the rapid development of applications, it is desirable to support a declarative interface with clear processing semantics, such as window SQL. This paper examines the programming interfaces of the state-of-the-art data stream management systems and discusses the necessity of SQL support to help users write stream queries easily and in an integrated manner.

Microblog platform, such as Twitter and Sina, has been one of the major ways of information diffusion in modern society. However, although microblog has been proven to contain lots of information, it is really hard for people to find useful information on it. Besides, some information in microblog such as user IDs is not allowed to publish due to privacy policies. Thus the queries on microblog data can be regarded as privacy-preserved queries. One of the challenged issues is the poor performance of answering privacy-preserved queries over microblog data, which owes to the large and increasing volume and the complex social network structure of microblog data. In this paper, we propose a basic idea to optimize the privacy-preserved queries on microblog data. We use a query-specific approach to treat the queries, i.e., the microblog data is first preprocessed according to the specific requirements of different types of queries, which are then organized through some indexing structures. Our preliminary experiments on real microblog data show that this approach has reasonable performance.

The Nearest Surrounder (NS) query is to find a set of all the visible objects that are not obstructed by other objects around the query location. Formally, for a given query Q and dataset D, NS(Q) returns a set of tuples, ni : <Oi,Θi→,Θi←> if and only if (i) Oi □ R and Oi is visible to Q, and (ii) □Oj□R□{Oi} , dist (Q,Oi)□ dist (Q,Oj) at angular range [Θi→,Θi←), where i≠ j. Using the R-tree index, NS query processing algorithm is based on the not only distance bounding properties but also angle-based bounding properties. Although NS query has a wide spectrum of applications, such as surveillance and augmented reality services, the existing work is not able to effectively support the 3D geospatial environments. Motivated by the weaknesses, we suggests a solution, termed 3D Nearest Surrounder (3DNS) query, to maintain NS query result from a dataset of 3D geospatial objects. In this paper, we propose heuristics for 3DNS query based on R-tree spatial index without pre-computing the visible region.

In order to realize the idea of pay-as-you-go (PAYG) data management which dataspace emphasizes, the very loosely structured data model is usually used to describe the massive, heterogeneous and dynamic data in dataspace. However, the present study mainly concentrates on the applications of the data model, and the query theory research is less. The query algebra is a theoretical foundation for query and its optimization in a PAYG fashion, so that how to establish a complete algebra based on the characteristics of loosely structured data model is an important problem need to be solved. In this paper, a formal definition of very loosely structured data model is given, then the query model and query algebra based on the model are proposed, which support to not only the operations such as set operators, selection, projection and join, but also association query in dataspace.