Earticle

초록

영어

Frequent Itemset Mining (FIM) is one of most fundamental techniques in data mining with extensive applications to a variety of data mining problems such as association rule mining, correlations, clustering and classification. Since the first proposal of frequent itemset mining, numerous serial algorithms have been proposed in order to improve mining performance, yet most of them cannot scale to massive datasets which are very common nowadays. In this paper, we propose a new parallel FIM algorithm named PFIN based on Nodeset which is a more efficient data structure for mining frequent itemsets. PFIN can intelligently decompose a large-scale FIM problem into a set of tasks, where each task can be executed in parallel without unnecessary communication overheads. Moreover, a hash-based load balancing strategy has been adopted to optimize resource use and maximize throughput. For evaluating the performance of PFIN, we have conduct extensive experiments on Spark which is an emerging distributed in-memory processing framework to compare it against PFP which is one of state-of-the-art parallel FIM algorithms on a range of real datasets. The experimental results demonstrate that our proposed PFIN are highly competitive with PFP in scalability performance, outperforming PFP in speed performance.

목차

Abstract
 1. Introduction
 2. Background
  2.1. Frequent Itemset Mining
  2.2. Apache Spark Framework
 3. Related Work
 4. PFIN: The Proposed Method
  4.1. FIN Algorithm
  4.2. PFIN Outline
  4.3. Parallel Counting
  4.4. Grouping Items with Load Balancing
  4.5. Generating Conditional Transactions
  4.6. Constructing Local POC-trees and Generating the Nodesets of 2-itemset
  4.7. Constructing Local Set-enumeration Trees and Mining Frequent Itemsets
 5. Experiments
  5.1. Experiment Setup
  5.2. Speed Performance Analysis
  5.3. Scalability Performance Analysis
 6. Conclusion
 Reference

키워드

저자

• Chen Lin [ Department of Computer Science and Technology East China Normal University ]
• Junzhong Gu [ Department of Computer Science and Technology East China Normal University ]

참고문헌

발행기관

간행물

표지보기 관심저널 등록

• 간행물명

  International Journal of Database Theory and Application

• 간기

  격월간

• pISSN

  2005-4270

• 수록기간

  2008~2016

• 십진분류

  KDC 505 DDC 605