Earticle

초록

영어

Duplicate detection is the special case of data matching that discovers groups of records within a single database that belong to same real world entity. It is also an inevitable part of data cleansing because duplicate records can strongly influence the results of later data mining or processing. In this process, one record is compared to all other records. Different data representations, formats, terminologies and data entry errors make this task complex. Involvement of heavy volume databases adds more complexity. To reduce comparison of records, indexing algorithms are used that partition the data and perform comparisons with in that partition. Sorted Neighborhood Method (SNM) is a standard indexing algorithm that sorts dataset by using defined “sorting key” and moves fixed size window to compare records within that window. Duplicate Count Strategy-Multi record increase (DCS++) is latest improvement in SNM that adapts the window size for every duplicate in current window. We propose Innovative Windows (Inn Win) algorithm that assumes i) detected duplicate in sorted dataset raises the probability of finding more duplicates in neighborhood ii) Series of consecutive non-duplicates drops the probability of duplicates in neighborhood. Using this concept, it adapts window both for duplicates and non-duplicates and avoids unnecessary comparisons without losing effectiveness. We prove that Inn Win is a better alternative in windowing algorithms.

목차

Abstract
 1. Introduction
 2. Problem and Proposed Idea
 3. Innovative Windows
 4. Experimental Evaluation
  4.1. Evaluation Metrics
  4.2. Algorithm Compared
  4.3. Data Set
  4.4. Similarity Function
  4.5. Tool Kit
  4.6. Results and Discussion
 5. Related Work
 6. Conclusion and Future Work
 References

키워드

저자

• Humaira Bano [ College of Electrical and Mechanical Engineering National University of Sciences and Technology, H-12, Islamabad, Pakistan ]
• Farooque Azam [ College of Electrical and Mechanical Engineering National University of Sciences and Technology, H-12, Islamabad, Pakistan ]

참고문헌

발행기관

간행물

표지보기 관심저널 등록

• 간행물명

  International Journal of Software Engineering and Its Applications

• 간기

  월간

• pISSN

  1738-9984

• 수록기간

  2008~2016

• 등재여부

  SCOPUS

• 십진분류

  KDC 505 DDC 605