Earticle

초록

영어

As non-formal language, texts containing dirty words are widespread in Web reviews. Due to their bad effects on users of communication, it is essential to perform automatic analysis on Chinese texts containing dirty word. In this paper, we first crawled over millions of evaluating sentences which contain a lot of dirty words from the Web. Second, we manually annotated 40 typical dirty words with weights. And then proposed a machine learning-based approach for collecting dirty word texts corpus. Overall, more than 6000 sentences were collected from the huge amount of Web reviews to form a corpus on Chinese texts containing dirty words. With the corpus, we present SVM (Support Vector Machine) and ME (Maximum Entropy) classifiers to automatic detect Chinese texts containing dirty words. Empirical studies demonstrate that the SVM and ME classifiers are both effective for this task and the recall and precision are both over 97%.

목차

Abstract
 1. Introduction
 2. Related Work
 3. Constructing the Corpus on Dirty Word Texts
  3.1. Dirty Word Texts
  3.2. Collecting Review Texts
  3.3. Solution
  3.4. A High-Precision Classifier
  3.5. Preprocessing
  3.6. Features Recommendation
  3.7. Corpus Constructing
 4. Automatic Detecting Chinese Dirty Word Texts
  4.1. SVM and ME Classifiers
  4.2. Feature Selection
  4.3. Experiments
 5 Conclusion
 References

키워드

저자

• Xiaoxu Zhu [ School of Computer Science & Technology, Soochow University, Suzhou 215006, China ]
• Peide Qian [ School of Computer Science & Technology, Soochow University, Suzhou 215006, China ]

참고문헌

발행기관

간행물

표지보기 관심저널 등록

• 간행물명

  International Journal of Hybrid Information Technology

• 간기

  격월간

• pISSN

  1738-9968

• 수록기간

  2008~2016

• 십진분류

  KDC 505 DDC 605