Earticle

초록

영어

With the rapid development of network and information technology, there is a wealth of huge amounts of data on the internet. But it’s a major problem faced by the majority of researchers how to effectively filter out a particular subject or field of information from these data. In this paper, we try to builder a focused crawler based on vector space model and TF-IDF text correlation analysis. We take the seed URL as a collection entrance and fetch web pages from internet. Then analysis page information though technological like web content extraction, page link analysis technology and get the main content of one page. By the correlation analysis method based on VSM and TF-IDF text, we calculation the correlation between pages and the topics what have been defined, so we can achieve the purpose of the focus areas of the web.

목차

Abstract
 1. Introduction
 2. The Process of Focused Crawler
 3. Key Technologies
  3.1. Web Page Pretreatment
  3.2. Web Page Analysis
  3.3. The TF - IDF Text Relevancy Analysis based on the VSM
 4. Experiment
  4.1. Evaluation Index
  4.2. Experimental Process
  4.3. Results Analysis
 5. Conclusion
 References

키워드

저자

• Qiuli Qin [ Logistic Technology and Management Lab, School of Economics and Management，Beijing Jiaotong University, Beijing 100044 ]
• Xin Peng [ Logistic Technology and Management Lab, School of Economics and Management，Beijing Jiaotong University, Beijing 100044 ]

참고문헌

발행기관

간행물

표지보기 관심저널 등록

• 간행물명

  International Journal of Future Generation Communication and Networking

• 간기

  격월간

• pISSN

  2233-7857

• 수록기간

  2008~2016

• 십진분류

  KDC 505 DDC 605