Normalizing web product attributes and discovering domain ontology with minimal effort

Tak Lam WONG, Lidong BING, Wai LAM

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

We have developed a framework aiming at normalizing product attributes from Web pages collected from different Web sites without the need of labeled training examples. It can deal with pages composed of different layout format and content in an unsupervised manner. As a result, it can handle a variety of different domains with minimal effort. Our model is based on a generative probabilistic graphical model incorporated with Hidden Markov Models (HMM) considering both attribute names and attribute values to extract and normalize text fragments from Web pages in a unified manner. Dirichlet Process is employed to handle the unlimited number of attributes in a domain. An unsupervised inference method is proposed to predict the unobservable variables. We have also developed a method to automatically construct a domain ontology using the normalized product attributes which are the output of the inference on the graphical model. We have conducted extensive experiments and compared with existing works using prouct Web pages collected from real-world Web sites in three different domains to demonstrate the effectiveness of our framework. Copyright © 2011 ACM.
Original languageEnglish
Title of host publicationProceedings of the 4th ACM International Conference on Web Search and Data Mining
Place of PublicationNew York
PublisherAssociation for Computing Machinery
Pages805-814
ISBN (Print)9781450304931
Publication statusPublished - 2011

Fingerprint

Ontology
Websites
Hidden Markov models
Experiments

Citation

Wong, T.-L., Bing, L. & Lam, W. (2011). Normalizing web product attributes and discovering domain ontology with minimal effort. Proceedings of the 4th ACM International Conference on Web Search and Data Mining (pp. 805-814). New York: Association for Computing Machinery.

Keywords

  • Information extraction
  • Graphical models
  • Web mining