Eye movement as an interaction mechanism for relevance feedback in a content-based image retrieval system

Yun ZHANG, Hong FU, Zhen LIANG, Zheru CHI, Dagan FENG

Research output: Chapter in Book/Report/Conference proceedingChapters

21 Citations (Scopus)

Abstract

Relevance feedback (RF) mechanisms are widely adopted in Content-Based Image Retrieval (CBIR) systems to improve image retrieval performance. However, there exist some intrinsic problems: (1) the semantic gap between high-level concepts and low-level features and (2) the subjectivity of human perception of visual contents. The primary focus of this paper is to evaluate the possibility of inferring the relevance of images based on eye movement data. In total, 882 images from 101 categories are viewed by 10 subjects to test the usefulness of implicit RF, where the relevance of each image is known beforehand. A set of measures based on fixations are thoroughly evaluated which include fixation duration, fixation count, and the number of revisits. Finally, the paper proposes a decision tree to predict the user's input during the image searching tasks. The prediction precision of the decision tree is over 87%, which spreads light on a promising integration of natural eye movement into CBIR systems in the future. Copyright © 2010 by the Association for Computing Machinery, Inc.
Original languageEnglish
Title of host publicationProceedings of ETRA 2010: ACM Symposium on Eye-Tracking Research & Applications
Place of PublicationNew York
PublisherAssociation for Computing Machinery
Pages37-40
ISBN (Print)9781605589947
DOIs
Publication statusPublished - 2010

Citation

Zhang, Y., Fu, H., Liang, Z., Chi, Z., & Feng, D. (2010). Eye movement as an interaction mechanism for relevance feedback in a content-based image retrieval system. In Proceedings of ETRA 2010: ACM Symposium on Eye-Tracking Research & Applications (pp. 37-40). New York: Association for Computing Machinery.

Keywords

  • Eye tracking
  • Relevance feedback (RF)
  • Content-based image retrieval (CBIR)
  • Visual perception

Fingerprint Dive into the research topics of 'Eye movement as an interaction mechanism for relevance feedback in a content-based image retrieval system'. Together they form a unique fingerprint.