Content-based image retrieval using a combination of visual features and eye tracking data

Zhen LIANG, Hong FU, Yun ZHANG, Zheru CHI, Dagan FENG

Research output: Chapter in Book/Report/Conference proceedingChapters

14 Citations (Scopus)

Abstract

Image retrieval technology has been developed for more than twenty years. However, the current image retrieval techniques cannot achieve a satisfactory recall and precision. To improve the effectiveness and efficiency of an image retrieval system, a novel content-based image retrieval method with a combination of image segmentation and eye tracking data is proposed in this paper. In the method, eye tracking data is collected by a non-intrusive table mounted eye tracker at a sampling rate of 120 Hz, and the corresponding fixation data is used to locate the human's Regions of Interest (hROIs) on the segmentation result from the JSEG algorithm. The hROIs are treated as important informative segments/objects and used in the image matching. In addition, the relative gaze duration of each hROI is used to weigh the similarity measure for image retrieval. The similarity measure proposed in this paper is based on a retrieval strategy emphasizing the most important regions. Experiments on 7346 Hemera color images annotated manually show that the retrieval results from our proposed approach compare favorably with conventional content-based image retrieval methods, especially when the important regions are difficult to be located based on visual features. Copyright © 2010 by the Association for Computing Machinery, Inc.
Original languageEnglish
Title of host publicationProceedings of ETRA 2010: ACM Symposium on Eye-Tracking Research & Applications
Place of PublicationNew York
PublisherAssociation for Computing Machinery
Pages41-44
ISBN (Print)9781605589947
DOIs
Publication statusPublished - 2010

Citation

Liang, Z., Fu, H., Zhang, Y., Chi, Z., & Feng, D. (2010). Content-based image retrieval using a combination of visual features and eye tracking data. In Proceedings of ETRA 2010: ACM Symposium on Eye-Tracking Research & Applications (pp. 41-44). New York: Association for Computing Machinery.

Keywords

  • Eye tracking
  • Content-based image retrieval (CBIR)
  • Visual perception
  • Similarity measure
  • Fixation

Fingerprint

Dive into the research topics of 'Content-based image retrieval using a combination of visual features and eye tracking data'. Together they form a unique fingerprint.