Recognition of attentive objects with a concept association network for image annotation

Hong FU, Zheru CHI, Dagan FENG

Research output: Contribution to journalArticlespeer-review

11 Citations (Scopus)


With the advancement of imaging techniques and IT technologies, image retrieval has become a bottle neck. The key for efficient and effective image retrieval is by a text-based approach in which automatic image annotation is a critical task. As an important issue, the metadata of the annotation, i.e., the basic unit of an image to be labeled, has not been fully studied. A habitual way is to label the segments which are produced by a segmentation algorithm. However, after a segmentation process an object has often been broken into pieces, which not only produces noise for annotation but also increases the complexity of the model. We adopt an attention-driven image interpretation method to extract attentive objects from an over-segmented image and use the attentive objects for annotation. By such doing, the basic unit of annotation has been upgraded from segments to attentive objects. Visual classifiers are trained and a concept association network (CAN) is constructed for object recognition. A CAN consists of a number of concept nodes in which each node is a trained neural network (visual classifier) to recognize a single object. The nodes are connected through their correlation links forming a network. Given that an image contains several unknown attentive objects, all the nodes in CAN generate their own responses which propagate to other nodes through the network simultaneously. For a combination of nodes under investigation, these loopy propagations can be characterized by a linear system. The response of a combination of nodes can be obtained by solving the linear system. Therefore, the annotation problem is converted into finding out the node combination with the maximum response. Annotation experiments show a better accuracy of attentive objects over segments and that the concept association network improves annotation performance. Copyright © 2010 Elsevier Ltd. All rights reserved.
Original languageEnglish
Pages (from-to)3539-3547
JournalPattern Recognition
Issue number10
Early online date28 Apr 2010
Publication statusPublished - Oct 2010


Fu, H., Chi, Z., & Feng, D. (2010). Recognition of attentive objects with a concept association network for image annotation. Pattern Recognition, 43(10), 3539-3547. doi: 10.1016/j.patcog.2010.04.009


  • Image annotation
  • Concept association network (CAN)
  • Attentive objects
  • Visual classifier
  • Neural network


Dive into the research topics of 'Recognition of attentive objects with a concept association network for image annotation'. Together they form a unique fingerprint.