Learning with privileged information for photo aesthetic assessment

Yangyang SHU, Qian LI, Shaowu LIU, Guandong XU

Research output: Contribution to journalArticlespeer-review

18 Citations (Scopus)

Abstract

Privileged information (PI), known as teacher providing students helpful comments, comparisons, and explanations to improve students’ performance, has been widely applied in various machine learning tasks, resulting in great success. Existing approaches utilizing attributes either fail to leveraging the attributes information thoroughly, or suffer from the complex network structures for automatically attributes learning. Therefore, we propose a new Deep Convolutional Neural Network with Privileged Information (PI-DCNN) for photo aesthetic assessment by utilizing the prior knowledge of photo and photographic elements as privileged information. This paper is the first to systematically summarize all the attributes (i.e., photo and photographic attributes) related to aesthetics assessment. Specifically, we first explore the privileged information of photo and photography attributes, which is available at the training stage but it is not available for the test set. After that, we transfer the probabilistic dependency relations as constraints, and formulate photo aesthetics assessment in a deep convolutional neural network. Lastly, we propose a new pair-wise ranking loss that can exploit the relationship of photo aesthetic quality within a pair of photos. Experimental results on two widely benchmark databases of aesthetic assessment, AADB and AVA, demonstrate the effectiveness of the proposed PI-DCNN method on photo aesthetic assessment task. Copyright © 2020 Elsevier B.V. All rights reserved.

Original languageEnglish
Pages (from-to)304-316
JournalNeurocomputing
Volume404
Early online dateMay 2020
DOIs
Publication statusPublished - Sept 2020

Citation

Shu, Y., Li, Q., Liu, S., & Xu, G. (2020). Learning with privileged information for photo aesthetic assessment. Neurocomputing, 404, 304-316. https://doi.org/10.1016/j.neucom.2020.04.142

Fingerprint

Dive into the research topics of 'Learning with privileged information for photo aesthetic assessment'. Together they form a unique fingerprint.