Privileged multi-task learning for attribute-aware aesthetic assessment

Yangyang SHU, Qian LI, Lingqiao LIU, Guandong XU

Research output: Contribution to journalArticlespeer-review

4 Citations (Scopus)

Abstract

Aesthetic attributes are crucial for aesthetics because they explicitly present some photo quality cues that a human expert might use to evaluate a photo's aesthetic quality. However, the aesthetic attributes have not been largely and sufficiently exploited for photo aesthetic assessment. In this paper, we propose a novel approach to photo aesthetic assessment with the help of aesthetic attributes. The aesthetic attributes are used as privileged information (PI), which is often available during training phase but unavailable in prediction phase due to the high collection expense. The proposed framework consists of a deep multi-task network as generator and a fully connected network as discriminator. Deep multi-task network learns the aesthetic attributes and score simultaneously to capture their dependencies and extract better feature representations. Specifically, we use ranking constraint in the label space, similarity constraint and prior probabilities loss in the privileged information space to make the output of multi-task network converge to that of ground truth. Adversarial loss is used to identify and distinguish the predicted privileged information of a deep multi-task network from the ground truth PI distribution. Experimental results on two benchmark databases demonstrate the superiority of the proposed method to state-of-the-art. Copyright © 2022 Elsevier Ltd. All rights reserved.

Original languageEnglish
Article number108921
JournalPattern Recognition
Volume132
Early online dateJul 2022
DOIs
Publication statusPublished - Dec 2022

Citation

Shu, Y., Li, Q., Liu, L., & Xu, G. (2022). Privileged multi-task learning for attribute-aware aesthetic assessment. Pattern Recognition, 132, Article 108921. https://doi.org/10.1016/j.patcog.2022.108921

Fingerprint

Dive into the research topics of 'Privileged multi-task learning for attribute-aware aesthetic assessment'. Together they form a unique fingerprint.