by A. Helbling, C. Rozell, M. O'Shaughnessy and K. Fallah
Abstract:
Isolating and controlling specific features in the outputs of generative models in a user-friendly way is a difficult and open-ended problem. We develop techniques that allow a user to generate an image they are envisioning in their head by answering a sequence of relative queries of the form \textit"do you prefer image $a$ or image $b$?" Our framework consists of a Conditional VAE that uses the collected relative queries to partition the latent space into preference-relevant features and non-preference-relevant features. We then use the user's responses to relative queries to determine the preference-relevant features that correspond to their envisioned output image. Additionally, we develop techniques for modeling the uncertainty in images' predicted preference-relevant features, allowing our framework to generalize to scenarios in which the relative query training set contains noise.
Reference:
Oracle Guided Image Synthesis with Relative QueriesA. Helbling, C. Rozell, M. O'Shaughnessy and K. Fallah. April 2022.
Bibtex Entry:
@Conference{helbling.22,
title={Oracle Guided Image Synthesis with Relative Queries},
author={Helbling, A. and Rozell, C. and O'Shaughnessy, M. and Fallah, K.},
booktitle={ICLR Workshop on Deep Generative Models for Highly Structured Data},
year = {2022},
month = apr,
abstract = {Isolating and controlling specific features in the outputs of generative models in a user-friendly way is a difficult and open-ended problem. We develop techniques that allow a user to generate an image they are envisioning in their head by answering a sequence of relative queries of the form \textit{``do you prefer image $a$ or image $b$?''} Our framework consists of a Conditional VAE that uses the collected relative queries to partition the latent space into preference-relevant features and non-preference-relevant features. We then use the user's responses to relative queries to determine the preference-relevant features that correspond to their envisioned output image. Additionally, we develop techniques for modeling the uncertainty in images' predicted preference-relevant features, allowing our framework to generalize to scenarios in which the relative query training set contains noise.},
url = {https://openreview.net/forum?id=rNh4AhVdPW5}
}