Data augmentation via photo-to-sketch translation for sketch-based image retrieval

Takahiko Furuya; Ryutarou Ohbuchi

doi:10.1117/12.2524230

6 May 2019 Data augmentation via photo-to-sketch translation for sketch-based image retrieval

Takahiko Furuya, Ryutarou Ohbuchi

Proceedings Volume 11069, Tenth International Conference on Graphics and Image Processing (ICGIP 2018); 1106925 (2019) https://doi.org/10.1117/12.2524230
Event: Tenth International Conference on Graphic and Image Processing (ICGIP 2018), 2018, Chengdu, China

Abstract

Sketch-based image retrieval (SBIR) technique has progressed by deep learning to learn cross-modal distance metrics that relate sketches and photos from a large number of sketch-photo pairs. However, datasets of sketch-photo pairs are small, as acquisition of a large number of such pairs is expensive. To alleviate the issue, data augmentation via image transformation such as scaling, flipping, rotation, and deformation has been widely adopted. Still, insufficiency in training set seems to have impeded deep learning from achieving its full potential for SBIR. In this paper, we propose a novel data augmentation approach dedicated for SBIR. A deep neural network called Photo2Sketch (P2S) converts photos into line drawings that are visually similar to those sketched by human. An artificially augmented training dataset of sketch-photo pairs is generated at low cost by feeding photos from a large image corpus into the P2S. Experiments evaluate quality of sketch-like images generated by the P2S as well as efficacy of the proposed data augmentation algorithm under SBIR scenario. In particular, retrieval accuracy is significantly improved when the proposed algorithm is combined with the data augmentation by image transformation

Citation Download Citation

Takahiko Furuya and Ryutarou Ohbuchi "Data augmentation via photo-to-sketch translation for sketch-based image retrieval", Proc. SPIE 11069, Tenth International Conference on Graphics and Image Processing (ICGIP 2018), 1106925 (6 May 2019); https://doi.org/10.1117/12.2524230

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
10 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Image retrieval

Edge detection

Feature extraction

Neural networks

Information technology

Data modeling

Computer vision technology

Show All Keywords

Keywords/Phrases

Search In:

Publication Years