Texture synthesis and transfer using convolutional neural networks
Leon Gatys, doctoral candidate at Bethge Lab, Tübingen, Germany
Abstract: We introduce a new model of natural textures based on the feature spaces of convolutional neural networks optimised for object recognition. Samples from the model are of high perceptual quality demonstrating the generative power of neural networks trained in a purely discriminative fashion. Within the model, textures are represented by the correlations between feature maps in several layers of the network. We show that across layers the texture representations increasingly capture the statistical properties of natural images while making object information more and more explicit. Extending this framework to texture transfer, we introduce StyleNet, an algorithm that can separate and recombine the image content and style of natural images. Finally, we demonstrate how StyleNet allows us to produce new images of high perceptual quality that combine the content of an arbitrary photograph with the appearance of numerous well-known artworks.