neural-style

neural-style This is a torch implementation of the paper A Neural Algorithm of Artistic Style by Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge. The paper presents an algorithm for combining the content of one image with the style of another image using convolutional neural networks. Here's an example that maps the artistic style of The Starry Night onto a night-time photograph of the Stanford campus: Applying the style of different images to the same content image gives interesting results. Here we reproduce Figure 2 from the paper, which renders a photograph of the Tubingen in Germany in a variety of styles: Here are the results of applying the style of various pieces of artwork to this photograph of the golden gate bridge: Content / Style Tradeoff The algorithm allows the user to trade-off the relative weight of the style and content reconstruction terms, as shown in this example where we port the style of Picasso's 1907 self-portrait onto Brad Pitt: Style Scale By resizing the style image before extracting style features, we can control the types of artistic features that are transfered from the style image; you can control this behavior with the -style_scale flag. Below we see three examples of rendering the Golden Gate Bridge in the style of The Starry Night. From left to right, -style_scale is 2.0, 1.0, and 0.5. Multiple Style Images You can use more than one style image to blend multiple artistic styles. Clockwise from upper left: "The Starry Night" + "The Scream", "The Scream" + "Composition VII", "Seated Nude" + "Composition VII", and "Seated Nude" + "The Starry Night" Style Interpolation When using multiple style images, you can control the degree to which they are blended: Transfer style but not color If you add the flag -original_colors 1 then the output image will retain the colors of the original image; this is similar to the recent blog post by deepart.io. Setup: Dependencies: torch7 loadcaffe Optional dependencies: For CUDA backend: CUDA 6.5+ cunn For cuDNN backend: cudnn.torch For OpenCL backend: cltorch clnn After installing dependencies, you'll need to run the following script to download the VGG model: sh models/download_models.sh This will download the original VGG-19 model. Leon Gatys has graciously provided the modified version of the VGG-19 model that was used in their paper; this will also be downloaded. By default the original VGG-19 model is used. If you have a smaller memory GPU then using NIN Imagenet model will be better and gives slightly worse yet comparable results. You can get the details on the model from BVLC Caffe ModelZoo and can download the files from NIN-Imagenet Download Link You can find detailed installation instructions for Ubuntu in the installation guide. Usage Basic usage: th neural_style.lua -style_image <image.jpg> -content_image <image.jpg> OpenCL usage with NIN Model (This requires you download the NIN Imagenet model files as described above): th neural_style.lua -style_image examples/inputs/picasso_selfport1907.jpg -content_image examples/inputs/brad_pitt.jpg -output_image profile.png -model_file models/nin_imagenet_conv.caffemodel -proto_file models/train_val.prototxt -gpu 0 -backend clnn -num_iterations 1000 -seed 123 -content_layers relu0,relu3,relu7,relu12 -style_layers relu0,relu3,relu7,relu12 -content_weight 10 -style_weight 1000 -image_size 512 -optimizer adam To use multiple style images, pass a comma-separated list like this: -style_image starry_night.jpg,the_scream.jpg. Note that paths to images should not contain the ~ character to represent your home directory; you should instead use a relative path or a full absolute path. Options: -image_size: Maximum side length (in pixels) of of the generated image. Default is 512. -style_blend_weights: The weight for blending the style of multiple style images, as a comma-separated list, such as -style_blend_weights 3,7. By default all style images are equally weighted. -gpu: Zero-indexed ID of the GPU to use; for CPU mode set -gpu to -1. Optimization options: -content_weight: How much to weight the content reconstruction term. Default is 5e0. -style_weight: How much to weight the style reconstruction term. Default is 1e2. -tv_weight: Weight of total-variation (TV) regularization; this helps to smooth the image. Default is 1e-3. Set to 0 to disable TV regularization. -num_iterations: Default is 1000. -init: Method for generating the generated image; one of random or image. Default is random which uses a noise initialization as in the paper; image initializes with the content image. -optimizer: The optimization algorithm to use; either lbfgs or adam; default is lbfgs. L-BFGS tends to give better results, but uses more memory. Switching to ADAM will reduce memory usage; when using ADAM you will probably need to play with other parameters to get good results, especially the style weight, content weight, and learning rate; you may also want to normalize gradients when using ADAM. -learning_rate: Learning rate to use with the ADAM optimizer. Default is 1e1. -normalize_gradients: If this flag is present, style and content gradients from each layer will be L1 normalized. Idea from andersbll/neural_artistic_style. Output options: -output_image: Name of the output image. Default is out.png. -print_iter: Print progress every print_iter iterations. Set to 0 to disable printing. -save_iter: Save the image every save_iter iterations. Set to 0 to disable saving intermediate results. Layer options: -content_layers: Comma-separated list of layer names to use for content reconstruction. Default is relu4_2. -style_layers: Comma-separated list of layer names to use for style reconstruction. Default is relu1_1,relu2_1,relu3_1,relu4_1,relu5_1. Other options: -style_scale: Scale at which to extract features from the style image. Default is 1.0. -original_colors: If you set this to 1, then the output image