Jovian
Sign In
Learn practical skills, build real-world projects, and advance your career

Deep Learning & Art: Neural Style Transfer

In this assignment, you will learn about Neural Style Transfer. This algorithm was created by Gatys et al. (2015).

In this assignment, you will:

  • Implement the neural style transfer algorithm
  • Generate novel artistic images using your algorithm

Most of the algorithms you've studied optimize a cost function to get a set of parameter values. In Neural Style Transfer, you'll optimize a cost function to get pixel values!

Updates

If you were working on the notebook before this update...
  • The current notebook is version "3a".
  • You can find your original work saved in the notebook with the previous version name ("v2")
  • To view the file directory, go to the menu "File->Open", and this will open a new tab that shows the file directory.
List of updates
  • Use pprint.PrettyPrinter to format printing of the vgg model.
  • computing content cost: clarified and reformatted instructions, fixed broken links, added additional hints for unrolling.
  • style matrix: clarify two uses of variable "G" by using different notation for gram matrix.
  • style cost: use distinct notation for gram matrix, added additional hints.
  • Grammar and wording updates for clarity.
  • model_nn: added hints.
import os
import sys
import scipy.io
import scipy.misc
import matplotlib.pyplot as plt
from matplotlib.pyplot import imshow
from PIL import Image
from nst_utils import *
import numpy as np
import tensorflow as tf
import pprint
%matplotlib inline

1 - Problem Statement

Neural Style Transfer (NST) is one of the most fun techniques in deep learning. As seen below, it merges two images, namely: a "content" image (C) and a "style" image (S), to create a "generated" image (G).

The generated image G combines the "content" of the image C with the "style" of image S.

In this example, you are going to generate an image of the Louvre museum in Paris (content image C), mixed with a painting by Claude Monet, a leader of the impressionist movement (style image S).
alt

Let's see how you can do this.

2 - Transfer Learning

Neural Style Transfer (NST) uses a previously trained convolutional network, and builds on top of that. The idea of using a network trained on a different task and applying it to a new task is called transfer learning.

Following the original NST paper, we will use the VGG network. Specifically, we'll use VGG-19, a 19-layer version of the VGG network. This model has already been trained on the very large ImageNet database, and thus has learned to recognize a variety of low level features (at the shallower layers) and high level features (at the deeper layers).

Run the following code to load parameters from the VGG model. This may take a few seconds.