.png)
Painting has played a great part in human life. Since thousand years human represents their culture, art of living, and various more through painting. Some people express their feelings with the help of paintings putting into the canvas. There is a saying “a picture is worth a thousand words”, means many ideas can be shared through a painting. Let’s how we can generate an artistic image automatically within a short period.
Figure 1: An artistic painting ( credit: Olha Darchuk art)
After the rise of machine intelligence, we have taken it to another level whereby the help of Neural Network which designed in such a way that mimics the human brain can synthesize images and gives a new flavor to it in an artistic manner.
Convolutional Neural Network (CNN) is a powerful tool in the field of image processing. Each hidden layer in CNN extracts a unique pattern form the image and the output of CNN is consist of feature maps in the other word it differently filtered from the input image. Feature space is used for the representation of the style which was originally designed to capture texture information. It is built on the top of the filter responses in each layer of the network and it is consist of correlation of various filter response. Stationary, multi-scale representation of an input image is obtained which includes the correlation of multiple layers that can get the texture information.
Figure 2: Neural Style Transfer CNN (Figure 1 ofGatys et. al. 2015)
Generating a new style of an image by two different images is called neural style transfer.
Image1 and image2 are called as the content image (C) and style image (S) respectively together forms generated image (G). We can measure how similar content generated image and style to the generated image by J(G). where alpha and beta are weighting factors of content and style images respectively.
To find the generated image G we have to initialize G randomly and use gradient descent to minimize the function J(G).
There are two types of the cost function in neural style transfer are described below.
1. content cost function
2. style cost function
Content Cost function:
Let’s assume we have a hidden layer ‘l’ to compute the content cost. In general, we take the hidden later ‘l’ neither too shallow nor too deep. Then we will use our convNet model or a pre-trained model (e.g. VVG network) for the computation Let a[l](C) and a[l](G) are the activation layer ‘l’ on the images where ‘l’ represents the hidden layer and C and G re content and generated image respectively. If a[l](C) and a[l](G) are similar then we can say images have similar content.
Style Cost Function:
Let’s say we are using layer ‘l’ ‘s activation to measure style. So, define style as a correlation between activations across different channels. The correlation tells which of these high-level texture components tend to occur or not occur together in a part of an image and that’s the degree of correlation that gives you one of the ways of measuring these different high-level features. Style matrix can be represented as a[l] i.e. activation at (I,j,k) and style image is described below.
Gikk’ will measure how correlated are the activations in channel k compared to activation ink’.
Figure 3: (a) Content image (b) Style image (c) Generated image
Now you can visualize the generated image by using neural style transfer. The content cost function and style cost function for the first image and second image are 5.59, 1.25, 3.49, 2.09 respectively. Whereas the total loss is 7.16 5.78. which can be further fine-tuned by correctly tuning the hyperparameters of the neural network. The more the loss function minimizes it gets more attractive.
Summary:
A deep neural network can create an artistic painting without any human help that looks realistic and high perceptual quality. The algorithm uses both content and style arbitrary images to separate and recombine and creates an artistic image.
LinkedIn: Kalyan Mohanty
Twitter: Kalyan_0nline
Reference:Gatys et. al. 2015A Neural Algorithm of Artistic Style