Lyffe Style

Inspiration

We were greatly inspired by Artificial Intelligence and Machine Learning and wanted to create an interesting, unique, and fun project. We decided to develop a project based on a concept we had heard about called style transferring. We were also new to Hackathons, so we decided to create a straightforward, but impressive piece of software. The end result is Lyffe Style, an extension for Google Chrome that lets you manipulate the images you see on screen.

What it does

Lyffe Style is a Chrome extension that takes the style of one image and transplants it onto any image you mouse over while browsing Google. For example, you can input the URL of Vincent Van Gogh's The Starry Night and move your mouse over the Google Image search results of "dog," and see pictures of dogs that look like they were painted with that iconic blue swirl look.

How we built it

On the Chrome side of things, we needed a JSON file, an HTML file, several Javascript files, and a CSS file. The JSON communicated settings to Chrome, the HTML and CSS described the UI of our extension, and the Javascript did most of the hard work. On the Python side of things we used Flask to create a server that can accept post requests containing the URL's of the image and style we want to transfer onto it. For the actual style transfer we used Tensorflow Hub's magenta/arbitrary-image-stylization-v1-256 a prebuilt model that can do arbitrary style transfer.

Challenges we ran into

None of us were very familiar with Javascript, HTML, or CSS, so we had to Google a lot of answers, finagle with so many syntax errors, and respond to baffling error messages. Another large challenge that we ran into was that we initially had no idea how to connect the Python side of things to the actual chrome extension, and when we finally decided that a Flask server was to way to go we had no idea on how to make one. Finally, we had problems where the chrome extension was sending already styled images back to the server, but we managed to solve it by modifying the images listeners to only activate if the src URL of the image is the same as it was when the web page loaded.

Accomplishments that we're proud of

In the limited time of 36 hours, our group was able to build a working neural network style transfer that can be applied to the Chrome Browser through a Chrome extension. We were very proud of our accomplishments with the capabilities of the chrome extension where the user is able to select specific images to style transfer. Similarly, we allow the user to upload specific styles they want to apply to the images. We are really pleased that when we paste the image address of the art style, we are able to store the correct information into the chrome extension and let the algorithm process the style accordingly. Furthermore setting up the Flask server was an ordeal and it was very gratifying to be able to connect our chrome extension to the Flask server which then runs a python neural network script on the passed images from the chrome extension. We were never sure if this would actually be possible and ran into many errors trying to get it working. Overall, we were ecstatic that our chrome extension was able to be created exactly as we imagined. We were completely new to most of the components used in this project so it was very amazing to be able to accomplish what we did.

What we learned

We definitely learned to read HTML, CSS, and Javascript, and now we can write these three languages with some help from Stack Overflow or Google. We first had to learn the syntax and language we need such as javascript and CSS to create the front end of the extension. The language of Javascript, structure of the Chrome extension, TensorFlow library, and Flask server are completely new for some of our group members.

What's next for Lyffe Style

In the future of Lyffe Style, various changes can be made to improve the flow and design of the chrome extension. To make Lyffe Style ready for public use, we would need to upgrade our Flask Server, due to it being hosted on our local machines, making it run very slow and mostly inaccessible for most people. The Flask Server would have to be modified to run on the cloud using some cloud database hosting service. Similarly, using Tensorflow hub, we built our neural network to style transfer various images onto the web pages, which would need to be upgraded in order to allow for an increased volume of inputs. Currently, it is not optimized fully to handle large volumes of images. We could also allow the user to save various styles that they like and possibly automate these styles and their transfers. Another next step for our Chrome extension, Lyffe Style, would be clean up the UI for the chrome extension, with smoother transitions between styled and regular images.

Built With

• css
• flask
• html5
• javascript
• python
• tensorflow
• tensorflow-hub