$MMM

Visualise the public's opinions about major companies -- with memes

Comment

Inspiration

A paper by Indian Institute of Technology researchers described that stock predictions using sentiment analysis had a higher accuracy rate than those analyzing previous trends. We decided to implement that idea and create a real-time, self-updating web-app that could visually show how the public felt towards the big stock name companies. What better way then, than to use the most popular and relatable images on the web, memes?

What it does

The application retrieves text content from Twitter, performs sentiment analysis on tweets and generates meme images based on the sentiment.

How we built it

The whole implementation process is divided into four parts: scraping data, processing data, analysing data, and visualizing data. For scraping data, we were planning to use python data scraping library and our target websites are the ones where users are active and able to speak out their own minds. We wanted unbiased and representative data to give us a more accurate result. For processing data, since we will get a lot of noise when we scrape data from websites and we try to make sure that our data is concise and less time-consuming to feed our algorithm, we planned to use regular expression to create a generic template where it ignores all the emoticons.

Challenges we ran into

We encountered some technical, architectural, and timing issues. For example, in terms of technical problems, when we try to scrape data from twitter, we ran into noise issues. To clarify, a lot of users use emoticons and uncommon symbols when they post tweets, and those information is not helpful for us to find how users actually react to certain things. To solve this challenge, we came up with a idea where we use Regular Expression to form a template that only scrapes useful data for us. However, due to limited time during a hackathon, we increased efficiency by using Twitter’s Search API. Furthermore, we realized towards the end of our project that the MemeAPI had been discontinued and that it was not possible to generate memes with it.

Accomplishments that we're proud of

  • Designing the project based on the mechanism of multi servers
  • Utilizing Google Cloud Platform, Twitter API, MemeAPI

What we learned

  • Google Could Platform, especially the Natural Language and Vision APIs
  • AWS
  • React

What's next for $MMM

  • Getting real time big data probably with Spark
  • Including more data visualization method, possibly with D3.js
  • Designing a better algorithm to find memes reflecting the sentiment of the public towards the company
  • Creating more dank memes
Share this project:

Updates