SentiMarket | Devpost

Inspiration

As demonstrated by the recent Equifax breach and the subsequent stock market fallout, large changes in company stock prices are often accompanied by significant public discourse on the subject. We wanted to take advantage of this by building a tool that would visualize public sentiment in social media, enabling more informed trading decisions.

What it does

SentiMarket queries Twitter for tweets mentioning a company of interest, and tracks the sentiment of such posts over time using multiple sentiment analysis engines. It then creates a time graph showing public sentiment vs. stock price, allowing users to see how and to what extent a particular company's stock price may be linked to public opinion.

How we built it

We built our application as a web service, with a backend powered by Python Flask running on Heroku. Upon receiving a company to query for, we multiple perform Twitter searches for tweets mentioning the company, with each search covering a one-week interval. We then group each week's tweets and send them to the Microsoft Cognitive Services text analysis API as well as the Vivekn sentiment API, allowing us to receive numeric scores corresponding to positivity or negativity. On the frontend, we use chart.js to visualize this data against stock prices retrived from AlphaVantage.

Challenges we ran into

We originally intended to retrieve tweets using the Twitter's official search API, but unfortunately the API only provides access to the last seven days of tweets. Since we wanted to visualize sentiment over a longer period of time, we had to search for unofficial solutions. We found a project on Github (https://github.com/Jefferson-Henrique/GetOldTweets-python) that did the job, albeit more slowly than we would have liked.

We also ran into some issues getting Python development going - two of our team members were new to Python and there were problems getting the necessary Python packages in Windows. We ultimately resorted to creating cloud-hosted Linux VMs that could be used over SSH.

Accomplishments that we're proud of

After completing the project, we were surprised at how well trends in sentiment matched with stock price, providing confirmation of our hypothesis that public opinion and stock price is often correlated. After an overnight train ride followed by 24 hours of coding, it was extremely gratifying to see it in action.

What we learned

Originally we used only Microsoft Cognitive Services, but we later noticed that utilizing multiple sentiment analysis engines produced better results than using just a single one as it can balance out the occasional incorrect interpretations of a single engine.