Opinion Diversifier

Using presidentobama Twitter
Using speakerpelosi Twitter

Inspiration

With the issues arising recently in the news from radicalized individuals due to echo chambers on social media, we took on the Marshal Wace challenge to have a positive impact using technology.

What it does

You supply the webpage with your public Twitter or Reddit name and our web app provides opposing views based off of other user's (anonymous) views with the goal of stimulating critical thinking about your political views.

How we built it

The program has 4 aspects to it:

The web scraper which uses the social media handles to gather data from the user's posts. These are fed into a clustering algorithm which places the user into a specific cluster. Based on the cluster position, the program selects another user's views which oppose yours. This is then fed to a flask program that interacts with AJAX to get client input and output.

Challenges we ran into

Each component and hence each developer had its own technical challenges. On the backend side, verifying and connecting our own domain to Google Cloud took a long time. Using Twitter's and Reddit's APIs implied some rate limits which are tough to get around. The clustering algorithm was difficult to debug and validate because of the lack of a baseline model. Also assembling the pieces was harder than expected, due to some misunderstanding.

Accomplishments that we're proud of

We all had some past experience with our own projects, but this topic and the related technologies were mostly new for all of us. Choosing this project idea was a tricky choice, but we learned a lot by working on it and eventually succeeded in all parts.

What we learned

We learned that the input-output format of each component should be fixed at the beginning precisely. Working as a team and deploying to the cloud with a database and continuously added data was a great learning experience.

What's next for Opinion Diversifier

At the moment we have just two social media options (Twitter, Reddit), we are planning to extend it. Also, the clustering and post suggesting algorithm should be refined paying more attention to the actual behavior of social media users, not just on graph theory aspects.

Built With

Submitted to

Hex Cambridge 2021

Created by

I worked on connecting the main program with Datastax. This was my first time using Cassandra, and NoSql in general so it was a bit challenging, but it all worked out in the end.

Viktor Mirjanic
I worked on obtaining the data for the clustering. This involved scraping Twitter and Reddit using their respective APIs. Rate limits were a nuisance, although we did get some interesting results.

Nikola Georgiev
I worked on the front-end. First time using AJAX and was happy to touch my development and UI skills throughout the process

Skye Purchase
I was responsible for the natural language processing and clustering algorithm using sklearn.

Gábor Pituk
I worked on the Flask backend connecting all the pieces of the project. I was also responsible for deploying the project on Google Cloud.

Vári-Kakas Andor
Nikola Georgiev