Fact Checker

Machine learning based SDK to distinguish between mainstream opinions and noise

Comment

Inspiration

We love sharing information with each other online, and quite frankly, user generated information has become the major news source for us. Websites, such as Reddit, Twitter and etc. provide us with a unconventional and extremely existing perspective of the world and deliver the most unexpected information in such an expedient way. But all of us have encountered and suffered from online trolling activities-- obnoxious and sometimes malicious behaviours that undermine the prosperity of the community, and without which the world wide web would have have been a much better place. Hence our team embarked upon the endeavor to develop a Machine Learning based bad behaviours detection system for online communities, and along the way we have adopted a easy-to-use, user-friendly web interface for our project

What it does

The system records streams of comments, and temporarily stored them with Amazon DynamoDB service. Periodically, the learning algorithm would be invoked, to improve the large-scale anomolies detection system, leveraging data streaming techniques and online learning models. As a result, the well-trained model would be able to assign a score to each text block, reflecting the possibility of good behaviors, and to raise reg flags for extreme outliers. Fact Checker is the watch dog for your online community.

How I built it

Instead of building a full-stack server, we fully optimized our design architects and have implemented two lambda functions with AWS services. We separated out the back-send services and compartmentalized them into public APIs to enhance the flexibility of the system such that we could effortlessly respond to any streaming events, and therefore minimize the costs of operation. Furthermore, we implemented our own version of online machine learning - clustering algorithm, catering specifically towards the problem in question, in an effort to achieve a better outcome and accuracy.

Meanwhile, our front end easy-to-use user-friendly web interface serves both as a secure path to our database as well as a neat demonstration of the functionalities

Challenges I ran into

Build a cloud-first system is completely different from implementing a full-stack server. For instance, the standardized means of communications, parameters and return values, are replaced by serialized data transmission JQuery and JSON for instance, and hence insinuate a different design architect. Additionally, getting familiar with the wide variety of AWS services is a great hurdle for us.

Accomplishments that I'm proud of

All of the team members have learned substantial amount of new knowledge through this experience. We all stepped out our comfort zones and tackled a rather challenging problem together for social good deeds.

What I learned

We learnt to prototype and utilize design thinking techniques in brainstorming, designs and implementation process. And we learnt a lot from each other about good coding practices and useful problem-solving strategies.

What's next for Fact Checker

It would be really nice if we can connect our services to the big online forums and communities. It would really nice if one day we could work with Twitter and Reddit, given the amount of daily activities on these websites and consequently the substantial amount of threats and pressures they face

Share this project:

Updates