Inspiration
Countless people's most private and intimate moments, in the form of private photos shared with significant others, are being used for the purpose of bullying, sexual harassment and shaming. We feel that the various websites where this occurs online have a responsibility to prevent these images from being posted to the internet.
We understand that the task of identifying this type of activity isn't so simple, and we hope Censei can provide the facility to curb this harassment.
What it does
Censei is a service developers can communicate with while processing a new Post/Photo Upload.
By simply asking Censei.isThisSafe({ image, text }), we can confirm through our API, by using a combination of Image Analysis (with Clarifai), Watson's Tone Analysis service, and our own algorithms, if an image is safe to publish on the web.
How we built it
The Censei Client is currently build as an npm module, which provides the facility for any Node server to communicate with our Censei Server (Node.JS). The Client library could easily be ported to other languages and frameworks.
The Censei Server itself encodes image in base64, running it through Clarifai's NSFW model, while simultaneously running the coupled text through Watson's Tone Analysis engine.
If an image is tagged NSFW from Clarifai over a tuned threshold, we further process the emotional intent and feeling analysis from Watson, and decide if an image (and it's text), is safe to post. It's important to note as well that the server could be configured instead to return a scored decision, rather then a binary one.
Challenges we ran into
We attempted to train a natural language text classifier, based on training data from conversations surrounding these harassment occurrences, but have yet to accurately classify text samples.
We ran into some issues around image processing, but managed to get past them fairly quickly.
Accomplishments that we're proud of
We're proud of all that we accomplished with Censei after an unsuccessful attempt to build a gesture controlled quadcopter, and how well our team handled the pivot.
What we learned
One of the most valuable things we learned from this experience, is that often some of the most troubling issues in our society are left unsolved, merely because they exist in tabooed topics of conversation.
What's next for Censei
We hope to extend Censei not only to more platforms, but to more forms of classification. There are plenty of other types of harassment and bullying that occur online, and Censei could grow into a service to detect and prevent all of them.
Log in or sign up for Devpost to join the conversation.