FightBite

Diagnosing deadly bug bites with the power of machine learning

Comment

Inspiration

The inspiration behind FightBite originated from the Brave Wilderness youtube channel, particularly the Bites and Stings series. When watching the series, we were terrified by the amount of destruction that could be caused by such minuscule beings. We were also inspired by the overwhelming 725,000 yearly deaths from mosquito-borne diseases. As a group, we decided to think of a solution, and this solution eventually became FightBite.

What it does

FightBite is a modular and interactive phone application that allows users to quickly and effectively take a picture of a bug bite or take an existing picture and get instant feedback on the type of bug bite, whether it be a mosquito, tick, or even bed-bug bite. In addition to detecting the bug type, FightBite also pulls up the relevant medical information for bite treatment. To use FightBite, simply tap on the start button, and then choose to either take a picture from the phone’s camera, or directly pick the bug bite from the gallery! Once an image has been selected, the user has the option of saving the image for future reference, discarding it and selecting a new image, or if they are satisfied, scanning the picture with our own AI for bite analysis.

How we built it

As FightBite is a phone based application, we decided to use react native for our front end design. As we intend for FightBite to work on both IOS, and android operating systems, react native allows us to write a single codebase that renders to native platforms, saving us the problem of creating two separate applications. Our neural network was created and trained with Pytorch, and was built on top of the DenseNet121 model. We then used transfer learning in order to adapt this pretrained network to our own problem. Finally, we created an API Endpoint with Flask and deployed it with Heroku.

Challenges we ran into

Over the past 48 hours, we faced various issues, mainly relating to the overall setup of react native and its many modules that we implemented. As this is our first time creating a phone application using React Native, we first had to take time to learn the documentation from scratch. Furthermore, we ran into issues regarding react native camera being deprecated due to lack of maintenance, so we were forced to use expo-camera instead, causing many delays. In addition to front end issues, we did not have any major access to a pre-existing dataset, so the majority of our data was compiled manually. This led to the size of our dataset being limited, which hurt the training of the model greatly.

Accomplishments that we're proud of

After completing HackThe6ix, our team is extremely proud that we managed to create our first ever functioning full stack mobile application in less than 48 hours. Although our team had some experience with web development such as HTML, CSS and Javascript, we never worked with react native before, so being able to implement a new language in creating a fully functional phone application is a huge accomplishment to our learning. Furthermore, this is our first ever “real” machine learning project with Pytorch, and we are extremely proud that we were able to build and deploy a machine learning model within 36 hours.

What we learned

We have learned many new skills after participating in HackThe6ix. Mainly, we learned more about phone app development through using React Native, and developed the ability to create an aesthetically pleasing application for both IPhone and Android devices. In addition, we also learned a lot about preparing and collecting data, along with training and evaluating a machine learning model. We also further enhanced our capabilities with Flask, as our team had very little experience with the framework coming into Hack The 6ix.

What's next for FightBite

Our first priority of FightBite is to ultimately expand our dataset, with more bug bites, and more images per type of bite, in order to quickly and accurately diagnose bug bites, for faster treatment and recovery. In particular, we plan to add some of the more deadly variants of bug bites(like black widow bites, brown recluse spider bites) in order to save as many lives as possible before it becomes too late. We also hope to add more depth to our bite analysis, like detecting potential diseases(an example of such could be detecting skin-lesions in tick bites like here.

Share this project:

Updates