Upside Healthcare

Demonstration of app's function without any uploaded image
Demonstration of app's function with an uploaded image (an option is presented to use the webcam as well)

Inspiration

Our inspiration for Upside Health stems from our passion to empower individuals to take charge of their healthcare and to revolutionize the current healthcare system by transitioning from the traditional "fee for service" model to a value-based approach. Our goal is to create a user-friendly and cost-effective platform that leverages the latest in artificial intelligence and machine learning technologies, fostering a more efficient, effective, and personalized healthcare experience for patients and providers alike.

What it does

Upside Healthcare provides patient-centric care by using state-of-the-art AI to empathetically listen to patient concerns and pain points. The user opens up the app and communicates their symptoms to the AI doctor via text or image input. The app then analyzes the image against a trained Computer Vision model to classify the skin disease that may be present. Alongside, it sends the textual symptoms to ChatGPT, which has been primed to produce advice based on the symptoms presented. We compare the diagnoses given by both models to give a confidence level of the most susceptible medical condition, presenting suggestions on which tests and steps to take.

This makes the healthcare experience more efficient and thorough by adding that extra image and AI dimension to 1) allow the patient to weigh the need for an expensive doctor's visit, and 2) helps a doctor narrow down the diagnosis or triage the case.

How we built it

We used a ChatGPT API and computer vision to diagnose the symptoms presented.

Challenges we ran into

We ran into a challenge in data pre-processing with finding images of pale skin.

Frontend-wise, we tried integrating a mobile application that is able to take a picture using the phone's camera and upload it to the cloud where the backend will process it. However, that task encountered many blockers and had to be dismissed in favor of developing a web app and using the laptop's webcam to take a picture, and storing it locally.

Accomplishments that we're proud of

Learning how to encode an image into a base64 string was bewildering as now we can send the image through the internet (to the backend) like any text data. We're also proud to go through the arduous boring data science task of finding and cleaning the dataset which took up most of our time, but it taught us a lot of good practices and tools like 7zip when it comes to the task.

What we learned

Getting and cleaning a dataset is very difficult, but also a very important part of the process. Further, being able to capture a photo and transmit it over the Internet via a POST request and creating a good UI to make the app's feedback more friendly