SpotSense | Devpost

Inspiration💡

We created this app because we saw how scary and confusing it can be for people who notice a new spot or mole and aren’t sure what it means. We wanted to give everyone an easy way to check their skin, find answers fast, and connect with medical help if needed. By combining AI with simple, user-friendly tools, our goal is to make it less overwhelming to stay on top of your skin health and catch potential issues early—because a little peace of mind can make a big difference.

What it does🔍

Meet SpotSense, a user-friendly web app that takes the guesswork out of checking for skin cancer. Simply upload or snap a photo through your webcam, and our TensorFlow model will analyze it to see if it might be malignant or benign. Your images are stored securely in an SQL database, and there’s even a “Help” section, powered by Google Map that help find nearby hospitals. Need more guidance? Our Cohere-powered chatbot is on hand to answer questions about skin diseases, and Google OAuth keeps everything secure for a smooth experience.

How we built it🛠️

We built SpotSense by combining several powerful tools and frameworks to ensure accuracy and ease of use. At its core is a TensorFlow-trained model, which relies on neural networks and machine learning to analyze user-submitted or webcam-captured images and classify them as potentially malignant or benign. These images are stored in an SQL database for future review or follow-up. We also integrated a “Help” section that uses the Google Maps API to help users quickly locate hospitals if they need additional medical support, and employed a Cohere-powered chatbot to answer general questions about skin diseases. Finally, we implemented Google OAuth to deliver a secure login experience, ensuring user data remains protected while still offering a streamlined workflow.

Challenges we ran into🚧

Finding a diverse, high-quality dataset for our TensorFlow model to accurately predict whether the image was malignant or benign.
Maintaining quick response times during the image analysis.
Storing user images and data in an SQL database with security in place, making sure everything remains private and trustworthy.
Managing multiple services (Google Maps, Cohere chatbot, Google OAuth) introduced extra challenges and compatibility issues.

Accomplishments that we're proud of🏆

We designed a simple, good-looking UI to help you focus on your health
We developed a TensorFlow model powered by advanced neural networks and machine learning techniques, enabling accurate classification of images as malignant or benign.
We integrated webcam captures and image uploads into a straightforward user flow, making skin checks simple and accessible.
Implemented several API's and services into our project (Google Maps, Cohere, Google OAuth)

What we learned🧠

How to efficiently use GitHub/Git for collaboration and part integration
How to use Tensorflow and Keras to create neural networks and deep learning models
Using SQL to store images in databases
Setting up webcam features
Using google maps api to retrieve routes to nearest hospitals based on user address
Using Cohere API to create a friendly chatbot

What's Next for SpotSense?

Next for SpotSense, we aim to enhance our AI model's accuracy by incorporating a broader dataset to improve predictions across diverse skin types and conditions. We plan to integrate telemedicine features, allowing users to connect directly with dermatologists for professional advice. Additionally, we’re working on a personalized health tracking dashboard where users can monitor skin changes over time and receive tailored health tips. With future updates, we envision SpotSense becoming a comprehensive tool for proactive skin health management.

Built With

css3
flask
google-maps
google-oauth
html5
javascript
jinja
keras
numpy
pillow
python
sql
tensorflow
werkzeug

Submitted to

DeltaHacks XI

Created by

I worked on the main front end of the project using HTML, CSS, JavaScript, and Flask, where I designed and developed a visually stunning and user-friendly UI. Also created the main feature of uploading images and taking images through JavaScript that saves it into a database using SQL. I also played a key role in integrating various components, ensuring cohesive functionality
throughout the application.

Sahir Zaman
I developed the Python Flask backend and created the routes to handle application functionality. I designed and implemented the database system, and built the login functionality for user authentication. Additionally, I integrated these components with other elements of the project including the google maps api and our trained tensorflow model, ensuring they worked seamlessly to deliver the final product.

Maximilian Miller
I contributed to developing a comprehensive solution by retrieving and preparing images to train and test a TensorFlow model. Additionally, I integrated the Google Maps API to help users locate and navigate to the nearest hospital. The system provides multiple route options across various transportation methods, ensuring users receive the most reliable and efficient assistance. Additionally, I helped with the integration of the various components, ensuring seamless transitions.

Eishan Ashraf
I built a neural network using TensorFlow to classify cancerous skin images as either malignant or benign. The model, which uses convolutional neural networks (CNNs), was trained on a prepared dataset of images, with preprocessing steps like resizing and data augmentation to improve performance. After fine-tuning, the model achieved an accuracy of about 85% in distinguishing between malignant and benign cases, showing promising results. I also helped bring everything together, making sure all the parts worked smoothly.

Farhan Sifar