MediSage | Devpost

Login page
Home page
Ai catagory
ChatBot/Translator
Website

Inspiration We were inspired by the serious risks language barriers create in healthcare. Misunderstandings between doctors and patients can lead to medical errors, poor treatment outcomes, and even fatalities. We wanted to build a tool that not only translates but also empowers patients to understand their care and actively participate in it — no matter what language they speak.

What it Does MediSage is a medical assistant app that:

Provides real-time speech-to-speech translation between patient and doctor.
Summarizes prescriptions and care instructions in the patient's native language.
Suggests personalized questions or concerns based on the patient’s medical records to help them advocate for their own care. MediSage transforms a medical appointment from a confusing experience into a clear, confident conversation.

How We Built It We built MediSage using a full-stack web development approach. On the frontend, we designed a dynamic and responsive user interface with HTML, CSS, and JavaScript, using Poppins fonts and animations like particle backgrounds. The backend was developed with Node.js and Express, where we handled chat communication, language translation, speech recognition, medical image analysis, and recommendation generation. We used the OpenAI API (GPT-3.5-turbo and GPT-4 models) for natural language translation, explanations, and follow-up question generation. For image uploads and analysis, we used Multer for handling files and GPT-4 Vision for medical image interpretation. We also built a custom REST API with Axios to connect everything together, hosted it locally with plans for cloud deployment, and securely managed API keys with dotenv.

Challenges We Ran Into One of the biggest challenges we ran into was dealing with weird WebKit bugs in the browser’s speech recognition. Sometimes events like onend or onerror wouldn’t fire properly, which made it really hard to know when a user actually stopped talking. I had to write a bunch of custom fallback code to catch those edge cases and keep the experience smooth. Another challenge was hitting the ChatGPT rate limits during translation and explanation requests. To get around it, we tweaked how we sent prompts and also lowered the temperature setting to make responses faster and more lightweight. It definitely took a lot of trial and error, but it was worth it.

Accomplishments That We're Proud Of MediSage is like a personal translator and medical helper for patients who don’t speak English fluently. When a doctor speaks in English, MediSage listens, translates it into the patient’s language, and shows it clearly on the screen. Then when the patient responds in their language, it translates their words back into English for the doctor. It can also help explain medical information in simpler words if the patient is confused, and even generate smart follow-up questions they might want to ask. Plus, it can analyze medical images like X-rays or test results and explain them in the patient's language too.

What We Learned

Building healthcare tech demands extreme attention to accuracy, privacy, and usability.
AI models must be trained or adjusted specifically for fields like medicine — general translation isn’t enough.
Real-world problems are complex, and the best solutions require both technology and empathy.
Cross-disciplinary collaboration between tech, healthcare, and language experts is key.

What's Next for MediSage

Expand language support to include a wider range of dialects and regional variations.
Pilot test in real hospitals and telehealth systems to collect feedback and improve.
Launch a patient-facing app version so individuals can use MediSage independently in any medical setting.
Pursue partnerships with hospitals, NGOs, and governments to bring MediSage to underserved communities.
Long-term vision: Expand into legal services, education, emergency services, and global travel to create a true universal communication bridge.