Salus Sync | Devpost

Salus Sync Promotional Poster
App Intro
Login for all users
Menu
Dashboard
Appointments
Medical Records
Chat
Emotions

Inspiration 🌟

Scheduling appointments and going to appointments take up time and even though doctors provide instructions for how patients should utilize their prescriptions they don't always follow them properly. This is where Salus Sync comes in. Salus is the Latin word for health. We hope that we can get doctors and patients on the same page and sync in terms of the patients' health.

What it does ✅

The goal of Salus Sync is to improve medical adherence and doctor-patient relationships through the use of AI. This is mainly utilized through the speech to text recording feature via the chat between the patient and doctor. We also incorporated the user interface to include medicine tracking/records, appointments, and an emotions feature. The emotions feature utilizes Hume AI to gather the emotions a patient feels while recording and represents the emotions in a word cloud for easy visibility to doctors.

How we built it 🧑‍💻

We utilized Hume AI to allow patients to record their interpretation of the medical advice they were given to communicate to their doctors to make sure that both sides are on the same page. This also allows doctors to gain emotional information of their patients' recordings.

Challenges we ran into 🪨

We had issues converting the web file from browser recording API to ArrayBuffer and blob to submit to different platforms took up a significant portion of time.

Accomplishments that we're proud of 🤩

We were able to successfully implement speech to text and convert it into embeddings. We used Hume AI to do sentiment analysis for patient's voice recordings and used LLM to summarize and analyze the recordings.

What we learned 💪

We learned to handle audio files recorded through the browser and convert these files to different file formats. We also figured out how to stream these audio files to different services and APIs leveraging the latest technology - OpenAI function call.

What's next for Salus Sync 👀

For the hackathon, we used Next.js to create a prototype. We hope to build a native mobile app or a progressive web app in the near future. We also planned to have different dashboards for doctors and patients but were unable to complete this in time for the hackathon demo but hope to expand and build this out in the future.