About SyDeTre
Hi, I'm Toni, and this is my project called SyDeTre, a medical model finetuned for symptom detection and treatment.
Project Motivation
I chose this task because pharmaceutical drugs and various treatments are often missing in medicinal LLM datasets. To address this gap, I started by gathering a comprehensive multi-point database containing thousands of illnesses and their corresponding medications.
Data Collection and Processing
I began by querying questions to these data points using Mistral 8x22b and incorporated data from Medquad. This effort resulted in an MIT Licensed medical dataset consisting of 72,000 Q&A pairs.
Finetuning Approach
For finetuning the model, I experimented with different hyperparameters, finding that a higher number of training steps and a smaller learning rate were beneficial. By iterating the process based on wandb results, I achieved positive outcomes.
Deployment
The final model was deployed with a Next.js interface, allowing users to chat with SyDeTre and utilize its medical insights.
Future Plans
I am pleased with the results of the dataset and finetuning efforts. I plan to continue improving the model and will test SyDeTre on medical LLM benchmarks in the future.
Conclusion
This project provided valuable insights and a unified approach to finetuning with the Mistral API. I am grateful for the opportunity to share my work.
Log in or sign up for Devpost to join the conversation.