Project Story: Neurolytics
Inspiration
Our journey began with a shared passion for medical imaging and artificial intelligence. We were inspired by the challenges radiologists face in analyzing complex 3D medical scans, particularly CT and MRI images. The idea of creating an interactive, AI-powered visualization tool that could help medical professionals better understand and analyze these scans became our mission.
What We Learned
Throughout this project, we gained invaluable knowledge across multiple domains:
- Medical Imaging: We delved deep into DICOM file formats, understanding how medical images are stored and processed
- 3D Visualization: We mastered the use of Napari for creating interactive 3D visualizations of medical scans
- Full-Stack Development: We built a modern web application using Next.js for the frontend and Python for the backend
- AI Integration: We learned how to effectively integrate AI models with medical imaging data
- Data Processing: We developed robust systems for handling and processing large medical datasets
How We Built Our Project
Backend Development
We started by creating a powerful backend system that could:
- Load and process DICOM files efficiently
- Sort medical images using InstanceNumber for proper sequencing
- Create 3D volumes from 2D slices
- Implement advanced visualization features like MIP (Maximum Intensity Projection) and translucent rendering
- Add interactive features like center markers and coordinate axe
- Flask endpoints for complex computing
- AWS for cloud storage and encryption
Frontend Development
Our frontend was built using modern web technologies:
- Next.js for a responsive and performant user interface
- Tailwind CSS for beautiful, consistent styling
- TypeScript for type-safe development
- Custom hooks and components for reusability
- Integration with the backend visualization system
AI Integration
We incorporated AI by:
- Using using Gemini's Visual Language Model(VLM) for real-time MRI analytics
- Provide intelligent insights about the scans by using context JSONs
- Identifying possible tumor location in a CT scan
Challenges We Faced
Data Processing Complexity
- Handling large DICOM files efficiently
- Ensuring proper sorting and stacking of medical images
- Managing memory usage with large 3D volumes
AWS Set-up
- Creating IAM User Roles for production applications
- Utilizing S3 Buckets for data storage
- Rendering data based on S3 Bucket contents
Integration Hurdles
- Connecting frontend and backend systems seamlessly
- Managing state across different components
- Ensuring smooth data flow between AI models and visualization
Technical Learning Curve
- Mastering medical imaging standards
- Learning new visualization libraries
- Understanding complex 3D rendering concepts
Impact and Future Vision
Our project has the potential to revolutionize how medical professionals interact with 3D medical scans. By combining advanced visualization with AI capabilities, we've created a tool that can:
- Improve diagnostic accuracy
- Reduce analysis time
- Provide better patient care
- Enable more effective medical training
Looking ahead, we envision expanding our platform to include:
- More advanced AI analysis features
- Support for additional medical imaging modalities
- Collaborative features for medical teams
- Integration with electronic health record systems
- Further data security measures
This project represents not just a technical achievement, but a step forward in medical technology that could improve healthcare outcomes worldwide.
Log in or sign up for Devpost to join the conversation.