fillosophy

Agentic document workflows for first-gen immigrants struggling with medical documents. A user uploads a video explaining their situation to power a RAG automation agent to parse and fill documents.

Comment 3

fillosophy

Inspiration

As children of immigrants, we've experienced firsthand the struggles our parents faced navigating the tedious process of filing medical documents. The complex terminology and bureaucratic language create significant barriers for them. We wanted to leverage the capabilities of Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) to build a platform that helps first-generation immigrants complete essential medical paperwork with confidence and ease.

What it does

Fillosophy leverages Document Agentic Workflows for end-to-end automation of medical document filing. The user experience follows a simple flow:

  1. Document Upload: Users upload the specific medical documents they need to complete
  2. Intelligent Parsing: Our agent parses and analyzes these documents to extract all relevant fields
  3. Multilingual Input: Users can record a video in their native language simply explaining the information needed
  4. Supplemental Information: Additional supporting documents can be uploaded for context
  5. Automated Completion: Using RAG technology, Fillosophy synthesizes all inputs and automatically fills out the required forms

This approach eliminates language barriers and simplifies what would otherwise be a confusing and stressful process.

How we built it

Fillosophy integrates two core agentic features:

Document Parsing

  • Leveraged LlamaIndex's LlamaParse library powered by Google Gemini
  • Extracts relevant fields from complex medical documents
  • Identifies required information with high accuracy

Video Content Analysis

  • Implemented Google Gemini 2.0 Pro for video analysis and voice transcription
  • Converts native language explanations into structured data
  • Matches user input to document requirements

Tech Stack

  • Backend: FastAPI (Python)
  • Frontend: Next.js, Tailwind CSS, ShadCN
  • Database: MongoDB for secure storage of user documents and videos
  • Performance: Implemented streaming of LLM completion requests for a seamless user experience

Challenges we ran into

  • Technical Integration: Despite LlamaIndex documentation indicating support for various file types (.txt, .md, .docx), we discovered through extensive testing that only the PDF parsing functionality worked reliably
  • Workflow Complexity: Building a functional document workflow with Gemini and LlamaIndex required significant adaptation and workarounds

Accomplishments that we're proud of

Our proudest achievement is creating an Agentic RAG system capable of handling multiple input types and sources. When users upload videos to complete documents, they can also include multiple supporting materials which our system successfully synthesizes into accurate form completion. This flexible approach enables a much more natural user experience compared to traditional form-filling interfaces.

What we learned

  • Leveraging UI libraries like ShadCN accelerate frontend development
  • Strategies for creating effective RAG systems with multiple data sources

What's next for Fillosophy

We plan to expand Fillosophy beyond medical documents to serve other industries with complex paperwork requirements. Our team believes the application can provide valuable assistance to:

  • Law enforcement personnel
  • Restaurant owners and food service businesses
  • Small business operators
  • Immigration services
  • Tax preparation services

By addressing document complexity across multiple sectors, we aim to make essential processes more accessible to everyone, regardless of language background or familiarity with technical terminology.

Built With

+ 3 more
Share this project:

Updates