Chunky

Inspiration

Default RAG architecture is based on fixed chunks, either you get them right or you don’t

What it does

Chunky is a better way of chunking. We,

• Chunking by meaning, not by size
• Evaluate chunk relevance
• Re-chunk to provide more accurate response

How we built it

• Data pipelines: LangChain
• Model: OpenAI (Embedding and GPT3.5)
• Orchestrator: Custom scripts
• LLM Cache: GPT Cache
• ML Ops/Logging: MLFlow, Arize
• Vector Database: Mongo Atlas DB
• Deployment/Frontend: Heroku/Mercury

Challenges we ran into

• 8 person team
• 5 people left in the end
• 6 hours of hacking
• Frontend: Mercury framework was in-development, deployment to HuggingFace failed
• Backend: so much

Accomplishments that we're proud of

• Frontend working
• Backend near working

What we learned

What's next for Chunky

Built With

• arize
• embeddings
• gpt-cache
• gpt3.5
• heroku
• langchain
• mercury
• mlflow
• mongodb
• openai