Reeliable | Devpost

Inspiration

While completing at a unrelated hackathon I had an idea, what if I could create something to prevent the younger and older generations from consuming fake news. The more our would shifts to electronic news the more fake news there is on the internet, in a paper published by Digit Health in 2024, 44% of posts on Instagram concerning dental caries were classified as misinformation. This is critical since knowledge is power and if false knowledge falls into the wrong hands there can be dire consequences.

What it does

You download Reeliable extension and open Instagram reels, then two things happen simultaneously:

The caption text is grabbed from the DOM
The reel's audio is captured and streamed to deepgram for speech-to-text transcription Then the caption and transcribed audio is sent to the backend, then three things happen:
Claude Haiku reads the text and extracts any verifiable health claims
Each claim gets embedded and searched against a vector database (Qdrant) full of medical literature from sources like WHO, CDC, PubMed, and Cochrane
Claude Haiku writes a grounded verdict using only the retrieved evidence: Supported, Contradicted, Partially True, or Unverified Results stream back and appear as a card floating to the right of the reel video, showing the claim, verdict, and sources, the ultimate goal is to catch health misinformation on social media the moment someone watches it, without interrupting their experience.

How we built it

For the chrome extension we used:

Manifest V3, TypeScript, React (Shadow DOM overlay), Vite
For audio transcription we used:
Deepgram WebSocket (streaming ASR) For the backend server we used:
Fastify, TypeScript AI/LLM we used:
Anthropic Claude Haiku (claim extraction + verdict synthesis) For embeddings we used:
OpenAI The vector database we used:
Qdrant Package Manager
pnpm (monorepo workspace)
Infrastructure:
Docker (Qdrant locally), deployable to Fly.io / Railway

Challenges we ran into

Trying to obtain data for reels after the one a user is currently using. Due to Instagram's settings it is hidden but using react fibers we were able to connect to the DOM and extract all currently loaded reels, then caching them, giving users access to information instantaneously.

Accomplishments that we're proud of

We are proud to have implemented a tricky engineering feat of getting future reels and caching the fact checking response, so that users can can close to instant fact checking as they scroll. We are also proud to have correctly produced fact checking for health/fitness.

What we learned

We learned how to build a chrome extension, we also learned how to correctly fact check claims using info from different sources such as official corpus etc.