Joanna | Devpost

Inspiration

We all know journaling is good for us. It helps with self-awareness, motivation, and mental clarity. But most of us don't do it. Writing takes effort, life gets in the way, and traditional journaling apps just store your words without ever learning from them. We asked: what if your journal could listen like a friend, remember what matters, and bring your past experiences back into the conversation exactly when you need them? That's where Joanna came from -- a journal you speak to, not write in.

What it does

Joanna is a voice-first AI life logger. You talk to it like you would a close friend about your day, your goals, your wins, and your frustrations. Joanna listens, extracts the meaningful details, and stores them as long-term memories. The next time you talk, Joanna doesn't start from scratch. It remembers. "Last week you mentioned struggling with motivation at the gym -- did today feel different?" That's not a script. That's someone paying attention. You can also browse your past entries in an interactive diary view, an open book with a mini calendar on the left and your memories on the right.

How we built it

Voice pipeline: Live voice input is transcribed in real time using Google Gemini 2.5 Flash Native Audio over a WebSocket connection. Responses are spoken back through ElevenLabs text-to-speech, making the experience feel like a real conversation. Agentic RAG loop: At its core, Joanna runs a multi-stage ReAct (Reasoning + Acting) loop: Transcription: Voice is live-transcribed via Gemini Synthesis: Before responding, the agent "thinks" in a separate pass: extracting new memories, detecting user intent, and generating follow-up leads Retrieval: Using Backboard.io, it retrieves relevant past memories via RAG, re-ranked by relevance. Recent memories are stored with high detail; older ones are progressively summarized Memory commit: New memories are immediately stored in Backboard for future recall Response generation: An enriched prompt combining conversation context, retrieved memories, and strategic directives produces an empathetic, grounded response Frontend: Built with Next.js 15, React 19, and Tailwind CSS 4. The mobile voice interface is a phone-style UI with an animated orb. The diary view features a 3D open-book design with CSS-masked hole punches threaded through spiral coils, stacked page edges for depth, gutter shadows, paper texture, and page-flip animations. Stack: TypeScript end-to-end, tRPC for typesafe APIs, PostgreSQL with Drizzle ORM, Docker Compose for infrastructure, Biome for linting, and Vitest for testing.

Challenges we ran into

Aligning agent with user preference through agent loop design and prompt engineering Balancing agent loop complexity with end-to-end inference latency

Accomplishments that we're proud of

he agent genuinely remembers and references past conversations in a way that feels natural and caring, not creepy. The diary view looks and feels like a real physical notebook, with 3D depth, paper texture, spiral binding that threads through hole-punched pages, and smooth page-flip animations. The full agentic RAG pipeline -- synthesize, retrieve, commit, respond -- works end-to-end in real time during a live voice conversation.

What we learned

RAG isn't just about retrieval; the synthesis step before retrieval (figuring out what to look for) is just as important as the retrieval itself. Voice-first UX requires a completely different design mindset. There's no "undo" button in a conversation -- the agent has to gracefully handle misunderstandings, short responses, and users who want to stop talking. CSS transform-style: preserve-3d and mask-image can produce surprisingly convincing 3D effects without WebGL or canvas. Progressive memory summarization is essential for long-term AI companions -- without it, context windows fill up with irrelevant details fast.

What's next for Joanna

Sentiment tracking: Detect and log emotional tone over time, with mood visualizations and trend insights. Goal tracking: Automatically extract and track goals mentioned in conversations, with progress reminders. Encrypted storage for privacy. Images: Store photos for daily Joanna entries.