My Hackathon Journey: Building XO Brain AI

Inspiration

A few years ago, I discovered two books that really helped with my approach to productivity: "Building a Second Brain" and "Getting Things Done". As someone with ADHD, these structured systems were a game-changer for me.

Unfortunately, I found that existing apps claiming to use the "second brain" concept for marketing didn't fully really follow the principles outlined in the book. The closest solution was a template that pushed Notion to its limits, but it fell short when I needed offline access, which is fairly frequent for me as I have my best thoughts offline.

I also love using ChatGPT, but I often found myself repeating information about my projects and personal life. While ChatGPT is awesome, it doesn't truly know me. I realized that an AI capable of leveraging my context would be exponentially more useful. For an AI to be truly effective, it needs to understand my projects, goals, and priorities. This is what sparked the inspiration behind XO Brain AI.

What it does

XO Brain AI is designed to handle my chaotic stream of consciousness and transform it into organized, actionable information. With real-time transcription and processing, XO Brain AI allows users to speak their thoughts, and the app intelligently splits and organizes the notes based on semantic content and topics. The app features a Notion-style editor called Blocknote, enabling users to edit and view their notes effortlessly.

The next step in XO Brain AI's roadmap is to automatically organize notes according to projects, areas, and resources, as outlined in the "Building a Second Brain" methodology. The AI will intelligently categorize notes, either finding an existing category or creating a new one as needed.

How I built it

XO Brain AI is built using a full-stack TypeScript approach, with Convex for the backend and Next.js for the frontend. I heavily relied on GPT-4 for the AI capabilities but designed the system to be model-agnostic, allowing for hotswapping of future language models. Zod through instructor is used to ensure data consistency, and techniques like reordering, merging groups, and re-summarizing are implemented to create a smooth note organising.

Challenges I ran into

One of the most significant challenges I faced was implementing the text editor and getting stuck for a long time. I initially started with Tiptap, but due to poor documentation and difficulties in implementing real-time collaboration, I had to pivot to Blocknote, which was very simple in comparison.

Unfortunately, I had to temporarily set aside the collaborative multiplayer features, the plan is to have the AI to act as a collaborator, making edits in real-time alongside human users with the cursor and highlight leveraging the prose mirror offline adaptor for this.

Understanding convex, this project has forced me to understand and learn how convex works. There was a bit of a learning curve around how mutations and actions call each other and should be used. I'm happy to have convex as a back and and it's enabling some really cool AI innovation, I will continue with it.

Accomplishments that we're proud of

I'm particularly proud of the real-time speech processing feature, which allows XO Brain AI to gather context and build a knowledge base as the user speaks. This work will be useful going forward because I can use the same logic to structure information from Chat messages and other inputs. This is the base level of an AI that processes and builds structured knowledge in real-time.

What I learned

I gained valuable experience working with chunking & real-time transcription managing processing queues using Convex.

I've gained more confidence nextjs and typescript.

I've also learnt how important and subtle prompting can be, how one minor tweaks can have massive consequences.

I also learned the importance of taking breaks, recharging and re-prioritising, it was frustrating getting stuck down a few dead ends and not getting as far as I'd like with the overall app but working harder and longer did not translate to better output. Big lesson to have boundaries around dropping the work completely, not thinking about it and picking it up again later.

What's next for XO Brain AI

I'm exited about the territory trajectory of AI generally and I'd like XO to be my interface with the latest models and techniques.

I'm eager to complete the chat interface and have the Ai be aware of all the notes and tasks. It's a critical mass in terms of usefulness. Over time the more context builds the more helpful XO will be.

I've recently found a startup mentor and he encouraged me to validate the demand for XO Brain AI, so I've set up a waitlist to capture early interest. I will invite those who are interested having a say on features and giving feedback into the XO Tribe discord.

If you're interested in joining the early access join the waitlist at xoBrain.Ai.

Visit xobrain.ai/tryme to test out the braindump transcription feature.

Built With

• convex
• instructor
• nextjs
• openai