Socratical

An intelligent whiteboard for visual problem solving.

Comment

Inspiration

In subjects like math or physics, working through practice problems is often crucial for learning concepts effectively. This works sometimes without external help but eventually students will get stuck or be left wondering if they're on the right track. A tutor could help, but tutors are expensive and aren't always available. ChatGPT exists, but all it can do is give students the answers, there is no learning process involved by just being given a solution.

Socratical brings the following to the table:

1. A pedagogy conducive to all learning

Socratical makes the Socratic method scalable. Most students may never get a tutor that's this patient or accessible. Now they can work through any problem visually, on a canvas that feels natural, while an AI guides their thinking rather than just checking their answers. Unlike human tutors, Socratical isn't limited to math. Any problem that can be worked out on a whiteboard is fair game with Socratical: calculus, physics, biology, discrete math, Leetcode, System Design, the same method still applies.

2. Why vs What

The turn-based dialogue the interface provides creates a richer experience than ChatGPT. The tutor can ask direct questions in the chat window and also directly manipulate the canvas to simulate an actual live tutoring session. The tutor doesn't stop at the answer however, it constantly probes for the user's input on edge cases and real-life applications. The user will understand why the problem solving process works rather than just knowing that it works.

3. Performance anxiety

A student can work at their own pace, make mistakes privately, and get helpful guided questions instead of being told that they're wrong. That's psychologically different from a student raising their hand in class.

How I built it

I started by extending an open-source lightweight interactive canvas library called tldraw. The goal of the implementation was to enable the AI to see and interact with the canvas to provide the best learning experience for the user using the canvas' APIs. I used Claude Sonnet 4 as the multi-modal model along with Vercel's AI SDK to implement features such as streaming and tool calls seamlessly. I hosted it on Vercel making use of Vercel functions and Fluid Compute to make API calls feel instant.

Challenges I ran into

Working with a canvas library and its APIs was a first for me. I had to design helper functions to create screenshots and calculate annotation positions on the canvas. I also had issues with synchronization and timing LLM responses with rendering. Thankfully the Vercel AI SDK abstracted away some elements such as streaming but I had to implement my own custom hooks and components to render annotations.

Accomplishments that I'm proud of

This is the first hackathon I've ever done solo and so, this is a project that I completely own and am proud of in so many ways. To me this is a totally new interaction model that no chat bot like ChatGPT can easily replicate. Technically speaking, I'm proud of the decisions I made regarding the tech stack and architecture. This project was complex but scoped down enough to fit inside a weekend-long hackathon and I'm glad I made the right calls so that the project didn't become an insurmountable task.

What I learned

Even though I'd say I have a decent amount of experience in web dev, interfacing with a non-standard interaction model (that being a drawable canvas) was a new experience for sure. I had to consider rendering strategies, complex state management, and finally understanding what useRef was good for.

What's next for Socratical

Right now, Socratical exists on a session by session basis, meaning conversations and previous work aren't stored anywhere. Data persistence was intentionally left out for the sake of time, but is an important feature to implement in the future. As frontier models improve, their 'vision' may get better but I imagine messy handwriting or complex diagrams may overwhelm Socratical as of now. I hope to come up with ways to inject more context (perhaps making use of tldraw's APIs) into Socratical so that it could understand more complex problems. I also want to use Vercel's AI SDK to its fullest by implementing agentic workflows that could help the AI make better decisions such as using Desmos to plot graphs or an external calculator to validate answers.

Built With

Share this project:

Updates