nomi | Devpost

Inspiration

We’ve all been stuck at 2am with a full stack of notes we couldn’t turn into understanding: office hours closed, group chat asleep, and ChatGPT handing over answers with the wrong method and no idea what’s on our actual exam. We wanted the opposite of a chatbot: something that sits next to you while you work on paper, knows your course, and only speaks up when you’re actually stuck. That’s nomi.

What it does

Handwriting-first notebook: Apple Pencil, paper or PDF backgrounds, crisp vector ink, scribble-to-erase, and paste-anywhere images you can drag/resize and write over.
A tutor that shadows your work: it watches the page, grounds itself in your uploaded notes/slides/past exams, and nudges you (out loud) the moment you go off track.
Hands-free voice: real back-and-forth conversation that scans your page with every question.
Exam Prep: generates a likely exam as a PDF from your materials, runs a timed Exam Mode with countdown nudges, then auto-grades it with a per-question breakdown and correct approaches.
A living Nomi: a hand-built vector mascot that breathes, blinks, and shifts mood with how you’re doing.

How we built it

iOS (SwiftUI + PencilKit): per-page zoomable canvas; images layered below the ink so the pencil writes over them, manipulated with finger-only gestures.
Living mascot: a 24-point Catmull-Rom blob with a pose model driving gaze, blinks, mouth, and squash-and-stretch, rendered every frame.
Backend (FastAPI): provider-agnostic LLM layer (OpenAI/Gemini) with a vision + retrieval pipeline. Sources are chunked, embedded, and sent with each turn so answers stay grounded. Exams generate/grade via structured JSON; grading transcribes the filled pages with vision.
Extras: on-device exam PDFs (duration inferred at $t \approx 1.5 \times \text{marks}$ min), plus a GitHub Pages landing page reusing the same Nomi art.

Challenges we ran into

PencilKit auto-inverted ink in dark mode (black strokes vanished on white paper). We fixed it by pinning the canvas to light.
Making images live below the ink but still draggable/resizable, and killing a snap-back bug from persisting a stale frame.
Continuous voice broke speech recognition because of an audio-session race between playback and record. We fixed it with a clean session reset and a “wait for speech” mic.
Grades that didn’t add up. We now derive the score from the per-question marks.

Accomplishments that we’re proud of

A mascot that genuinely feels alive, a full exam loop (generate → timed sitting → auto-grade → breakdown) from a student’s own materials, a tutor that stays grounded and non-naggy, all real and dogfooded on an actual iPad.

What we learned

How to bend PencilKit to our will, that “feeling alive” is mostly timing, that grounding an LLM in a student’s own sources beats raw model power, and that audio sessions are a minefield where small delays make or break the magic.

What’s next for nomi

We’re continuing to build nomi toward an App Store launch and looking for funding to scale: bigger source libraries, faster on-device inference, shared class notebooks, and richer spoken tutoring. nomi should be the study companion every student wishes they had at 2am.