Chorémon | Devpost

home page, coin mode, tile mode
chore selection, trash chore, laundry chore
mode selection, quest completed, leaderboard
unity augmented reality development
webxr + three.js testing

Inspiration

As Purdue University students, life gets hectic real fast. We all have classes, organizational activities, and everything happening on campus, and by the time we get back to our dorm we're already exhausted. The last thing on our minds is doing chores. The dishes pile up. The floor hasn't been vacuumed in days. The laundry is just sitting on the corner of the room.

But ignoring it only makes the guilt worse and it's not just us. Getting kids to do chores is a universal struggle every parent faces. Nagging doesn't work. Chore charts get forgotten. Bribes only last so long. Kids need something that sparks their interests, such as: fun, rewards, and a little competition.

We wanted to fix that feeling for everyone, not by making chores feel like a responsibility, rather by making them feel like a reward and satisfaction. We grew up with Pokémon Go, the thrill of discovering new Pokémons to put in the Pokédex, defending the gym. So we figured: what if our messy dorm room or your kid's disaster of a bedroom, gave us a quest instead of giving you anxiety?

Choremon was born from that idea. Whether you're a burnt-out college student or a parent trying to get their 8-year-old to sweep the floor, your chores shouldn't feel like a burden. They should feel like a boss fight waiting to happen and once you complete it, the satisfaction takes over and you can enjoy the rest of your night minding your own business without needing to worry about doing chores.

What it does

Choremon is a gamified chore app that uses AR & AI to turn your living space into a quest board. It doesn't require any manual logging, chore charts or nagging.

The experience is simple. You open the app, point your camera at any room, and take a photo. The software has a few features to select from, including: vacuum, sweep, mop, wipe, trash and laundry. Depending on the action that you chose, the implementation will be different. For example, when you choose vacuum/sweep/mop/wipe, you will be required to scan your room and Choremon will randomly put coins on the floor for you to collect while doing your chores. Additionally, we have another game mode which is called "Tile Mode". The difference with "Coin Mode" is that "Tile Mode" lets you create a zone by placing points on the corner of the room. Once the zone is created, it will be highlighted green and when your vacuum goes on top of it, it will delete a part of the highlighted zone. This mode has a progress bar to see which part that you still need to vacuum.

As for declutter/throw trash, you will be asked to take a picture of the mess before you throw it into the trash. Choremon will then analyze it to determine what are the trash that needs to be thrown away. After that, Choremon creates a list of the trash and you can complete the check list or ignore some elements if you think it is not trash.

Finally for laundry, we created a check list for steps starting from putting your clothes into the washing machine to folding it after it gets out of the dryer. In order to get the XP, you need to complete all of the steps and click "Finish Quest".

Once you complete all the chores, your streak will be updated as well as the XP progress bar in the home page. Furthermore, you can check the family leaderboard. Streaks multiply your rewards and push you up the family leaderboard, which makes it competitive between roommates, siblings, or households. For kids, that leaderboard is everything. For college students, it turns a dreaded task into something that at least feels worth doing after a long day. The core idea is that Choremon removes the two biggest friction points: remembering what needs to be done, and finding the motivation to do it.

How we built it

Choremon is built on multiple layers of technology working together. For starters, the main app lives in Next.js 14 with the App Router, React 19, Tailwind CSS, and Framer Motion for animations, deployed on Vercel. The moment you land on the dashboard, you see your streak, level, and XP bar and Rascal greets you. From there, a chore selector routes you into one of four modes: Trash, Laundry, Vacuum/Sweep, or Mop/Wipe.

The AR features are where things get more interesting. We built two distinct AR game modes, and they're actually built on completely different technology. Coin Mode runs entirely in the browser. Using WebXR + Three.js, we perform a floor hit-test to anchor real 3D coins to your actual floor surface in augmented reality. You physically walk through the room collecting them as you clean, and gathering every last coin triggers a quest complete fanfare. It works on any AR-capable Android browser. It also works on iOS, however we need an extension which is an app called XRViewer.

Tile Mode is a native Unity APK built in C# using AR Foundation + ARCore, sideloaded onto the demo device. The flow starts with the web app firing window.location.href = "choremon://ar?mode=tile", then Android opens the pre-installed Unity APK, and Unity reads Application.absoluteURL to get the mode parameter. From there, ARCore detects the horizontal floor plane, renders it as an overlay mesh, and the user taps to confirm it. Unity then calculates the total surface area in m² from the plane boundary vertices and sets it as the cleaning target. The floor is divided into a grid of virtual tiles, each one starts highlighted (dirty). As you physically walk over the area, the camera and device position are projected onto the floor plane each frame, tiles within radius are marked clean and the highlight disappears in real time, and a coverage percentage is calculated as cleaned tiles divided by total tiles in real time.

For trash and declutter detection, we built a multi-layered AI vision pipeline. The primary model is the Gemini API, which analyzes your photo and identifies what qualifies as trash. If that call fails, we fall back to FeatherlessAI running a fine-tuned Gemma model. If that also fails, we reroute through OpenRouter with the same model as a second safety net. On top of that, we layer NVIDIA's vision models, specifically nvidia/llama-nemotron-embed-vl-1b-v2 and nvidia/nemotron-nano-12b-v2-vl, for more precise object detection and classification. The result is a resilient pipeline that generates a checklist of items to throw away, which you can complete or dismiss based on your own judgment.

For laundry, there's no camera involved, just a carefully designed 8-step checklist walking you from sorting through putting clothes away. Each step has its own XP value: Sort (+5), Load (+10), Detergent (+5), Wash cycle (+15), Move to dryer (+10), Dry cycle (+15), Fold (+20), Put away (+20), totaling a max of 100 XP. Rascal fires commentary at steps 1, 2, 4, 5, 7, and 8 to keep you moving.

The audio layer is built around Rascal, our AI companion with a full voice architecture. His lines are pre-written per chore category: greeting, general, dishes, laundry, sweeping, trash, and idle, that is tracked in a set so he never repeats himself until every line in a category is exhausted. What makes him feel alive is the emotion tagging: lines are written with inline tags like [annoyed], [sarcastic], [sighs heavily], and [passive aggressive] integrated directly into the strings, passed raw to ElevenLabs eleven_multilingual_v2 using the Finn voice. Voice settings are tuned carefully: stability 0.35, similarity boost 0.75, style 0.6, speaker boost on. The useRascalChatter hook fires his first quip 8–15 seconds after a chore starts, then continues randomly every 25–60 seconds via chained setTimeout calls. If you stop moving which is detected via the devicemotion API watching accelerationIncludingGravity magnitude drop below 1.5 for 12 seconds, Rascal will call you out. Sound effects for coin collection and quest fanfares are generated using the Web Audio API with a sine-wave oscillator at zero latency.

Challenges we ran into

First of all, it would be sleep. We didn't get much of it. Our brain is running at half speed, and the fact that we're debugging a Unity APK at 3am wondering if the floor plane detection is broken or if you're just too tired to hold the phone steady. The biggest challenge of this entire project was simply executing on an ambitious idea under a brutal time constraint.

The second major challenge was figuring out which AR technology to even use. We went back and forth for a long time on this (WebXR, AR.js, MindAR, 8th Wall, Unity, Kivicube, React Native with ViroAR) as each one had tradeoffs in terms of device support, ease of integration, and what we could actually pull off in the time we had. Settling on the right stack for each mode took longer than we'd like to admit, and it ate into precious build time as we tried each one.

Once we committed to Unity for Tile Mode, that became the single longest part of the entire build. Getting AR Foundation and ARCore to reliably detect a floor plane, divide it into a grid of virtual tiles, project the device position onto that plane each frame, and track coverage percentage in real time was genuinely hard. The deep link handoff between the web app and the Unity APK added another layer of complexity, making sure choremon://ar?mode=tile correctly launched the APK and that the URL callback on completion properly passed coverage data back to the web app for XP to be awarded. Then there was iOS. Neither of us has a MacBook, which means no Xcode, which means no ARKit, which means WebXR simply doesn't work on Safari out of the box. Our workaround was XRViewer, Mozilla's experimental WebXR browser for iOS, which let us test the browser-based AR experience without needing a Mac in the loop. This also means that since there are no ARKit, we need to make our own AR in Unity and that's why it took us the longest time. It works, but it's not exactly a smooth path, and it added friction every time we needed to verify something on an iPhone.

Finally, model reliability was a constant battle. When you're chaining together Gemini, Gemma, ElevenLabs, TensorFlow.js, and a Unity APK, the failure surface is enormous. We learned the hard way that if any one model is down or slow, the whole experience breaks. That pushed us to build out a proper fallback chain (if Gemini fails on room analysis, we serve mock data; if the primary trash detection model is unavailable, we route through backup models) but designing and testing all those fallbacks was a significant chunk of work we hadn't fully anticipated at the start.

Accomplishments that we're proud of

We did all of this without a single energy drink or cup of coffee, somehow. No caffeine, no shortcuts, literally just water, willpower, and the quiet fear of missing a deadline. In a hackathon culture that practically glorifies sleep deprivation and energy drinks, we're genuinely proud of that. We tried to be more healthy about it, and somehow we still shipped.

We're also proud of the fact that we wrapped up all the technical work with 10 hours to spare, and we used every minute of it. That buffer went into writing the video script, planning and filming the pitch video, preparing for the face-to-face presentation, and putting together this Devpost. Building something is one thing. Being able to clearly explain why it matters and how it works to a room of judges is a completely different skill, and we're glad we gave ourselves enough time to do it properly.

On the technical side, the accomplishment we're most proud of is the sheer breadth of what we actually got working. We shipped four distinct chore modes, each built on a different technology stack, which are: WebXR, Unity AR Foundation with ARCore, Gemma 3 vision, and a guided checklist system. Somehow, they all work. The deep link handoff between the Next.js web app and the Unity APK, where the web fires a custom URI scheme and gets coverage data back via a URL callback, is the kind of thing that sounds straightforward on paper and absolutely isn't in practice.

Finally, we're proud of Rascal. He started as a nice-to-have and ended up being the soul of the product. The emotion-tagged voice lines, the gyroscope-based idle detection that calls you out when you stop moving, the per-category line tracking so he never repeats himself, none of that was strictly necessary, but all of it makes Choremon feel alive in a way that a chore app has no right to feel.

What we learned

Coming into this hackathon, we were already stepping outside our comfort zones.

For Winner, this was his first time ever touching Unity and building anything in augmented reality. A week ago, AR Foundation and ARCore were just words he'd seen online. By the end of this hackathon he was projecting device positions onto floor planes and calculating tile coverage in real time. It wasn't pretty at first, and there were hours where he genuinely didn't know if it was going to work, but there's something about a hard deadline that forces you to figure things out faster than any tutorial ever could.

For Farell, this was his very first hackathon. Jumping straight into a 44-hour build with multiple moving parts, tight integration between systems, and the pressure of a live demo, that's not exactly a gentle introduction. But he handled it, and watching someone navigate their first hackathon experience is a reminder of how much you can grow in a compressed period of time when you're just thrown into it.

Together, we learned tools we had genuinely never heard of before this hackathon started. WebXR, AR Foundation, FeatherlessAI, TensorFlow.js COCO-SSD, Framer Motion and the list goes on. A week ago these were foreign names. Now they're things we've actually shipped with, debugged at 3am, and built real features on top of. Maybe the biggest lesson of all though is this: you don't need to know everything before you start. You just need to be willing to figure it out as you go.

What's next for Choremon

We're just getting started. The feature we're most excited about is real-world brand collaborations. Imagine grinding your XP all week (vacuuming, doing laundry, taking out the trash) and hitting a milestone that unlocks a free scoop at Ben & Jerry's. Not virtual coins. Not a badge. Actual ice cream. We want Choremon to be the bridge between keeping your space clean and getting rewarded in the real world, partnering with brands to make high XP streaks mean something tangible. For kids, that's the ultimate motivation. For college students, free food is basically a currency.

Beyond that, we want to grow Choremon in a few directions:

Choremon companions --> raise a virtual pet that evolves as your household keeps up with chores. Neglect your quests and your companion gets sad. Keep your streak alive and it levels up alongside you.
Streak multipliers and weekly boss challenges --> a "Disaster Kitchen" boss that requires the whole family or dorm floor to coordinate and chip in XP together (could be cleaning up after cooking or after having a big feast).
A proper iOS build --> now that we know what we're doing with AR Foundation and ARCore, the next step is an ARKit version so iPhone users get the full native "Tile Mode" experience without needing XRViewer workarounds.

Smart display glasses AR integration would be nice to have as it would make household chores more efficient.

The dream is simple: a world where nobody hates chores, kids actually want to help around the house, and a clean dorm room gets you free ice cream on a Friday afternoon.