LivingInHell | Devpost

City Page
Post Page
Post Result
AI generating City Digital Image
An event is going on in University of Waterloo
Post from other users
Interaction with other users' posts
Points Shop
User's own city where they can decorate with things they bought from point shop
Flower placed in UW Environmental Reserve
Tornado in the City!!!
User's post about Tornado, it was those posts made AI realize Tornado is in the city and pointed it out for other users

Inspiration

Every app that knows where you are gives you a list. Lists are good at telling you that a patio was busy on Friday night. They are bad at the thing we actually wanted, which is to look at our own city and see that Uptown is loud tonight while Downtown Kitchener is quiet.

The second half came from the other side of the same problem. City staff find out what residents are experiencing through forms, phone lines and complaint portals, which are some of the least enjoyable software anyone ever has to use. Almost nobody reports a fallen tree twice. So we asked whether the fun version and the useful version could be the same product: people post because building a city out of their own photos is fun, and the structured picture that falls out the other end is the thing a city actually needs.

What it does

Living City is Waterloo Campus rendered as a cartoon 3D miniature, divided along the two cities' official planning boundaries. Every block is a real neighbourhood.

You post from your phone: a photo, a caption, and optionally a few seconds of voice. Your post lands in the block you are standing in. An AI call reads it and writes a compact structured record of what that place feels like right now. A deterministic aggregator combines it with the other recent posts in that block. A second AI call decides how the block should look, and that decision is validated, clamped and handed to an engine that builds the geometry. The block rebuilds while you watch, with lights coming on, a crowd appearing, music notes floating up.

Posting earns points. Points buy decorations in a shop. Decorations go into fixed slots on any block in your own private copy of the city, so you can decorate your street without touching what anyone else sees.

There is also a government view. When a post describes something real and reportable, like flooding or a fallen tree or a blocked road, the same analysis that drives the visuals files a structured incident. City staff open a separate page, see the report with its photo and its block, and mark it verified. Nothing on that page is written by a model. It is the same analysis the city itself runs on.

How we built it

One rule shaped every decision: AI interprets reality and plans visual intent, and it ends at JSON. There are exactly two model calls on the path between a post and a block changing. Everything geometric, procedural, scored, placed, sold or reported is ordinary deterministic code.

The system is four modules plus two deterministic layers around them.

The map module keeps two layers that never mix. The city you look at is hand-drawn: twelve to twenty simplified blocks traced over the real map, built to be readable rather than accurate. Assignment uses the untouched official planning-area polygons from Kitchener and Waterloo, with point-in-polygon through Turf.js, so a post always lands where it really happened even though the visual city is a caricature.

Post processing runs the first call inline, validates the output against a versioned schema, and stores it. The aggregator is plain arithmetic: a weighted mean per block, weighted by the model's own confidence, by how much a post is really about that location, by a bonus for engagement, and by recency decay. The planning module builds an input package per block and makes the second call, and then a validator does the unglamorous work that keeps the demo alive: enum checks, composition normalized to 100, height clamped to the block's capacity, mood free to swing but identity locked unless two consecutive windows support the change. A rejected plan falls back to the previous one.

Modeling lays a regular grid over the block polygon, keeps the cells whose centres fall inside it, and fills them from the plan. Buildings by composition and density, vegetation in the gaps, one plaza cell for crowds and the stage. Everything is seeded by the block id, so the same plan always builds the same block. The renderer is React Three Fiber with flat toon materials and a raised slab per block.

There is no long-running process on Vercel, which shaped two things. Analysis runs inline in the post request behind a concurrency guard. Planning is driven by a ticker in the operator panel that replans only the blocks whose input hash changed, and the scene polls a cheap version endpoint every five seconds and rebuilds only the blocks whose plan id moved. A judge can post and watch their own block update without anyone touching a laptop.

Around that: the OpenAI API powers both calls with a strict response schema at temperature zero. Voice posts go through a Huawei OMNI multimodal call that returns a transcript plus what it heard in the background, which folds into the analysis alongside the photo. GPTZero scores caption and transcript so a synthetic report never gets weighted like a resident standing in the rain. Elasticsearch holds every analysed post with its embedding, geo point and timestamp, and a civic agent searches it to merge duplicate reports of the same event, attach corroborating posts and flag reports that disagree with each other. Sentry traces the whole path from a phone tapping post to a block rebuilding.

Challenges we ran into

Making one post visibly change a block, without letting the model redraw the city, took more tuning than anything else. Too strict and the demo is invisible. Too loose and a single sarcastic post turns a quiet neighbourhood into a festival. The answer was to split what a plan may change: mood, lighting, effects and activity move quickly, while archetype, density and building mix need sustained evidence across windows.

Handling conflict was the other hard one. Two residents post about the same block in the same ten minutes and say opposite things, and both are telling the truth about their own corner. We weight by recency, by whether the location was verified, and by corroboration, and when reports still disagree we record the disagreement instead of averaging it away.

Then the ordinary weekend problems. Browser audio records in different formats on iOS and Android.

Accomplishments that we're proud of

What we learned

The constraint did the work. Deciding early that AI stops at JSON meant every hard failure had exactly one place to be caught, and the things that usually break a hackathon demo (a model returning nonsense on stage, geometry it cannot draw, cost running away) turned into a validator, a clamp and a previous-plan fallback.

What's next for LivingInHell

Adding a city means a boundary file, a centre point and a timezone, so the next one is a data task rather than a code task. Beyond that: trend history and CSV export for city staff, live weather driving the city's sky instead of a fixture, incident markers on the blocks, and a report flow for residents who want to file something without posting about it.