TakeOver 1600-SF22

Co-located MR cooking game built with the Meta XR SDK using shared anchors, hand-tracking, spatial audio, and Photon Fusion to turn any room into a chaotic kitchen serving hilariously picky customers.

Comment

Order’s Up

Inspiration

We knew from the start that we wanted to build something co-located: an experience that feels better when people are physically together, not just online. For us, mixed reality isn’t just placing objects in a room. It’s about creating shared presence, laughter, and chaos, where the people in the room matter as much as the digital content.

Early brainstorms ranged from a bomb defusal puzzle to a *stacking challenge, but the pattern was clear: we wanted something intuitive, physical, and instantly fun. We wanted a game anyone could understand in seconds, where the learning curve is tiny and the chaos curve is huge.

A key inspiration came from a teammate who wanted something he could play with his kids. We needed something simple enough to learn instantly, playful enough to laugh over, and impressive enough to say “again!”. That became our north star: build a co-located experience where anyone can jump in and have fun together.

The result is Order’s Up: a mixed-reality cooking game where players grab scattered ingredients, chop food with full hand-tracked gestures, throw bacon across the room, and collaborate under pressure.

What It Does

Order’s Up transforms any room into a shared mixed-reality kitchen. When the game starts, ingredients appear around the room like tomatoes, bread, cheese, lettuce, and meats.

Players use hand-tracking shortcuts to pick up utensils (a knife and a pan), prepare ingredients by chopping and frying them, and assemble fully custom burgers and sandwiches. Customers appear and place orders that print on virtual receipts. Players work together to find, prepare, and assemble ingredients as per the orders. When dishes are ready, the customers rate them.

Your job is to:

  • Read the order ticket
  • Assemble the dish based on requirements
  • Match the dish to the correct customer
  • Satisfy the picky customers

Customer ratings depend on:

  • Ingredient correctness
  • Preparation level
  • Assembly accuracy

The fun isn’t just scoring points but the physical comedy of cooking together, yelling instructions, and panicking under time pressure.

How to Run Orders Up

Setup

  1. Build the project to Meta Quest headsets.
  2. Put the build on all headsets that will be used.
  3. Start the game on one headset. This headset becomes the host.
  4. Other players join by launching the game while standing near the host in the same room.

Starting the Game

  1. All players put on headsets.
  2. Wait for the voice line that says Orders Up.
  3. The game begins when everyone hears the phrase.

Core Interactions

Hand Gestures

  • Knife gesture
    Right hand forward like a karate chop, thumb pointing forward, fingers together. This turns your hand into a knife for chopping.

  • Pan gesture
    Fingers and thumb closed together, palm facing up. This creates a frying pan in your hand.

Using the Kitchen

  • All ingredients are interactable. Pick them up, chop them, and cook them.
  • Go to the order printer and press the top button to print a ticket.
  • Follow the ticket to collect ingredients and prepare the dish.
  • Assemble ingredients on a plate.

Completing the Recipe

  1. When the dish is ready, place the printed recipe ticket on top of the completed plate.
  2. The game will convert it into the finished meal.
  3. Move fast and work together.

How We Built It

We built Order’s Up using the Meta XR SDK and their Building Blocks as the foundation for mixed reality features:

  • Colocation via shared spatial anchors so every player sees the same virtual kitchen mapped into their physical environment
  • Hand-tracking and pose detection for grabbing, slicing, flipping, and throwing without controllers
  • Meta Spatial Audio for positional kitchen sounds: sizzling pans, chopping, and customer reactions
  • Photon Fusion Networking to synchronize ingredients, utensils, and interactions across headsets in real-time

Each ingredient has around 3 states: raw, prepared, burnt. (Choppable ingredients do not get burnt!)

Each state changes how the ingredient looks, scores, and reacts.

Additional systems include:

  • Physics-based ingredient throwing
  • Collision-aware sandwich assembly
  • Customer spawning and rating logic
  • Timer-based difficulty scaling

Together, these create a full start-to-finish loop: prep → assemble → match ticket → customer rating.

Challenges We Ran Into

The first major challenge was co-location. Early prototypes showed drift and anchor inconsistencies, which caused ingredients to misalign between players. We had to learn:

  • Which objects required strict network synchronization
  • Which could live locally
  • How to minimize network traffic while staying responsive
  • How to pass a simple cube back and forth

We also underestimated the difficulty of gesture reliability. Detecting the difference between a grab, slice, and throw requires tuned thresholds and noise smoothing. Hand-tracked design is only fun when interactions feel forgiving and responsive.

Lastly, building an end-to-end experience under hackathon time meant stripping down mechanics to what was instantly readable: > “If you can grab it, you can cook it.”

TLDR: Simple design is the hardest design.

Accomplishments We’re Proud Of

  • A fully working co-located MR game built in a weekend supporting up to 20 users
  • A complete game loop: prep → assemble → ticket match → rating
  • Hand-only interactions: no controllers, no menus
  • Real-time networking with Photon Fusion
  • Stable shared spatial anchor for a full session

The moment someone catches a flying tomato and yells “Cut this!” at their teammate is the exact magic we were aiming for.

What We Learned

We learned that co-location is the experience. When multiple players share a real space with virtual objects, the game stops feeling like VR and becomes a shared physical game that just happens to use digital ingredients.

Building Order’s Up taught us practical lessons about mixed reality as a product and as a technical discipline. The experience required us to think about spatial computing as a system that blends human behavior, networking constraints, and real world environments.

The strongest response came from shared moments. Two people arguing about how to cook virtual bacon created more engagement than any shader or physics improvement. Mixed reality is most compelling when it creates a feeling of being together in a world that exists between physical space and digital logic.

What’s Next for TakeOver

We’re excited to continue development with features like:

  • More utensils (spatula, grill, toaster, blender)
  • Procedural customer personalities and orders
  • Ingredient physics “chaos mode”
  • Multiplayer scoring and leaderboards
  • Custom dishes and community recipes
  • Cosmetic upgrades for chefs
  • Interactions between chefs (like high fives!)

Long term, we see Order’s Up as a platform for social mixed reality. The core loop, grab and prepare and deliver, is simple enough for anyone to learn in seconds, but the underlying systems have the depth to keep people laughing, arguing, and trying new strategies. Our goal is to grow the experience into something families and friends can pull out in any room for a shared moment, and to show that co-located MR can create real engagement in a casual setting where the fun comes from being together.

Built With

Share this project:

Updates