Ninhando

Play arcade classic using just your hand gestures! Load up your favorite retro arcade game, move your hands for the webcam, and enjoy :)

Comment

Inspiration

Ninhando is a new way to play old games! Play your favorite arcade classics using just hand gestures.

What it does

Ninhando uses your web cam to detect hand poses and then maps them to NES controls in realtime. These inputs are sent into an emulator running inside the web app, allowing you play the game without touching your keyboard!

How we built it

  • Tensorflow.js hand pose detection model
  • Javascript NES emulator loading ROMs using JNES
  • Everything inside a React web app

Challenges we ran into

Running everything in real time in the browser. This took a lot of optimization :)

Accomplishments that we're proud of

Creating a general system that can work for any game and set of poses. Being able to serve all of this in the browser.

What we learned

  • Be careful with useEffect in React.js
  • Google's Teachable Machine is very cool
  • Javascript NES simulators exist

What's next for Ninhando

  • Multiplayer!
  • Full body pose detection for games like Mortal Kombat
  • Custom poses

Built With

Share this project:

Updates