Coin Grabber | Devpost

Inspiration

We have worked with simple games in the past, but we wanted to implement a vision-based API that would allow for more fun and interactive experience. We recently noticed many widespread "coin tapping" games on mobile, so we want to make something similar but more enjoyable.

What it does

When the game is played, a countdown timer displays at the top of the screen and counts down until the game begins. When the game starts, it displays the current coins captured and the time remaining in the game at the top of the screen. The player collects the randomly generated coins that bounce around the play screen by closing their hands around the coins. We use hand and finger tracking to detect when the user's hands are closed around the coin, adding points as you collect more coins. The program then displays the score and pauses at the point of finishing to get a snapshot of those last few seconds.

How we built it

We built this program almost entirely in python, and our primary tool utilized was the OpenCV API. OpenCV allowed us to track objects on the screen and changes in live camera capture. We used this to our advantage to capture hand structures and map the movements of our hands in real-time. We then programmed our game to register captures when our hand is completely closed and award points based on the coins captured in the given time. We overlaid the main game over the real-time video as someone plays, so you can interact with it more accessible and play with multiple users at once. This allowed us to combine the OpenCV tech with a fun and interactive game to provide endless entertainment.

Challenges we ran into

We were initially trying to create a keyboard game using OpenCV and hand tracking. We then decided we wanted to push forward and make an even more complex program and game. We started by converting our motion tracking to a hand tracker that could see if our hands were closed or open and where they were at the time of recording. We then created a live demo version that allowed us to track our hands as objects in real-time. This is where we ran into issues relating to the game's hitboxes and tracking. The program could not correctly track our hand motions, and overlaying the game was difficult because of all the other moving parts of the program. Accomplishments that we are proud of We were proud to accomplish a fun and working prototype of what could be a fantastic game. We played it a lot ourselves and are really happy with our results.

What we learned

We learned how to use motion tracking to capture real-time video and decipher between different body and hand structures. We also learned how to overlay a game and interact with those moving parts on the backend.

What is next for Coin Grabber

As the program is now, it is a working and fun prototype of what could be a huge game. We would love to see more development and could overlay it over many other apps like Snapchat or Instagram as a filter which could bring the program more wide adoption.