KinectWatch | Devpost

Inspiration

I saw a streamer one day who was playing overwatch in a really unconventional way and wanted to try out something similar.

What it does

It uses gestures to play the game Overwatch.

How we built it

We used C# and the corresponding kinect libraries to work together. Then we trained "sets" and created one database to compare and contrast between values we have stored under the database versus what gesture was being used in live feed. Then those gestures were hotkeyed to a specific key map via making virtual key presses and playing the game without actually having to use the keyboard or the mouse.

Challenges we ran into

The lack of resources and the newness of the technology for creating the database made it extra hard to set up as it asked for very specific instructions than have more freedom.
The inconsistency of what we have found for sample codes to reference to versus the library as it had a different way to set up the same thing.
The entire difference in the library between the Kinect V1 versus the V2 was marginal such that there is absolutely no way to integrate them both to work.
A lot of libraries dont work together as well as we wanted so we had to think of different ways to try out different things. ## Accomplishments that we're proud of The game runs (although not with the most accurate gestures, we can see it working) ## What we learned I have learned that it is really really hard to make something from scratch especially just by only relying on libraries without the useage of samples or trying to find resources for some languages (depending on what we were making) ## What's next for KinectWatch We will try to make the machine learning of the database for gestures more accurate and refined so it wont confuse itself with other gestures working together.