Jestur | Devpost

Inspiration

After we found out that HackTJ was offering Leap Motions, we decided that we wanted to build a motion interface for computer usage. After building Jestur, we decided on the name because it uses hand gestures to manipulate the cursor.

What it does

Jestur controls your computer's desktop using hand motion/position and finger gestures. Pinch the tips of your index finger and thumb to click, pinky and thumb to right click, swipe down with your entire hand to scroll down, swipe up to scroll up, and move your fingers in a counter clockwise fashion to minimize the focused window. Moving your fingers clockwise closes the current window.

How we built it

We used Leap Motion's developer SDK, Quartz CoreGraphics, Apple Appscript, and general Python in conjunction to build Jestur.

Challenges we ran into

Many issues were encountered along the way. In the initial steps, we tried to simply implement the predetermined gestures provided by Leap Motion. However, many of these made basic functionality annoying. For instance, one gesture is a simple pointer finger jab. This instinctively can be used to left-click, but when used in that manner the cursor moves slightly and the target is no longer under the cursor. Most code was simple enough, but heavy testing was needed to find correct values for the proper calibration of the functions.

Accomplishments that we're proud of

We are proud of making an app at HackTJ from scratch and with hardware that we had no prior experience with. We were able to persevere through bugs and difficulties to produce a product we believe properly exemplifies our skills.

What we learned

We learned how to control Mac OS X desktop and keyboard inputs with Python as well as how to use the Leap Motion SDK.

What's next for Jestur

In the future, we plan on perhaps porting Jestur to C++ to improve efficiency, create a Windows port, implement cooler and more specific gestures (copy paste!!!), and improving quality of life for users.