HanDrone

What if you could control drones intuitively by having them follow the movements of your hand? We used a LeapMotion in conjunction with an MX Master mouse to create an immersive drone controller.

Comment

Inspiration

We believe that the drone experience is pretty immersive thanks to FPV goggles, but you still need to control them with a regular controller which we don't believe to be a very intuitive method. This is because you must control movement in three different axis and the throttle. That's why we came up with HanDrone, to provide a more natural and fun alternative.

What it does

The basic function of HanDrone is very simple. Put your hand on top of the LeapMotion, and the drone will mirror your movements! We also offer support to control some or all of the axis through a Logitech MX Master 3 mouse, for a two-handed control experience. Our software allows adding more control methods easily as well.

How we built it

The main challenge was to hack a drone controller to send commands from the computer. We did this by modifying the firmware, which we had the source for but it had no documentation except some comments written in Chinese. We managed to hack in some code to communicate with the PC via USB (which wasn't possible before) so we could override the values sent to the drone by the controller through that interface. This way we completely bypass the controller and use it as a simple transmitter.

To use the LeapMotion and the Logitech MX Master 3 we used the manufacturer provided SDKs and adapted the different control planes they offered to use with the drone. After some finessing with the parameters, we had it working!

Challenges we ran into

During the process of hacking the controller, we faced some challenges with the firmware. We had to work with a microcontroller nobody had worked before, the comments were sparse and in Chinese, and a lot of things had to be done. We discovered that the chip had a connection with the USB controller through an USART interface, and we had to configure the chip's CSRs to use the DMA controller and read the data the PC was sending through a USB-Serial interface. That took some trial and error!

We had some set-up problems as well to find the intersection of the set of computers that could run LeapMotion with the set that could run the new Logitech SDK.

What we learned

This was a very fun mix of hardware and software challenges, and while we hustled we found out a lot about drone controllers and the protocols involved, how the different values are calculated, and how to build controllers that feel natural.

What's next for HanDrone

We have open-sourced our modifications to the drone controller firmware so anyone can take inspiration or use it to control their drone.

Possible future improvements to the project could be:

  • Adding more control methods.
  • Use the feedback from the drone camera to do something interesting :)

Built With

Share this project:

Updates