Pi-Casso | Devpost

Pi-Casso - The Accessible Solution to Drawing Through Eye-Tracking Technology
Pi-Casso Build - A 3 layered platform moving on both the x and y axis.
Eye Tracking Software - A display demonstrating the the precise location of the user's eye focus.
Website Feature - Our project featured on the Hack the North 2023 Website

Inspiration

Our inspiration for Pi-Casso came from our tedious experiences with freehand drawing in design class at school. We realized that sketching and drawing can be very challenging, even for fully able people. Therefore, we looked at HTN 2022 as an opportunity to design a system that would allow disabled people to create “freehand” sketches.

What it does

The Pi-Casso uses the “Adhawk Mindlink” vision tracking system and a 2 axis actuating bed to convert the user’s eye movements into a drawing. The vision tracking system is calibrated to detect where the user is focusing their vision with respect to a predetermined grid of coordinates. As the user changes their point of focus, the system detects the new direction. Pi-Casso will use this change in vision to create a drawing. If the user looks towards the right, the Pi-Casso will begin to draw a line towards the right. When running, the Pi-Casso will continuously track the user’s vision and create a physical image from it. To pause the system, the user takes a long blink. If another long blink is taken, the system will resume.

How we built it

First we set goals and expectations for the controls and mechanical aspects of the project’s design. We separated our team into groups of two, one for the controls and the other for the mechanical, then prioritized and delegated responsibilities accordingly. For the mechanical aspects, we began with preliminary sketches to rapidly develop several possible designs. After discussion, we settled on a specific design and began CAD. We went through multiple design iterations as new problems arose. As construction finished, we made numerous changes to our original plan because of unforeseen issues, such as failures in 3D printing.

Challenges we ran into

Our original design required a 30” x 21.5” base, which we quickly discovered was unfeasible with the resources available. We then revised the design to use a smaller base by reworking the placement of our motors.

Another challenge we faced was designing the mount for the pen. Due to severe sleep deprivation, one of our designers did not realize the measurements used were way too small, and the other team members were too tired to notice. As a result, it took 3 iterations to create a working pen mount.

Another issue arose with the availability of 3D printing. The printing of a critical design component failed several times. Since we were under tight time limitations, we elected to use stock components instead. This proved to be an inferior mechanism, but it was a necessary compromise given the time and material constraints.

Accomplishments that we're proud of

Our team is very proud to have had excellent communications between the controls and the mechanical team. Throughout the project, all team members kept each other updated on the major events and goals from the respective subsections. We are especially proud that we’ve been able to work together effectively in an intense competitive environment and through several setbacks and mistakes.

What we learned

One of the biggest takeaways from HTN 2022 is that we need to test as early and as often as possible. We lost at least 5+ hours of productive work because we based many of our designs on assumptions that proved to be faulty when actually implemented. Another key takeaway is the necessity of having strategized alternate ways of solving problems. During this build process, we often ran into issues where we kept trying to implement the original plan instead of switching to a new method. As a collective, we also gained a huge amount of technical skills. Specifically, we learned to work with eye tracking & laser cutting software and improved our abilities to CAD, rapid prototype, build and test software and hardware systems.

Our team also learned critical project management skills. Namely, we actively worked to improve our ability to analyze and break down a problem from several perspectives, allowing the problem to be systematically overcome.

What's next for Pi-Casso

We’ve identified several key areas in which Pi-Casso could be improved. Most notably, stability of the system leaves much to be desired. The instability leads to rough and often low resolution prints. Another great feature to add would be the ability to detract the pen from the paper. The necessity of having the pen in continual contact with the paper is a key limitation in making high quality prints. The Pi-Casso would also benefit from increasing the amount of control. For example, adding sensors could help create more feedback, allowing the system to create more detailed images. Increased control could also help the system avoid self damage.