IntelliScroll

Accessible Chrome Browsing through Facial Recognition and Machine Learning

Comment

Inspiration

We were inspired by people with mobile disabilities In the modern era, the ability to effectively use a computer is crucial, yet there are those with physical impairments that struggle to do so. We wanted to fix this.

What it does

Our Chrome extension allows users to scroll and navigate through their browsers with simple head gestures. This provides accessibility to members of society with physical disabilities and impairments.

How we built it

We built our google chrome extension using machine learning, javascript and tensorflow.js

Challenges we ran into

Lots. Loading tensorflow models in javascript was painful due to packaging, saving, space, and loading issues. We also ran into many transpilation issues with babel and webpack.

Accomplishments that we're proud of

Our application works effectively! Any user is able to train his browser to use his/her gestures to navigate the browser.

What we learned

We learned how to develop chrome extensions, how to load and use machine learning models in javascript.

What's next for IntelliScroll

Voice commands, IOT, Audio and hands-free HCI, Facial Recognition, Customizability

Built With

Share this project:

Updates