Inspiration
Our goal was to help Parkinson patients, who suffer from poor user experience on mobile devices. User interface design guidelines for smartphone applications for people with Parkinson’s disease
Problems
Natural touch interfaces are common:
- Accuracy way worse
- Cumbersome to select text
- Stated by Patrick Pantel in 2016 (smart select)
What it does
We help Parkinson patients with better user experience on mobile devices. The first iteration includes a smart text selection based on AI. The user has to only tap the word and we expand the selection to the desired text selection and copy it right to the clipboard.
- Smart Algorithm detects if the selected word is associated with other nearby words
How we built it
We use a Flutter mobile app, which connects to a Azure web app, which again connects to either the Text Analytics API or an Ubuntu VM with the latest Zalando Research Flair SequenceTagger running as a Flask API. The app sends the sentence and the selected position and gets the extended selection as a response.
Challenges I ran into
- Azure problems with the free plan
- Bad data for this use case from Microsoft Open Data
- Defining of deep learning task and finding suitable frameworks / libraries
Accomplishments that I'm proud of
- Built and end-to-end MVP
- Usage of Cognitive API and own deep learning API
What I learned
- The interfaces should be defined better in the beginning even in the agile approach
What's next for Select AI
- Auto Response (Smart Reply)
- Complete sentences, while writing
- Selects the words according to user preference/history
Log in or sign up for Devpost to join the conversation.