TL;DR

Severe neurological conditions such as ALS, advanced MS, brain strokes, and high spinal cord injuries often paralyze the muscles required for speech, stripping patients of their ability to communicate. While specialized AAC and BCI devices can help, they frequently cost over $10,000 and require extensive training.

Our goal is to build an accessible alternative using the $200 consumer-grade Muse 2 headset. By analyzing data from the TP9 and TP10 sensors behind the ears, our system classifies the electrical signals produced by intentional jaw clenches and eye blinks (which the Muse sensors pick up very clearly because they rest right next to the jaw muscles behind the ear). We use these specific movements to control a custom AAC sequential scanning dashboard. As the interface highlights available words, the system decodes a user's blink as the command to 'Select Word' and a jaw clench to 'Complete Sentence'.

The tricky part about using a device like the Muse 2 is that the raw EEG data is incredibly noisy, and everyone's biological signals look a bit different. Our main goal is to train a model that can reliably tell the difference between a user deliberately blinking, clenching their jaw, or just resting idly. Since this data is so personalized, we pulled an existing Muse dataset from Kaggle and combined it with our own labeled recordings. We tested out a few different machine learning approaches, including XGBoost and Decision Trees. In the end, we went with EEGNet—a neural network built specifically for this kind of brainwave data—because it gave us the best accuracy by far.

Inspiration

Imagine being fully conscious and aware, knowing exactly what you want to say, yet being unable to say even a single word. This is the daily reality for tens of millions of people worldwide living with conditions such as stroke, ALS, cerebral palsy, paralysis, or non-verbal autism. In the United States alone, more than 5 million people live with aphasia, and globally, 90 to 100 million individuals could benefit from alternative communication tools. Among people with autism, 20 to 30 percent never develop functional speech, despite having intact understanding and thoughts. The impact goes far beyond inconvenience. Research shows that individuals with severe communication impairments experience higher rates of social isolation, depression, and loss of independence. When expressing a simple sentence requires intense physical effort, precise motor control, or long delays, many people gradually stop trying to communicate. Not because they have nothing to say, but because saying it becomes too exhausting. LyoText was inspired by this silent loss of connection. We set out to build a system for people whose minds are active but whose bodies limit expression, where communication begins with intent rather than speech or movement. By combining EEG-based intent detection with AI-assisted language generation, LyoText aims to make communication faster, less effortful, and more independent. At its core, this project is about restoring something fundamental. The ability to be heard.

What it does

Hardware Interface

We use a $200 consumer-grade Muse 2 headset, specifically pulling data from the TP9 and TP10 sensors located just behind the ears.

Signal Classification

Because these sensors sit near the facial muscles, they clearly capture the electrical spikes generated by intentional jaw clenches and eye blinks. Our system detects these artifacts and converts them into actionable commands.

AAC Dashboard Navigation

These movements control a custom sequential scanning interface. As words are highlighted on the screen, the user blinks to select a word and clenches their jaw to complete the sentence.

Predictive Text Integration

To speed up communication, we integrated an n-gram language model that predicts and recommends the next likely word, similar to the predictive keyboard on an Apple device.

How we built it

We built LyoText as an end-to-end, real-time brain–computer interface that turns EEG signals into natural language communication. We used the Muse 2 EEG headset to capture brain activity and streamed the data live using muselsl over the Lab Streaming Layer (LSL) protocol. This setup allowed us to reliably collect low-latency EEG data for both live inference and offline training. The backend is implemented in Python, where a continuous processing loop listens to the LSL stream, buffers incoming EEG packets, and applies signal preprocessing. Using NumPy and SciPy, we filter the raw EEG to reduce noise and stabilize the signal, while also handling common artifacts such as eye blinks and facial muscle activity so that unintentional movements do not trigger false selections. For intent detection, we trained a lightweight deep learning model based on EEGNet, a well-established architecture for EEG classification with a small number of parameters. We trained the model in PyTorch on recorded Muse 2 sessions and balanced the dataset across idle and intentional states to avoid bias. Once trained, the model is loaded into the real-time pipeline to classify user intent directly from the live EEG stream. To make communication faster and less cognitively demanding, we designed a structured sentence-building system instead of free typing. Words are organized into grammatical groups such as subject, adverb, adjective, and action, guiding users to form valid sentences step by step. After each selection, the system generates context-aware recommendations for the next word, reducing the total number of interactions required to express a complete idea. For these recommendations, we integrated the Gemini API as an AI language engine. The backend sends the current partial sentence, topic constraints, and grammar slot to Gemini, which returns short, ranked suggestions that fit both context and structure. This approach provides intelligent autocomplete behavior while keeping vocabulary safe and relevant for assistive communication use cases. The front end is a lightweight interactive web interface that highlights options in real time and updates the sentence as selections are made. It communicates with the backend through an API layer and provides immediate visual feedback so users can understand and trust the system’s responses. Once a sentence is finalized, it is spoken aloud using text-to-speech, allowing users to communicate naturally with people around them before the system resets and continues listening for the next message.

Tech Stack:

Use FastAPI framework with WebSockets for real-time, bidirectional communication.
Python for backend and React for Front-End
Muesls package to collect data from the EEG device
Use Redis (real-time database) to store signals collected from the EEG device
Use Basic bandpass + epoching to prepare the preprocessing pipeline (used in EEG/MEG).
Inference process with the EEGNet/XGBoost/Decision Tree model to train data collected from Kaggle.
GUI with flashing interface.
Use an n-gram for word suggestions.
Use Gemini for speech-to-text.

Challenges we ran into

Working with real-time EEG data was our biggest challenge. Consumer EEG signals are noisy, highly variable between users, and easily affected by eye blinks, jaw movement, or slight changes in headset position. Balancing sensitivity and stability required many iterations to avoid false triggers while still detecting intentional selections reliably.

Another challenge was latency. The system needed to feel responsive without rushing the user, which meant optimizing the entire pipeline from signal streaming and model inference to UI updates and AI recommendations. Designing a sentence-building interface that was both structured and flexible was also difficult, especially when accounting for different communication speeds and cognitive loads.

Accomplishments that we're proud of

We successfully built a full end-to-end system that converts live brain signals into meaningful language using affordable, off-the-shelf hardware. LyoText runs in real time, integrates EEG-based intent detection with AI-assisted language generation, and produces spoken output without requiring speech or fine motor control.

We’re especially proud of designing a communication flow that reduces effort for the user by combining intent detection with smart word recommendations, making sentence construction faster and more natural. Most importantly, we demonstrated that brain-driven communication is feasible outside of lab settings and can be built with accessibility and inclusivity in mind.

What we learned

This project taught us that assistive technology is as much about human experience as it is about technical performance. Small delays, false positives, or confusing UI elements can dramatically affect usability for someone who relies on the system to communicate.

We also learned how powerful lightweight EEG models can be when paired with strong preprocessing and thoughtful interaction design. Combining brain–computer interfaces with modern AI language models opens new possibilities, but only when carefully constrained to ensure reliability, safety, and user trust.

What's next for LyoText

Next, we plan to improve model robustness through multi-user training and adaptive calibration so the system can personalize itself over time. We also want to expand language support, improve recommendation accuracy, and explore additional intent signals such as blink patterns or sustained attention for more expressive control.

Long-term, our goal is to collaborate with clinicians and patients to validate LyoText in real-world settings and evolve it into a practical communication tool for people living with severe speech and motor impairments.