ALS | Devpost

ALS

Inspiration

ALS, also known as amyotrophic lateral sclerosis, is a neurological disease where patients lose the ability to move their limbs and speak. They can only communicate by typing through eye-tracking technology. A famous patient, Stephen Hawking, is a well-known case. Even with technological support, such communication is limited to controlling a keyboard through eye movement, one letter at a time, to speak or issue commands. This game allows players to simulate the daily life of an ALS patient while experiencing the classic game Tetris.

What it does

It requires the user to use eye control to play the classic game Tetris.

How we built it

We used the technique of machine learning to train a dataset to recognize the eye movement of a user. Then the user can use the information to control the game by looking up, left, or right.

Challenges we ran into

We had difficult implementing the data retrieved by the eye movement technique to the game when we were trying to develop the game.

Accomplishments that we're proud of

We are proud of imitating the life of disabled people.

What we learned

~ We successfully converted the frames received from the camera front-end into readable information to be processed by the backend. This was a wonderful practice in front-end and back-end integration! ~ We reviewed the principles of eye-tracking: by marking 68 points on the facial images in the training set, we can locate the position of the eyes and mask other facial areas. By judging the relative position of the iris and eyelids, we can infer whether the gaze is moving left or right. ~ We encountered many difficulties along the way, but we adapted to the situation and completed this task!