NeuralNurse | Devpost

Inspiration

The NeuralNurse project is an innovative initiative designed to assist individuals with dementia by enhancing their ability to recognize familiar faces and remember their relationships with those people.

What it does

At the heart of this system is a Raspberry Pi computer, which works in conjunction with a camera and a microphone to observe and interact with the user's environment. The camera, equipped with cutting-edge YOLO and OpenCV technologies, scans the surroundings for faces. YOLO, a state-of-the-art object detection system, quickly identifies human faces in the camera's view, while OpenCV helps in recognizing these faces by comparing them against a pre-existing database of known individuals. This database stores crucial information linking each face to a name and their relationship to the user, such as "Peter, your son."

Simultaneously, the microphone listens to the user's spoken words, converting them into text for the system to understand and respond to queries or commands. This feature is particularly useful for users to interact naturally with the NeuralNurse system. Upon recognizing a familiar face, the Raspberry Pi engages its text-to-speech capability to inform the user of the person's identity and their relationship to them. For instance, if the system identifies Peter, the user might hear a reminder saying, "Hello Mary, I'm Peter, your son." This prompt aims to support users with dementia in social interactions, making it easier for them to connect with friends and family members despite their memory challenges.

How I built it

The system is built using a Raspberry Pi computer, a camera, and a microphone. YOLO (You Only Look Once) and OpenCV technologies are integrated for face detection and recognition. A pre-existing database of known individuals is created to store information about familiar faces, including their names and relationships to the user. The microphone and speech-to-text functionalities enable natural interaction, while text-to-speech capabilities provide verbal reminders to the user.

Challenges I ran into

One of the primary challenges was ensuring accurate and quick face recognition in varying lighting conditions and environments. Integrating YOLO and OpenCV for seamless operation was also complex. Additionally, creating a robust database that correctly links faces with names and relationships posed a challenge, as did ensuring the microphone accurately converts spoken words to text.

Accomplishments that I'm proud of

Successfully integrating YOLO and OpenCV technologies for efficient face recognition and creating a reliable system that supports users with dementia in maintaining social connections are significant accomplishments. Developing a user-friendly interface that allows natural interaction and providing real-time verbal prompts are also notable achievements.

What I learned

Throughout the development of NeuralNurse, I learned about the complexities of implementing real-time face recognition and the integration of various technologies to create a cohesive system. Additionally, I gained insights into the challenges faced by individuals with dementia and the ways technology can be harnessed to improve their quality of life.

What's next for NeuralNurse

Future developments for NeuralNurse include expanding the database to include more familiar faces and relationships, enhancing the accuracy of face recognition in diverse environments, and improving the system's ability to understand and respond to more complex user commands. Additionally, exploring ways to integrate the system with other assistive technologies for a more comprehensive support solution is a key goal.