Croak Quest | Devpost

Inspiration

Our project was inspired by the desire to help others learn sign language in an engaging, interactive way. We wanted to create an experience that not only teaches sign language but also enhances motor skills and hand-eye coordination through gaming

What it does

The game allows players to interact with the platformer using real-world hand gestures, facilitated by the Leap Motion controller. While playing, users can learn and practice sign language, improving their hand-eye coordination, motor control, and dexterity.

How we built it

We built the game using Unity and C#. The Leap Motion controller was integrated into the system, allowing users to control in-game movements using hand gestures. We used version control and collaboration tools to build this project together, although we encountered some challenges along the way.

Challenges we ran into

Version control: Managing the codebase and assets was difficult at times, especially with multiple developers working on different aspects of the project.
Collaborating on one project: Ensuring seamless collaboration and communication between team members while maintaining individual tasks was a challenge.
Integrating hand gestures with character movement: Mapping Leap Motion hand gestures to control in-game actions was a complex task, requiring fine-tuning of input recognition and movement translation.
Setting up our dev environment: Establishing a consistent and functional development environment took some time, particularly when configuring Leap Motion with Unity.

Accomplishments that we're proud of

Successfully integrating Leap Motion into our movement system, allowing for realistic and responsive - character control based on hand gestures.
Creating a creative and original approach to teaching sign language while also helping players improve their motor skills and dexterity.
As first-time game developers, we’re proud to have successfully built and launched a full-fledged game, overcoming technical and creative challenges along the way.

What we learned

C#: We learned how to write code in C#, focusing on game mechanics, logic, and integrating external hardware.
Integrating external devices: Working with the Leap Motion controller gave us hands-on experience in integrating external devices with Unity.
Game development: We gained a deeper understanding of the game development process, including design, coding, and debugging.
Collaborating on one project: Through collaboration, we honed our communication skills, learning how to divide tasks and bring our individual efforts together into a cohesive project.

What's next for Croak Quest

More levels: We plan to expand the game with additional levels that incorporate more ASL signs, creating an even more educational experience.
Integrating ML models: We aim to integrate machine learning models to better recognize hand gestures, improving the accuracy and responsiveness of sign language recognition.
Different modes for teaching: We’re considering adding various modes to the game, such as beginner, intermediate, and advanced, to cater to different learning speeds and techniques for teaching sign language.