Punkify

SteamPunk theme App for Therapeutic Application

Comment

Inspiration

The journey of Punkify began with a simple yet profound idea: making therapy engaging and accessible for individuals with motor disabilities. We noticed that traditional motor skill exercises often lack excitement, making it challenging for users to stay motivated. Inspired by the captivating aesthetics of steampunk and the potential of AI-driven technologies, we envisioned an application that transforms mundane exercises into an immersive experience. Our goal was to merge fun with purpose, turning therapy into a game that users would look forward to playing.

What It Does

Punkify is an interactive application that helps users improve their hand movements through gesture-based activities infused with steampunk flair. Here's how it works:

Steampunk Image Transformation: Users can upload any image, and our app transforms it into a steampunk masterpiece while keeping the main subject recognizable. The transformation applies a palette of brass, copper, brown, and deep burgundy to evoke an industrial, vintage look.

Gesture Recognition and Interaction:

OK Gesture: Triggers a steampunk hat to appear on all detected faces in real-time. We have four unique hats that switch every five seconds, adding a playful element to the experience. Fist Gesture: Displays random, fascinating facts about steampunk culture, enhancing the educational aspect of the app.

One Finger Up Gesture: Allows users to ask our AI any question related to steampunk or other topics, providing an interactive learning platform.

Steampunk-Themed Game: We developed a game using Pygame where steampunk characters harness elemental reactions like fire, water, and electricity. Players control these elements using hand gestures, promoting hand-eye coordination and motor skills development.

How We Built It

We utilized a combination of cutting-edge technologies to bring Punkify to life:

Machine Learning: Trained a decision tree model to accurately recognize different hand gestures. Computer Vision: Used OpenCV for hand and face detection, ensuring real-time interaction and accurate placement of steampunk hats. Image Processing: Applied filters and transformations to convert uploaded images into steampunk art. Game Development: Created the steampunk-themed game with Pygame, incorporating gesture controls for an immersive experience. AI Integration: Implemented an AI assistant to answer user queries, enhancing interactivity. Challenges We Ran Into Gesture Detection Accuracy: Ensuring the machine learning model accurately recognized hand gestures was a significant hurdle. We experimented with various algorithms to find the optimal balance between speed and accuracy. Real-Time Face Tracking: Positioning the steampunk hats correctly on moving faces required precise face tracking and alignment. Multiplayer Gesture Tracking: Accurately tracking hand gestures for both player one and player two in the game was challenging, necessitating advanced tracking algorithms. Technical Issues with Integration: We initially planned to use the domain punkify.tech, but due to technical issues, we opted for punkify3.tech. Major League Hacking (MLH) is aware of this change. Accomplishments That We're Proud Of Creating an Engaging Therapeutic Tool: We're thrilled to have developed an application that can make motor skill therapy enjoyable and accessible. Innovative Use of AI and Machine Learning: Successfully integrated AI-driven gesture recognition and image transformation features. Interactive Steampunk Game: Developed a unique game that combines entertainment with motor skill development. Overcoming Technical Hurdles: Despite challenges, we managed to create a seamless user experience. What We Learned Machine Learning Optimization: Gained deeper insights into training machine learning models for real-time applications. User Experience Design: Learned the importance of creating intuitive interfaces, especially for applications in therapy. Technical Resilience: Overcame integration issues and learned to adapt quickly to technical setbacks. Potential of AI in Therapeutic Applications: Discovered new ways AI can enhance therapy and rehabilitation processes.

What's Next for Punkify

Integration with Privacy-First Blockchain (Midnight): We aim to incorporate Midnight's privacy-first blockchain to enable secure image transformations and anonymous sharing. Although we faced issues with the testnet faucet being non-operational, we plan to implement these features once the network is functional. Expanded Therapeutic Features: Intend to collaborate with therapists to develop more gesture-based exercises tailored for specific motor disabilities. Augmented Reality (AR) Enhancements: Explore AR capabilities to make the experience more immersive, potentially turning Punkify into an entry point for AR interfaces on standard hardware. Community Engagement: Build a community around Punkify where users can share their creations, progress, and support each other's therapeutic journeys. Visit us at punkify3.tech to experience Punkify yourself!

Check out our demo videos: https://www.youtube.com/watch?v=G5Jkr0Apf4g https://www.youtube.com/watch?v=Oxt569xvuZ4 https://www.youtube.com/watch?v=gu2EE_gFVZ4

Built With

Share this project:

Updates