FroggyFlow | Devpost

Website Landing Page
Notification for bad posture (over a period of time)
FroggyFlow Logo
Hardware

Inspiration

FroggyFlow is inspired by the constant bother of bad posture, back, and shoulder pain which hampers one's productivity. We want to renovate the way people are sitting in front of their laptop, desktop, or other devices while improving their productivity and health through a series of connected activities and interactions with our web application.

What it does

Our revolutionizing technology serves users through the form of a web app with the ability for users to sign in or create an account associated with their Gmail powered by Auth0 to smooth out their user experience. Our server then records user's log-in information and other associated activities on our website and store using MongoDB Atlas, including logged study-related activities during each session. With security and efficient run-time, our website allows users to experience features of one of the best productivity app as while as a wellness app available. Users are asked to utilize our hardware technology (integrated with gyroscope and accelerometer on Arduino) if they wish to, later, analyze their posture trend from the on-going productive session. Users are able to enter a productivity session with an aesthetic froggy background and non-disturbing timer countdown. Users are able to either ends a session voluntarily or complete the productive session and enters through a short rest break from their work.

• During the session, while the user clips their 3D printed case with hardware inside, user will be notified through a notification system linked to the user's account to make sure that the user sit up well IN REAL TIME. How does it work? We created training data from prelabeled data as good or bad. Then we input these feature values (xyz angle and acceleration values from the gyroscope) into a RandomForestClassifier model along with their respective labels for training. The model was able to score (accuracy score) of 98% against the test data.

• When an user ends their productive session, they are able to view their past posture trend via MATLAB graphing tool as well as analyze their past history posture progress log (with time stamps) to any past sessions associated with the account via MongoDB Atlas.

• When an user completes a productive session, they are automatically transferred to a rest period with a webcam that prompts the user to do 5 jumping jacks with a system counter. The webcam is developed with AI-generated and OpenCV framework that analyze real-time user's body motion while keeping track of the completeness of user's jumping jack to avoid any accidents by improper form.

• After the user completes their rest period, they are able to go back to a new study/productive session or go view their profile (with analyzed trends of their postures from previous sessions).

Overall, the web app is fully-responsive with many interactive features allowing users to enjoy the smoothest experience while improving their posture and productivity!

How we built it

• Hardware Integration: Utilized gyroscopes and accelerometers with Arduino to monitor and analyze user posture. In addition, we utilizes 3D printing to create customized cases for the hardware.

• Data Analysis: Employed MATLAB to generate and visualize posture trend graphs from collected data.

• Data Management: Used MongoDB for logging session data securely.

• User Authentication: Implemented Auth0 for secure user account management.

• Real-Time Monitoring: Integrated AI and machine learning with OpenCV to track body motion and ensure accurate exercise counts. Frameworks used are OpenCV (mainly Mediapipepose), Numpy, and TensorFlow.

• Web Interface: Developed a real-time counter and feedback system displayed on the webcam feed for immediate exercise validation.

Challenges we ran into

• Making the components (ML models) integrated to front-end (e.g. webcam feature in rest session took very long to configure to the web app due to different API requirements.

• We had to make sure the gyro-data ML model was accurate to be able to detect bad posture in real time and send a notification to the user.

• Making hardware work well with our software components. Firstly, 3D printing took many tries (including designing the hardware, prototyping the hardware with different versions to fit with streamlined user experience design.

• Endpoint Not Responding: The server endpoint might not respond due to incorrect URL paths or errors in routing.

• Our project has A LOT of components, and a big challenge was to finish them in the span of 36 hours. We had to downsize after ideation since we had so many ideas! Getting these components to work together was also a challenge since we have multiple backends and they may require different versions of python for example.

Accomplishments that we're proud of

• Getting the gyroscope data trained and implemented a machine learning model to determine if posture if bad or good, and notifying the user when their posture is bad

• Successfully implemented Mediapipe Pose that uses a webcamera to monitor and count the number of jumping jacks that you do during your break (they have to be proper jumping jacks!) Staying active increases concentration!

• Logged all the posture data (how well your posture was during your study session) and graphed them in Matlab after each session

• Logged data for each user from previous sessions and display this in the profile page

• Implementing FastAPI to connect all the components to the front-end, making sure they work seemelessly