ProPoser | Devpost

Inspiration

In this ever-changing world, there may be nothing eternal. However, if we had to pick one thing, wouldn't it be photos? Many of us have experienced awkward moments when we couldn't think of cute poses for pictures with loved ones. Moreover, sometimes we would freeze when we are taking a picture with someone who we don't really know! With a desire to take cute pictures, I created this web application, which recommends poses randomly based on the type of group. This idea not only solves the problem that I originally had, but also serves an opportunity to quickly get closer to people who may still feel awkward with each other.

What it does

The web application recommends cute and suitable selfie poses based on the party size (type of group such as individual, couple, family, friends, club, etc.) and assigns an accuracy score based on synchronization level.

Afterward, it allows you to save photos locally. With these photos, you can always go back and reminisce about the past. On the results page, you can view photos of other people who had the same poses as you. Since it arranges photos in descending order of scores, you'll keep striving to achieve high scores!

How we built it

We detect one or more people using the Yolov3 pre-trained model. We also used the Mediapipe Holistic model to extract their face, body, and hand poses. In addition, we used Auth0 to authenticate users for our service and store it in the database. In order to communicate with Django, we have implemented and utilized REST APIs from Next.js for seamless data processing from backend.

Challenges we ran into

We had to use the Yolo model to detect more than one person in a frame. Despite the attempts to reduce video delay by using the Yolov3-tiny or lower versions, we ended up using the Yolov3 pre-trained model because the other models did not detect multiple people properly.
Figuring out how to video stream on port 3000 while performing image processing on the backend took a significant amount of time.
Resizing collected sample photos online with different sizes (height and width) to fit the camera screen proved challenging.

Accomplishments that we're proud of

Firstly, we're proud of devising an idea that everyone found interesting and collaboratively working on it with passion.

Secondly, although the performance may not be perfect, considering the limited time, we are pleased to have implemented this feature.

Lastly, we want to commend ourselves for the growth we've experienced over the past year, evident in the projects submitted to ConUHacks last year and this year.

What we learned

Unlike last year, where we used React and Django, this year, we explored Next.js and TypeScript. It was also our first time incorporating artificial intelligence into a hackathon project. We realized that creating a project with a level of completeness suitable for a service requires more ample time.

What's next for ProPoser

We plan to gather more poses to offer a variety of random poses. Additionally, improving camera performance to provide a better user experience is on our agenda!