Rexflection

Remember Google's offline dinosaur game? Relive it here alongside your Google Photos! Traverse through your personal art gallery, and enjoy the stories.

Comment

Inspiration

We remembered Google's offline dinosaur game and were able to re-discover Google Photos from our childhood in this hackathon. Since the overall theme this year was nostalgia, we thought: why not make a fun memory gallery in this fashion?

What it does

Users can upload photos to the Rexflection software, which utilizes a retro "you are offline" dinosaur character to take users on a journey through their photos. The dinosaur walks through and provides descriptions of each photo in the art gallery.

How we built it

Our stack consisted of React Frontend and Flask Backend. We utilized Google Photos API to obtain the photos necessary. We also tried a YOLOv8 object detection model trained on general everyday object dataset COCO128 to generate object data on each image. Then, we fed the resulting information into an engineered Cohere prompt to generate the resulting art gallery story caption. Our UI was created with just CSS.

Challenges we ran into

A teammate got ill by the end of the first day, so they were unable to attend the rest of the Hackathon. Thus, we had to change our initial idea to accommodate for our capabilities. The majority of us were new to the tech that we worked on, so it took a bit of a learning curve at the start. Figuring out which tools would lead to the best implementation also took a while. We also ran into challenges properly implementing APIs and integrating the frontend and backend due to OS and CORS issues. At the end of the day, we overcame all of these challenges and are proud of our product.

Accomplishments that we're proud of

All the 36 hours put into the completion of our project :D It was tiring, but fun, we liked the process of working on building our idea into reality. It felt extremely rewarding to use new technologies and look back on what we learned throughout. It also looks really cute!

What we learned

We came with varying levels of experience, yet all of us were able to learn something new. Angela delved into the usage of Google Photos and OAuth, as well as the capabilities of pure CSS in UI design. Allison learned about APIs and how to test them with Postman, along with prompt engineering with Cohere. Charlie was introduced to using and training ML models, along with discovering the various resources and capabilities of ML models. All of our learnings were with the great assistance of the many mentors at UofTHacks, as well as the wonderful sponsors we had the chance to talk to.

What's next for Rexflection

The next steps for Rexflection include deployment, adding more features, enhancing descriptions and statistics, and improving various elements of the project. It could also move towards the game aspect, where it traverses through all your photos in the classic style of the game.

Built With

Share this project:

Updates