Project Baymax | Devpost

Thumbnail
Tools and Technologies
The Team!
CoHere API

💫 Inspiration

Big Hero 6, was both an inspiration and a goal for us ever since we first watched the movie a decade ago. Considering the leaps and bounds technology has made in the past few years, Baymax is becoming closer and closer to becoming a reality.

😮 What is Project Baymax

Project Baymax, based on Baymax himself is the base of the Artificial mind and Voice that can interact with the user. Designed as the bedrock of an artificial mind and voice system, it aims to revolutionize the medical relations of patients through seamless communication and empathetic understanding.

🔨 How we built it

Each of us implemented an Artificial Intelligence API that focused on 2 different aspects of this project. We utilized Play.ht voice models to clone and train the perfect AI voice using actual feed from the film. For the mind and executive functions, we implemented CoHere's API and a 2-way communication script between user and bot roles, with an additional system role to reinforce the message context!

😰 Challenges we ran into

This was our first time working with AI in general and independently integrating 2 different APIs into the same project, that fed each other information based on the other's outputs. Our biggest hurdle was assembling every component to seamlessly function together, as opposed to crashing randomly. There were other small hiccups along the way:

Creating the perfect and identical AI voice model
Fitting the AI model to accurately respond to various randomized prompts with the correct style and manner of speech
Returning and handling streams of text and speech
Next.js 13 is significantly different so routing was difficult to understand and there were various package management issues.
Incorporating a 3D model of Baymax!

😤 Accomplishments that we're proud of

Successfully cloning Baymax's Voice Model !
Successfully training Baymax's Brain Model to 98.33% !

🧠 What we learned

While the AI model has been fine-tuned to embody the persona of Baymax, it does it a bit too well... Initially, the model was under-fitted as it did not resemble Baymax's comforting health care providing personality. Tampering with the model's parameters allowed for a better and more precise Baymax tone, however it lead to overfitting, meaning it responds well with predictable inputs, however lacks the necessary flexibility for untrained & real-world data. With a bit of tampering and some deeper studying on parameter optimization, we can both become better developers and train models to balance precision and flexiblity.

In addition, there was a great grievance using the new Next.js version since much of the syntax and server-side/api architecture has changed. Along with the troubles of learning the new version, the package manager could not manage Cohere and PlayHT. This is where we can take the next big step and find different front-end/full-stack solutions such as Nuxt.js or Sveltekit to expand our horizons.

💙 What's next for Project Baymax

The current speech model still needs to be appropriately fitted and tuned and the voice model needs to be trained to keep the tone consistent and clearer. In the future, we hope to recreate Baymax at his finest, truest and cuddliest form. With hardware and hard work, we'll give him eyes to scan his patients and be the best AI healthcare provider he can be just like Tadashi would've wanted.