Airi - Your AGI Secretary

Inspiration

• Originally thinking of personal coach (for gym, singing, programming, etc)
• POC is aggregating today's information --> B2B potential for the AGI secretary, with lots of potential for GPT plug-ins to support the enterprise customer's multiple third-party integrations

What it does

Airi - Your AGI Secretary!

• a 3D "talking head" avatar
• performs text-to-speech and outputs it with a natural voice, supporting plug-ins like AWS Polly (default), Eleven Labs (once its text-to-speech becomes much faster...!)
• utilizes text content powered by ChatGPT + Plugins
  • weather data
  • horoscope data
  • news data

How we built it

• Made "plug-ins" (API endpoints) to be used by AutoGPT to shortcut the common Airi functionalities.
• Get user profile data from MongoDB + inject it into the prompt to provide personalized context, and hence personalized responses!
• Unity to generate 3D "talking head" avatar
• AWS Polly (default) and Eleven Labs (tested and gave up) for text-to-speech output

Challenges we ran into

• We did not have ChatGPT Plugins access, so used AutoGPT.
• Issues getting JSON POST working on Unity, switched to query form variable.

Accomplishments that we're proud of

• Get AutoGPT working with the helper functions we write
• A suite of useful POC helper functions, which can be easily made into a single plug-in.
• Generalizable 3D "talking head" interface which could work with any text-based input! So any kind of conversational responses, which many plug-in outputs could be converted into.

What we learned

• The promise offered by GPT Plugins is incredibly exciting, even without Plugins access.
• Lots come down to prompt engineering still, to have the user context properly injected + AutoGPT fulfill the request as expected.

What's next for Airi

• Support additional roles -- additional 3D avatar and response creation patterns
  • e.g. a gym coach, eg for strength training, marathons, whatever -- would ask about the user's fitness profile + inject that into prompts for customized fitness recommendations + generate a dynamic strength training plan

Built With

• chatgpt
• python