All in GPT | Devpost

Inspiration

We all love all in podcast. Join our enthusiastic hosts as they share valuable insights and expert advice on finance, entrepreneurship, and personal growth. Unlock your potential, increase wealth, and attain financial freedom. Join us today to transform your financial outlook. Go All-In!

What it does

The Allin podcast was born out of a deep passion for the chosen subject matter, a desire to share knowledge and expertise, and a longing to connect with like-minded individuals. Inspired by successful podcasts and the potential to create a platform for meaningful conversations, the hosts embarked on a journey to educate, empower, and entertain their audience.

How we built it

We utilized the Cockatoo API to transfer audio data into CSV (Comma-Separated Values) format. The Cockatoo API offers powerful functionality for converting audio files into machine-readable data, enabling efficient processing and analysis.

Once the audio data was transformed into CSV format, we proceeded to save the resulting scripts into the Pinecore vector database. The Pinecore vector database serves as a storage system for various types of data, including text. By saving the scripts, we ensured easy access and retrieval of the information at a later stage.

To enhance the prompts we sent to the ChatAPI, we integrated the Featureform API into our workflow. The Featureform API facilitated the retrieval of relevant information from the stored CSV data. By extracting key features such as speaker identification, sentiment analysis, or speech patterns, we obtained valuable insights from the audio scripts.

Next, we embedded the extracted information into the prompts we sent to the ChatAPI. This embedding process involved incorporating the relevant details retrieved through the Featureform API into the prompt text itself. By doing so, we provided contextual information to the ChatAPI, enabling it to generate more informed and accurate responses.

Finally, the enhanced prompts, containing the embedded information from the Featureform API, were sent to the ChatAPI. The ChatAPI, powered by the ChatGPT language model, leveraged this input to understand the context and generate human-like responses. By utilizing the Cockatoo API, Pinecore vector database, Featureform API, and ChatAPI together, we created a comprehensive pipeline that transformed audio data into actionable prompts for engaging conversations.

Challenges we ran into

At first, we wanted our API to distinguish between different voices from different speakers. We had a really hard time distinguishing different speakers when they speak over each other. We tried several APIs from Microsoft to Azure, and it took us a lot of effort to integrate different APIs.

Accomplishments that we're proud of

We successfully shorten the processing time of requests by utilizing pre-trained LLM. Instead of a large text training, we extract keywords from the client's question and search them in the pre-built online vector library.

What we learned

We managed to make efficient use of APIs, including Featureform, Pinecorn, Openai, and e.t.c. We learned the tremendous potential of LLM and some good applications of these models.

What's next for AllinGPT

AllinGPT will be able to provide precise answers according to specific speakers. It will also be used for educational purposes to help students more efficiently learn from lecture recordings.