VibePrompting | Devpost

Landing
Prompt training

Inspiration

With bigtech embracing AI agentic models in their workflow and working towards seamless integration with the infrastructure, there can be many errors during this phase. We wanted to work on this very phase of shifting and deploying AI models and also not break production while doing it.

There are several cases wherein agents do not perform to their full potential due to reasons completely unrelated to the design or deployment but due to improper prompting. Even though it may seem like debugging these cases can be straightforward, our team has experienced designing near perfect workflows (of course from a human standpoint) and can still have erratic results due to semantic differences and language processing errors. That is how we thought of VibePrompting. We aim to automate the entire phase of designing agentic workflows.

What it does

VibePrompting is a dashboard which takes as input the AI model that the user is working on, and works on refining the workflow to help the AI model perform better. Our product has the following features:

Real time AI agent refinement: On uploading the agent to the platform the user can see the workflow and input commands to customise the agent without the additional burden of overcoming precise prompting.

Enabling editing different configs of the agent: Our platform allows the user to revise specific part of the field of the json config of the agent thereby giving the user more control and flexibility and mimicking the experience of working on the json itself rather than a black box intermediary which is not the best experience for devs.

Version control and activity logs: We enabled version control features like branching and building off from different branches so that the dev can easily test out models made from the different branches rather than struggle with figuring the most optimised prompts which give the best results. The platform even has a log history highlighting the different changes made for better UI.

Reinforcement learning with workflow performance metrics: We used RL to analyse the performance of different agents designed by the user and automatically improve the agentic prompts under the hood.

How we built it

We built a robust frontend using Next.js, React, and TypeScript. The responsive frontend features an interactive graph for managing prompts, an AI-powered interface for modifying the prompts, and a JSON editor. The interface is styled with TailwindCSS and uses Framer Motion for smooth animations.

The backend server is built with Flask and Python, which is designed to handle the infrastructure logic, version control, and branching for agent workflows. The backend provides API endpoints that integrate with Gemini API and custom Gemini agents to refine prompts and apply reinforcement learning based on feedback loops to improve user prompts.

It features a versioning system with branching and history logs, allowing users to track and manage different agent versions. The server seamlessly interacts with Google ADK, enabling users to run tests, refine their agents, and switch between the dashboard and ADK for custom evaluations.

Challenges we ran into

Managing complex state: One of the major challenges was creating our own version control software for tracking changes. This meant maintaining history of the different changes in the activity log. We needed to work on maintaining the different states of the agentic workflows as well as a way to quickly and seamlessly transition between them. The logic behind this took a lot of time to streamline and develop.

Integration with google ADK: While working on this platform there were a lot of design considerations in terms of the dev kit to be used and how we could integrate it into our platform. One of the errors we faces was triggering the google adk every time the user changes the workflow.

Accomplishments that we're proud of

Finishing our project in 24 hours 🥳
Integrating reinforcement learning with real-time prompt refinement
Implementing our own version control system

What's next for VibePrompting

The next focus for Vibe Prompting is to step out of the MVP and scale the infrastructure to enable traffic and deployment. Currently, our platform allows agent development in the Google ADK framework but we will be working on compatibility with more frameworks. We will also be working on making the version control more seamless and improve the state changes to make them faster.