Raja AI

Automate tickets to pull requests in minutes.

Comment 1

Inspiration

“If you're busy at work, odds are you will eventually be replaced by a robot” - Nicholas Nassim Taleb.

Observing that up to 50% of a software engineer's tasks consist of working through backlogs, fixing pesky bugs and performing chores, we realized there's a vast potential for code generation models to intervene and reshape the labor dynamics within software engineering teams.

Raja was conceived as an agent to tackle this backlog, automating a significant share of routine tickets that traditionally consumed much of an engineer's time. By shouldering this burden, Raja allows engineers to devote more of their attention to high-level tasks, where their expertise can make a profound difference. For medium to large software engineering teams, Raja is the engineering manager’s wet dream that reduces turnaround time for tickets, accelerates development sprints and ultimately, reduces developer cost.

Rather than displacing human labor, this AI-human collaboration redefines it, enabling software engineering to be more productive, efficient, and cost-effective. Raja represents an exciting step towards a future where AI and human capabilities complement each other in harmony, enhancing the software engineering process.

What it does

Raja is an AI-powered agent that transforms your Trello tickets into ready-to-review pull requests, functioning like your dedicated Junior Developer. Swiftly deciphering ticket requirements, Raja navigates your entire Git repository to generate and submit a pull request within minutes. This seamless automation not only expedites the development process but significantly enhances productivity.

How we built it

Raja has been engineered using the robust gpt-3.5-turbo-16k-0613 model, which provides a context length four times greater than its 4k base model. This enhanced context length empowers Raja to tackle extensive tasks, interpret Trello tickets, generate viable solutions, and submit pull requests seamlessly.

A crucial component of Raja's infrastructure is Pinecone, a vector database. Pinecone plays a pivotal role in storing the contents of the entire Git repository, functioning as a valuable long-term memory for Raja. By preserving semantic information in vector embeddings, Pinecone aids Raja in understanding and recalling data, crucial for executing intricate tasks.

Integrating Langchain, we created custom prompt templates that draw from this memory. Langchain, when combined with the chain of thought prompting technique, enables Raja to decompose multi-step problems into manageable intermediate steps. This technique, explored in the study “Chain of Thought Prompting Elicits Reasoning in Large Language Models,” substantially improves the reasoning abilities of our AI.

To ensure that Raja fits seamlessly into current software development workflows, we have fine-tuned it to work harmoniously with popular tools like Trello and GitHub, harnessing their REST APIs for fluid interaction. This integration facilitates the automatic transformation of Trello tickets into GitHub pull requests, dramatically accelerating the software development cycle.

Challenges we ran into

Creating an AI tool like Raja presented us with multiple challenges. One of the significant hurdles was integrating Raja with Trello and GitHub. We initially attempted to use pythonic wrappers for REST APIs with GitHub, but we quickly discovered their limitations, particularly when dealing with file changes and reflecting these changes in the pull request.

Managing permissions also presented a considerable obstacle. Balancing the need for Raja to have sufficient access with the necessity of maintaining secure and controlled environments was complex. We had to devise solutions that allowed Raja to perform its tasks without compromising security.

Another challenge arose from the task of retrieving relevant files to make changes based on the ticket requirements. Understanding these requirements and translating them into appropriate code changes was no small feat. Additionally, we had to ensure that the files we modified were the correct ones to meet these requirements.

Despite these challenges, we discovered that leveraging the chain of thought prompting technique was instrumental in guiding Raja to arrive at the correct answer. Through this method, Raja could decompose multi-step problems into manageable intermediate steps, enabling it to understand and execute tasks more effectively.

Accomplishments that we're proud of

In the span of just two days, we successfully built a proof of concept for Raja, an AI tool that serves as a virtual Junior Developer. We navigated through a host of models including StarCoder and gpt-3.5-turbo, experimenting, testing, and optimizing until we achieved the optimal balance of speed, accuracy, and efficiency in ticket-to-pull-request transformation. The accomplishment isn't just the creation of Raja, but the agility and adaptability we demonstrated in rapidly developing a viable and highly functional AI solution.

What we learned

Our experience building Raja offered us invaluable insights into constructing end-to-end systems, from front-end user interactions to the intricacies of interacting with large language models via REST APIs. As we worked to ensure our prompts returned the correct responses, we learned to fine-tune our approach, meticulously crafting and adjusting our input to achieve the desired output. This iterative learning process sharpened our skills and gave us a profound appreciation for the nuances of large language model behavior.

Beyond technical learnings, we also took away some qualitative lessons. We appreciated the value of persistence, the necessity of continual learning, and the importance of a collaborative mindset when confronting the vast and exciting landscape of AI-powered software development. Ultimately, our endeavor to build Raja reinforced our belief in the transformative potential of human-AI collaboration.

What's next for Raja

Raja is only at the beginning of its journey. As we onboard initial users, we're committed to not only improving its current capabilities but also expanding its scope and reach. We plan to integrate it with a wider array of project management tools like Asana, Linear, Notion, and Jira, making it even more useful to teams with diverse toolsets.

We're also planning to finetune Raja's performance on different types of tickets, whether they're bug fixes, features, or chores. The goal is to have Raja adapt and respond optimally to a wide range of task types, boosting its effectiveness in managing diverse workloads.

Looking further into the future, we're excited about the prospect of developing specialized agents. For instance, QA agents could test the changes made by Raja, while backend engineer agents and information systems specialist agents could identify and handle more complex, architecture-related tasks.

But we don't intend to stop there. As large language models continue to evolve and improve, we envision exploring higher-order agents like senior managers, engineering managers, principal engineers, and even research agents. These AI-powered roles could take Raja's capacity for augmenting human effort to the next level, reshaping how software development teams function.

Our ultimate vision for Raja goes beyond automating tasks to empowering humans. By taking care of repetitive tasks, Raja frees developers and product managers to focus on what they do best: innovating and creating solutions that make a real impact. As we continue to refine and expand Raja, we're excited about its potential to transform software development and redefine what it means to be a productive, innovative team in the age of LLMs and AI.

Built With

Share this project:

Updates