TrialbyCode | Devpost

Logo
Loading Page
Code Annotations
Correct Answer
Incorrect Answer
Extra Resources

Inspiration

If you've ever programmed, we've all been there, staring at code that "should work" but doesn't, getting generic error messages that don't actually help us understand why something broke. Traditional code review feels like being handed a list of things you did wrong without understanding the reasoning behind better approaches. We wanted to create something that doesn't just tell you what's broken, but helps you discover the solution through guided questions, like having a patient mentor sitting next to you.

What it does

Trial by Code transforms your VS Code editor into an interactive learning environment. As you write code, it automatically detects potential issues - security vulnerabilities, missing error handling, type mismatches, performance problems - and presents them as challenges rather than just errors. Instead of saying "this is wrong," it asks you questions like "What happens if this user input is null?" or "How could an attacker exploit this query?" When you answer correctly, it automatically fixes the code and rewards you with XP. Wrong answers trigger educational resources like relevant tutorials or documentation.

How we built it

We built this as a VS Code extension using TypeScript, integrating multiple sophisticated systems. The core uses Google's Gemini AI for natural language processing, but the real innovation is in our RAG (Retrieval-Augmented Generation) pipeline. We implemented tree-sitter parsers to understand code structure across multiple languages, created semantic chunking algorithms to break code into meaningful pieces, and built a vector database using HNSWLib for intelligent pattern recognition. The UI is a custom webview with three tabs - active challenges, your learning profile with XP/levels, and answer history. We spent considerable time on the gamification system, implementing streaks, difficulty scaling, and educational resource generation.

Challenges we ran into

Getting the AI to ask good questions was harder than expected. Early versions either asked obvious questions or were too abstract to be helpful. We had to fine-tune prompts extensively to generate questions that were challenging but fair. The tree-sitter integration was technically complex - each programming language required custom parsing logic and different AST traversal patterns. VS Code's extension API has some quirks around webview communication and file system access that required creative workarounds. Performance was another hurdle - analyzing large codebases in real-time without blocking the editor required careful optimization of our analysis pipeline and smart caching strategies.

Accomplishments that we're proud of

We're most proud of creating something that genuinely changes how people think about their code. The "aha moments" when users realize why a certain pattern is problematic - not just that it is - feel incredibly rewarding. Technically, we're proud of the RAG pipeline's sophistication - it can recognize design patterns across different languages and identify inconsistencies in coding approaches across a project. The gamification system actually works - people want to earn more XP and level up, which keeps them engaged with learning. We also built a robust fallback system that gracefully handles API failures and still provides value even when the AI components aren't working perfectly.

What we learned

Building an educational tool taught us that good learning experiences require both challenge and support - too easy and people get bored, too hard and they get frustrated. We learned extensively about VS Code's extension ecosystem, modern AI/ML integration patterns, and the complexities of parsing and analyzing code at scale. On the technical side, we gained deep experience with tree-sitter, vector databases, and building responsive web UIs inside desktop applications. Most importantly, we learned that the best developer tools don't just solve problems - they help developers become better at solving problems themselves.

What's next for TrialbyCode

We're planning team collaboration features like shared challenges, leaderboards, and the ability for senior developers to create custom challenge sets for their teams. We want to expand language support beyond our current six languages and add integration with popular code review platforms like GitHub. The AI system will get smarter with difficulty adaptation based on individual learning patterns and more sophisticated pattern recognition. We're also exploring integration with computer science education - imagine automatically generating homework problems from students' actual code, or tracking learning progress across an entire course. The long-term vision is making TrialbyCode the standard way developers learn and improve throughout their careers.