Graft | Devpost

Inspiration

The modern web is built on text, but human understanding is visual. Students, researchers, and professionals have to sift through dense articles, reports, and textbooks, trying to connect the dots hidden within paragraphs. The cognitive load is immense. Manually creating diagrams to aid understanding is slow, disruptive, and requires switching to separate design tools, breaking concentration and slowing progress. This not only makes comprehension harder, especially for visual learners or anyone short on time, but also makes communicating ideas harder. Complex concepts get buried in paragraphs when they could be shared more clearly and accurately through visual structure.

What it does

Graft is a Chrome extension that turns highlighted text into clear, customizable diagrams directly on the webpage, without switching tools or tabs.

Highlight any passage and Graft analyzes the content to generate a diagram
Customize your diagram manually on the canvas
Refine the diagrams with natural language commands like “remove the node containing 'Photosynthesis'", "add a node with the label 'Oxygen'", "Connect the nodes that contain 'Energy'"
Generate supplemental images or icons for your visual diagram
Export the final diagram as a PNG or SVG for use in presentations, notes, or articles
View your history of generated diagrams

This makes complex information not only easier to understand, but also easier to verify and share.

How we built it

AI-Powered Diagram Generation
We used Chrome's built-in Prompt API to convert highlighted text into diagram data. When a user selects or pastes text, the API analyzes it and returns JSON containing information about nodes (with labels) and arrows (showing which nodes connect to each other).

Smart Positioning
The initial AI output often had nodes placed randomly or overlapping each other. To fix this, we adjusted the prompt and applied a force-directed layout algorithm that automatically spreads nodes apart and positions them in a way that makes sense visually.

Arrow Calculations
We calculate the exact coordinates and control points for each arrow based on where the nodes are positioned and how big they are. This ensures arrows connect cleanly to the edges of nodes.

Excalidraw Integration
We convert all of this data into Excalidraw elements and inject them into an Excalidraw canvas. This means users get access to all of Excalidraw's features—they can drag elements around, change colors and styles, add text, and export their work.

Natural Language Editing
Users can modify their diagrams by typing commands in plain English. We send these commands back to the Prompt API, which returns updated JSON with the changes. Then we run the same positioning and arrow calculation process to update the canvas.

Summarization Feature
We used Chrome's Summarizer API to add a summarization feature. When the button is clicked, it generates a summary of the original text and provides context about what the diagram shows.

Image Generation
For supplemental images and icons, we built a small Express.js server that calls the Gemini Imagen API. When users request images related to their diagram, the server generates them and we add them directly to the Excalidraw canvas.

Challenges we ran into

Getting Clean-Looking Diagrams
Our biggest challenge was making diagrams that actually looked good. Early versions had nodes overlapping each other, arrows crossing through unrelated nodes, and generally messy layouts that were hard to read. We solved this through:

Experimenting with different ways to prompt the AI to get better initial structure
Implementing the force-directed layout to automatically space things out
Writing custom logic to calculate arrow coordinates and magnitudes to connect cleanly to the edges of nodes.

Slow Response Times
The Prompt API can take 20-30 seconds to generate results, especially with longer text. We found that shorter inputs help speed things up, but it's still slower than we'd like. This remains an ongoing challenge.

Accomplishments that we're proud of

Seamless In-Browser Experience
We built something that works entirely in the browser without needing to copy-paste into another tool or open a new tab. Highlight text, get a diagram, edit it, export it all on the same page you're reading.

AI + Manual Control
Users can generate diagrams automatically with AI, refine them with natural language, and still have full manual control through Excalidraw. This combination of automation and customization makes the tool flexible for different needs.

Quality Outputs
Through our layout algorithm and arrow rendering, we turned rough AI-generated structures into polished diagrams that people can actually use in presentations and reports.

Solving a Real and Interesting Problem
We've created something that genuinely addresses a daily pain point for students, researchers, and knowledge workers: making sense of complex information quickly and communicating it effectively.

What we learned

AI Outputs Need Guidance
Raw outputs from LLMs like that of the Prompt API aren't always ready to use. We learned that treating AI output as a starting point (not the final result) and applying our own algorithms on top made a huge difference in quality.

How You Prompt Matters
Small changes in how we structured our prompts, such as being more specific about the output format we wanted or keeping inputs concise, had big impacts on both the quality of results and how long generation took.

Performance Constraints are Important
Working with Chrome's AI APIs taught us about performance constraints. We had to think about things like showing progress indicators and setting user expectations around processing time.

What's next for Graft

Generate multiple diagram options to choose from
Make diagram nodes for hyperlinked concepts clickable, allowing the diagram to expand by analyzing the linked page's content.
Visually compare two text selections (great for revisions or versioning)
Right click any image or slide to generate a diagram from its text
Turn diagrams into interactive quizzes to reinforce learning
Allow users to save diagrams to an account and share interactive versions with others via a link.