Panelmu Comic Storyboarder

Panelmu is an AI-powered storyboard generator that transforms your story ideas and character sketches into a complete comic book layout. It bridges the gap between narrative text and visual storytelling by generating not just the story structure, but the actual panel images themselves.

Inspiration

The process of creating a comic is incredibly demanding, often requiring a unique blend of skills in writing, illustration, and cinematography. For writers who can't draw, or artists struggling with story structure, bringing an idea to life can be a monumental challenge. We were inspired to create a tool that could act as a collaborative partner, an "AI Art Director" that could interpret a story, establish a consistent visual language, and produce a first-draft storyboard, dramatically accelerating the creative workflow from concept to comic.

What it does

Panelmu offers a streamlined, multi-step process for comic creation:

1. Input Story & Characters: Users provide a short story (3-8 sentences), reference images for their main characters (with names), and optional images to guide the art style.
2. AI Storyboard Generation: The application sends this multi-modal input to the Gemini 2.5 Pro model. A sophisticated system prompt instructs the AI to act as an expert art director, analyzing the inputs and outputting a structured JSON object that defines the entire storyboard. This JSON includes:
   • Canonical descriptions of each character to ensure consistency.
   • A detailed style guide (colors, rendering style, lighting).
   • A panel-by-panel breakdown with shot types, actions, captions, and dialogue.
3. Visual Panel Creation: The app then uses Gemini 2.5 Flash Image to generate the image for each panel. It combines the specific panel instructions with the canonical character descriptions and style guide, ensuring visual continuity. It supports two modes:
   • Sequential Mode: Generates panels one by one, using the previous panel as visual context for maximum consistency.
   • Parallel Mode: Generates all panels simultaneously for faster results.
4. Edit & Refine: Users can click on any generated panel to edit captions and dialogue, or to provide a text prompt to regenerate the image (e.g., "Change the character's expression to be happier").
5. Stitch & Export: Once the storyboard is complete, the built-in "Comic Stitcher" allows the user to arrange the panels into a final comic page, customize the layout, and export the result as a high-resolution PNG.

How we built it

Panelmu is a modern web application built with React and powered by the Google Gemini API.

• Frontend: The user interface is built with React (TypeScript) and styled using Tailwind CSS for a responsive and clean design. The state management is handled within React using useState and useEffect hooks to manage the complex flow of data from user input to final output.
• AI Core Logic (Gemini 2.5 Pro): The "brain" of the operation. We use gemini-2.5-pro for its advanced reasoning and multi-modal capabilities. A carefully engineered system prompt guides the model to analyze the user's inputs and return a structured JSON object. This is the key to creating a predictable and reliable workflow.
• Image Generation (Gemini 2.5 Flash Image): All visual generation is handled by gemini-2.5-flash-image. Its ability to take multiple image and text inputs in a single call is crucial for our consistency strategy. For both initial generation and editing, we provide the model with the character/style references alongside the text prompt.
• Comic Export (HTML Canvas): The "Comic Stitcher" feature uses the browser's native HTML Canvas API. We dynamically draw each generated panel onto a canvas element based on user-defined layout settings (columns, gutter, margins), and then convert the canvas to a PNG data URL for previewing and downloading.

Challenges we ran into

The primary challenge was maintaining visual consistency across panels—a common problem in AI image generation. We tackled this with a multi-pronged strategy:

1. Canonical Descriptions: Instead of relying on the image model to remember faces from one generation to the next, we use Gemini 2.5 Pro to first create a detailed textual description of each character.
2. Programmatic Prompting: This canonical description is then programmatically injected into the image prompt for every single panel that features that character.
3. Persistent References: The original character reference images are also included in every call to the gemini-2.5-flash-image model, constantly reminding it of the target appearance.
4. Sequential Context: In "Sequential Mode," we take it a step further by passing the previously generated panel as an additional image input, which helps maintain continuity in lighting and character posing.

Accomplishments that we're proud of

We are proud of creating a tool that provides a tangible and useful output. The visual consistency engine is the core accomplishment that makes Panelmu more than just a novelty. The complete workflow, from a simple text idea to a downloadable comic page, feels empowering and genuinely useful for creatives. The interactive editing loop, which allows for human-in-the-loop refinement, was critical to making the tool practical.

What we learned

This project was a deep dive into the power of multi-modal AI and chained AI workflows. We learned that the most powerful applications often come from using different models for different tasks—a large reasoning model like Gemini Pro for structure and logic, and a specialized image model like Flash Image for visual execution. The importance of structured outputs (like JSON mode) cannot be overstated for building reliable AI-powered applications.

What's next for Panelmu

• Advanced Inpainting: Allow users to mask a specific area of a panel and regenerate only that part for more precise edits.
• AI-Suggested Layouts: Have the AI recommend a layout in the Comic Stitcher based on the story's pacing and mood.
• Expanded Export Options: Support for PDF export or even layered files (e.g., PSD) for further editing in professional software.
• Character Sheets: Generate a full character turn-around sheet from the initial reference images to provide an even more robust guide for consistency.

Built With

• aistudio