FIBO Lightning Director

From guesswork to precision. Control cinematic lighting in AI-generated scenes.

Comment

Inspiration

Watching professional cinematographers struggle to translate their intricate lighting visions into AI-generated visuals was our catalyst. We noticed a fundamental disconnect: most AI image models excel at imagination but fail at precise control. Text prompts like "make it moodier" are ambiguous and unreliable. Our goal was to bridge this gap by building a tool that gives creators deterministic, repeatable control over cinematic lighting—essentially, a "virtual gaffer" powered by AI.

What it does

The FIBO Lightning Director transforms cinematic lighting design from an art of guesswork into a science of precision. It provides a visual interface where users can:

  • Control Professional Parameters: Adjust key light intensity (0-200%), color temperature (2000K-10000K), direction (0-360°), and softness in real-time.
  • Generate with FIBO: Send these precise parameters as a structured JSON prompt to the FIBO model, which deterministically generates high-fidelity, HDR-ready images.
  • Refine Iteratively: Use natural language commands (e.g., "make it darker," "add rain," "change to a 24mm lens") to intelligently modify only the relevant parts of the scene JSON and regenerate, eliminating prompt drift.
  • Export for Production: Generate Nuke scripts or Datasmith files that include camera and lighting data for direct use in professional VFX and virtual production pipelines.

How we built it

We built a full-stack application with a clear, production-ready architecture:

  • Frontend (React/Vanilla JS + Three.js): A cinematic "virtual gaffer board" UI built with React for complex state management and Three.js for real-time 3D lighting previews.
  • Backend (Python FastAPI): A robust API layer that orchestrates the AI pipeline, handles JSON transformations, and manages caching. Deployed on Render.com.
  • AI Pipeline:
    1. User inputs are converted into a rich, structured JSON schema.
    2. This JSON is sent via the official fal-client library to the FIBO (fal-ai/fibo) API endpoint.
    3. For the "Refine" feature, the Gemini API parses natural language instructions and surgically edits the scene JSON.
  • Key Technologies: FIBO by Bria AI, fal-client, FastAPI, React, Three.js, Render.com, Gemini API.

Challenges we ran into

  • The "JSON Wall": FIBO's power is its structured prompts, but its required schema is complex and strict. We spent significant time reverse-engineering and validating the exact JSON structure (short_description, lighting.conditions, etc.) to avoid 422 validation errors.
  • API Client Confusion: Navigating the correct usage of the fal-client Python library was tricky—differentiating between submit(), subscribe(), and handling async patterns caused several iteration loops.
  • State Synchronization: Keeping the React frontend's lighting preview, the parameter sliders, and the backend's JSON builder perfectly in sync required careful state management design.
  • Deployment & Configuration: Ensuring environment variables (API keys for Fal.ai and Gemini) were securely managed and correctly injected into the Render.com deployment.

Accomplishments that we're proud of

  • Demonstrating True Controllability: We successfully showcased FIBO's core differentiator—making a specific, repeatable change to a complex scene (like adjusting light temperature) and getting a predictably altered output, which is nearly impossible with standard diffusion models.
  • Professional-Grade UX: Creating an interface that feels intuitive for filmmakers, not AI researchers, with real-time visual feedback.
  • A Working, Deployed Full-Stack App: Delivering a polished, end-to-end application from UI to deployed backend within the hackathon timeline.
  • Innovative "Refine" Pipeline: Integrating a secondary LLM (Gemini) to enable natural language refinement, pushing FIBO's capabilities toward an interactive, conversational workflow.

What we learned

  • The Future is Structured: The shift from brittle text prompts to structured, JSON-native generation (as with FIBO) is a fundamental leap for professional AI workflows, enabling version control, collaboration, and deterministic results.
  • API Integration is a Skill: Successfully leveraging a cutting-edge model like FIBO meant deeply understanding its API, authentication, error handling, and cost management (serverless credits).
  • HDR is a Game-Changer: Working with FIBO's 16-bit color space potential revealed how crucial high dynamic range is for achieving truly cinematic, theatrical quality in AI-generated imagery.
  • The Power of Simplicity: The most impactful feature ("make it darker") was often the simplest to articulate but the most complex to implement correctly, teaching us to prioritize user-centric design.

What's next for FIBO Lightning Director

The current version is a powerful proof-of-concept. To turn it into an industry-standard tool, we would:

  1. Integrate with Professional Software: Build native plugins for Nuke, Unreal Engine, and DaVinci Resolve, allowing lighting setups to be exported and used directly in live projects.
  2. Develop a Full "Virtual Scouting" Suite: Expand from lighting to include location generation, asset placement, and camera choreography for full pre-visualization.
  3. Implement Multi-Shot Continuity: Develop algorithms to maintain lighting and aesthetic consistency across a sequence of shots for storyboard generation.
  4. Create a Shared Preset Library: Build a community-driven platform where cinematographers can share and rate lighting presets (e.g., "Blade Runner 2049," "Wes Anderson Symmetry").
  5. Optimize the Cost Pipeline: Implement smarter caching, preview generations, and batch processing to make the tool cost-effective for studio-scale use.
# Example of the core JSON structure our app builds for FIBO
fibo_scene_description = {
    "short_description": "A detective in a noir office at night",
    "background_setting": "rain-streaked window, wooden desk, shadowy corners",
    "lighting": {
        "conditions": "low key dramatic",
        "direction": "45 degrees from camera right",
        "shadows": "deep, high contrast"
    },
    "aesthetics": {
        "composition": "close up, rule of thirds",
        "color_scheme": "monochromatic, desaturated amber and black",
        "mood_atmosphere": "suspenseful, lonely"
    },
    "style_medium": "photography",
    "artistic_style": "film noir, cinematic"
}

The FIBO Lightning Director proves that AI can be a precise collaborator in the creative process, not just a source of random inspiration. We're giving the director of photography a new kind of light to work with.

Your AI Cinematographer Check it out : [https://thriving-squirrel-bbbc9d.netlify.app]

Built With

Share this project:

Updates