Iconix is a human-AI co-creative system that organizes icon generation along two axes: semantic richness (what is depicted) and visual complexity (how much detail). Given a user-specified concept, Iconix constructs a semantic scaffold of related analytical perspectives and employs chained, image-conditioned generation to produce a coherent style of exemplars. Each exemplar is then automatically distilled into a progressive sequence, from detailed and elaborate to abstract and simple. The resulting two-dimensional grid exposes a navigable space, helping designers reason jointly about figurative content and visual abstraction.

• [2025.12.23]: /back_end/example.ipynb notebook released
• [2026.1.21]: backend released
• [2026.1.22]: frontend released
• [2026.1.23]: lora weights released

Create a conda environment and install dependencies:

conda create -n iconix python=3.10 -y
conda activate iconix

cd Iconix
pip install -r requirements.txt

Download the following model files and place them in the specified locations（back_end/iconix_config.yaml）:

1. FLUX.1-Kontext-dev (for image generation)

   • Edit the path in back_end/iconix_config.yaml under model_paths.flux_kontext
   • Or set via environment variable: export FLUX_KONTEXT_MODEL_PATH="/your/path/to/FLUX.1-Kontext-dev"

2. Stable Diffusion v1.5 (for image simplification)

   • Edit the path in back_end/iconix_config.yaml under model_paths.stable_diffusion
   • Or set via environment variable: export STABLE_DIFFUSION_MODEL_PATH="/your/path/to/stable-diffusion-v1-5"

3. SAM2 Hiera Large (for image segmentation)

   • Edit the path in back_end/iconix_config.yaml under model_paths.sam2_hiera_large
   • Or set via environment variable: export SAM2_HIERA_LARGE_MODEL_PATH="/your/path/to/sam2.1-hiera-large"

4. LoRA Weights

   • Download the following LoRA weights and place them in back_end/pretrained_loras/:

   back_end/
   ├── pretrained_loras/
   │   ├── windowsColor_mask2outline.safetensors
   │   ├── ios_outline2filled4.safetensors
   │   └── windows_outline2color.safetensors
   

Create a .env file in the back_end/ directory:

# LLM configuration (for concept exploration and semantic analysis)
LLM_PROVIDER=openai
LLM_API_KEY=your_openai_api_key_here
LLM_BASE_URL=https://api.openai.com/v1
LLM_MODEL_ID=gpt-4o

Recommended: Edit back_end/iconix_config.yaml to configure model paths, GPU settings, and hyperparameters. This configuration file is shared by both Django server and notebooks (e.g., example.ipynb).

model_paths:
  flux_kontext: "/path/to/FLUX.1-Kontext-dev"
  stable_diffusion: "/path/to/stable-diffusion-v1-5"
  sam2_hiera_large: "/path/to/sam2.1-hiera-large"

Alternative: You can also override via environment variables (environment variables take precedence over the config file):

# Linux/Mac
export FLUX_KONTEXT_MODEL_PATH="/path/to/FLUX.1-Kontext-dev"
export STABLE_DIFFUSION_MODEL_PATH="/path/to/stable-diffusion-v1-5"
export SAM2_HIERA_LARGE_MODEL_PATH="/path/to/sam2.1-hiera-large"

# Windows PowerShell
$env:FLUX_KONTEXT_MODEL_PATH="C:\path\to\FLUX.1-Kontext-dev"
$env:STABLE_DIFFUSION_MODEL_PATH="C:\path\to\stable-diffusion-v1-5"
$env:SAM2_HIERA_LARGE_MODEL_PATH="C:\path\to\sam2.1-hiera-large"

For more detailed configuration options, see back_end/CONFIGURATION_GUIDE.md

For a quick demonstration, you can run the example notebook:

cd back_end
jupyter notebook example.ipynb

For a complete setup, run both servers simultaneously:

1. Terminal 1 - Backend:

   cd back_end
   python manage.py runserver

2. Terminal 2 - Frontend:

    cd front_end
   
    # Install dependencies (first time only)
    npm install
   
    # Start the development server
    npm run dev

For detailed setup instructions and troubleshooting, refer to back_end/RUNNING_GUIDE.md or back_end/RUNNING_GUIDE_CN.md.

@inproceedings{10.1145/3772318.3790694,
author = {Sun, Zhida and Wang, Xiaodong and Zhang, Zhenyao and Lu, Min and Lischinski, Dani and Cohen-Or, Daniel and Huang, Hui},
title = {Iconix: Controlling Semantics and Style in Progressive Icon Grids Generation},
year = {2026},
isbn = {9798400722783},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3772318.3790694},
doi = {10.1145/3772318.3790694},
abstract = {Visual communication often needs stylistically consistent icons that span concrete and abstract meanings, for use in diverse contexts. We present Iconix, a human-AI co-creative system that organizes icon generation along two axes: semantic richness (what is depicted) and visual complexity (how much detail). Given a user-specified concept, Iconix constructs a semantic scaffold of related analytical perspectives and employs chained, image-conditioned generation to produce a coherent style of exemplars. Each exemplar is then automatically distilled into a progressive sequence, from detailed and elaborate to abstract and simple. The resulting two-dimensional grid exposes a navigable space, helping designers reason jointly about figurative content and visual abstraction. A within-subjects study (N = 32) found that compared to a baseline workflow, participants produced icon grids more creatively, reported lower workload, and explored a coherent range of design variations. We discuss implications for human-machine co-creative approaches that couple semantic scaffolding with progressive simplification to support visual abstraction.},
booktitle = {Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems},
articleno = {250},
numpages = {18},
keywords = {Icon, Semantics, Style, Generative AI, Human-AI Co-creation},
location = {
},
series = {CHI '26}
}