TL;DR: SVG code as a Visual Representation

See our demo video for fun!

• [2025.12.20] 🌟 Added GPT-5.2 to our benchmark, showing solid performance, below Gemini-3-Pro but outperforming Claude-4.5-Sonnet.
• [2025.11.21] 🔥 Added Gemini-3-Pro to our benchmark, showing excellent performance.
• [2025.11.08] 🎥 Released our demo video featuring lots of fun memes and reaction images converted into SVGs.
• [2025.11.08] 🚀 We now offer a free trial API on our 🤗 HuggingFace Space.
• [2025.11.05] 🔥 We are honored to be featured as 🤗 HuggingFace Daily Paper #1.

• 🛠️ Installation
• 🚀 Quick Start
• 🔮 Evaluation
• 📌 Citation

Environment

git clone -b main --single-branch https://github.com/CSU-JPG/VCode.git
cd VCode
conda create -n vcode python=3.10.2 -y
conda activate vcode
conda install pytorch=2.5.1 torchvision=0.20.1 torchaudio=2.5.1 pytorch-cuda=12.4 -c pytorch -c nvidia
pip install -r requirements.txt

VCode-suite is a comprehensive toolkit that automates the full image-to-SVG-to-render workflow. It includes both integrated pipelines and independent modules for generation, rendering, and revision. Users can either run the end-to-end pipelines for batch processing, or execute individual scripts for customized control.

📁 vcode-suite/
├── filter.py
├── img2svg.py
├── img2svgthinking.py
├── img2svg-w-visual-tool.py
├── img2text2svg.py
├── pipeline.sh
├── revision_pipeline.sh
├── revision.py
└── svg_render_img.py

💡 Tip: The pipelines (pipeline.sh, revision_pipeline.sh) perform fully automated batch processing, while the Python scripts (img2svg.py, img2text2svg.py, revision.py, etc.) can be run independently to support flexible and modular experimentation within the VCode framework.

pipeline.sh orchestrates the full image-to-SVG-to-render workflow. It can connect to different generation modules — img2svg, img2text2svg, or img2svgthinking — to convert images into SVGs, then filter and render them into pixel images.

chmod +x pipeline.sh
./pipeline.sh

revision_pipeline.sh automates the revision and optimization process. It takes the previously generated SVGs (generated_svgs/) and rendered images (generated_imgs/), calls the API-based revision module, and outputs the optimized SVGs and renders to optimized_svgs/ and optimized_imgs/.

chmod +x revision_pipeline.sh
./revision_pipeline.sh

Both generation and revision scripts can be executed independently for flexible and customized workflows.

Each core generation script — img2svg.py, img2text2svg.py, img2svgthinking.py, and img2svg-w-visual-tool.py — can directly convert input images into SVG code. Similarly, revision.py can be run independently to optimize previously generated SVGs through visual feedback.

Run img2svg.py

python vcode-suite/img2svg.py \
/path/to/input_images \
./generated_svgs \
--model gpt-5 \
--base-url https://openrouter.ai/api/v1 \
--api-key <OPENROUTER_API_KEY> \
--max-tokens 16384

Run revision.py

python vcode-suite/revision.py \
--svg-folder ./generated_svgs \
--original-folder ./input_images \
--rendered-folder ./generated_imgs \
--output-folder ./optimized_svgs \
--analysis-folder ./visual_analysis \
--base-url https://openrouter.ai/api/v1 \
--api-key <OPENROUTER_API_KEY> \
--model gpt-5 \
--max-tokens 16384

💡 Tip: The revision.py script refines existing SVGs based on visual comparison feedback, while generation scripts (img2svg.py, img2text2svg.py, img2svgthinking.py, img2svg-w-visual-tool.py) create SVGs from input images_folder. You can flexibly mix and match these tools depending on your pipeline needs.

Use the VCode-suite pipeline (or standalone scripts) to render images for each dataset. Original images are already in data/:

• MM-Vet: data/mm-vet/images
• CV-Bench: data/cv-bench
• MMMU: data/mmmu/mmmu_dev_processed_single_img_subset

Running your pipeline will produce, per dataset, a folder like:

generated_svgs/
generated_imgs/  ← used by the evaluators

Each evaluator is a shell script under evaluation/…. They all follow the same usage:

chmod +x evaluation/mm-vet/mmvet_eval.sh
./evaluation/mm-vet/mmvet_eval.sh

chmod +x evaluation/cv-bench/cvbench_eval.sh
./evaluation/cv-bench/cvbench_eval.sh

chmod +x evaluation/mmmu/mmmu_eval.sh
./evaluation/mmmu/mmmu_eval.sh

These scripts will read your generated_imgs/ and compute scores.

💡 Reference: For directory organization and example script configuration, see example_results/ (it shows a working layout you can mirror).

Full Command with Options

python metrics.py \
--folder1 /path/to/reference_images \
--folder2 /path/to/model_outputs/gpt-4o \
--ckpt google/siglip2-so400m-patch14-384

Command Line Arguments

Expected Directory Layout:

Reference Images Folder (--folder1)

Location: data/mm-vet/images (example path - can be customized)

folder1/
├── category1/
│   ├── image001.png
│   ├── image002.jpg
│   └── ...
├── category2/
│   ├── image003.png
│   └── ...
└── ...

Model Output Folder (--folder2)

Location: example_results/mm-vet/Gemini-2.5-Pro (example path - can be customized)

folder2/
├── generated_imgs/           # Generated/rendered images
│   ├── category1/
│   │   ├── image001.png
│   │   ├── image002.jpg
│   │   └── ...
│   ├── category2/
│   │   ├── image003.png
│   │   └── ...
│   └── ...
│
└── generated_svgs/           # SVG source files
   ├── category1/
   │   ├── image001.svg
   │   ├── image002.svg
   │   └── ...
   ├── category2/
   │   ├── image003.svg
   │   └── ...
   └── ...

If you find our work useful, please cite:

@article{vcode,
  title={VCode: a Multimodal Coding Benchmark with SVG as Symbolic Visual Representation},
  author={Lin, Kevin Qinghong and Zheng, Yuhao and Ran, Hangyu and Zhu, Dantong and Mao, Dongxing and Li, Linjie and Torr, Philip and Wang, Alex Jinpeng},
  journal={arXiv preprint arXiv:2511.02778},
  year={2025}
}