VisionFoundry
Teaching VLMs Visual Perception with Synthetic Images
Dataset
Vision-language models (VLMs or MLLMs) still struggle with visual perception tasks such as spatial understanding and viewpoint reasoning. We hypothesize that this perception bottleneck stems from insufficient supervision in natural image datasets for visual perception skills.
We introduce VisionFoundry, a task-aware synthetic data generation pipeline that uses LLMs to generate questions, answers, and T2I prompts, then synthesizes images and verifies consistency with a VLM, requiring no reference images or human annotation. Using VisionFoundry, we construct VisionFoundry-10K, a 10k synthetic VQA dataset with 10 visual perception tasks.
VisionFoundry-10K improves visual perception under both SFT and reinforcement learning across three VLM backbones. The pipeline also remains effective with open-source image synthesis and self-verification by Qwen2.5-VL-3B-Instruct.
Visual Perception: A Data Problem?
VLMs still struggle with visual perception tasks. We hypothesize that this perception bottleneck stems from insufficient supervision in natural image datasets for low-level visual skills. Natural image-text corpora may not systematically cover the full combinatorial space of spatial relations, viewpoint variations, and depth orderings.
Can we synthesize targeted supervision from scratch to address these weaknesses, without relying on reference images or expensive human annotation?
Main Results
Supervised Fine-Tuning
Finetuning on VisionFoundry-10K consistently improves visual perception benchmarks across three VLMs, while general-purpose benchmarks show benchmark-dependent changes.
Reinforcement Learning
| GRPO backbone | MMVP Pair | MMVP Single | CV-Bench-2D | CV-Bench-3D | RealWorldQA |
|---|---|---|---|---|---|
| Qwen2.5-VL-3B-Instruct | 46.0 (+10.7) | 71.3 (+7.0) | 72.9 (+5.6) | 72.3 (+6.3) | 67.2 (+2.2) |
| Llama-3.2-11B-Vision-Instruct | 45.3 (+2.6) | 70.7 (+0.4) | 72.4 (+2.0) | 76.3 (+1.9) | 63.8 (+0.8) |
| MiMo-VL-7B-SFT | 48.0 (+4.7) | 73.3 (+6.6) | 78.7 (+4.4) | 75.8 (+3.5) | 69.2 (+3.3) |
GRPO on the same 10-task synthetic dataset improves all five visual-perception metrics for every backbone. Parentheses report absolute gains over each pretrained baseline.
VisionFoundry Pipeline
Text Generation
GPT-5.2 generates task-aware Q&A pairs and detailed T2I prompts from an entity pool.
Image Synthesis
Gemini-2.5-Flash-Image (Nano Banana) synthesizes images conditioned on T2I prompts.
Verification & Filtering
A multimodal verifier checks alignment between each image and its answer-determining visual statement. The pipeline supports both proprietary and open-source judges.
Open-Source Self-Verification
VisionFoundry does not depend on a stronger proprietary verifier. With image synthesis fixed to Qwen-Image-2512, an open-source Qwen2.5-VL-3B-Instruct verifier remains competitive with Gemini-3-Pro and trades wins across benchmarks.
Self-Verified 10K Evolution
| Setting | MMVP Pair | MMVP Single | CV-Bench-2D | CV-Bench-3D | RealWorldQA |
|---|---|---|---|---|---|
| Qwen2.5-VL-3B baseline | 35.3 | 64.3 | 67.3 | 66.0 | 65.0 |
| Self-verifier 10K SFT | 40.7 (+5.4) | 68.7 (+4.4) | 69.3 (+2.0) | 75.2 (+9.2) | 65.9 (+0.9) |
Here the downstream Qwen model also serves as the verifier. This self-verified, fully open-source 10K pipeline improves all five visual-perception benchmarks, demonstrating a practical self-evolution loop without a stronger external judge.
VisionFoundry-10K
Each task targets a distinct low-level visual perception skill identified as a persistent weakness in contemporary VLMs. Each produces 1,000 verified samples, yielding 10,000 QA pairs in total.
Dataset Examples
Data-Size Effects of Synthetic Supervision
Performance improves predictably with more synthetic data. Results show an upward trend on visual perception benchmarks as data size increases, demonstrating that VisionFoundry-10K provides reliable, high-quality supervision without saturation.
Equal-Sized Mixture vs. Pure Natural Data
Equal-sized synthetic-natural mixture outperforms pure natural data on visual perception benchmarks while maintaining comparable general-purpose performance. This confirms that verifier-filtered synthetic images provide complementary signals that are hard to obtain from natural data alone.
Epoch Trade-off
With a single-task 1k subset, performance converges after ~8 epochs. With the full 10-task set, convergence is reached sooner. Larger and more diverse datasets enable more efficient training.
Key Findings
The visual perception bottleneck in VLMs is, to a large extent, a data problem.
On small-scale datasets, synthetic images can demonstrate preliminary task-aware scaling behavior.
The same verifier-filtered supervision consistently improves visual perception through GRPO across Qwen, MiMo, and Llama backbones.
Open-source Qwen image synthesis and self-verification retain the gains, enabling a fully open self-evolution pipeline.
