A Review of Multi-Conditional Controllable Diffusion Models for Generative Vision: Based on the Where-What-When Framework

Hanjia Chen

doi:10.47297/taposatWSP2633-456908.20260704

PDF
导出
分享
收藏
专辑

A Review of Multi-Conditional Controllable Diffusion Models for Generative Vision: Based on the Where-What-When Framework

7卷第4期

浏览量: 0 Downloads: 2

A Review of Multi-Conditional Controllable Diffusion Models for Generative Vision: Based on the Where-What-When Framework
A Review of Multi-Conditional Controllable Diffusion Models for Generative Vision: Based on the Where-What-When Framework
Taylor's University, Petaling Jaya, Selangor, 46150, Malaysia
2026年7卷第4期页码：41-46
纸质出版：2026-04-30
DOI：10.47297/taposatWSP2633-456908.20260704
Accepted：

Scan QR Code

Hanjia Chen. (2026). A Review of Multi-Conditional Controllable Diffusion Models for Generative Vision: Based on the Where-What-When Framework. 科技理论与实践（英文版）, 7(4), 41-46.

Chen Hanjia.A Review of Multi-Conditional Controllable Diffusion Models for Generative Vision: Based on the Where-What-When Framework[J].Theory and Practice of Science and Technology,2026,07(04):41-46.
Hanjia Chen. (2026). A Review of Multi-Conditional Controllable Diffusion Models for Generative Vision: Based on the Where-What-When Framework. 科技理论与实践（英文版）, 7(4), 41-46. DOI： 10.47297/taposatWSP2633-456908.20260704.

Chen Hanjia.A Review of Multi-Conditional Controllable Diffusion Models for Generative Vision: Based on the Where-What-When Framework[J].Theory and Practice of Science and Technology,2026,07(04):41-46. DOI： 10.47297/taposatWSP2633-456908.20260704.

摘要

Abstract

In recent years

diffusion models have emerged as a key technical approach in the field of generative vision; however

real-world applications often rely not only on a single text prompt but also require the coordinated constraints of multiple conditions

such as structure

layout

reference images

style

or editing instructions. Although existing research has proposed numerous control frameworks

there remains a lack of a concise and systematic review of unified classification

conflict resolution mechanisms

and fusion strategies when multiple conditions act simultaneously. This paper focuses on multi-condition controllable diffusion models in generative vision and proposes a three-dimensional 'Where-What-When' analytical framework: 'Where' describes the location where conditions are injected; 'What' describes the type of condition and the granularity of control; and 'When' describes the timing of the condition's application during the diffusion process. Building upon this

the paper provides a unified review of representative methods such as ControlNet

T2I-Adapter

IP-Adapter

Uni-ControlNet

UniControl

GLIGEN

Composer

MultiDiffusion

Zero-Painter and DC-ControlNet

and further summarises issues such as semantic conflict

structural conflict

condition competition and temporal mismatch in multi-condition scenarios. The review concludes that current research is shifting from 'single-condition supplementary control' to 'unified multi-condition modelling'

and from 'static superposition' to 'attention-driven

test-time optimisation and hierarchical fusion'; however

significant shortcomings remain in terms of explicit time-step scheduling

combinatorial generalisation

unified evaluation and inference efficiency. Finally

this paper summarises and looks ahead to future development directions for multi-condition controllable diffusion models.

关键词

Keywords

references

Ho J, Jain A N, Abbeel P. Denoising Diffusion Probabilistic Models[C]//Advances in Neural Information Processing Systems. 2020.

Rombach R, Blattmann A, Lorenz D, Esser P, Ommer B. High-Resolution Image Synthesis with Latent Diffusion Models[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 10684-10695.

Croitoru F A, Hondru V, Ionescu R T, Shah M. Diffusion Models in Vision: A Survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.

Jiang R, Zheng G C, Li T, Yang T R, Wang J D, Li X. A Survey of Multimodal Controllable Diffusion Models[J]. Journal of Computer Science and Technology, 2024, 39(3): 509-541.

Podell D, English Z, Lacey K, Blattmann A, Dockhorn T, Muller J, Penna J, Rombach R. SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis[J/OL ] . arXiv: 2307.01952 , 2023.

Zhang L, Rao A, Agrawala M. Adding Conditional Control to Text-to-Image Diffusion Models[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023: 3836-3847.

Mou C, Wang X, Xie L, Wu Y, Zhang J, Qi Z, Shan Y, Qie X. T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models[J/OL ] . arXiv: 2302.08453 , 2023.

Ye H, Zhang J, Liu S, Han X, Yang W. IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models[J/OL ] . arXiv: 2308.06721 , 2023.

Zhao S, Chen D, Chen Y C, Bao J, Hao S, Yuan L, Wong K Y K. Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models[C]//Advances in Neural Information Processing Systems. 2023.

Qin C, Zhang S, Yu N, Feng Y, Yang X, Zhou Y, Wang H, Niebles J C, Xiong C, Savarese S, Ermon S, Fu Y, Xu R. UniControl: A Unified Diffusion Model for Controllable Visual Generation in the Wild[C]//Advances in Neural Information Processing Systems. 2023.

Huang L, Chen D, Liu Y, Shen Y, Zhao D, Zhou J. Composer: Creative and Controllable Image Synthesis with Composable Conditions[C]//Proceedings of the 40th International Conference on Machine Learning. 2023: 13753-13773.

Li Y, Liu H, Wu Q, Mu F, Yang J, Gao J, Li C, Lee Y J. GLIGEN: Open-Set Grounded Text-to-Image Generation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 22511-22521.

Zheng G, Zhou X, Li X, Qi Z, Shan Y, Li X. LayoutDiffusion: Controllable Diffusion Model for Layout-to-Image Generation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 22490-22499.

Bar-Tal O, Yariv L, Lipman Y, Dekel T. MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation[C]//Proceedings of the 40th International Conference on Machine Learning. 2023: 1737-1752.

Brooks T, Holynski A, Efros A A. InstructPix2Pix: Learning To Follow Image Editing Instructions[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 18392-18402.

Hertz A, Mokady R, Tenenbaum J, Aberman K, Pritch Y, Cohen-Or D. Prompt-to-Prompt Image Editing with Cross Attention Control[J/OL ] . arXiv: 2208.01626 , 2022.

Chefer H, Alaluf Y, Vinker Y, Wolf L, Cohen-Or D. Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models[J/OL ] . arXiv: 2301.13826 , 2023.

Ohanyan M, Manukyan H, Wang Z, Navasardyan S, Shi H. Zero-Painter: Training-Free Layout Control for Text-to-Image Synthesis[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 8764-8774.

Yang H, Han W, Zhou Y, Shen J. DC-ControlNet: Decoupling Inter- and Intra-Element Conditions in Image Generation with Diffusion Models[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2025: 19065-19074.