SynthLight: Portrait Relighting with Diffusion Model by Learning to Re-render Synthetic Faces

CVPR 2025
Sumit Chaturvedi1, Mengwei Ren2, Yannick Hold-Geoffroy2, Jingyuan Liu2, Julie Dorsey1, Zhixin Shu2
1Yale University,2Adobe
Paper arXiv

SynthLight relights human portraits using an environment map lighting. By learning to re-render synthetic human faces, our diffusion model produces realistic illumination effects on real portrait photographs, including distinct cast shadows on the neck and natural specular highlights on the skin.

Abstract

We introduce SynthLight, a diffusion model for portrait relighting. Our approach frames image relighting as a re-rendering problem, where pixels are transformed in response to changes in environmental lighting conditions. Using a physically-based rendering engine, we synthesize a dataset to simulate this lighting-conditioned transformation with 3D head assets under varying lighting. We propose two training and inference strategies to bridge the gap between the synthetic and real image domains: (1) multi-task training that takes advantage of real human portraits without lighting labels; (2) an inference time diffusion sampling procedure based on classifier-free guidance that leverages the input portrait to better preserve details. Our method generalizes to diverse real photographs and produces realistic illumination effects, including specular highlights and cast shadows, while preserving the subject's identity. Our quantitative experiments on Light Stage data demonstrate results comparable to state-of-the-art relighting methods. Our qualitative results on in-the-wild images showcase rich and unprecedented illumination effects.

Method

Method figure

(a) We formulate relighting as re-rendering synthetic faces. (c) In SynthLight, we propose multi-task training to: (1) re-render synthetic faces, and (2) generate portraits from text without image conditioning. (c) During inference, we apply classifier-free guidance on the input portrait. (d) To test the generality of our formulation, we train a simplified DiT-based variant, SynthLight-DiT, solely on synthetic data to predict both the input and relit portraits, using channel conditioning as in ReCamMaster. This variant performs competitively, underscoring the strength of our core approach.

Results

Close-up Portraits

SynthLight relights close-up portraits by smoothly rotating an environment map. Training on synthetic data enables it to realistically capture effects like specular highlights on skin, subtle subsurface scattering in ears under backlighting, rim lighting in hair, and clear cast shadows, all while preserving the original subject's identity and overall visual coherence.

Generalization Capabilities

SynthLight generalizes surprisingly well beyond its training domain: it reliably handles half-body portraits and painted faces, despite training solely on synthetic data. Moreover, stress tests on figurines and non-photorealistic 3D characters reveal its remarkable ability to produce plausible shadows and accurate lighting effects in diverse scenarios.

Half-body Portraits Face Paint Miniatures and Figurines

Comparisons

Comparisons

User Study Results

User Study Results

User preference rates indicate how often SynthLight was preferred over baselines; e.g., our method was preferred 92% of times in terms of lighting over IC-Light. SynthLight outperforms baselines in lighting, image quality, and subject identity as all preference rates exceed 50%.

BibTeX


        @article{chaturvedi2025synthlight,
          title={SynthLight: Portrait Relighting with Diffusion Model by Learning to Re-render Synthetic Faces},
          author = {Chaturvedi, Sumit and Ren, Mengwei and Hold-Geoffroy, Yannick and Liu, Jingyuan and Dorsey, Julie and Shu, Zhixin},
          journal={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
          year={2025}
        }