NVIDIA Unveils AI-Powered Lighting Control Tool for Videos: DiffusionRenderer
NVIDIA Research has introduced DiffusionRenderer, a groundbreaking AI tool that enables precise lighting control in videos. Capable of transforming bright daytime scenes into moody nightscapes, swapping sunny skies for overcast clouds, or softening harsh fluorescent light into natural tones, this innovation is set to redefine video editing and synthetic data generation.
At its core, DiffusionRenderer is an advanced neural rendering technique. Unlike traditional methods, it merges two key processes — inverse rendering (understanding lighting from images) and forward rendering (simulating light in scenes) — into a single, unified framework. This integration allows the system to achieve highly realistic lighting edits and outperforms existing state-of-the-art techniques.Designed for a wide range of uses, DiffusionRenderer offers powerful tools for both creative and technical fields. Video creators in advertising, filmmaking, and game development can seamlessly alter lighting in real or AI-generated footage. Meanwhile, AI researchers and robotics developers can use the system to enhance synthetic datasets by introducing varied lighting conditions — a critical step for training models in applications such as autonomous driving.
DiffusionRenderer is one of over 60 innovative research papers that NVIDIA is presenting at the 2025 Computer Vision and Pattern Recognition (CVPR) Conference, held June 11–15 in Nashville, Tennessee.
Creating AI That Delights
DiffusionRenderer tackles the challenge of de-lighting and relighting a scene from only 2D video data.
De-lighting is a process that takes an image and removes its lighting effects, so that only the underlying object geometry and material properties remain. Relighting does the opposite, adding or editing light in a scene while maintaining the realism of complex properties like object transparency and specularity — how a surface reflects light.Classic, physically based rendering pipelines need 3D geometry data to calculate light in a scene for de-lighting and relighting. DiffusionRenderer instead uses AI to estimate properties including normals, metallicity and roughness from a single 2D video.
With these calculations, DiffusionRenderer can generate new shadows and reflections, change light sources, edit materials and insert new objects into a scene — all while maintaining realistic lighting conditions.
Using an application powered by DiffusionRenderer, AV developers could take a dataset of mostly daytime driving footage and randomize the lighting of every video clip to create more clips representing cloudy or rainy days, evenings with harsh lighting and shadows, and nighttime scenes. With this augmented data, developers can boost their development pipelines to train, test and validate AV models that are better equipped to handle challenging lighting conditions.