# Luce: Relightable Gaussians for 3D Asset Generation

[Apple](https://yomu.fyi/company/apple) · Aug 26, 2026

**Type:** Problem & solution

## Summary

High-fidelity image-to-3D asset generation requires 3D representations that capture geometry alongside physically based rendering modalities for seamless pipeline integration and relighting. Luce addresses this challenge by unifying geometry and material properties into a voxelized multimodal Gaussian cloud with dedicated primitives for albedo, metallic-roughness, and surface normals. A variational autoencoder compresses these multimodal primitives into a unified material-aware latent space. A rectified-flow transformer subsequently generates this latent from a single input image using multi-layer features extracted from a pretrained image encoder. On the Toys4K dataset, Luce improves FID by 28% over the leading baseline while maintaining fine details such as text and logos.

## Context

High-fidelity image-to-3D generation requires 3D representations that capture geometry and appearance while supporting relighting and standard rendering pipeline integration via physically based rendering (PBR) modalities such as albedo, metallic-roughness, and surface normals.

## Approach / What changed

Luce unifies geometry and PBR materials within a voxelized multimodal Gaussian cloud using dedicated Gaussian primitives for each modality. A variational autoencoder compresses the representation into a unified material-aware latent space, which a rectified-flow transformer generates from a single image conditioned on multi-layer features from a pretrained image encoder. The generated latent decodes into relightable PBR Gaussians and an optional textured mesh with a tangent-space normal map.

## Takeaways

- Luce represents geometry and physically based rendering modalities, including albedo, metallic-roughness, and surface normals, using dedicated primitives inside a voxelized multimodal Gaussian cloud.
- The architecture uses a variational autoencoder to compress multimodal 3D Gaussians into a latent space and a rectified-flow transformer conditioned on multi-layer image encoder features for generation.
- On the Toys4K benchmark, Luce improves FID by 28% over the strongest baseline and achieves a CLIP image-alignment score of 0.8519 compared to 0.8299 on AI-generated images.

**Tags:** [Machine Learning](https://yomu.fyi/topic/machine-learning)

[Read original post](https://machinelearning.apple.com/research/relightable-gaussians-3d-generation)