Adapt Geometry, Freeze the Light: Anatomy of a 3D Generation Fine-Tune
A layered architecture view of fine-tuning a 3D generation model: which flow DiT branches get LoRA adapters, which stay frozen, and how data, compute, and evaluation wrap the adapted core.
Fine-tuning a 3D generation model is mostly a decision about where to adapt and where to freeze. In the TRELLIS.2 footwear example, LoRA adapters are attached only to the sparse-structure and shape flow DiTs, while the texture flow DiT stays frozen because its diffuse color maps carry baked lighting; data curation, compute scheduling, and held-out evaluation sit outside the model. The payoff is measurable: mean F-score rises from 0.276 to 0.803 and Chamfer distance falls from 0.174 to 0.033. Use this layered view to decide what to train, what to freeze, and what to keep as reproducible infrastructure.
FAQ
Why freeze the texture flow DiT instead of adapting all three branches?
The texture branch learns from diffuse color maps that include baked lighting, so adapting it would bake dataset-specific lighting into the model. The reported geometry gains come from the sparse-structure and shape flows alone, and freezing the texture branch also shortens the training run.
How do I know the fine-tune actually improved geometry?
Evaluate on objects the model never trained on and compare geometry metrics before and after. In the footwear example, mean F-score rose from 0.276 to 0.803, Chamfer distance fell from 0.174 to 0.033, and normal consistency rose from 0.530 to 0.804 across nine held-out objects.
What does one category fine-tune cost in compute?
The sparse-structure run took 7.2 hours and the shape run 3.6 hours on a single A100 80 GB GPU, roughly 11 GPU-hours total, using supplied Docker containers and Hugging Face Jobs. Because stages differ in cost, match compute to each stage rather than provisioning one uniform cluster.
ARC Region Switch is a managed failover orchestrator that runs team-owned plans in dependency order. This diagram maps its trigger, orchestration, execution, and region layers and the ownership boundary.
Active learning is a loop that spends a fixed labeling budget on the samples most worth annotating. This diagram maps its stages — pool, score, label, train, evaluate — and the return path that closes the cycle.
AWS Cloud WAN is a managed core network that joins Regions, VPCs, and on-premises sites through attachments. This architecture view shows where the per-Region egress decision is made and how one link can be drained safely.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.