---
title: "The Definitive Fine-Tuning Guide for 3D Generation Models"
description: "Fine-tuning a general-purpose 3D generation model like TRELLIS.2 for a specific object category can correct geometry errors, such as inaccurate shoe soles. The authors add LoRA adapters only to the sparse-structure flow DiT and the shape flow DiT, freezing the texture flow DiT to avoid training on diffuse color maps that include baked lighting. Using 188 footwear objects from the Google Scanned Objects dataset, the fine-tuned model improves the mean F-score from 0.276 to 0.803 on nine held-out objects. Chamfer distance falls from 0.174 to 0.033, and normal consistency rises from 0.530 to 0.804. The approach uses supplied Docker containers and Hugging Face Jobs, with the sparse-structure run taking 7.2 hours and the shape run taking 3.6 hours on one A100 80 GB GPU."
---

# The Definitive Fine-Tuning Guide for 3D Generation Models

[Hugging Face](https://yomu.fyi/company/hugging-face) · Suvaditya Mukherjee, merve · Nov 12, 2026

**Type:** Tutorial

## Summary

Fine-tuning a general-purpose 3D generation model like TRELLIS.2 for a specific object category can correct geometry errors, such as inaccurate shoe soles. The authors add LoRA adapters only to the sparse-structure flow DiT and the shape flow DiT, freezing the texture flow DiT to avoid training on diffuse color maps that include baked lighting. Using 188 footwear objects from the Google Scanned Objects dataset, the fine-tuned model improves the mean F-score from 0.276 to 0.803 on nine held-out objects. Chamfer distance falls from 0.174 to 0.033, and normal consistency rises from 0.530 to 0.804. The approach uses supplied Docker containers and Hugging Face Jobs, with the sparse-structure run taking 7.2 hours and the shape run taking 3.6 hours on one A100 80 GB GPU.

## Context

General-purpose 3D generation models like TRELLIS.2 can produce inaccurate geometry for specific object categories. The authors use a subset of 188 footwear objects from the Google Scanned Objects dataset to improve sole accuracy.

## Approach / What changed

Add LoRA adapters to the sparse-structure flow DiT and the shape flow DiT of TRELLIS.2, freeze the texture flow DiT, preprocess GLB/OBJ meshes into latent representations and render input images, then run separate fine-tuning jobs on Hugging Face Jobs using supplied Docker containers.

## Takeaways

- Adding LoRA adapters to the sparse-structure flow DiT and shape flow DiT of TRELLIS.2, while freezing the texture flow DiT, improved geometry accuracy for footwear generation.
- On nine held-out objects, the fine-tuned model raised the mean F-score from 0.276 to 0.803, reduced Chamfer distance from 0.174 to 0.033, and increased normal consistency from 0.530 to 0.804.
- The texture flow DiT was frozen because the dataset’s diffuse color maps include baked lighting, which would be incorrect training targets and could cause catastrophic forgetting.

## FAQ

### Why is the texture flow DiT frozen during fine-tuning?

The texture flow DiT is frozen because the dataset contains diffuse color maps with baked lighting, not separate material maps needed for training. Training on these maps could cause the model to learn lighting as a surface property and suffer catastrophic forgetting.

### Why use LoRA instead of full fine-tuning?

LoRA trains small adapter matrices while keeping original weights frozen, reducing trainable parameters and memory. For a small dataset of 188 objects, full fine-tuning would cost more and risk overwriting learned priors. The configuration trains about 9.3 million adapter parameters total.

### How long did each fine-tuning run take?

The sparse-structure run took 7.2 hours and the shape run took 3.6 hours, each on one A100 80 GB GPU on Hugging Face Jobs. The sparse-structure loss fell quickly then slowly; the shape loss stayed approximately level.

**Tags:** [3D Generation](https://yomu.fyi/topic/3d-generation), [Fine-Tuning](https://yomu.fyi/topic/fine-tuning), [Hugging Face](https://yomu.fyi/topic/hugging-face), [LoRA](https://yomu.fyi/topic/lora), [TRELLIS](https://yomu.fyi/topic/trellis)

- Source: [Hugging Face](https://huggingface.co/blog/3d-lora-finetuning)
- Source URL: https://huggingface.co/blog/3d-lora-finetuning
- Ingested by Yomu: 2026-11-12T15:00:13.782Z

[Read original post](https://huggingface.co/blog/3d-lora-finetuning)
