---
title: "Adapt Geometry, Freeze the Light: Anatomy of a 3D Generation Fine-Tune"
description: "A layered architecture view of fine-tuning a 3D generation model: which flow DiT branches get LoRA adapters, which stay frozen, and how data, compute, and evaluation wrap the adapted core."
---

# Adapt Geometry, Freeze the Light: Anatomy of a 3D Generation Fine-Tune

**Type:** [Architecture](https://yomu.fyi/visual-explainers/category/architecture.md)

![Layered architecture diagram of a 3D generation fine-tune: a data layer feeds a base model whose sparse-structure and shape flow DiTs get LoRA adapters while the texture flow DiT stays frozen, followed by training, evaluation, and harness layers.](https://yomufyi.s3.us-east-1.amazonaws.com/visual-explainers/c51b6b4d-61f0-4b85-8e3c-057aafb986fe/image-91a47411-d530-4966-97fc-7cbe146ccebb.webp)

> A layered architecture view of fine-tuning a 3D generation model: which flow DiT branches get LoRA adapters, which stay frozen, and how data, compute, and evaluation wrap the adapted core.

## Explanation

Fine-tuning a 3D generation model is mostly a decision about where to adapt and where to freeze. In the TRELLIS.2 footwear example, LoRA adapters are attached only to the sparse-structure and shape flow DiTs, while the texture flow DiT stays frozen because its diffuse color maps carry baked lighting; data curation, compute scheduling, and held-out evaluation sit outside the model. The payoff is measurable: mean F-score rises from 0.276 to 0.803 and Chamfer distance falls from 0.174 to 0.033. Use this layered view to decide what to train, what to freeze, and what to keep as reproducible infrastructure.

## FAQ

### Why freeze the texture flow DiT instead of adapting all three branches?

The texture branch learns from diffuse color maps that include baked lighting, so adapting it would bake dataset-specific lighting into the model. The reported geometry gains come from the sparse-structure and shape flows alone, and freezing the texture branch also shortens the training run.

### How do I know the fine-tune actually improved geometry?

Evaluate on objects the model never trained on and compare geometry metrics before and after. In the footwear example, mean F-score rose from 0.276 to 0.803, Chamfer distance fell from 0.174 to 0.033, and normal consistency rose from 0.530 to 0.804 across nine held-out objects.

### What does one category fine-tune cost in compute?

The sparse-structure run took 7.2 hours and the shape run 3.6 hours on a single A100 80 GB GPU, roughly 11 GPU-hours total, using supplied Docker containers and Hugging Face Jobs. Because stages differ in cost, match compute to each stage rather than provisioning one uniform cluster.

**Tags:** [3D Generation](https://yomu.fyi/topic/3d-generation.md), [Fine-Tuning](https://yomu.fyi/topic/fine-tuning.md), [GPU](https://yomu.fyi/topic/gpu.md), [LoRA](https://yomu.fyi/topic/lora.md), [TRELLIS](https://yomu.fyi/topic/trellis.md)

## Related reading

### [Modular regression testing framework built on AWS for large-scale systems](https://yomu.fyi/post/modular-regression-testing-framework-built-on-aws-for-large-scale-syst.md)
- Company: [AWS](https://yomu.fyi/company/aws.md)
- Author: Hannah Zhang
- Published: Nov 12, 2026
- Type: Tutorial

A modular regression testing framework built on AWS replays a sample of production traffic against new code and compares results to validate no behavior changes before deployment. It uses Step Functions for orchestration, Lambda for replay, DynamoDB for capturing requests and responses, and CloudWatch for logging and alarming on match fraction. The design is minimally invasive, requiring no changes to stateless systems and only a side-effect skip for stateful ones, and can be inserted at API, service, database, or message queue boundaries. In over six months of production use, the framework caught five regressions with zero false positives, using a 99.9% match fraction alarm to block problematic deployments via a deployment gate. The post covers three use cases—refactoring, feature toggle deployments, and impact boundary validation—and provides code examples for capture, replay, and evaluation functions.


### [Scale distributed AI workloads with Ray and Anyscale on Amazon EC2](https://yomu.fyi/post/scale-distributed-ai-workloads-with-ray-and-anyscale-on-amazon-ec2.md)
- Company: [AWS](https://yomu.fyi/company/aws.md)
- Author: Sindhura Palakodety
- Published: Nov 12, 2026
- Type: Problem & solution

Building multimodal AI/ML pipelines on Amazon EC2 that process text, video, and audio presents challenges in scaling, cost management, and keeping up with evolving hardware. The Ray-powered Anyscale application on AWS addresses these challenges by providing a fully managed service that deploys within the customer’s AWS account. Anyscale offers three deployment options: Amazon EC2 for maximum flexibility, Amazon EKS for Kubernetes-native environments, and Amazon SageMaker HyperPod for resilience-critical training. It supports heterogeneous clusters mixing Amazon EC2 Spot Instances, AWS Graviton, NVIDIA GPUs, AWS Trainium, and AWS Inferentia, matching compute to each pipeline stage. The same Ray code runs from a single instance to a multi-node cluster without modification, and Anyscale handles cluster management, fault tolerance, autoscaling, and resource orchestration.


### [How H6.ai builds AI-powered financial automation with Lakebase Postgres](https://yomu.fyi/post/how-h6-ai-builds-ai-powered-financial-automation-with-lakebase-postgre.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Benji Borg, Mauro Pirrone
- Published: Nov 12, 2026
- Type: Problem & solution

H6.ai automates accounts payable by reading invoices, matching them to purchase orders, assigning line items to accounts, and checking against company policies. The company faced a challenge: financial data changes constantly, so a result that differs from an earlier run could be due to code changes or data changes. H6.ai adopted a lakebase architecture with one dedicated Lakebase Postgres database per customer, each holding the customer’s full financial context. When an issue is found, the team branches the database to create a frozen snapshot; test runs fork a short-lived branch from that snapshot, process the invoice against the real pipeline, judge the result, and delete the branch. This approach provides isolation, frozen state, cheap disposable copies, and schema-safe replays, enabling trustworthy regression testing for AI-powered financial automation.


### [Agent harness engineering: Separating model capabilities from enforcement](https://yomu.fyi/post/agent-harness-engineering-separating-model-capabilities-from-enforceme.md)
- Company: [Red Hat](https://yomu.fyi/company/red-hat.md)
- Author: Mark Vaykhansky, Uri Shaket
- Published: Nov 12, 2026
- Type: Explainer

A breach at Hugging Face showed that capable agents can chain vulnerabilities across trust boundaries when the surrounding harness gives them paths around security controls. The post argues that capabilities models can reliably learn should be absorbed into the model, while enforcement properties such as permissions, identity, isolation, and budgets must be hardened as infrastructure outside the model. It classifies harnesses into four types—workflow, coding, research, and agent runtime—each with different architectures for different jobs. The better the model gets, the more dangerous a weak harness becomes, because teams often remove enforcement scaffolding as the model absorbs planning and tool use. The post recommends diagnosing gaps in an existing agent’s harness before applying tooling such as a gateway layer, sandboxed containers, or distributed tracing.


### [The Definitive Fine-Tuning Guide for 3D Generation Models](https://yomu.fyi/post/the-definitive-fine-tuning-guide-for-3d-generation-models.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Suvaditya Mukherjee, merve
- Published: Nov 12, 2026
- Type: Tutorial

Fine-tuning a general-purpose 3D generation model like TRELLIS.2 for a specific object category can correct geometry errors, such as inaccurate shoe soles. The authors add LoRA adapters only to the sparse-structure flow DiT and the shape flow DiT, freezing the texture flow DiT to avoid training on diffuse color maps that include baked lighting. Using 188 footwear objects from the Google Scanned Objects dataset, the fine-tuned model improves the mean F-score from 0.276 to 0.803 on nine held-out objects. Chamfer distance falls from 0.174 to 0.033, and normal consistency rises from 0.530 to 0.804. The approach uses supplied Docker containers and Hugging Face Jobs, with the sparse-structure run taking 7.2 hours and the shape run taking 3.6 hours on one A100 80 GB GPU.
