3D Gaussian Splatting
A technique that reconstructs 3D scenes from 2D images by representing geometry as millions of tiny Gaussian blobs, enabling fast rendering and novel view synthesis.
Loading post…
NVIDIAMichelle HortonTutorial
Scale AV Perception Across Vehicle Platforms with NVIDIA Omniverse NuRec
Summary
Autonomous-driving perception changes when the same software moves across carlines because sensor placement, calibration, fields of view, occlusions, geometry, timing, and coverage change. The post presents NVIDIA Omniverse NuRec as a way to adapt a stack before a complete target-carline dataset exists, while retaining real-world data for grounding and validation. NuRec reconstructs recorded drives with 3D Gaussian splatting, renders novel views through a target rig with configurable camera models and calibration, and preserves trajectories, annotations, and map data for label adaptation. The workflow pairs a reconstructed scene with a rig, renders target views, refines frames with NVIDIA Harmonizer, and trains a perception model on the output. It also documents containerized rendering and auxiliary-data generation, including semantic masks, metric depth, and ego-vehicle masks, while noting that views outside observed coverage may be lower quality.
Context
Moving a perception stack to another carline changes how the world appears because the vehicle’s sensor placement, calibration, fields of view, occlusions, body geometry, timing, and coverage differ. Collecting and labeling a new real-world dataset for every variant is expensive and may be impractical before a target vehicle or fleet is available, although real-world data remains important for grounding and validation.
Approach / What changed
NuRec reconstructs recorded drives and renders them from a target vehicle’s sensor rig. The workflow pairs a reconstructed USDZ scene with a rig configuration, exports target-camera trajectories, renders selected views, refines frames with NVIDIA Harmonizer, and trains a perception model on the result. Its retained trajectories, calibration, dynamic-object tracks, and map data support adapting labels, while auxiliary-data tooling can generate semantic segmentation, metric depth, and ego-vehicle masks.
Takeaways
NuRec can alter camera extrinsics, intrinsics, field of view, and supported lens models, including pinhole, fisheye, and f-theta configurations, when synthesizing target views.
Reconstructed scenes retain rig trajectories, per-camera calibration, dynamic object tracks, and map data that can help align objects, lanes, traffic lights, and road-boundary labels.
Cameras positioned far outside the observed trajectory or aimed toward areas with limited source-camera coverage may produce lower-quality renders.
Source and freshness
- Source
- developer.nvidia.com
- Published
- Added to Yomu
- Last updated
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
Visual explainers
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.