Fine-Tuning SOTA Object Detection Models on Real-World Datasets

Summary

This practical comparison examines how YOLO12, YOLO26, and RF-DETR behave when moved from COCO to specialized real-world object-detection datasets, and how fine-tuning changes the outcome. The authors first evaluate six pretrained checkpoints on COCO val2017, then use cable-damage, bone-fracture, and soda-bottles datasets from RF100-VL for zero-shot testing, fine-tuning, and accuracy/latency comparison. Because the model implementations require incompatible dependencies, the workflow uses one PyCharm project with three isolated uv environments and remote GPU execution; the reported latency stays in native frameworks rather than TensorRT. All models perform well on COCO, but direct performance on specialized data falls close to zero, while 10 fine-tuning epochs adapt the detectors well to cable damage and soda bottles. Bone-fracture remains difficult despite fine-tuning, with RF-DETR performing best, indicating that domain shift may require more data, longer training, or domain-specific pretraining.

Takeaways

1. 1

   On COCO val2017, RF-DETR Base recorded the highest mAP50-95 at 0.5325, while YOLO26-N had the lowest listed latency at 12.3 ms.

2. 2

   Native-framework latency is not directly comparable with optimized paper benchmarks because the run used different hardware and did not compile models with TensorRT.

3. 3

   Soda-bottle predictions were generally tight and accurate; cable-damage localization was less consistent, and fewer than half of bone-fracture images had an identified break.