We Got Claude to Fine-Tune an Open Source LLM

Summary

Hugging Face Skills equips coding agents like Claude Code, OpenAI Codex, and Google's Gemini CLI to manage end-to-end language model fine-tuning. Using the hf-llm-trainer skill, an agent validates dataset formats, selects cloud hardware, configures authentication, and submits jobs to Hugging Face Jobs. Supported techniques include Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Group Relative Policy Optimization (GRPO) for models ranging across various parameter sizes. The integration incorporates Trackio for real-time monitoring and automates LoRA configuration for larger models. Once training completes, the agent pushes artifacts to the Hugging Face Hub and can convert models to GGUF format for local execution.

Takeaways

1. 1

   The hf-llm-trainer skill supports Supervised Fine-Tuning, Direct Preference Optimization, and Group Relative Policy Optimization for models including text and vision language models.

2. 2

   For models exceeding 3B parameters up to 7B, the skill automatically applies LoRA on instances like a10g-large or a100-large to fit GPU memory limits.

3. 3

   Trained models can be automatically converted to GGUF format with quantization such as Q4_K_M for local inference using tools like llama.cpp and Ollama.