Loading…
We Got Claude to Fine-Tune an Open Source LLM
Hugging Faceben burtenshaw, shaun smith
Summary
Hugging Face Skills equips coding agents like Claude Code, OpenAI Codex, and Google's Gemini CLI to manage end-to-end language model fine-tuning. Using the hf-llm-trainer skill, an agent validates dataset formats, selects cloud hardware, configures authentication, and submits jobs to Hugging Face Jobs. Supported techniques include Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Group Relative Policy Optimization (GRPO) for models ranging across various parameter sizes. The integration incorporates Trackio for real-time monitoring and automates LoRA configuration for larger models. Once training completes, the agent pushes artifacts to the Hugging Face Hub and can convert models to GGUF format for local execution.
Context
Training and fine-tuning language models typically requires specialized manual decisions regarding hardware selection, authentication, training script creation, job monitoring, and hyperparameter choices.
Approach / What changed
Package training instructions, domain knowledge, and scripts into the hf-llm-trainer Hugging Face Skill, allowing coding agents like Claude Code to execute data validation, cloud GPU job submission, Trackio monitoring, and Hub deployments via conversational prompts.
Takeaways
- The hf-llm-trainer skill supports Supervised Fine-Tuning, Direct Preference Optimization, and Group Relative Policy Optimization for models including text and vision language models.
- For models exceeding 3B parameters up to 7B, the skill automatically applies LoRA on instances like a10g-large or a100-large to fit GPU memory limits.
- Trained models can be automatically converted to GGUF format with quantization such as Q4_K_M for local inference using tools like llama.cpp and Ollama.
Related reading
Codex is Open Sourcing AI models
Hugging Face Skills equips AI coding agents like OpenAI Codex to execute end-to-end machine learning workflows. By reading AGENTS.md files and interfacing via the Model Context Protocol, Codex automates dataset validation, training script updates, and job submissions to Hugging Face Jobs. The workflow supports methods including supervised fine-tuning, direct preference optimization, and reinforcement learning for models ranging from 0.5B to 7B parameters. Throughout execution, Codex tracks live metrics via Trackio, records benchmark evaluations against baselines, and maintains Markdown reports. Once training concludes, Codex merges LoRA adapters, applies GGUF quantization, and publishes the resulting models to the Hugging Face Hub for local deployment.
ben burtenshaw, shaun smithSyGra: The One-Stop Framework for Building Data for LLMs and SLMs