---
title: "Active Learning with AutoNLP and Prodigy"
description: "Active learning requires iteratively adding labeled data, retraining models, and serving them to end users. Building such pipelines often demands substantial effort in data labeling, model selection, hyperparameter tuning, and training infrastructure. The author demonstrates a low-code active learning pipeline using Explosion's Prodigy for entity annotation and Hugging Face's AutoNLP for automatic training and evaluation. After first training a news categorization model achieving 98.67% accuracy on Kaggle's BBC News dataset, the author manually annotated named entities across iterative batches. An export script converted annotations to JSONL with IOB tags, showing progressive metric improvements from 20 samples to 250 samples, where the token classification model attained 95.9% accuracy, 0.73 precision, and 0.79 recall."
---

# Active Learning with AutoNLP and Prodigy

huggingface.co · Abhishek · Dec 23, 2021

**Type:** Tutorial

## Summary

Active learning requires iteratively adding labeled data, retraining models, and serving them to end users. Building such pipelines often demands substantial effort in data labeling, model selection, hyperparameter tuning, and training infrastructure. The author demonstrates a low-code active learning pipeline using Explosion's Prodigy for entity annotation and Hugging Face's AutoNLP for automatic training and evaluation. After first training a news categorization model achieving 98.67% accuracy on Kaggle's BBC News dataset, the author manually annotated named entities across iterative batches. An export script converted annotations to JSONL with IOB tags, showing progressive metric improvements from 20 samples to 250 samples, where the token classification model attained 95.9% accuracy, 0.73 precision, and 0.79 recall.

## Context

Active learning involves iterative data annotation and continuous model retraining, which traditionally requires manual model selection, hyperparameter tuning, optimizer configuration, preprocessing, and evaluation.

## Approach / What changed

The author combined Prodigy for interactive named entity recognition annotation with Hugging Face's AutoNLP for automated model training and hyperparameter optimization. A Python script using spaCy extracted Prodigy database annotations into JSONL format with IOB tags, enabling iterative token classification model training on the BBC News dataset.

## Takeaways

- Training 15 multi-class classification models on the BBC News dataset via AutoNLP completed in approximately 15 minutes, with the best model reaching 98.67% accuracy.
- A custom Python script converted Prodigy annotations into AutoNLP token classification JSONL format by generating token arrays and IOB tag sequences via spaCy.
- Iterative active learning on named entity recognition progressed from 0 precision/recall at 20 samples to 95.9% accuracy, 0.73 precision, and 0.79 recall at approximately 250 annotated samples.

**Tags:** [Machine Learning](https://yomu.fyi/topic/machine-learning), [Python](https://yomu.fyi/topic/python)

- Source: [huggingface.co](https://huggingface.co/blog/autonlp-prodigy)
- Source URL: https://huggingface.co/blog/autonlp-prodigy
- Ingested by Yomu: 2026-08-27T15:07:52.330Z

[Read original post](https://huggingface.co/blog/autonlp-prodigy)