---
title: "Thinking of ACE? We Can Do It with Fewer Tokens"
description: "ALTK-Evolve and ACE are agentic-memory systems that turn an LLM agent’s past trajectories into reusable lessons without weight updates or human labels. Both preserve individual lessons and attach evidence counts rather than compressing experience into a short summary, but ACE injects its full evolving playbook at every step while ALTK-Evolve consolidates typed, provenance-linked guidelines and retrieves a model-appropriate subset or the full set. On AppWorld’s 168-task test_normal split with the same ReAct setup, ALTK-Evolve reached 89.3 TGC and 80.4 SGC on DeepSeek-V3.2 at 263K tokens per task, versus ACE’s 80.4 and 73.2 at 634K. With gpt-oss-120b, it reached 56.0 TGC and 37.5 SGC at 116K tokens, narrowly exceeding ACE’s 54.8 and 35.7 at 777K. The comparison uses single pass@1 runs and in-house ACE results on matched models, while the authors attribute the cost difference to delivery rather than lesson quality."
---

# Thinking of ACE? We Can Do It with Fewer Tokens

[Hugging Face](https://yomu.fyi/company/hugging-face) · Vatche Isahagian, Jayaram Radhakrishnan, Vinod Muthusamy, Gaodan Fang, Punleuk Oum, G Thomas, Ashwath Vaithinathan Aravindan, Evelyn Duesterwald, Merve Unuvar · Aug 11, 2026

**Type:** Benchmark

## Summary

ALTK-Evolve and ACE are agentic-memory systems that turn an LLM agent’s past trajectories into reusable lessons without weight updates or human labels. Both preserve individual lessons and attach evidence counts rather than compressing experience into a short summary, but ACE injects its full evolving playbook at every step while ALTK-Evolve consolidates typed, provenance-linked guidelines and retrieves a model-appropriate subset or the full set. On AppWorld’s 168-task test\_normal split with the same ReAct setup, ALTK-Evolve reached 89.3 TGC and 80.4 SGC on DeepSeek-V3.2 at 263K tokens per task, versus ACE’s 80.4 and 73.2 at 634K. With gpt-oss-120b, it reached 56.0 TGC and 37.5 SGC at 116K tokens, narrowly exceeding ACE’s 54.8 and 35.7 at 777K. The comparison uses single pass@1 runs and in-house ACE results on matched models, while the authors attribute the cost difference to delivery rather than lesson quality.

## Context

LLM agents can fail realistic multi-step tasks because they misuse known APIs, select the wrong entity, or return an unrequested value. The source presents these errors as usage patterns that can be learned from the agent’s own trajectories. It compares ACE and ALTK-Evolve because both address this problem through reusable agentic memory, while differing in how that memory reaches the model.

## Approach / What changed

ALTK-Evolve mines trajectories for lessons, extracts typed strategy, recovery, and optimization guidelines with causal attribution and provenance, clusters near-duplicates while conserving support counts, and retrieves a fixed core plus task-relevant guidelines or the full set. This contrasts with ACE’s Generator, Reflector, and Curator loop and its full playbook injected at every inference step.

## Takeaways

- Both systems retain distinct lessons instead of reducing an agent’s experience to a small summary; ACE uses per-bullet helpful or harmful counters, while ALTK-Evolve uses guideline support counts.
- On DeepSeek-V3.2, ALTK-Evolve recorded 89.3 TGC and 80.4 SGC at 263K tokens per task, compared with ACE’s 80.4 TGC and 73.2 SGC at 634K.
- On gpt-oss-120b, selective ALTK-Evolve retrieval used about 29 guidelines and 116K tokens per task, achieving 56.0 TGC and 37.5 SGC versus ACE’s full-playbook 54.8 and 35.7 at 777K.

**Tags:** [LLMs](https://yomu.fyi/topic/llm), [Performance](https://yomu.fyi/topic/performance)

- Source: [Hugging Face](https://huggingface.co/blog/ibm-research/altk-evolve-sldd)
- Source URL: https://huggingface.co/blog/ibm-research/altk-evolve-sldd
- Ingested by Yomu: 2026-08-29T05:41:05.600Z

[Read original post](https://huggingface.co/blog/ibm-research/altk-evolve-sldd)