---
title: "How Much Memory Does Your Agent Actually Need?"
description: "The post examines how much agentic memory different models can use effectively, evaluating ALTK-Evolve's self-distilled guidelines across eight models on AppWorld's 585 multi-step tasks. Memory is produced by extracting and consolidating lessons from successful and unsuccessful trajectories, then delivering either the full guideline set on every ReAct step or a fixed core plus task-relevant retrieval, without updating model weights. Results identify three patterns: strong models with headroom benefited from the full set, gpt-oss-120b gained 16.1 percentage points in task completion from curated retrieval, and GLM-5 showed no measurable gain; stricter scenario completion sometimes improved more. Curated retrieval gave gpt-oss-120b that gain with only 5% more tokens versus 51% for its full set, while prompt caching is presented as a way to reduce recurring input cost, supporting calibration to model capability rather than simply accumulating memory."
---

# How Much Memory Does Your Agent Actually Need?

[Hugging Face](https://yomu.fyi/company/hugging-face) · Vatche Isahagian, Gaodan Fang, Jayaram Radhakrishnan, Punleuk Oum, Ashwath Vaithinathan Aravindan, Evelyn Duesterwald, G Thomas, Vinod Muthusamy, Merve Unuvar, Ayhan Sebin · Aug 18, 2026

**Type:** Benchmark

## Summary

The post examines how much agentic memory different models can use effectively, evaluating ALTK-Evolve's self-distilled guidelines across eight models on AppWorld's 585 multi-step tasks. Memory is produced by extracting and consolidating lessons from successful and unsuccessful trajectories, then delivering either the full guideline set on every ReAct step or a fixed core plus task-relevant retrieval, without updating model weights. Results identify three patterns: strong models with headroom benefited from the full set, gpt-oss-120b gained 16.1 percentage points in task completion from curated retrieval, and GLM-5 showed no measurable gain; stricter scenario completion sometimes improved more. Curated retrieval gave gpt-oss-120b that gain with only 5% more tokens versus 51% for its full set, while prompt caching is presented as a way to reduce recurring input cost, supporting calibration to model capability rather than simply accumulating memory.

## Context

The work addresses the observation that adding more agentic memory does not consistently improve performance. Across models, the useful amount and delivery strategy vary with factors including capability, benchmark headroom, context-window size, architecture, guideline quality, and task distribution. The evaluation uses AppWorld's 585 multi-step tasks across nine simulated apps, with Task Goal Completion and the stricter Scenario Goal Completion as metrics.

## Approach / What changed

ALTK-Evolve extracts behavioral guidelines from successful and unsuccessful agent trajectories, consolidates them into a reusable set, and injects them at inference time without changing model weights. The evaluation compares a no-memory baseline, the full guideline set injected on every ReAct step, and curated retrieval consisting of a fixed high-confidence core plus task-relevant guidelines. Guidelines are mined only from the AppWorld training split, and prompt caching is proposed for reducing the cost of repeated static context.

## Takeaways

- Strong models with remaining benchmark headroom generally benefited from the full guideline set; DeepSeek-V3.2 gained 9.5 percentage points in TGC and 16.1 points in SGC.
- gpt-oss-120b performed best with curated retrieval, gaining 16.1 percentage points in TGC while increasing tokens per task by only 5%, compared with a 51% increase for its full guideline set.
- GLM-5 showed no measurable gain from the tested memory configurations, illustrating that additional context did not improve every evaluated model.

**Tags:** [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Performance](https://yomu.fyi/topic/performance)

- Source: [Hugging Face](https://huggingface.co/blog/ibm-research/altk-evolve-hmm)
- Source URL: https://huggingface.co/blog/ibm-research/altk-evolve-hmm
- Ingested by Yomu: 2026-08-29T05:41:39.475Z

[Read original post](https://huggingface.co/blog/ibm-research/altk-evolve-hmm)