Topic

Self-Improvement

1 post about Self-Improvement. Every summary links to the original.

Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.

Newest first

Posts

Apple
RLTL;DR: Self-Improvement by Internalizing Self-Generated Feedback
Standard reinforcement learning with verifiable rewards (RLVR) fails when tasks are so difficult that agents have no chance of success and no teacher models exist. RLTL;DR addresses this by having the policy write its own TL;DR insight after each failed attempt, conditioning the next rollout on all previous insights. Backpropagation on the in-context insights internalizes a direct task-to-insight mapping. On challenging tool-calling and coding datasets filtered to Pass@128=0, standard GRPO training stays flat at 0-1% Pass@1, while RLTL;DR achieves 14-31% Pass@1 with insights in context during training and 12-13% without at eval time. Reducing to SFTL;DR, training on only 4k (task, insight) tuples without rollouts, recovers almost full performance of RLTL;DR and classical SFT, demonstrating a promising compacted training paradigm.

The best engineering reads, once a week.

Topic

1 post about Self-Improvement. Every summary links to the original.

Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.

Newest first

Apple
RLTL;DR: Self-Improvement by Internalizing Self-Generated Feedback
Standard reinforcement learning with verifiable rewards (RLVR) fails when tasks are so difficult that agents have no chance of success and no teacher models exist. RLTL;DR addresses this by having the policy write its own TL;DR insight after each failed attempt, conditioning the next rollout on all previous insights. Backpropagation on the in-context insights internalizes a direct task-to-insight mapping. On challenging tool-calling and coding datasets filtered to Pass@128=0, standard GRPO training stays flat at 0-1% Pass@1, while RLTL;DR achieves 14-31% Pass@1 with insights in context during training and 12-13% without at eval time. Reducing to SFTL;DR, training on only 4k (task, insight) tuples without rollouts, recovers almost full performance of RLTL;DR and classical SFT, demonstrating a promising compacted training paradigm.

The best engineering reads, once a week.