Scaling patterns for self-organizing multi-agent clusters with Kiro

Summary

Traditional multi-agent architectures rely on a central supervisor agent to decompose tasks and aggregate results, creating single points of failure and context-window throughput bottlenecks. To address these scaling limits on decomposable workloads, kiro-flock implements self-organizing clusters of headless Kiro CLI agents on Amazon EC2 coordinated entirely through shared Amazon S3 state. Agents operate in bounded peer topologies such as logical rings, reading neighbor append-only logs and generating artifacts without direct inter-agent messaging or central brokers. Each agent starts fresh per iteration without conversational memory, reading current direction and peer logs to prevent drift and preserve steering control. This shared-state pattern enables resilient parallelism for tasks requiring diverse independent contributions, such as code reviews or codebase migrations.

Context

Multi-agent systems using central supervisors suffer from context-window size limits, single points of failure, and fixed task decompositions that struggle with large-scale, highly parallel, or open-ended workloads.

Approach / What changed

Implemented kiro-flock, an open-source system running headless Kiro CLI agents on Amazon EC2 that coordinate without an orchestrator by reading and appending to shared state and logs stored in Amazon S3.

Takeaways

• Agents in kiro-flock communicate strictly through append-only logs and artifacts in Amazon S3 rather than direct network connections or message brokers.
• Starting every agent iteration as a fresh session without conversational memory prevents individual agent drift and ensures steering through shared logs.
• Bounded peer visibility in logical rings allows diverse approaches to develop gradually and avoids premature consensus across the agent cluster.

Related reading

• Docker · May 18, 2026

  Coding Agent Horror Stories: The Security Crisis Threatening Developer Infrastructure

  Autonomous AI coding agents increasingly handle developer workflows, but their execution model introduces critical security risks. Operating with the executing user's full permissions, these agents can read ambient secrets, inherit cloud credentials, and run destructive shell commands without human intervention. Documented incidents between October 2024 and February 2026 reveal systemic hazards, including unintentional home directory deletions and increased security vulnerabilities in generated code. To address these vulnerabilities, Docker Sandboxes isolate agent execution within dedicated microVMs, preventing access to the host filesystem and blocking common credential paths by default. Routing changes through Git worktrees and restricting network egress ensures that agent actions remain strictly bounded and reviewable before affecting main branches or production infrastructure.

  Jennifer Kohl

• Grab · Jun 22, 2026