From Agent Folder to Shippable Agent: Five Checkpoints
An Agent SDK agent is a folder of instructions, tools, skills, and channels. This walkthrough follows five checkpoints from local validation to gateway-gated tool calls and replayable evals.
DatabricksLisa Cao, Sudhanva HuruliBenchmark
AI agents now create over 70% of Spark data pipelines on Databricks, making cheap iteration and readable code critical. A benchmark comparing procedural PySpark with declarative pipelines tested 22 tasks across 12 datasets each, totaling 528 builds using Opus 4.8 for agent coding and an automated grader. Declarative pipelines enable dry-run validation that catches structural errors before execution, reducing cluster compute time by roughly 34x compared to procedural pipelines, which used 596 executor-seconds versus 17. Declarative code was about half the size (median 64 vs 131 lines) with less variability, while silent error rates were similar within about 15%. The declarative approach trades 2.3x higher LLM token costs ($0.50 vs $0.26 median) for eliminating wasted cluster time, a trade-off that improves as data volumes grow because compute waste scales while token costs are fixed.
Declarative pipelines describe the datasets you want and leave Spark to decide how to build them. They can be validated before they run via dry-run, catching structural errors like unresolved column references, missing table dependencies, and ordering issues.
Declarative pipelines reduced cluster compute time by roughly 34x compared to procedural pipelines because dry-run validation catches structural errors before execution, preventing wasted cluster time on flawed builds.
Declarative pipeline code was about half the size of procedural code (median 64 vs 131 lines) with less variability across implementations, reducing the amount of code that needs review and maintenance.
Silent logic error rates were similar between declarative and procedural pipelines (within about 15%), so both approaches still require tests and review to catch incorrect output that runs cleanly.
No, because the declarative agent only writes declarative pipeline code describing tables, and Spark checks that description before anything runs. You trust the platform's evaluation, not the model. The agent never runs code or reads your data itself.
It applies more broadly to any declarative approach where you can separate 'here is what I want' from 'now go build it,' so the first part can be checked independently. Examples include SQL, infrastructure-as-code tools, and schema-first APIs. The benchmark uses Spark Declarative Pipelines, but the reasoning holds for any such tool.
Not at realistic pipeline sizes. The 2.3x token increase is about $0.50 vs $0.26 median build at Opus 4.8 list rates, while one failed procedural run cost about 521 executor-seconds compared to half a second for a rejected declarative build. Avoiding a few bad runs covers the extra token cost, and the gap widens as data volumes grow.
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
An Agent SDK agent is a folder of instructions, tools, skills, and channels. This walkthrough follows five checkpoints from local validation to gateway-gated tool calls and replayable evals.
Cekura generates simulated caller scenarios to test voice-agent prompts, so failures surface before real calls. The trade-off: fast, repeatable iteration covers only the failures you thought to script.
A case study sells a peer's outcome; a benchmark proves a repeatable delta. This side-by-side shows what each kind of evidence establishes, what it hides, and how to pick the right proof before you commit.
Keep reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.