Loading…
How Salesforce Built an Agentic Engineering Enablement Strategy for Thousands of Software Engineers
SalesforceScott Nyberg
Summary
Salesforce addressed the challenge of scaling agentic engineering across thousands of software developers by focusing on organizational learning rather than tooling. While individual engineers initially developed disparate, uncoordinated workflows, the Technology, People, Innovation, and Learning (TPIL) team unified these approaches into the Proficiency Level (PL) Framework. The framework maps developer growth across four mindset-driven stages: AI-Assisted, AI-Validating, AI-Orchestrating, and AI-Native. To facilitate progress across these stages, TPIL introduced global AI camps, weekly working sessions, and manager coaching guides. Rather than tracking course completions or telemetry scores, Salesforce measured transformation through behavioral indicators, such as shifts toward validating agent output, designing multi-agent workflows, and asking higher-level questions regarding human judgment.
Context
At Salesforce, individual engineers across a large organization developed fragmented, local agentic workflows without a shared standard of quality, leading to inconsistent practices and difficulty compounding productivity gains across teams.
Approach / What changed
Salesforce's TPIL team created the four-stage Proficiency Level (PL) Framework (AI-Assisted, AI-Validating, AI-Orchestrating, AI-Native) focusing on behavior and mindset over tooling, supported by global AI camps, weekly sessions, and manager coaching guides.
Takeaways
- The four-stage Proficiency Level Framework covers AI-Assisted, AI-Validating, AI-Orchestrating, and AI-Native mindsets.
- Progress stalled around code validation because generating AI code proved easier than evaluating accuracy and interpreting requirements correctly.
- Adoption success was evaluated through behavior change and shifting question complexity rather than attendance, completion rates, or telemetry dashboards.
Related reading
Building Reliable Production AI with Durable Workflows
AI prototypes rely on simple prompt-response interactions, but production AI systems handling thousands of records must manage distributed execution issues like worker crashes, rate limits, and transient failures. Rather than treating long-running operations as single large jobs or rebuilding state from logs, systems require durable workflows that preserve execution history across failures. In Agentforce Grid, execution is organized into a hierarchy where column runs divide into recoverable batches and rows composed of discrete activities. Each batch checkpoints after every successful row, retrying up to ten times with exponential backoff and resuming strictly at incomplete work to prevent duplicated model calls. Exposing progress across these hierarchical boundaries ensures that both users and engineers can distinguish completed, failing, and retryable units.
Scott NybergWhy AI-Generated Code Is Easy but Engineering Trust Is Hard
At Salesforce, engineering teams discovered that AI coding agents could produce internally consistent code and passing test suites that still failed to meet actual requirements. To ensure correctness before writing code, the team developed an agentic Spec-Driven Development workflow centered on explicit specifications and gated verification. The system separates evidence-based repository lookups handled by agents from subjective judgment calls escalated to human engineers. Implementation plans must cite repository evidence reviewed by a Skeptic Agent, while a Compliance Matrix traces each success criterion directly to executable test proof. Finally, a multi-agent review system evaluates the implementation under an asymmetric rule allowing an independent judge to downgrade passing checks but never override failed gates.
Scott NybergHow AI Rebuilt Salesforce’s Decades-Old Localization Pipeline
Salesforce needed to deliver major releases across 34 languages as product localization volume surged by more than 35% under fixed budgets and unchanged delivery windows. To address these scaling constraints, the engineering team replaced their legacy translation workflows with an LLM-powered orchestration pipeline integrating prompt engineering, context engineering, and multi-stage validation. The architecture transforms existing style guides, glossaries, and terminology databases into structured context, routing interface strings through specialized prompt stages—up to 85 stages for some languages—to enforce product context, grammar, and branding rules. Combined with an automated validation loop spanning AI editing, AI validation, and human review, this system reduced localization costs by 50 to 90% while accelerating turnaround times across enterprise releases.
Scott NybergFrom Prediction to Action: How to Turn AI Outputs Into Decisions
Salesforce addressed an operational challenge where sellers faced roughly 12,000 dashboards and over 20 applications outputting machine learning predictions without clear next steps. The engineering team reframed machine learning outputs as raw signals rather than standalone answers. To bridge the gap between assessment and action, they built a Next Best Action layer that combines model signals, business logic, and contextual institutional knowledge into actionable recommendations. They integrated this layer with an AI agent using Model Context Protocol (MCP) tool contracts, enabling dynamic discovery and explicit handling of missing data. Finally, rather than introducing a separate dashboard destination, the agent serves on-demand recommendations directly inside Slack where sellers already collaborate.
Scott Nyberg