Loading…
Salesforce
The world's leading cloud-based Customer Relationship Management (CRM) platform that helps businesses manage customer data, interactions, and internal workflows.
- 24 posts summarised
- latest
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
From their blog
Latest articles
Apache Spark Resource Optimization: Lessons From Eight Million Jobs a Day
Salesforce Data 360 processes about eight million Apache Spark executions daily, where successful completion alone does not confirm efficient resource use. The team instrumented the shared Data Processing Controller (DPC) gateway with a DataFlint Spark listener plugin to capture per-executor CPU, memory, shuffle, and zero-task metrics without application code changes. Metrics stream through Kafka into an Iceberg lakehouse, and a curated Silver analytic layer ensures analysis uses only complete, comparable runs and correctly handles retries and missing data. Investigation revealed workloads that repeatedly allocated executors performing zero tasks, enabling targeted reductions like lowering initial executors from forty to four. After staged production rollout monitored for runtime, spill, retries, and platform pressure, weekly compute-cost run rate fell 31% while DPC job starts grew over 7%.
AI Agents Need Memory: Building Persistent Memory for Agentforce
Enterprise AI agents without persistent memory suffer from the “Groundhog Day” problem, repeatedly asking users to re-explain themselves. Agentforce Memory addresses this by giving agents persistent memory that retains user preferences, conversation history, and context across sessions. The team first established a shared terminology document to define concepts like semantic and episodic memory before architectural discussions could proceed. The architecture evolved incrementally, starting with conversation history injection and later adding semantic retrieval to find relevant past information regardless of recency. Memory retrieval uses keyword search during pre-orchestration to balance context quality with low latency, and moving memory into the Reasoner simplified configuration for administrators. Privacy and user trust were treated as architectural requirements from the start, enabling individual user controls over memory creation and management.
How Deterministic Controls Turn AI Output Into Reliable Prompt Templates
Salesforce’s Assistive Authoring aimed to reduce prompt-template creation from 25 minutes to under a minute by using Agent Graph. A router node dispatches natural-language requests to specialized sub-agents for creation, refinement, or clarification while state variables carry context. Deterministic controls prevent the LLM from corrupting routing, record identity, or structured output through guarded transitions and a server-side transpose layer. A discovery service normalizes available data fields into an allowlist, and the LLM can only select from validated references. A human-in-the-loop gate ensures the agent never saves or executes, presenting output as an accept-or-reject diff only.
Engineering Multi-Agent AI Teams That Build and Test Themselves
Salesforce Marketing Cloud’s AI Delivery Lead Sohini Arya and team built Agent Designer, a system that automates the creation and testing of multi-agent AI teams. The orchestrator cannot write agent files itself; instead, seven specialist agents analyze architecture, failure modes, and compliance before the orchestrator synthesizes findings for human review. The system transforms an engineer’s request into a tested multi-agent team in 15-30 minutes, down from 2-4 hours manually, and helped approximately 200 engineers adopt the manager-of-agents model. Governance challenges included enforcing write-scope boundaries via prompts, reconciling independently produced design conflicts, and applying topology-specific budget formulas that account for flat, swarm, or team structures. Verifier agents return structured PASS/WARN/FAIL blocks, and self-healing is limited to two fix attempts and a $3 ceiling to prevent runaway repairs, with the orchestrator’s 500-turn budget divided across analysis, approval, generation, testing, and retries.
How Intelligent Load Shedding Prevents Cascading Failures in Tier-0 Systems
Salesforce’s Cloud Atlas team operates a globally distributed identity data store that must maintain five nines of availability, roughly five minutes of downtime per year. The team redesigned service protection after per-instance rate limiting proved insufficient under autoscaling and increasingly bursty, unpredictable agent-driven traffic. The new approach combines intelligent load shedding, which monitors queue time as an early indicator of system pressure before error rates spike, with coordinator-free global quota management. Servers independently measure and share customer usage to throttle noisy tenants without a centralized coordinator, preventing one customer’s surge from affecting others. This design prevents cascading failures by shedding lower-priority work before overload spreads across dependent services, ensuring graceful degradation rather than widespread outage.
AI Agent Observability: Making Silent Production Failures Visible and Actionable
Agentforce Health Monitoring (AHM) addresses silent availability failures in production AI agents that traditional monitoring misses because agents can appear healthy even when unreachable. The team unified telemetry from approximately five or six locations across sessions, reasoning steps, tool calls, errors, and escalations into a coherent representation. End-to-end alerting latency was reduced from roughly 20 minutes to several minutes by converting Data 360 ingestion to streaming and simplifying query complexity. AHM provides 16+ built-in metrics including agent availability, error rate, response/escalation rate, engagement, and latency, with a drill-down interface connecting alerts to session context. The roadmap includes cost, TTFT/TTLT, toxicity, feedback metrics, RAG Quality Monitoring integration, and eventually a runtime insight agent for root-cause analysis and automated remediation.
How Data 360 Builds Trusted Context: The Enduring Layer for Enterprise AI
Enterprise AI needs a way to assemble the smallest sufficient set of current, relevant, and authorized evidence for each agent turn instead of sending an entire data estate to a model. Salesforce presents Data 360 as a shared Trusted Context runtime spanning Salesforce, external platforms such as Databricks and Snowflake, operational systems, documents, knowledge bases, and live signals. Its progressive pipeline connects data through ingestion and Zero Copy, then harmonizes, unifies, models, retrieves, reconciles, governs, and compiles evidence into a token-fit Context Pack with citations and action contracts. The Agent Context Engine also returns typed interaction traces, promoting only approved facts into durable memory and validated outcomes into learnings. Q2 FY27 production usage is reported at 104.3 trillion connected records, including 81.5 trillion Zero Copy rows, while proposed benchmarking covers correctness, authorization, provenance, freshness, latency, cost, and memory continuity.
How AI Agents Get Trusted Customer Context with Data 360 Data Graphs
Salesforce’s AI Engineering team uses Data 360 data graphs to close the context gap that prevents agents from reliably personalizing customer interactions. The graphs join, aggregate, and apply business logic across fragmented identities, accounts, products, contracts, orders, cases, services, and entitlements, while partitioned data spaces provide a filtered customer-success view. The team designs smaller graphs or multi-graphs around agent access patterns, adds indices, supports semantic or keyword search, and curates context so agents can traverse extensible many-to-many relationships without receiving unnecessary data. Reusable deployment automation and validation practices helped deliver five major releases in six months, with integrations spanning Postman, Apex, Agentforce, and low-code or no-code workflows. Live monitoring later showed P50 performance below 200 milliseconds, compared with an earlier benchmark of about 400 milliseconds.
Engineering Real-Time Mobile Personalization Across iOS, Android, React Native, and Flutter
Building real-time mobile personalization across iOS, Android, React Native, and Flutter required one architecture to handle different rendering models, lifecycles, identity flows, consent requirements, and release cycles without disrupting native app experiences. The team standardized schemas, event semantics, identity behavior, fallback handling, and SDK APIs, using bridge layers so React Native and Flutter could access native iOS and Android capabilities. Developers instrument apps once with constrained content zones and approved native components, while marketers configure templates, targeting, and content through Salesforce UI; metadata is published via a global CDN and rendered dynamically by the SDK. Data 360 performs server-side identity resolution and decisioning, limiting unnecessary raw profile storage on devices. An early customer implemented the system in two months and generated more than 150 million decisions within three weeks, while QR-code previews enabled testing against live profiles without app recompilation.
Enterprise AI Accuracy: Building a More Trustworthy RAG Application
An enterprise RAG pipeline scored above 90% on standard text-retrieval benchmarks but approximately 46% on complex, customer-representative documents, with failures spanning parsing, chunking, enrichment, embeddings, retrieval, and answer generation. The proposed diagnostic traces wrong answers backward from generated response to retrieved context, chunks, and parsed source, using human-validated end-to-end evaluation and changing one stage at a time. Intelligent Parsing routes pages according to complexity and preserves relationships in structures such as multi-page tables; in a staged benchmark, it raised accuracy from 65.1% to 84.4%, while Enriched Indexing raised it to 86.8%. Retrieval improvements include dynamic metadata pre-filters for explicit constraints and GraphRAG for multi-hop evidence, with GraphRAG slated for general availability in November 2026. Across broader improvements, complex-document accuracy exceeded 90%, and service-based Just-in-Time indexing reduced sub-100 MB payload indexing from 30+ minutes to approximately seven to ten minutes.
How Metadata and Runtime Telemetry Reveal Which Enterprise System Problems to Fix First
Salesforce Health Insights addresses enterprise org health monitoring by combining configuration metadata with runtime telemetry to identify and prioritize remediation. The team evolved a static report into an in-app experience with durable finding dispositions, approximately 400+ signals across domains, and standards informed by Well-Architected principles and implementation anti-patterns. A shared harmonization layer reconciles fragmented datasets, connector patterns link runtime activity to metadata components, and the system preserves data origin, freshness, and access requirements while keeping deterministic findings distinct from externally supplied context. Prioritization combines findings with request volume, application and database CPU, call behavior, and peak-business-hour traffic to rank next best actions by criticality and effort. A headless JSON schema supports interfaces including an MCP connection, while the deterministic core is intended to produce dependable recommendations and extend access beyond the primary interface.
How Agentforce Achieves 100% Deterministic Rendering for AI Agent UX
Agentforce faced a production problem: LLM-driven rendering could recognize a required form or regulated disclosure yet choose a different output because its reasoning was stochastic. The Agentforce Connections layer addresses this by separating probabilistic reasoning and orchestration from deterministic presentation: builders associate an action with a connection-specific response format through the render: directive, and the platform applies that format instead of asking the LLM to select a component. Multi-action, long-running, and supervised flows required the system to identify the configured action and render only after approval and execution. The same abstraction supports Lightning Web Components, third-party interfaces, and headless structured JSON outputs. Per-action, per-session event logs record the action, format, connection, and conversation point, providing auditable evidence that required rendering occurred.
Why AI Agents Get the Right Facts but the Wrong Answer—and How GraphRAG Helps
The post examines why an agent can cite accurate return-policy evidence yet reject an eligible customer when decisive facts are distributed across systems. Its washing-machine scenario combines a product classification, an extended-policy rule, membership tier, purchase amount, and purchase date to show that retrieving enough evidence differs from retrieving relevant evidence. GraphRAG addresses the gap by extracting entities and relationships into a knowledge graph, then using multi-hop retrieval to follow connections from WM-500 to the applicable policy and supporting records. The implementation also uses explicit pointers from graph entities to structured data, while separating the TBox blueprint from ABox instances and requiring business validation of the schema. The diagnostic framework distinguishes missing retrieved context, omitted business rules, and broken record mappings, while noting that these examples are not benchmark results and that correct evidence can still be used incorrectly.
How AI-Powered Attacks Led Salesforce to Reinvent Hyperscale DDoS Defense
Salesforce built DREAM (DDoS Response and Mitigation) to defend its distributed cloud against hyperscale, machine-speed DDoS attacks, with a stated goal of responding in seconds while protecting more than 4.5 million domains and 3 million customer orgs at 99.99% availability. The platform targets rapidly evolving application-layer attacks in a shared, multi-tenant environment by combining AI-driven inference with distributed orchestration for telemetry, policy deployment, notifications, retries, and fault-tolerant execution across regions. The team selected Temporal rather than rebuilding distributed-systems primitives, and used AI coding assistants to migrate about 100,000 lines at roughly ten times prior velocity; more than 90% of the rewrite was AI-assisted, with human reviews and test coverage above 95%. Production lessons moved bulk data outside orchestration and bounded workflow history, while DREAM reduced time to first mitigation fivefold.
From Prediction to Action: How to Turn AI Outputs Into Decisions
Salesforce addressed an operational challenge where sellers faced roughly 12,000 dashboards and over 20 applications outputting machine learning predictions without clear next steps. The engineering team reframed machine learning outputs as raw signals rather than standalone answers. To bridge the gap between assessment and action, they built a Next Best Action layer that combines model signals, business logic, and contextual institutional knowledge into actionable recommendations. They integrated this layer with an AI agent using Model Context Protocol (MCP) tool contracts, enabling dynamic discovery and explicit handling of missing data. Finally, rather than introducing a separate dashboard destination, the agent serves on-demand recommendations directly inside Slack where sellers already collaborate.
Why AI-Generated Code Is Easy but Engineering Trust Is Hard
At Salesforce, engineering teams discovered that AI coding agents could produce internally consistent code and passing test suites that still failed to meet actual requirements. To ensure correctness before writing code, the team developed an agentic Spec-Driven Development workflow centered on explicit specifications and gated verification. The system separates evidence-based repository lookups handled by agents from subjective judgment calls escalated to human engineers. Implementation plans must cite repository evidence reviewed by a Skeptic Agent, while a Compliance Matrix traces each success criterion directly to executable test proof. Finally, a multi-agent review system evaluates the implementation under an asymmetric rule allowing an independent judge to downgrade passing checks but never override failed gates.
How to Evaluate Production AI Agents: Measure System Outcomes, Not Conversations
Evaluating AI agents solely on conversational transcripts fails when agents must invoke tools to modify external production systems. Language models can convincingly narrate that an action occurred while completely omitting the underlying tool call, leaving database or billing records unchanged. Outcome-based evaluation addresses this divergence by treating external system mutations rather than generated dialogue as the actual source of truth. Frameworks such as CRMAgentBench implement persistent, stateful environments with strict all-or-nothing validation, verifying tool arguments, execution sequence, final record modifications, and the total absence of unauthorized actions. Furthermore, measuring repeated success across independent executions using pass^k metrics exposes operational variance and inconsistency that standard single-shot scores and pass@k metrics conceal.
How Agentforce-Powered AI Security Workflows Accelerate Incident Response
Salesforce engineers expanded Security Center from a basic conversational interface into an Agentforce-powered stateful investigations platform for incident response. Managing enterprise security incidents required handling long-running lifecycles, disparate telemetry formats, and finite LLM context windows that risk reasoning pocket collapse. To prevent overflowing context limits, the architecture partitions telemetry into contextual segments and uses AI-driven summarization pipelines before feeding data to the agent. Because non-deterministic LLM outputs break traditional deterministic testing, the team implemented AI-driven evaluation pipelines using simulated customer interactions to evaluate response behavior rather than exact wording. Ongoing work focuses on grounding public foundation models with curated, Salesforce-owned security knowledge to improve platform-specific remediation guidance.
How Standardizing Product Telemetry Reduced Time to Insight by 97%
Salesforce faced scaling bottlenecks when individual engineering teams maintained fragmented, custom telemetry pipelines requiring manual data transformations and dashboard creation every release. To resolve these silos, the engineering organization established the Product Data Platform (PDP) using a standardized custom schema built atop Monitoring Cloud infrastructure. This framework mandates core telemetry fields for consistent metric generation while offering governed optional attributes for product-specific flexibility. An AI-driven Model Context Protocol tool assists developers by recommending compliant instrumentation patterns directly within their workflows. Today, the unified platform processes 45 billion rows of data daily across 19,000 distinct events and 2,000 product features, reducing dashboard refresh latency by 97 percent from one month to daily updates.
How Salesforce Eliminated Single-Region Risk and Reduced Downtime Blast Radius at 4B Metrics/Min
Salesforce redesigned its internal observability platform, Argus, to eliminate single-region failure risks while ingesting approximately 4 billion metrics per minute. Operating out of a single AWS region previously created a global blast radius and generated high cross-region data transfer costs. To mitigate these risks without the prohibitive expense of full multi-region replication, the engineering team implemented a geo-local architecture that processes and stores telemetry closer to its origin across production geographies. A newly introduced federation query layer relies on Elasticsearch mappings to route queries selectively to relevant regional clusters, avoiding broad fan-out. The platform also adopted metadata caching for wildcard queries and implemented HTTP 206 partial-response handling to maintain visibility when individual regions experience downtime.
Topics they write about
All topics →Weekly digest
The best engineering reads, once a week.