Treating Pricing Changes Like Code Deploys

Summary

Thumbtack manages marketplace pricing by updating hundreds of thousands of bid settings stored in DynamoDB across service offerings. Previously executed through individual custom Go scripts, these pricing rollouts lacked automated runtime state tracking, dynamic database throughput pacing, and durable records for reliable rollbacks. To eliminate manual operational risk, engineering introduced a shared deployment-like engine that accepts declarative target states and executes pure Go transformation functions without direct I/O. The engine enforces idempotency, conducts dry runs with validation reports to S3, coordinates atomic DynamoDB updates with audit logging, and dynamically manages write capacity. This platform standardizes pricing updates into reviewable, resumable, and reversible operations that safely skip already written rows and check for production drift during rollbacks.

Context

Thumbtack regularly updates hundreds of thousands of bid settings in DynamoDB to adjust marketplace pricing. Historically, engineers used custom Go scripts for each rollout. While reviewed and tested in staging, scripts lacked runtime machinery to remember their execution progress when interrupted, gauge database capacity load, or create durable records for rolling back changes.

Approach / What changed

Engineering built a shared execution engine modeled after code deployments. Pricing logic was isolated into pure Go transform functions implementing generic interfaces for service and occupation bids. The engine uses declarative target states to ensure idempotency. It executes dry runs to S3 without writes, controls DynamoDB capacity with pacing intervals and backoffs, and performs atomic writes pairing bid updates with audit snapshots. Rollbacks verify audit records against production values while ignoring engine-normalized fields to prevent unintended overwrites. Failed rows are output to BigQuery and S3 for direct reruns.

Takeaways

• Declarative target-state inputs make bulk data updates idempotent, allowing failed or interrupted runs to safely resume without re-applying relative changes.
• The engine temporarily increases DynamoDB write capacity at startup, registers a deferred reset handler for safe cleanup on exit or panic, and paces writes using computed sleep intervals.
• Rollback safety checks must account for internal data normalization to prevent drift detection from falsely rejecting unmodified records.

Related reading

• Thumbtack · Apr 29, 2026

  Evaluating AI at Scale: How Thumbtack Approaches Reliability, Safety, and Quality in GenAI

  Thumbtack evaluates generative AI features across customer and service professional experiences to prevent unsupported claims, tone inconsistencies, and safety issues. The engineering organization transitioned from decentralized evaluation efforts to a dedicated cross-functional Evals team that provides shared infrastructure while preserving product team autonomy. System architectures combine rule-based checks, AI-as-a-judge scorers powered by DeepEval, trust and safety assessments, and crowdsourced human reviews. Teams leverage three parallel orchestration paths, including an MLflow-tracked stack, nightly Databricks batch jobs integrated with data warehouses, and multi-layer human-in-the-loop pipelines. Future milestones target centralized LLM judge registries, full task journey evaluations, automated judge-writing workflows, and cost-aware evaluation orchestration.

  Thumbtack Engineering

• Thumbtack · Dec 3, 2025