ThumbtackAnanda Kanagaraj SankarProblem & solution
Treating Pricing Changes Like Code Deploys
Summary
Thumbtack manages marketplace pricing by updating hundreds of thousands of bid settings stored in DynamoDB across service offerings. Previously executed through individual custom Go scripts, these pricing rollouts lacked automated runtime state tracking, dynamic database throughput pacing, and durable records for reliable rollbacks. To eliminate manual operational risk, engineering introduced a shared deployment-like engine that accepts declarative target states and executes pure Go transformation functions without direct I/O. The engine enforces idempotency, conducts dry runs with validation reports to S3, coordinates atomic DynamoDB updates with audit logging, and dynamically manages write capacity. This platform standardizes pricing updates into reviewable, resumable, and reversible operations that safely skip already written rows and check for production drift during rollbacks.
Context
Thumbtack regularly updates hundreds of thousands of bid settings in DynamoDB to adjust marketplace pricing. Historically, engineers used custom Go scripts for each rollout. While reviewed and tested in staging, scripts lacked runtime machinery to remember their execution progress when interrupted, gauge database capacity load, or create durable records for rolling back changes.
Approach / What changed
Engineering built a shared execution engine modeled after code deployments. Pricing logic was isolated into pure Go transform functions implementing generic interfaces for service and occupation bids. The engine uses declarative target states to ensure idempotency. It executes dry runs to S3 without writes, controls DynamoDB capacity with pacing intervals and backoffs, and performs atomic writes pairing bid updates with audit snapshots. Rollbacks verify audit records against production values while ignoring engine-normalized fields to prevent unintended overwrites. Failed rows are output to BigQuery and S3 for direct reruns.
Takeaways
Declarative target-state inputs make bulk data updates idempotent, allowing failed or interrupted runs to safely resume without re-applying relative changes.
The engine temporarily increases DynamoDB write capacity at startup, registers a deferred reset handler for safe cleanup on exit or panic, and paces writes using computed sleep intervals.
Rollback safety checks must account for internal data normalization to prevent drift detection from falsely rejecting unmodified records.
Source and freshness
- Source
- medium.com
- Published
- Added to Yomu
- Last updated
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.