High Availability by Offloading Work Into the Background

Summary

Unpredictable traffic spikes, slow third-party payment requests, and resource-intensive image processing can overwhelm an application, making it slow or unavailable; the text presents background jobs as a way to preserve meaningful user interactions. High availability is defined not merely as server uptime, but as users being able to interact with the application when needed. The application server places a task message in a queue and continues serving requests, while workers process jobs asynchronously, with support for retries, parallelization, prioritization, and event- or time-based scheduling. Offloading improves response times and buffers spikes, but changes immediate guarantees: an upload may be accepted before processing succeeds, and errors may be handled later. For distributed systems, data-based messaging such as RabbitMQ can connect different services, while Kafka's append-only, replayable event log supports shared consumption and event sourcing.

Context

Slow, resource-intensive, error-prone, or unpredictable work can overwhelm the application server and prevent it from accepting and serving incoming requests. High availability requires meaningful user interactions when needed, not merely a server that remains running. Traffic spikes, including Shopify's reported peak of 170k requests per second, make this separation especially important.

Approach / What changed

Offload any task that does not need to finish before the response into a background job. The application server places a task message in a queue and continues handling requests, while worker processes execute jobs asynchronously. Queues can buffer spikes, and background job backends can provide retries, parallelization, prioritization, scheduling, and an abstraction that keeps concrete job code simple. For distributed services, a message broker such as RabbitMQ can carry data-based messages across different codebases; Kafka instead provides an append-only, replayable event log.

Takeaways

• A queue buffers work when workers cannot keep up, allowing the application server to continue accepting requests while queued image-processing or similar tasks wait.
• Offloading trades immediate completion guarantees for faster responses: the application can acknowledge an upload before processing succeeds and handle errors later.
• RabbitMQ-style messaging supports distributed task delegation across different services and codebases, while Kafka retains replayable events for shared consumption and event sourcing.

Related reading