Active Storage Is a Boundary Decision
See how Active Storage balances local simplicity, durable object storage, and preview-processing dependencies so you can choose the right boundary for a Rails file workflow.
Loading…
Topic
S3
6 posts about S3. Every summary links to the original.
- 6 posts
- 5 companies
- 1 explainer
See the idea
S3 explainers
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Building cloud-native PACS on AWS
Multi-hospital networks face rising imaging volumes, siloed on-premises PACS, radiologist routing bottlenecks, and storage that must expand for 7 to 10 years of retention. The proposed pattern uses a hub-and-spoke hybrid architecture: local PACS instances preserve LAN-speed reading, while a centralized AWS archive provides cross-facility discovery, disaster recovery, and lifecycle-managed storage. Modalities send DICOM through C-STORE to local VNA servers, which normalize and compress objects; asynchronous replication sends images to Amazon S3, while Amazon Aurora PostgreSQL stores metadata and routing information. The AWS deployment spans two Availability Zones, places PACS services behind Network Load Balancers, and includes CloudFront and AWS WAF. Storage guidance distinguishes S3 Intelligent-Tiering, Glacier Instant Retrieval, and Glacier Deep Archive according to access frequency and retention needs.
From zero-shot forecast to purchase order with Amazon Bedrock AgentCore
This post presents an inventory automation architecture that moves from zero-shot demand forecasts to validated purchase orders while applying safety stock, minimum order quantities, budgets, and promotional adjustments. It combines Amazon Chronos2, hosted on SageMaker Serverless Inference, with four Claude-powered agents built using the Strands Agents SDK and deployed on Amazon Bedrock AgentCore; deterministic Python tools perform computation and agents handle judgment. Product data and future covariates arrive through CSV files in S3, while business rules remain in JSON, enabling what-if forecasts and onboarding without per-SKU training. In internal testing across 50 SKUs over a four-week horizon, the system reached 12.3% median WAPE, reduced onboarding from 2–3 weeks to under five minutes, and lowered estimated monthly inference cost from about $1,091 to $15. A conditional retry loop surfaces budget violations for user decisions instead of silently truncating orders, with traces and evaluator scores supporting auditability.
Rebuilding Notion’s lexical search reindexer
Notion’s lexical search reindexer had become a multi-week, manually monitored process that produced about 90% data consistency. The old design also relied on Snowflake metadata lookups and a separate catchup pipeline. The redesign replaces ECS/Fargate workers and live Elasticsearch writes with an Apache Spark–native pipeline that routes documents to exact index-and-shard pairs and generates native snapshot files through embedded Elasticsearch nodes. Temporary indices capture live updates while the snapshot is built, after which aliases, external document versions, Elasticsearch’s _reindex, and phantom detection handle catchup, including alive-to-deleted transitions. The new system completes full reindexing in under two days, catchup in under an hour, reaches 100% consistency, removes Snowflake and ECS from the indexing path, and reduces manual intervention to under two hours per run.
Globally Distributed Object Storage with Tigris
Globally distributed object storage is presented as a way to avoid the latency and application complexity that arise when modern apps route uploads and downloads through a single-region S3 bucket. Tigris addresses this with redundant FoundationDB clusters for object tracking, Fly.io NVMe volumes as a first-level raw-byte cache, and a queueing system modeled on Apple’s QuiCK paper that sends data to replicas, in-demand regions, and third-party stores such as S3. Objects smaller than about 128 kilobytes are instantly global by default, while the platform also provides inter-region routing, consistency guarantees, instant purge, and relay regions. Its S3-compatible API means existing S3 libraries can be reused, and fly storage create injects the required AWS configuration and secrets. Fly.io says the partnership delivers object storage under its billing system without requiring a separate account.
Rails 5.2 Active Storage: Previews, Poppler, and Solving Licensing Pitfalls
Rails 5.2’s Active Storage adds file uploads and attachments for Active Record models, with backing services such as AWS S3 and previews for PDFs and videos, but its beta PDF support depended on MuPDF. Security and legal review found MuPDF’s AGPL licensing could require commercial licensing, so Terence Lee updated Active Storage to support Poppler, an open-source PDF backend merged before release. Deployment guidance uses S3 through aws-sdk-s3, configures :amazon rather than the non-scalable local disk, and installs a custom buildpack containing FFmpeg and Poppler. The example Rails app attaches files to a Bulletin model, accepts uploads, and chooses PDF or video previews, image variants, or downloads according to available binaries and MiniMagick.
Future Proofing Our Cloud Storage Usage
Shopify’s growth from 375,000 to over 600,000 merchants coincided with about 450,000 S3 operations per second and rising error rates plus read and write latency. Merchant-uploaded images and theme installations exposed a partitioning problem: S3 groups keys by shortest common asset prefixes, and abrupt throughput spikes could trigger SlowDown failures across a range of shops. Black-box observations suggested affected keyspaces were unlikely to recur, while newer shops were more often involved, though the latter was explicitly described as a shaky inference. Shopify prepended a hash-generated digest to new shops’ asset keys, distributing writes across more partitions instead of concentrating them under shop-specific prefixes. After the first post-change SlowDown event, exceptions nearly disappeared, while median latency for digested assets fell about 60% and the 95th percentile about 25%.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.