Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,671 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Office Hours with Engineering Managing Director Mae Santos
Mae Santos, Head of Critical Applications and Data Reliability Engineering at Two Sigma, leads a global team across New York, Houston, London, and Tokyo to support critical applications and data pipelines. While software teams frequently prioritize functional features over operational stability, designing systems with reliability in mind from the beginning allows for progressive enhancements without complete rewrites. System architecture should be guided by concrete goals across four key dimensions: availability, observability, scalability, and supportability. In addition, mapping out interconnections among systems, infrastructure, data, and human processes helps teams manage dependencies effectively. Applying automation across support and delivery workflows further reinforces reliability by minimizing human error and accelerating execution.
Optimizing Data Reliability: Heroku Connect & Drift Detection
Heroku Connect syncs Salesforce data with Heroku Postgres, but primary polling can miss changes when long-running Salesforce automations commit after the polling window advances, creating drift. The service uses secondary polls over the same bounds as primary polls to find and repair missed changes, with Bulk API or SOAP API selected according to dataset size and retries for failed polls. Recent improvements change Bulk API drift handling by creating one secondary bulk job for each primary bulk job, then adding corresponding batches instead of creating many jobs. Secondary tasks are now scheduled and executed with limited concurrency, reducing simultaneous database connections consumed while Postgres is updated and preserving capacity for applications. The post says these changes reduce load on Salesforce and improve operational reliability during heavy synchronization, while noting that Heroku Connect remains asynchronous rather than truly real-time.
AWS without Access Keys
The post describes how Fly Machines can access AWS resources such as an S3 bucket without distributing long-lived AWS access keys. It uses OpenID Connect (OIDC): AWS IAM trusts Fly.io’s OIDC identity provider, a role grants narrowly scoped permissions, and the app receives the role ARN as an environment variable. At boot, Fly’s Rust init requests a machine-specific OIDC token through the /.fly/api Unix socket, writes it to /.fly/oidc_token, and sets AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_SESSION_NAME so the AWS SDK can call STS AssumeRoleWithWebIdentity. AWS verifies the token, issues short-lived credentials, and evaluates the role’s IAM policy before serving the requested object. The trust policy can constrain tokens by audience and subject, including an organization, Fly App, or machine; the same pattern can authenticate against other OIDC-compliant providers, although Azure and GCP lack equivalent init integration.
Enhancing Cloud Usage Forecasting, Monitoring & Optimizing
After Etsy completed its 2020 migration from an on-premise data center to Google Cloud Platform (GCP), the resulting FinOps team developed a global approach to forecasting, monitoring, and optimizing cloud spend. Costs are grouped into core infrastructure, machine learning and product enablement, data enablement, and Dev, with trailing Cost Per Visit (CPV) serving as the primary forecasting metric after regression tests found visits most correlated with costs. BigQuery and Looker dashboards, alerts, weekly Finance comparisons, and recurring cost-owner and leadership readouts support variance detection, while costs are tracked centrally without chargebacks to preserve flexibility. Optimization work combines FinOps, engineering, and Architecture collaboration, producing GCS storage reductions through monitoring, automation, TTLs, and Intelligent tiering, plus compute, model-deployment, and network compression improvements. The next priority is better visibility into shared machine learning platform spend to improve experiment ROI analysis and identify optimization opportunities.
eBay’s Christine Su Named to Fast Company’s 2024 Queer 50 List
eBay Head of Product Christine Su has been named to Fast Company’s 2024 Queer 50 list for the second consecutive year. The list recognizes LGBTQ leaders across industries, while Su’s role covers product work for AI Recommendations, Homepage, and My eBay. She describes using AI to personalize the eBay experience for buyers and sellers, aiming to connect people with products and communities aligned with their passions. Since her first inclusion in 2023, eBay has added AI-powered features including Shop the look, an immersive carousel that curates complementary fashion looks from a buyer’s past shopping history for fashion discovery within the marketplace. Su says the intended experience should feel personal, support discovery, and connect users with people who share their passions.
Make Better Decisions by Embracing Uncertainty
Ramp uses multi-armed bandits to choose among third-party bank-linking providers when provider performance varies by financial institution and available data is noisy, incomplete, or stale. Instead of maintaining separate, static A/B tests, it applies Thompson Sampling, modeling each provider’s success probability with a Beta distribution and repeatedly sampling, routing traffic to the highest sampled value, then updating the model from the result. Simulations examined changing success-rate trends, noise, volume, rolling-window size, trend type, and initial conditions; the algorithm generally exploited clearly better providers, split traffic more evenly when rates were close, and recovered eventually from incorrect starting conditions. In a sequential quasi-experiment, Thompson Sampling increased financial-institution linking success by 10% and reduced customers with manually uploaded bank statements by 25%.
Efficient Visual Representation Learning And Evaluation
Etsy uses image vector representations for visual search and similar-item recommendations, making low-latency, memory-aware models important for online query inference and downstream retrieval. The work compares EfficientNetB0, a compact CNN, with EfficientFormer-l3, which downsamples multiple blocks and applies attention only in its final stage; its representation uses the last block’s eight-head attention and average-pools the sequence. Fine-tuning adds an embedding and multiple classification heads, while multitask training supervises taxonomy, color, and review-photo labels; both models train on two 16GB P100 GPUs. Evaluation rebuilds a brute-force candidate index after each epoch and measures Recall@5 and Recall@10 on interaction and intra-item retrieval datasets, with early stopping available. EfficientFormer-l3 improves reported offline recall by about 5%, 17%, and 1.8% across three tasks, while experiments report lifts in CTR, purchase rate, click-recall@100, and clicks.
Profile-guided optimisation (PGO) on Grab services
Profile-guided optimisation (PGO) improves Go application performance by feeding production CPU profile data into subsequent compiler builds. Teams evaluated PGO across multiple Grab services, including TalariaDB and an orchestrator within the Catwalk service, using Go 1.20 and Go 1.21 builds. TalariaDB saw substantial resource savings after compiling Docker images with a 360-second pprof profile, resulting in at least a 10% drop in CPU usage, a 30% drop in memory consumption, and a 38% reduction in persistent volume usage for event ingestion queues. Conversely, testing on Catwalk showed only a 5% performance gain after adjusting the profiling duration from 59 seconds to 6 minutes, an outcome deemed insufficient to justify pipeline modifications. Broader adoption across monorepo services remains constrained until the central build process supports PGO arguments and automated profile retrieval.
Background Enhancement Tool Turns Any Photo Into a Studio-Quality Product Image
eBay’s background enhancement tool addresses the effort and expense of producing polished product photographs for listings. Sellers can remove an existing photo’s background with eBay’s background removal tool, choose from AI-generated scenes, and apply a new backdrop during listing in the iOS and Android mobile apps. The feature is available to iOS users in the U.S., U.K., and Germany, with Android rollout planned over the coming months, and remains in early rollout for feedback and iteration. It uses open source deep learning inpainting models, including Stable Diffusion, alongside eBay’s anonymous data and infrastructure; future visual generation models will be fine-tuned using seller feedback. Responsible AI collaboration, accuracy guidance, and transparency reminders are intended to reduce misrepresentation of products and their condition.
More problems
The post outlines research problems for the next phase of AI programming, centered on making coding systems predict actions, perform larger edits, understand codebases, and debug software. Proposed directions include low-latency next-action models for keystrokes, locations, files, terminal commands, and edits; background execution for “perfect edits” and pseudocode compilation; and multi-file changes supported by stronger tool use, reasoning, and agent trajectories. For context, retrieval alone is described as insufficient for codebase questions requiring multiple hops, while useful context may span documentation, source, history, UI pixels, logs, and messages. The post recommends combining retrieval, recurrence, and long-context attention, alongside specialized rerankers, multi-hop embedders, caching, and codebase-level retrieval. For reliability, it proposes calibrated bug detection, always-on AI linting, runtime-state tracking through cursor/debug, and improved debugger tool use, while noting trade-offs involving latency, false positives, infrastructure, and user experience.
How we evaluated the business impact of marketing campaigns
Grab needed a scalable, centralised way to evaluate the business impact of millions of marketing messages sent across expanding campaigns. Manual analysis by marketing analysts produced inconsistent results and failed to scale, leading to the creation of multi-touch attribution and controlled experiment methodologies. The team designed a Marketing attribution model (MAM) that credits touchpoints based on relevance and recency, while using stratified hold-out groups to assess single campaign lift against baseline metrics. Statistical significance calculations, implemented using the jStat library based on the central limit theorem, ensure observed business metric differences like gross merchandise value are not random. The system provides campaign managers with direct access to impact analyses while the team works on challenges surrounding smaller sample sizes and hold-out group constraints.
Editing Files at 1000 Tokens per Second
Large code edits expose coding agents to laziness, inaccuracy, and latency, especially when frontier models must rewrite hundreds of lines. The post presents fast apply, a specialized full-file code-edit model that separates planning from applying: Cursor's frontier model plans the change, while the apply model rewrites the current file. It evaluates roughly 450 edits on files under 400 lines, using Claude-3 Opus as grader, and finds llama-3-70b-ft nearly matches claude-3-opus-diff while outperforming GPT-4-Turbo and GPT-4o. Rather than generate conventional diffs, it uses full-file rewrites and search/replace hunks, which avoid line-number errors and give models more output tokens for reasoning. A custom speculative decoding method, deployed with Fireworks, reaches approximately 1,000 tokens per second on a 70b model, about 13x faster than vanilla Llama-3-70b inference and 9x faster than the previous GPT-4 deployment.
Evolving the Backend Storage for Platform Metrics
Heroku migrated MetaaS, its internal Metrics as a Service platform metrics backend, from self-hosted Apache Cassandra to AWS’s DynamoDB. MetaaS collects application observations, aggregates them into per-application, per-minute statistics such as median, maximum, and 99th-percentile response time, and stores time-series data for dashboard metrics, alerting, and autoscaling. The migration first duplicated Kafka stream-processing writes into DynamoDB, then used “Science Mode” to compare Cassandra and DynamoDB read results in production without changing returned data. After edge cases were fixed and results matched, retention-period TTL expiry avoided a separate historical data transfer, while gradual query cutovers provided another safeguard. Since 100% of MetaaS queries moved to DynamoDB in May of last year, DynamoDB has remained reliable at scale and been faster and cheaper than the Cassandra clusters, with no noticeable effect on users.
Picture This: Open Source AI for Image Description
Nolan Darilek presents a self-hosted image-description service intended to help blind screen-reader users understand photos, diagrams, menus, and other visual material. Humans often provide inadequate alt text, and the post contrasts that history with increasingly capable LLM-generated descriptions from apps and browser add-ons. The implementation combines Ollama, running the Apache-licensed LLaVA 34B multimodal model, with a PocketBase API over SQLite and a small Python client; collection hooks send images and follow-up questions, retain chat history, and enforce per-user access rules. In testing, a cold request on an A100-40GB Fly Machine took about 45 seconds, while stopping idle GPU Machines reduced resource use at the cost of startup delay. The example produced useful scene descriptions but could not reliably identify leafless tree species, illustrating both the prototype’s value and its limitations.
eBay’s Mobile App Offers a Goldmine of Insights for Sellers
eBay’s Product Research, formerly called Terapeak, brings three years of sold-items data into its mobile app so sellers can research products and create listings from anywhere. The service reports actual sold prices, including accepted Best Offer prices, along with average and range, free-shipping share, average shipping cost, and the number of sellers who sold a product. Sellers can search, sort, filter, inspect a price-trend graph, and use “Sell one like this” to start a listing pre-populated from a sold listing. Product Research is available in the latest eBay app for iOS and Android, while desktop sellers can use it through Seller Hub; planned additions include sell-through rate, more trend graphs, and active-listing views.
How to Use Okta's PowerShell Module to Manage Your Okta Org
The tutorial shows how to use Okta’s official PowerShell module to administer an Okta org, covering authentication, user operations, group management, and broader API automation. It requires a PowerShell terminal, the module installed from the PowerShell Gallery, Chocolatey, or GitHub, plus an Okta Developer Edition account for the setup described. Authentication uses an OIDC Native application with the Device Authorization grant, a client ID, the org base URL, and the okta.users.manage and okta.groups.manage scopes; Invoke-OktaEstablishAccessToken completes authorization. Examples search and retrieve users, create and update profiles, create groups, add users, and inspect memberships with PowerShell pipelines and formatting cmdlets. The demonstrated workflow connects successfully and manages users and groups, while the same techniques are presented as applicable to other Okta API endpoints.
Enter eBay’s 2024 University Machine Learning Competition
eBay’s 2024 University Machine Learning Competition invites U.S. college and university students to form teams and tackle a yearly ecommerce challenge, with top participants offered a shot at a 2025 summer internship at eBay. This year’s task focuses on eBay Motors: building a model that identifies which vehicles are compatible with particular parts and accessories. Participants must extract “fitment” information from listing titles, item specifics, and descriptions in the eBay Parts and Accessories categories, where noisy or extraneous data makes the task difficult. The competition is in its sixth year; last year, 1,439 students from 206 universities competed across 887 teams, and several winners later moved into full-time eBay roles.
No version left behind: Our epic journey of GitLab upgrades
Grab's self-hosted GitLab instance fell approximately 14 months behind official releases, serving thousands of engineers and monorepos reaching up to 39TB in total footprint. To address security updates and system stability, the team established a structured upgrade routine using Terraform, Packer, and Ansible across GitLab's 5,000-user reference architecture. They implemented staged deployments, upgraded stateful Gitaly nodes via in-place rotation, and resolved primary node clustering bottlenecks under Praefect. By utilizing official GitLab Upgrade Paths and parallelizing component deployments, the team caught up on 24 months of releases in 11 months and reduced release lag from 396 days down to 35 days.
Okta at RSA Conference 2024
Okta’s RSA Conference 2024 presence includes a May 7th session on identity as a defense against breaches and a booth featuring the Shared Signals Framework (SSF) and Universal Logout. Attackers can exploit weak passwords, phishing credentials, and privileged access, while applications may lack the cross-service context needed to identify suspicious account behavior. SSF, developed by the OpenID Foundation, enables software to send and receive signals about user behavior so services can respond according to their threat models. Universal Logout lets administrators and automation end a user’s existing sessions with an identity provider, complementing SSF and intrusion detection tools; Okta’s feature was not yet released. The post points to a build-your-own endpoint, SSF resources, and a pilot program for integrators.
Build and Integrate Universal Logout with Okta
The tutorial builds Universal Logout for an OIDC-enabled SaaS Todo application so enterprise administrators can end suspicious user sessions after account compromise or other detected risk. It explains how to create a /global-token-revocation endpoint, connect it to the Express app, identify a user from the revocation request, and invalidate that user's application session. Initial protection uses an API key, while the updated flow upgrades authorization to OAuth 2.0 Private Key JWT with the Okta JWT Verifier library. The tutorial then exposes the endpoint publicly, configures Okta Admin Console integration, and describes optional OIN submission with Private Key JWT and an email identifier for sub_id. In the test, an admin clears the user's sessions and tokens; the Todo app redirects the user to login when they attempt to add a task, demonstrating Universal Logout works.
Top topics
- Architecture834
- Performance816
- LLMs660
- Machine Learning622
- Open Source563
- AI Agents553
- AI470
- Developer Experience448
Weekly digest
The best engineering reads, once a week.