Loading…
Browse by type
Explainer reads
Explainer posts on Yomu, summarised newest first.
- 1,318 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Explainer posts
Iterating with shadow workspaces
Cursor’s shadow workspace addresses the problem of letting AI agents iterate on code without disrupting a developer’s active workspace. The design prioritizes LSP usability—especially lints, definition navigation, and language-server interaction—while also requiring independence, privacy, concurrency, universality, maintainability, and speed. A copied in-memory TextModel was rejected because sharing the user’s language server could introduce copied files into references, duplicate declarations, or miss errors. The implemented approach uses a reusable hidden Electron window for the workspace, routing AI edits through Cursor’s processes and returning lints independently of the visible editor. The post treats runnability as a harder future extension, with folder-proxy designs, platform limitations, and network-level independence still open questions.
What is Data Residency? Data Residency Concerns for Global Applications
Global applications can move customer data across borders, making data residency a compliance concern for organizations serving multinational users. The post defines data residency as legal requirements governing where data may be stored and processed, then distinguishes it from data sovereignty’s jurisdictional reach and data localization’s requirement that storage and processing remain within a jurisdiction. It notes that GDPR and CCPA focus on compliant handling and individual rights rather than necessarily mandating storage locations, while China’s Personal Information Protection Law requires certain Chinese citizens’ PII to be stored within China. The proposed approach uses cloud regions to align storage with local requirements, citing Heroku Enterprise Private Dynos across ten regions and Heroku Shield Private Spaces’ PCI, HIPAA, ISO, and SOC certifications; the post concludes that these features can make compliance more achievable while improving control and potentially application performance.
Approaches to keep sending OTP over SMS... for now
SMS and voice OTPs remain broadly applicable for identity verification, but the post argues they should eventually be replaced because SIM swapping, phishing, uncontrolled secret-delivery channels, device-linking gaps, and slower logins weaken them. Until then, Okta will require organizations to bring their own telephony provider, using a telephony inline hook to send codes and report outcomes. The implementation can be built in-house, outsourced, or delegated to a managed service, with regional regulations, traffic peaks, reliability, branding, security, and available resources shaping the choice. A hook has three seconds to return success or failure before Okta’s heavily rate-limited failover sends the message; teams must also secure endpoints and API keys, monitor logs, and handle vendor or delivery failures.
Building an Event-Driven Architecture with Managed Data Services
Event-driven architecture (EDA) addresses the need to process and respond to continuous streams of real-time activity from applications, e-commerce systems, wearables, and other IoT sensors. Using Apache Kafka, independent producers publish events to topics while consumers subscribe and trigger downstream processes, keeping services decoupled and independently scalable. Kafka also distributes large data streams in real time and replicates data across synchronized broker nodes, so a node failure does not lose data or interrupt operation. Apache Kafka on Heroku is presented as a fully managed service, with multi-tenant basic and single-tenant private plans, automatic broker scaling, and Standard-tier-and-above configurations starting with three brokers and extending to eight. Recommended integration practices include mapping producer-consumer flows, using transactions, topic and schema management, and delivery guarantees, plus monitoring performance and logging Kafka operations.
Machine Learning in Content Moderation at Etsy
Etsy describes a supervised-learning system for detecting potentially violating listings at marketplace scale, alongside community reporting and other automated controls. Training data combines logged and annotated positives, hard negatives, and randomly sampled easy negatives, with multimodal textual and imagery signals split by time for progressive evaluation. The architecture uses BERT-based text encoding with ALBERT for reduced compute, EfficientNet image encoding, late concatenation of modality representations, a softmax output, and focal loss for class imbalance. Etsy evaluates candidates offline and in production through API-serving, canary releases, and champion-challenger A/B tests, while using proxy recall because full manual review is infeasible. Frequent retraining and monitoring support ongoing iteration; the methodology has identified and removed more than 100,000 violations, alongside other tools and services.
Office Hours with Engineering Managing Director Mae Santos
Mae Santos, Head of Critical Applications and Data Reliability Engineering at Two Sigma, leads a global team across New York, Houston, London, and Tokyo to support critical applications and data pipelines. While software teams frequently prioritize functional features over operational stability, designing systems with reliability in mind from the beginning allows for progressive enhancements without complete rewrites. System architecture should be guided by concrete goals across four key dimensions: availability, observability, scalability, and supportability. In addition, mapping out interconnections among systems, infrastructure, data, and human processes helps teams manage dependencies effectively. Applying automation across support and delivery workflows further reinforces reliability by minimizing human error and accelerating execution.
Optimizing Data Reliability: Heroku Connect & Drift Detection
Heroku Connect syncs Salesforce data with Heroku Postgres, but primary polling can miss changes when long-running Salesforce automations commit after the polling window advances, creating drift. The service uses secondary polls over the same bounds as primary polls to find and repair missed changes, with Bulk API or SOAP API selected according to dataset size and retries for failed polls. Recent improvements change Bulk API drift handling by creating one secondary bulk job for each primary bulk job, then adding corresponding batches instead of creating many jobs. Secondary tasks are now scheduled and executed with limited concurrency, reducing simultaneous database connections consumed while Postgres is updated and preserving capacity for applications. The post says these changes reduce load on Salesforce and improve operational reliability during heavy synchronization, while noting that Heroku Connect remains asynchronous rather than truly real-time.
Enhancing Cloud Usage Forecasting, Monitoring & Optimizing
After Etsy completed its 2020 migration from an on-premise data center to Google Cloud Platform (GCP), the resulting FinOps team developed a global approach to forecasting, monitoring, and optimizing cloud spend. Costs are grouped into core infrastructure, machine learning and product enablement, data enablement, and Dev, with trailing Cost Per Visit (CPV) serving as the primary forecasting metric after regression tests found visits most correlated with costs. BigQuery and Looker dashboards, alerts, weekly Finance comparisons, and recurring cost-owner and leadership readouts support variance detection, while costs are tracked centrally without chargebacks to preserve flexibility. Optimization work combines FinOps, engineering, and Architecture collaboration, producing GCS storage reductions through monitoring, automation, TTLs, and Intelligent tiering, plus compute, model-deployment, and network compression improvements. The next priority is better visibility into shared machine learning platform spend to improve experiment ROI analysis and identify optimization opportunities.
Efficient Visual Representation Learning And Evaluation
Etsy uses image vector representations for visual search and similar-item recommendations, making low-latency, memory-aware models important for online query inference and downstream retrieval. The work compares EfficientNetB0, a compact CNN, with EfficientFormer-l3, which downsamples multiple blocks and applies attention only in its final stage; its representation uses the last block’s eight-head attention and average-pools the sequence. Fine-tuning adds an embedding and multiple classification heads, while multitask training supervises taxonomy, color, and review-photo labels; both models train on two 16GB P100 GPUs. Evaluation rebuilds a brute-force candidate index after each epoch and measures Recall@5 and Recall@10 on interaction and intra-item retrieval datasets, with early stopping available. EfficientFormer-l3 improves reported offline recall by about 5%, 17%, and 1.8% across three tasks, while experiments report lifts in CTR, purchase rate, click-recall@100, and clicks.
More problems
The post outlines research problems for the next phase of AI programming, centered on making coding systems predict actions, perform larger edits, understand codebases, and debug software. Proposed directions include low-latency next-action models for keystrokes, locations, files, terminal commands, and edits; background execution for “perfect edits” and pseudocode compilation; and multi-file changes supported by stronger tool use, reasoning, and agent trajectories. For context, retrieval alone is described as insufficient for codebase questions requiring multiple hops, while useful context may span documentation, source, history, UI pixels, logs, and messages. The post recommends combining retrieval, recurrence, and long-context attention, alongside specialized rerankers, multi-hop embedders, caching, and codebase-level retrieval. For reliability, it proposes calibrated bug detection, always-on AI linting, runtime-state tracking through cursor/debug, and improved debugger tool use, while noting trade-offs involving latency, false positives, infrastructure, and user experience.
Editing Files at 1000 Tokens per Second
Large code edits expose coding agents to laziness, inaccuracy, and latency, especially when frontier models must rewrite hundreds of lines. The post presents fast apply, a specialized full-file code-edit model that separates planning from applying: Cursor's frontier model plans the change, while the apply model rewrites the current file. It evaluates roughly 450 edits on files under 400 lines, using Claude-3 Opus as grader, and finds llama-3-70b-ft nearly matches claude-3-opus-diff while outperforming GPT-4-Turbo and GPT-4o. Rather than generate conventional diffs, it uses full-file rewrites and search/replace hunks, which avoid line-number errors and give models more output tokens for reasoning. A custom speculative decoding method, deployed with Fireworks, reaches approximately 1,000 tokens per second on a 70b model, about 13x faster than vanilla Llama-3-70b inference and 9x faster than the previous GPT-4 deployment.
NeurIPS 2023: Our Favorite Papers on LLMs, Statistical Learning, and More
Researchers reviewed prominent machine learning papers presented at NeurIPS 2023 covering large language models and statistical learning theory. One investigation showed that claimed emergent abilities in models such as GPT-3 often result from nonlinear evaluation metrics rather than fundamental shifts in model capability. To reduce the computational burden of model adaptation, QLoRA enables 65-billion-parameter model fine-tuning on a single 48-gigabyte GPU via 4-bit NormalFloat quantization and paged optimization. Direct Preference Optimization eliminates complex reward modeling by casting reinforcement learning from human feedback into a preference classification task. Additional work resolved statistical anomalies like double descent using effective parameter counts and introduced stochastic gradient approximations for Gaussian processes.
Improving Shopify App’s Performance
Shopify’s mobile app saw performance decline during its React Native migration, prompting a year-long effort to make critical screens load under 500ms at P75 and launch within 2s at P75. The team built real-time dashboards to filter results by device and operating system, then targeted excessive initial rendering, unnecessary background updates, launch overhead, and weak cache utilization. They introduced LazyScrollView and ListSource on FlashList, deferred work outside the viewport, enabled inlineRequires, optimized a native module, batched state updates, froze background components, and improved Restyle. They also fixed a graph QL cache issue and pre-warmed critical screen data, increasing cache-first delivery to as much as 90% for key screens. By year-end, app launch P75 was 44% faster and screen-load P75 times were reduced by 59%, with performance treated as an ongoing process requiring monitoring and repeated optimization.
Cutting Through the Noise: Three Things We've Learned About Generative AI and Developer Productivity
eBay describes three tracks for generative AI developer productivity: adopting a commercial offering, fine-tuning an open-source LLM on internal data, and building a knowledge base. Because no single metric is reliable, the exercise combines surveys with Git, DORA, Flow, code-quality, and performance measures. In a 300-developer Copilot A/B test, the treatment group had 27% code acceptance, 17% faster pull-request creation-to-merge time, and 12% lower Lead Time for Change; Sonar quality was unchanged. eBayCoder, based on Code Llama 13B and trained on eBay code and documentation, is presented as useful for software upkeep and broader context. Its internal GPT uses retrieval-augmented generation to answer questions from distributed sources, but early results remain preliminary and nonsensical answers require feedback and RLHF.
The End of Third
The post explains how browser phaseouts of third-party cookies can affect Okta authentication patterns and session features. Third-party cookies are issued by other domains and can support cross-site session introspection, federated login-related behavior, and convenience features, but browsers increasingly restrict them because of security and privacy concerns. Most Okta core authentication flows do not require them; however, applications using an Okta session across different top-level domains, customer-hosted sign-in experiences, ADSSO, and certain “remember me” functions may lose session introspection or extension while basic authentication and token return remain possible. The post recommends testing with Chrome’s third-party cookies disabled, using feature flags, and migrating away from these cookies before Google’s stated exemption ends in 2024.
A Packwerk Retrospective
Shopify’s retrospective examines Packwerk, a Ruby gem released in September 2020 to enforce package boundaries and help modularize Rails applications. It describes the tool as static analysis that checks constant references against a declared, non-circular dependency graph, with todo files allowing existing violations to be recorded while new ones are blocked. The authors recount how privacy checks shifted attention toward API design, introduced awkward file organization, and were removed in version 3.0. They also explain that Packwerk cannot determine whether package boundaries or violation fixes reflect runtime coupling, so semantic grouping can mislead developers and add harmful indirection. Shopify ultimately treats running code as the strongest progress signal and retains Packwerk mainly to prevent new dependencies at the application’s base layer.
Managing dynamic marketplace content at scale: Grab's approach to content moderation
Grab manages content moderation across GrabFood and GrabMart, where merchants update over 100,000 item listings daily across eight countries. The platform must adhere to internal rules, local government regulations, and external platform guidelines across these markets. To handle scale and regional nuance, Grab uses an in-house automated system powered by algorithms and machine learning to scan daily listings for violations. Items flagged as requiring subjective judgment or cultural awareness are routed to human moderators for manual review. Additionally, Grab collaborates with Google to maintain compliance with Play Store policies.
Macaroons Escalated Quickly
Fly.io built a Macaroons-based security token system after finding its API tokens too powerful for expressing scoped permissions and roles. The design starts with a user-linked, HMAC-signed bearer token and chains caveats through HMAC, while checking every caveat independently so each added restriction can only weaken authorization. It models organizations, apps, machines, and volumes with first-party caveats, and also includes third-party caveats, service-token transformations, machine locking, nonce-based revocation, and expiration handling across customer workloads. The resulting system lets users create roles without changing the platform and supports SSO, plugins, and traceable inter-service authorization, although the author acknowledges extra complexity and awkward ways to express some roles.
How Yoko Li makes towns, tamagoes, and tools for local AI
Yoko Li’s local AI work spans playful agent experiments and practical tools for running language-model workflows on personal hardware. AI Town uses conversations among AI agents to produce changing relationships and emergent behavior, while AI Tamago feeds game state and user inputs to an unmodified LLaMA 2 7B model and uses Ollama’s JSON mode to generate pet actions. The Local AI Starter Kit packages document ingestion, vector storage, and language-model responses into a private, local “chat with a knowledge base” pipeline, avoiding setup across at least four API providers. The post also describes unreliable JSON generation, hallucinations, and grammar-guided workarounds, concluding that randomness is a liability for factual outputs but useful for game behavior and creative experimentation.
Fly With Alpine
Alpine Linux can reduce container image sizes compared with Debian slim, but switching distributions involves compatibility and ecosystem trade-offs. The post compares Fly.io’s Debian-based baseline across Bun, Node, Python, and Ruby, then reports smaller Alpine base and application images, including 304MB versus 428MB for a minimal Rails/Postgresql/esbuild application. Migration consists of changing Dockerfile base images from -slim to -alpine, replacing apt-get with apk, and reviewing package-name differences; recent Bun, Node, and Rails generators can make these changes with an --alpine option. Alpine uses musl rather than Debian’s glibc, and issues can involve DNS, missing commands or packages, precompiled binaries, insufficient testing of release candidates, and software such as Chrome. Fly.io will retain Debian as its default while keeping Alpine available.
Top topics
- Architecture830
- Performance812
- LLMs648
- Machine Learning617
- Open Source561
- AI Agents542
- AI454
- Developer Experience446
Weekly digest
The best engineering reads, once a week.