Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,671 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Jekyll on Heroku
Jekyll can be deployed on Heroku to use its plugin system, but relying on jekyll serve would use Ruby’s single-threaded WEBrick rather than a production-oriented web server. The setup installs Ruby with ruby-install and chruby, then installs Bundler and Jekyll, creates a site, and tracks it in Git. Heroku’s official Ruby buildpack handles dependencies and runs a Rake assets:precompile task that executes jekyll build, while the Static buildpack serves the generated _site/ directory according to static.json. The example pins Ruby 2.4.2, adds Rake, enables clean URLs and HTTPS, and completes with a successful deployment of the generated site to a Heroku URL.
GrabShare at the Intelligent Transportation Engineering Conference
Grab presented a technical paper on the construction of its real-time ridesharing service, GrabShare, at the Intelligent Transportation Engineering Conference in Singapore. The platform pairs passengers heading along similar routes with drivers immediately while handling network drops, volatile supply and demand, and heavy traffic conditions in Southeast Asian cities. To deliver accurate pairings, the scheduling system generates and filters through hundreds of travel time estimates for each candidate match before finalizing an itinerary. Operational teams on the ground evaluate complaints about poor matches, enabling engineers to refine the online matching systems. Over the course of one month, the service cut more than 4.5 million kilometers of driving distance and brought in over 100,000 new users within two weeks.
Grabbing Growth: A Growth Hacking Story
Grab established a dedicated Growth Hacking team within its Technology organization to pursue high-risk, niche initiatives and scale impact across 68 million regional users. Positioning the team within engineering allowed rapid A/B testing and simultaneous multi-market deployments driven by structured growth loops. The team prioritizes and evaluates all initiatives using a Growth Factor metric, calculated as the increase in rides divided by the increase in costs. To improve driver engagement, the team implemented a Spin-to-Win game based on B.F. Skinner's variable ratio reinforcement principles, delivering probabilistic monetary and merchandise rewards upon meeting daily ride thresholds. Ongoing regional experiments continue to evaluate metrics including driver acceptance, cancellation rates, and driver ratings.
Namespacing Actions for Redux
Kickstarter’s front-end team encountered a Redux action-namespacing problem when multiple instances of the same functional React component responded to actions intended for only one instance. They rejected local state because the application used functional components and redux-loop, which routes state updates and side effects through reducers, and they sought programmatic namespacing compatible with combineReducers and a low-surface-area component API. The team compared nested reducers, higher-order reducers, and a closure-based module pattern; the latter met the stated constraints with namespaced actions, wrapper components, reducers, and a namespace returned by a factory that defaults to a UUID. Namespaces must be preserved when reducers trigger further actions, including redux-loop results, and complex composition can lose the required binding. The approach has worked for most reusable and compound components, while possible performance costs from many reducers remain a future concern.
How Shopify Merchants can Measure Retention
Shopify addresses customer retention for merchants operating non-contractual businesses, where churn is not directly observable. Common rules that label customers inactive after N days create arbitrary binary states, so the post proposes probabilistic modeling with the BG/NBD (Beta Geometric / Negative Binomial Distribution) model. It represents each customer with a purchase rate and churn probability, estimates their population distributions, and uses age, recency, and repeat-purchase frequency as sufficient statistics. Shopify fits a modified implementation in lifetimes using Apache Spark DataFrames, subsampling up to 50k customers per shop for performance, and produces activity probabilities and expected future purchases for more than 500 million historical merchant customers. The model is interpretable and supports customer ranking and targeted campaigns, but it handles seasonality poorly and does not easily incorporate additional variables such as country or products purchased.
FRP at Kickstarter and Kotlin
Kickstarter’s open-source iOS and Android apps use Functional Reactive Programming (FRP) patterns to keep views simple and move mutation and complex logic into thoroughly tested ViewModels. The post describes how this discipline helped a developer ramp up quickly on the Android codebase after working on Kickstarter’s backend and payments teams. It also introduces the Android team’s interest in Kotlin, a JetBrains language that runs on the JVM, while noting that dependency concerns had prevented conversion of core functionality. For a 2017 Functional Swift Conference talk in Berlin, the author used Kotlin to add a simple Android screen following the team’s established FRP patterns, presenting the implementation as live coding and an introduction to FRP and Kotlin.
Upgrading Kickstarter to Rails 5
Kickstarter upgraded its main Rails application to Rails 5, alongside payments app Rosie, to support site security and performance while moving a nine-year-old legacy codebase forward. The team planned a three-month project, upgraded Ruby 2.2.5 to 2.4.x, classified release-note changes, resolved dependency conflicts, used Rails’ upgrade script, and moved through twelve documented steps. Work included merging Rails 4.2-compatible fixes, repairing nearly 4,000 CI test failures from a roughly 70 percent coverage baseline, removing deprecations, smoke-testing major flows, backporting more than thirty pull requests, and preparing reviewers and deployment contingencies. The upgrade met its timeline without rollbacks or major downtime, although untested bugs increased Zendesk tickets and exposed issues involving ActionController::Parameters, tracking events, and mobile applications. A retrospective emphasized planning, documentation, cross-team communication, and broader native-team participation for future upgrades.
How We Enable Our Interns to Make an Impact
Shopify describes an internship model in which interns work on projects affecting merchants, partners, and developers, including merchant pain points such as selling across channels. The program combines autonomy comparable to that of full-time employees with mentorship, regular pair programming, unfamiliar challenges, and cross-team collaboration. Examples include launching the Amazon Channel, changing an Android app design pattern across multiple codebases while limiting effects on third-party developers, backfilling affiliate data with Elasticsearch and Ruby on Rails, and generating Java and Swift models from a strongly typed GraphQL schema. Interns are encouraged to investigate open-ended problems, consult documentation and stakeholders, and develop informed recommendations with mentor support. Returning interns can change teams or roles, while tools such as Slack enable direct communication across the organization; the stated result is practical experience, deeper technical knowledge, and potential full-time opportunities.
Tell Your Stories: The Benefits of Strategic Engineering Communications
Shopify created engineering communications in early 2016 as rapid growth and expansion from four to five offices made informal knowledge sharing insufficient. The program serves internal and external audiences through weekly technical talks, an internal newsletter, engineering-blog posts, conference presentations, and an editorial calendar, with Shopifolk developers treated as a primary audience. Communications staff hunt for stories by building relationships with project leads, then coach speakers and work alongside developers while preserving their voices; coaching covers narrative, breath control, body language, and calls to action. The reported effects include less risk of duplicated work or mistakes, cross-pollination of ideas, more confident speakers, increased interest in Shopify from some attendees, and a recurring pipeline in which storytelling generates further stories.
The Data and Science Behind GrabShare Part I: Verifying Potential and Developing the Algorithm
Expanding from point-to-point dispatch services to dynamic carpooling requires matching independent passenger requests traveling in similar directions without causing unacceptable delays. Grab evaluated the feasibility of its GrabShare service by analyzing historical trip data with DBSCAN clustering on coordinates projected into a Universal Transverse Mercator system. This analysis demonstrated that 35% to 46% of rides across typical daytime windows fell into tight geographic clusters with near-identical pickup and drop-off coordinates. The resulting matching framework adapts the baseline dispatch flow by searching for in-transit drivers and enforcing real-time seat reservation constraints. Route assignment decisions subsequently evaluate detour times, trip angles, and expected arrival times to ensure driver utilization improves while total driving distance decreases.
How Shopify Governs Containers at Scale with Grafeas and Kritis
Shopify describes using Grafeas and Kritis to govern container software across a platform handling more than 6,000 container builds per day and 330,000 images in its primary registry. Grafeas serves as a central store for metadata about container origins, builds, base images, layers, registry pulls, installed packages, vulnerabilities, and deployment status, supporting auditing and lifecycle strategies. During CI, audits produce attestations, which Shopify signs with PGP; Kritis uses named attestation authorities and Kubernetes policies to require specific attestations before deployment. Policies can require containers to be built by Shopify, sourced from a trusted repository, free of newly introduced vulnerabilities, tested, non-root, and deployed with the appropriate security context. Shopify also generates attestations from kubeaudit results for workload manifests, helping catch runtime-security regressions before deployment.
The Art of Hiring Good Engineers
Building and scaling engineering teams requires adapting recruitment strategies across different organizational sizes, from initial hires to hundreds of engineers. Teams often begin with small groups of battle-tested engineers before designing formal hiring pipelines. The recruitment approach outlines four key steps: sourcing top-tier talent from platforms like GitHub and LinkedIn, assessing technical fit through tailored screens or assignments, evaluating cross-functional and culture fit via group interviews, and optimizing the process. This optimization includes routing candidates strategically to strict bar-raiser interviewers to prevent burnout and expedite decisions.
In the Cloud, No One Can Hear Your OutOfMemoryError
Heroku introduces Heroku Exec and Language Runtime Metrics to improve production monitoring, inspection, and debugging for applications running on dynos. Heroku Exec creates secure TCP and SSH tunnels for SSH sessions, port forwarding, remote debugging, and diagnostic tools, while the Heroku Java CLI integrates jmap and VisualVM. A synthetic Java memory-leak example shows VisualVM tracking heap growth and jmap producing a downloadable HPROF heap dump that Eclipse Memory Analyzer identifies as Servlet-held FakeLeak objects. The metrics feature, enabled through the Heroku Metrics buildpack on paid Standard-or-above dynos, displays heap, non-heap, and garbage-collection activity in the dashboard through a Java agent using Prometheus; JVM-language support is generally available, while Go support is in beta.
Evolution of the Heroku CLI: 2008-2017
Between 2008 and 2017, Heroku's CLI evolved from Ruby (v1–v3) through Go/Node versions v4 and v5 to a pure Node v6 implementation. The rewrite addressed slower Windows performance, separation of the CLI's runtime from users' Ruby environments, migration to API v3, and limitations in Ruby-based plugin dependencies. V4 kept existing command syntax and output while adding features such as non-TTY color suppression and --json output; its Go core used Node for plugins, and the CLI became a collection of core plugins that could also be developed and installed independently. The team migrated commands incrementally, then added Flow, Babel, and improved tests, including Jest tests that run the flag parser. The resulting foundation supports modular, tested feature development, with cli-engine and more interactive interfaces planned.
Migrating Existing Datastores
Grab's Identity team faced imminent memory exhaustion on a single Redis node used to cache mobile authentication tokens under rapid user growth. Because read traffic outweighed write traffic by roughly 200 times, the team opted for an AWS ElastiCache cluster with three shards and two read replicas per shard. They executed a zero-downtime, six-phase migration plan while handling a peak load of 20,000 queries per second. The migration transitioned through initial one-time data replication, asynchronous shadow writes, synchronous dual writes, asynchronous read validation, switching primary reads, and final write cleanup. Controlled by feature flags and monitored with metrics at every stage, the migration completed without invalidating tokens or causing service disruptions.
So You Need to Hire Good Engineers
Hiring strategies in fast-growing technology startups dictate the quality and scalability of expanding engineering teams. Insights gathered from conversations with over one hundred engineering leaders identify crucial evaluation criteria across technical competence, cultural alignment, and problem-solving capability. Candidates rarely present a perfect technical fit, requiring hiring managers to weigh passion, willingness to learn, and risk appetite against baseline technical requirements. Rather than focusing solely on static factual questions, interviewers gain deeper insights by probing the rationale behind candidates' architectural and implementation choices. High-caliber engineers ultimately elevate team capability by simplifying complex challenges, writing cleaner code, and learning quickly.
Come and #hackallthethings at Grab
Grab has officially launched a public bug bounty program in partnership with HackerOne to strengthen the security of its platform. This rollout follows a private bounty initiative operated over the previous year, during which the organization worked with over 350 security researchers and resolved nearly 200 awarded bug reports. The new public program invites external researchers to scrutinize Grab's code for critical flaws, including remote code execution, SQL injections, and exportable cross-site scripting vulnerabilities. To support ethical and responsible disclosure, Grab offers payouts reaching up to $10,000 per valid vulnerability report based on severity and impact.
How We Scaled Our Cache and Got a Good Night's Sleep
Growing business load on the Common Data Service (CDS) created potential bottlenecks for its single-threaded Redis cache on ElastiCache, necessitating horizontal scaling for greater capacity and throughput. After ruling out master-slave replication and intermediate Twemproxy setups due to memory constraints and proxy I/O bottlenecks, the team implemented client-side sharding. Using an internal Go package for consistent hashing, CDS instances hash cache keys locally to determine the target shard. The implementation encapsulates hashing inside a thin `ShardedCache` wrapper sharing the original cache interface while supporting Ketama and custom hash functions. Deploying via double-writing cron jobs during off-peak hours reduced database read pressure and improved P99 latency.
Grab's Front End Study Guide
Rapid hiring and engineering expansion created a steep learning curve for newcomers and backend engineers unfamiliar with modern web architecture. Grab created an opinionated front-end study guide to help engineers navigate JavaScript tooling, state management, and modern application practices. The guide frames client-side single-page applications as a scalable alternative to traditional server-side rendering, enabling clear client-server separation for multiple client apps hitting the same backend API. It establishes core competencies in ECMAScript 2015 via Babel and explains how component-driven interfaces in React improve maintainability and performance through virtual DOM reconciliation. Ultimately, the resource aims to ramp up developer onboarding and accelerate shipping speeds across web platforms.
DNS Resolution in Go and Cgo
Go applications experiencing load balancing issues across AWS Elastic Load Balancer (ELB) nodes trace uneven traffic distribution to IP address sorting defined in RFC 6724. Comparing Go's native DNS resolver with Cgo and glibc's getaddrinfo shows that both initially sort destination addresses using Rule 9 longest matching prefix rules. Disabling IPv6 on the network interface causes C and Cgo resolvers to return IP addresses in randomized order, while the native Go resolver continues deterministic sorting. Examination of net/addrselect.go reveals that Go's native resolver implements only a subset of the RFC rules and omits dynamic source address selection. Achieving permanent parity requires modifying the Go source code directly.
Top topics
- Architecture834
- Performance816
- LLMs660
- Machine Learning622
- Open Source563
- AI Agents553
- AI470
- Developer Experience448
Weekly digest
The best engineering reads, once a week.