Loading…
Topic
Open Source
561 posts about Open Source. Every summary links to the original.
- 561 posts
- 36 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Supabrew - Never Code Thirsty
Supabase presented SupaBrew, a mock non-alcoholic beverage engineered specifically to address developer thirst during extended coding sessions and large pull request reviews. Developed following a survey of 80,000 registered developers, the recipe relies entirely on three ingredients consisting of barley, hops, and water in strict adherence to German beer purity laws. The mock packaging design features bottles manufactured from recycled cathode-ray tube monitors and labels repurposed from leftover Ubuntu conference stickers. The fictitious rollout introduces tiered pricing models, including free access for indie developers, Pay-as-you-Drink consumption billing, and dedicated enterprise tiers. Supabase concluded the announcement by confirming that the product release is an April Fools joke.
Best-in-Class Developer Experience with Vite and Hydrogen
Hydrogen, Shopify’s framework for custom React storefronts, chooses Vite to improve developer experience while targeting performance for end users and evergreen browsers. Traditional bundling with tools such as Webpack can become complex and slow during development because each change may require processing more project files; Vite instead serves user code through native ES Modules, while pre-bundling and caching third-party dependencies. Vite combines an HMR development server with sensible defaults, ESBuild compilation, Rollup’s plugin system, CommonJS-to-ESM conversion, and SSR support, including transformations that differ between browser and server environments. Hydrogen integrates Vite through four plugins covering Hydrogen configuration, React Server Components, SSR and RSC middleware, and React fast refresh, enabling server components, streaming requests, and caching; the text concludes that Vite is a suitable choice, while acknowledging Webpack remains appropriate for some advanced use cases.
Introducing Supabase Enterprise
Supabase announced the launch of its Enterprise Plan alongside new features integrated across existing tiers. The Pro Plan now merges pay-as-you-go billing with a default twenty-five dollar monthly spend cap, and users gain access to self-serve database add-ons for on-demand scaling. A new Log Explorer powered by the Logflare acquisition allows querying logs directly with SQL, providing ninety days of retention on the Enterprise tier. Dedicated enterprise capabilities include Point-in-Time Recovery powered by the open-source tool WAL-G, custom Prometheus scraping endpoints for observability, and guaranteed support service level agreements. Additionally, Supabase achieved SOC2 Type 1 compliance and introduced Elixir Livebooks to monitor PgBouncer connection pools.
GraphQL is now available in Supabase
Supabase announced the general availability of GraphQL support on its platform via pg_graphql, an open-source PostgreSQL extension that reflects SQL schemas into GraphQL types, fields, and relationships. Accessible through SQL functions or HTTP endpoints, the extension enforces PostgreSQL row-level security and filters introspection schemas according to the SQL roles assigned to request JWTs. To optimize performance, pg_graphql transpiles every GraphQL query into a single SQL query that aggregates results into JSON, avoiding the N+1 query problem and eliminating redundant join data payloads. Transpiled queries are cached as prepared statements to bypass planning overhead, and all generated mutations and queries support bulk operations. On Supabase Free Plan hardware, these optimizations achieve mean throughputs of 377.4 requests per second over the API and 656.2 queries per second over SQL connections.
Community Day
Supabase initiated its Launch Week with Community Day, announcing a series of ecosystem updates, security integrations, and open-source contributions. Through a partnership with GitHub, secret scanning now detects exposed Supabase service role API keys and automatically revokes them to prevent unauthorized database access. The pre-release of PostgREST version 10 introduces arrow operator queries for composite types and array elements, structured PostgreSQL-compliant error messages, and automated recovery from Linux EMFILE conditions. Authentication features expanded with four community-contributed OAuth providers, two phone providers, email one-time passwords via verifyOTP, and framework-specific supabase-auth-helpers starting with Next.js and Nuxt. Finally, Supabase launched a partner gallery powered by Postgres Full Text Search alongside educational courses and an upcoming ten-day virtual hackathon.
Introducing Decision Transformers on Hugging Face 🤗
Hugging Face has integrated the Decision Transformer architecture into its transformers library and Hub, expanding support for offline reinforcement learning. Unlike standard reinforcement learning methods that maximize returns by learning value functions or policies through active environment interaction, Decision Transformers model trajectories as conditional sequence problems. The model feeds past states, actions, and target returns-to-go over a context window into a GPT-2 autoregressive backbone to generate future actions. Evaluating pre-trained continuous control checkpoints requires standardizing observations with training set statistics and iteratively updating the target return based on received rewards. This sequence modeling paradigm enables practitioners to adjust agent behaviors and skill levels directly by specifying different target returns at runtime.
Should I Open Source my Company?
Software founders often hesitate to make their companies open source due to concerns regarding code scrutiny, security exploits, and intellectual property theft. Supabase operated as an open-source business from its inception, finding that community scrutiny actually accelerates code refactoring and surfaces security vulnerabilities early through public reporting channels. In addition, rapid execution and developer experience provide stronger competitive defenses against larger competitors than proprietary code obfuscation or software patents. Open repos also transformed the company's hiring strategy by replacing traditional take-home tests and LeetCode interviews with evaluation of real-world pull requests and asynchronous discussions. Finally, external contributors expand product capability by independently developing niche integrations and platform adapters that core startup teams lack the bandwidth to build.
Supabase Launch Week 4
Supabase announced the schedule and lineup for Launch Week 4, detailing daily product releases and events starting on March 28, 2022. The week initiated with Community Day to highlight community contributions and open-source tooling utilized across the platform. Technical additions across the week included the introduction of GraphQL data fetching for PostgreSQL databases, the rollout of Supabase Enterprise, and the launch of Edge Functions. Final announcements on April 1 revealed Supabrew alongside Supabase Realtime with multiplayer features, concluding with the start of a ten-day virtual hackathon running through April 9. For the hackathon, titled Bring the Func, Supabase partnered with the Future Forest Company to plant a tree for every submitted project on madewithsupabase.com.
Shopify’s Bug Bounty Program Raises Maximum Payout in 2022
Shopify’s Application Security team reports on its 2021 bug bounty results and announces program changes for 2022. Effective immediately, the maximum bounty for a CVSS 10.0 issue doubles from $50,000 to $100,000, with higher High and Critical brackets, while Shopify Plus, Shop, and Shop Pay services move into the Core asset category; mobile-client-specific issues remain Non-Core. The program received more than 3,000 reports, paid over $1 million in bounties and bonuses, and saw approximately 23% of reports judged valid, up from 19% in 2020. After its first full year using CVSS-based scoring, Shopify reports more consistent decisions and a reduction in average time to bounty from 249 hours to 137, while adding a CVSS guide, more resources, and further open-source support.
Announcing the 🤗 AI Research Residency Program
Hugging Face launched a nine-month Research Residency Program designed to train participants into impactful machine learning researchers. Residents collaborate directly with researchers from the Science Team to formulate research questions, develop novel machine learning techniques, and openly publish their findings. The initiative specifically encourages proposals from marginalized groups—including women, LGBTQ+ individuals, people of color, and working-class communities—to address disparities exacerbated by artificial intelligence progress. Applicants must demonstrate mathematical and programming abilities through coursework, open-source projects, and end-to-end proposals targeting positive societal impact. The full-time role is fully remote, provides benefits including medical coverage depending on location, and excludes concurrent student enrollment or employment.
Using Terraform to Manage Infrastructure
Shopify’s Conversations team used Twilio TaskRouter to route voice and chat tasks to agents according to routing rules and skills. As routing needs grew, configuration through Twilio’s GUI became difficult to understand and lacked clear change history, tests, and rollback support. Terraform provided an infrastructure-as-code workflow, backed by a reliable API, a Go client library, and a provider implementing CRUD operations; the team built a provider because Twilio did not have one at the project’s start. Resource definitions included schemas, dependencies, and an importer, while Terraform’s plan and apply commands previewed and executed changes. Integrated with Atlantis and GitHub pull requests, the workflow added plans, reviews, revision history, and easier rollbacks, while enabling business and support teams to submit infrastructure changes themselves.
Fine-Tune a Semantic Segmentation Model with a Custom Dataset
Fine-tuning a semantic segmentation model requires domain-appropriate training data and an efficient pipeline. Existing autonomous driving datasets feature roadway imagery captured by cars, creating a distribution mismatch for sidewalk-based delivery robots. To resolve this discrepancy, a dedicated dataset of sidewalk imagery is loaded from the Hugging Face Hub, split into training and test sets, and augmented on-the-fly using SegformerImageProcessor and torchvision. The smallest SegFormer architecture, B0, is fine-tuned using Hugging Face's Trainer API with mean Intersection over Union evaluation metrics. The final pipeline pushes the fine-tuned model to the Hub and executes inference by upsampling output logits to original image dimensions.
Image search with 🤗 datasets
Hugging Face datasets expanded its capabilities by introducing an Image feature type, enabling image processing and integration with vector indexing tools. The library was applied to a sample of historical book embellishments extracted via OCR from the British Library. Images were loaded using the ImageFolder loader, enriched with filename metadata, and pushed to the Hugging Face Hub. A FAISS index and CLIP embeddings were used to retrieve images matching natural language queries, such as categories, specific objects, and boolean operators. While the retrieval system demonstrated reasonable semantic search results across various prompts, full public deployment was avoided due to CLIP model card restrictions and potential bias in the historical dataset.
Creating a React Library for Consistent Data Visualization
Shopify found that teams across its more than 10,000 employees used different visualization tools, producing inconsistent chart styling and differing accessibility and print behavior. The Insights team built Polaris Viz, a React library intended to provide shared, accessible defaults while reducing repeated configuration, and focused this work on light and dark themes plus consumer-defined visual identities. The library replaces numerous visual-style props with a theme prop, stores themes through PolarisVizProvider, and uses createTheme to fill partial definitions with defaults and createThemes to preserve Default and Light themes. Themes can inherit from other themes, while a DataSeries colour can override theme series colors for dataset-specific gradients without creating additional themes. Polaris Viz was subsequently released as an open-source project.
Guiding Text Generation with Constrained Beam Search in 🤗 Transformers
Constrained beam search introduces direct control over generated text in Hugging Face Transformers. Standard beam search operates token-by-token without knowing the optimal step to force specific words or phrases, making it difficult to enforce mandatory vocabulary or choose between alternative expressions. To resolve this, the generate interface accepts constraints through arguments such as force_words_ids and a list of Constraint subclasses like PhrasalConstraint. This mechanism enables disjunctive constraints, where generation must include at least one phrase from a provided set, alongside strictly required sequences. Consequently, practitioners can inject prior knowledge or formatting requirements directly at generation time rather than filtering candidate outputs afterward.
After the Refactor: A Path to Faster Rendering with Liquid-C
Liquid-C is an extension to the Liquid gem that parses, tokenizes, and renders templates, but some Liquid features, including If and For tags, still render in Liquid rather than Liquid-C. The work explains how Liquid-C compiles templates into bytecode for a customer VM and addresses the difficulty of moving instruction and constant pointers when control flow requires forward or backward jumps. It refactors instructions to store each constant’s index in the constants array, using a hash table to avoid duplicate constants and remove the constant pointer. The change reduced one example constants array from 157 objects to 104, but did not drastically improve theme rendering speed; it slightly increased memory usage and parsing time while preparing for GOTO or JMP operations and later If and For support.
Postgres Auditing in 150 lines of SQL
Traditional PostgreSQL auditing methods duplicate source table structures, creating maintenance overhead when schemas change. To solve this, a compact SQL implementation stores insert, update, and delete events across multiple tables in a single audit table using JSONB columns. Query performance is maintained by using a BRIN index on naturally ordered insertion timestamps, a B-tree index on internal table OIDs, and UUIDv5 identifiers hashed from primary keys. Row-level PL/pgSQL triggers automatically populate the audit log and can be enabled or disabled dynamically with dedicated tracking functions. While trigger-based auditing introduces minimal overhead at rates below 1,000 writes per second, write-heavy workloads may benefit from logging changes outside SQL with tools like pgAudit.
RubyConf 2021: The Talks You Might Have Missed
RubyConf 2021: The Talks You Might Have Missed compiles talks presented by Shopify’s Ruby and Rails infrastructure engineers at the November conference in Denver, offering ways to revisit sessions for attendees who missed them. The program covers Ruby implementation topics including compiler history, Aaron Patterson’s pure-Ruby JIT compiler, YJIT’s incremental integration inside CRuby, parser generators, variable-width allocation, and Ruby memory layout. Other sessions address gradual Sorbet adoption, native extension compilation and security, pair programming, memoization performance, executable code as data, Ruby archaeology, fast CI, and Ractor-based parallel testing. The roundup reports early YJIT performance results, describes a CI environment with 2.8 million Rails monolith lines and 210,000 Ruby tests, and notes variable-width allocation passed Shopify’s Rails CI while serving more than 500 million requests in a week of limited production traffic.
BERT 101 - State Of The Art NLP Model Explained
Developed in 2018 by Google AI Language, Bidirectional Encoder Representations from Transformers addresses the historical challenge of machines lacking contextual understanding of human language. The model relies on an encoder-only Transformer architecture pre-trained on a 3.3-billion-word corpus consisting of Wikipedia and Google BooksCorpus. Training simultaneously combines masked language modeling, which hides 15% of tokenized words to enforce bidirectional context learning, with next sentence prediction across balanced sentence pairs. Pre-trained on Cloud TPUs over four days, BERT unifies solutions for more than eleven common NLP tasks and can be fine-tuned on task-specific annotated data within minutes. Unmasking experiments demonstrate that the model can also inherit distinct societal and gender biases from its underlying training corpora when predicting professions.
How to Get an Engineering Internship at Shopify: A Complete Guide
Shopify’s guide describes its engineering internship program as a way for students, recent graduates, career switchers, and bootcamp graduates to gain hands-on experience through real projects and practical feedback. Internships last three to eight months and span front-end, back-end, infrastructure, data, and mobile development, with three application cycles each year. Applicants submit an online application, complete an Intern Challenge or specialization-specific skills assessment, and progress through a two-phase interview involving a Life Story conversation and a technical solution. The guide identifies collaboration, lifelong learning, curiosity, GitHub experience, remote-work experience, communication, and accountability as important skills, while recommending practice challenges, project portfolios, and company research; 98% of respondents in Shopify’s latest intern satisfaction survey said they would recommend the program.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.