---
title: "Open Source"
description: "568 posts about Open Source, summarised, each linking to the original."
---

# Open Source
> 568 posts about Open Source, summarised, each linking to the original.

## Articles

### [Supabase Beta January 2021](https://yomu.fyi/post/supabase-beta-january-2021.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Feb 2, 2021
- Type: Announcement

Supabase released a collection of platform updates during January 2021 covering database tools, authentication integrations, regional availability, and infrastructure performance. Client libraries added support for PostgREST exact, planned, and estimated count queries to eliminate counting friction commonly experienced in platforms like Firebase. Authentication capabilities expanded to include Facebook and Azure login providers, an in-dashboard GoTrue audit trail, a helper function for user emails, and a prebuilt React Auth Widget. The platform also enabled new database hosting regions in London and Sydney while transitioning storage to GP3 for improved throughput. Furthermore, migrating API subdomains to Route53 alongside custom Let's Encrypt certificates yielded read performance benchmark gains of up to twelve percent.


### [Fit More and Train Faster With ZeRO via DeepSpeed and FairScale](https://yomu.fyi/post/fit-more-and-train-faster-with-zero-via-deepspeed-and-fairscale.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Stas Bekman
- Published: Jan 19, 2021
- Type: Benchmark

Machine learning model sizes frequently outpace consumer GPU memory, preventing practitioners from loading or fine-tuning large architectures on standard hardware. To address these resource constraints, the Hugging Face Trainer introduced experimental support for Zero Redundancy Optimizer features from DeepSpeed and FairScale starting in transformers v4.2.0. In dual-GPU benchmarks with a t5-large model, FairScale and DeepSpeed integrations expanded allowable batch sizes and shortened training durations compared to baseline DistributedDataParallel runs. Furthermore, DeepSpeed permitted single-GPU fine-tuning of a t5-3b model on a 24GB RTX-3090 card at batch size 20, whereas the baseline execution crashed immediately with an out-of-memory error. These optimizations manage memory allocation internally to curb memory fragmentation while requiring changes only to training scripts rather than model code.


### [How we sped up transformer inference 100x for 🤗 API customers](https://yomu.fyi/post/how-we-sped-up-transformer-inference-100x-for-api-customers.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Nicolas Patry
- Published: Jan 18, 2021
- Type: Problem & solution

Deploying large transformer models in production poses severe latency and scaling hurdles for machine learning engineers. Hugging Face achieved a cumulative 100x inference acceleration on its Accelerated Inference API by pairing library-level algorithmic adjustments with low-level hardware compilation. The initial 10x improvement reduces forward-pass computations, such as restricting attention calculations to the latest token in GPT generation tasks, while utilizing Rust-based tokenizers and caching mechanisms. The subsequent 10x speedup requires custom static graph compilation targeted to selected CPU or GPU hardware profiles, applying unused flow removal, instruction-specific layer fusion, and precision quantization via ONNX Runtime. These coordinated techniques enable real-time prediction latency across varied model sizes and demand profiles without sacrificing task accuracy.


### [Supabase Beta December 2020](https://yomu.fyi/post/supabase-beta-december-2020.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Jan 2, 2021
- Type: Announcement

Supabase transitioned to Beta following ten months of development concentrated on performance, security, and reliability. The platform introduced inline table documentation updates, storing descriptions directly as PostgreSQL comments and exposing them through OpenAPI specifications. Developers can now utilize a table view featuring immediate realtime update reflection alongside pagination for handling large datasets. Performance measurements show a 26 percent increase to 1460 requests per second in regions supporting AWS Graviton instances. Furthermore, the update introduces a new region in Sao Paulo, editing support for native Postgres arrays in the grid editor and side panel, and broader compatibility for custom database types.


### [Supabase Partners With Strive School To Help Teach Open Source](https://yomu.fyi/post/supabase-partners-with-strive-school-to-help-teach-open-source.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Rory Wilding
- Published: Dec 2, 2020
- Type: Announcement

Supabase has partnered with fellow Y Combinator Summer 20 alumni Strive School to teach upcoming software engineers open-source development practices. Strive School operates an eight-month training program in Europe covering full-stack web and artificial intelligence engineering, with students paying tuition only after securing employment. Through this collaboration, Supabase aims to improve relational database instruction using Postgres while producing exclusive educational content for the program. In addition to creating learning materials, Supabase is hiring a Strive School graduate as an intern to integrate new engineering talent into its team. Supabase is also introducing a Student Developer Pack offering two years of free platform credits, Slack community access, founder office hours, and merchandise.


### [Supabase Alpha November 2020](https://yomu.fyi/post/supabase-alpha-november-2020.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Dec 1, 2020
- Type: Announcement

After nine months of development, Supabase introduced several platform updates as it progressed closer to its beta release. The dashboard now supports manual user creation alongside administrative actions such as deleting users and sending password reset emails or magic links. Database management gained direct browser support through a full Monaco SQL editor and new foreign key controls within the table editor. Platform reliability and security were addressed through a DigitalXRAID audit, default email confirmations on signups, and a dedicated status page tracking uptime and latency. Additional progress included community integration work with Vercel, Stripe, and Redwood, as well as an updated benchmarking suite for realistic server workloads.


### [Adopting Sorbet at Scale](https://yomu.fyi/post/adopting-sorbet-at-scale.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Nov 19, 2020
- Type: Explainer

Shopify describes its adoption of Sorbet static typing across a Ruby monolith containing 37,000 files, 622,000 methods, and more than 2,000,000 calls. At the time of writing, Sorbet ran on every pull request, 80% of files were typed: true or higher, almost half of calls were typed, and half of methods had signatures. To handle Ruby and Rails idioms, the team built RuboCop Sorbet for compatibility rules, Tapioca for gem and DSL RBI generation, and Spoom for programmatic tooling, metrics, and LSP access. A controlled experiment found fewer production NoMethodErrors in files typed: true after typing about 20% of the application, although the results were preliminary and signatures had not yet been added. Shopify planned to reach typed: true across all files, improve Rails support, and continue collaboration around Sorbet and RBS.


### [Leveraging Pre-trained Language Model Checkpoints for Encoder-Decoder Models](https://yomu.fyi/post/leveraging-pre-trained-language-model-checkpoints-for-encoder-decoder.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Patrick von Platen
- Published: Nov 9, 2020
- Type: Tutorial

Pre-training sequence-to-sequence transformer models incurs massive computational costs, limiting development primarily to large institutions. To mitigate these expenses, encoder-decoder architectures can be warm-started using existing pre-trained checkpoints from encoder-only or decoder-only models like BERT and GPT-2. This walkthrough details the methodology and implementation of warm-starting sequence-to-sequence architectures using Hugging Face Transformers. By utilizing the EncoderDecoderModel framework alongside Seq2SeqTrainer, practitioners can construct and fine-tune models such as BERT2BERT on datasets like CNN/DailyMail. The resulting fully trained BERT2BERT model achieves a ROUGE-2 score of 18.22 on the full evaluation set, matching competitive sequence generation baselines at a fraction of standard pre-training costs.


### [Porting fairseq wmt19 translation system to transformers](https://yomu.fyi/post/porting-fairseq-wmt19-translation-system-to-transformers.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Stas Bekman
- Published: Nov 3, 2020
- Type: Tutorial

The fairseq WMT19 news translation system was ported to the Hugging Face transformers library under the FSMT architecture. Development began with an initial proxy notebook emulating the transformers API before implementing native configuration, modeling, tokenization, and checkpoint conversion scripts. The author first targeted English-Russian models to handle complex dual-vocabulary requirements and directly evaluate output quality during testing, later extending the implementation to merged-vocabulary language pairs. Although model ensembling was omitted due to library limitations, removing fairseq optimizer states reduced the final model download footprint from 13GB to 1.1GB. The implementation was validated locally against converted fairseq checkpoints and merged into the main codebase after two weeks of iterative review.


### [Supabase Alpha October 2020](https://yomu.fyi/post/supabase-alpha-october-2020.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Nov 2, 2020
- Type: Announcement

Eight months into building Supabase, the team released product updates focused on stability, reliability, and performance while preparing to transition from Alpha to Beta. The release introduces supabase-js version 1.0, incorporating extensive community feedback to deliver major developer experience improvements across client libraries. Platform interface updates include tabs and full-screen mode for the SQL Editor, alongside new keyboard shortcuts to help power users navigate and manipulate tables rapidly. For user authentication, Supabase added magic link capabilities, enabling passwordless login flows for both new and existing users either as a standalone method or alongside passwords. Under continuous improvement efforts, the team converted realtime-js to TypeScript, heavily optimized dashboard routing performance, and established a repository to benchmark all integrated open source tools.


### [Hyperparameter Search with Transformers and Ray Tune](https://yomu.fyi/post/hyperparameter-search-with-transformers-and-ray-tune.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: system
- Published: Nov 2, 2020
- Type: Tutorial

Hugging Face Transformers models often require hyperparameter tuning to achieve high performance, but practitioners frequently rely on basic grid search or bypass tuning entirely. The Transformers 3.1 release integrates Ray Tune into the Hugging Face Trainer to provide advanced hyperparameter optimization without losing framework customizability. By invoking hyperparameter\_search with the Ray backend, users can configure trial budgets, parallelize runs across multiple GPUs, and plug in search algorithms like HyperOptSearch alongside schedulers like ASHAScheduler. Experimental benchmarks on a BERT model using the RTE dataset show that Population-based Training reached 70.5% test accuracy in 48 GPU minutes, outperforming Grid Search at 65.4% test accuracy. The integration also supports experiment tracking tools like Weights and Biases out of the box.


### [Supabase.js 1.0](https://yomu.fyi/post/supabase-js-1-0.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Oct 30, 2020
- Type: Announcement

Supabase has launched Supabase.js 1.0, updating its client library ecosystem with breaking API changes, scalability enhancements, and new documentation tooling. The release introduces native TypeScript implementations across all underlying sub-libraries, including supabase-js, postgrest-js, gotrue-js, and realtime-js, which also enables auto-generated reference docs. In response to community feedback, database and authentication operations now return structured error objects alongside data payloads instead of throwing runtime exceptions. Realtime client scalability is improved by consolidating multiple active subscriptions into a single socket connection per Supabase client. Additionally, the release updates authentication workflows via the newly created gotrue-js library to support third-party logins and OAuth providers alongside revised method names.


### [Transformer-based Encoder-Decoder Models](https://yomu.fyi/post/transformer-based-encoder-decoder-models.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Patrick von Platen
- Published: Oct 10, 2020
- Type: Explainer

Natural language generation tasks map input sequences to target sequences whose lengths cannot be known in advance and vary by content. Standard deep neural networks struggle with these variable mappings because their inputs and targets require fixed-dimensional vector representations. While recurrent neural networks addressed this challenge by generating target sequences auto-regressively from a compressed context state, transformer-based encoder-decoder architectures became the standard paradigm in modern natural language processing. The framework processes sequence-to-sequence problems by encoding source sequences and auto-regressively decoding target sequences token by token using conditional probability distributions. A step-by-step implementation demonstrates greedy decoding with Hugging Face Transformers, passing cached encoder hidden states alongside previously generated target tokens to iteratively generate German translations from English inputs.


### [Supabase Alpha September 2020](https://yomu.fyi/post/supabase-alpha-september-2020.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Oct 3, 2020
- Type: Announcement

Seven months into development, Supabase announced a series of platform updates across authentication, database tooling, and client libraries. The release introduced OAuth logins supporting Bitbucket, GitHub, GitLab, and Google, alongside table cloning and one-click Postgres extension management. In the SQL editor, users can now save favorite queries and access locally stored query histories directly within the browser. The web dashboard adopted Next.js automatic static optimization for improved responsiveness, while postgrest-js migrated to TypeScript and an isomorphic gotrue-js TypeScript library was built for Netlify GoTrue integration. Supabase is prioritizing a transition from Alpha to Beta by tracking open-source tool performance in a dedicated benchmarks repository.


### [Supabase Hacktoberfest 2020](https://yomu.fyi/post/supabase-hacktoberfest-2020.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Thor Schaeff
- Published: Sep 11, 2020
- Type: Announcement

Supabase announced its participation in Hacktoberfest 2020 by organizing a series of community events and open-source contribution activities. Contributors can attend scheduled live broadcasts, including a GitHub Open Source Friday stream featuring @kiwicopple on Twitch alongside an online introductory meetup hosted on YouTube. To coordinate development, Supabase launched a GitHub project board containing tasks and idea notes, requiring participants to label new issues with the hacktoberfest tag for official challenge qualification. Available engineering efforts span creating framework examples for Blitz.js and Redwood.js as well as writing specialized Postgres templates for diverse use cases. Non-code contribution pathways are also supported, encouraging community members to submit documentation improvements, user experience adjustments, illustrations, custom loading pages, or Lottie animations.


### [Supabase Alpha August 2020](https://yomu.fyi/post/supabase-alpha-august-2020.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Sep 3, 2020
- Type: Announcement

Six months into developing its hosted database platform, Supabase announced updates focusing on authentication, table view management, and community developer tooling. Users can now create database tables and columns directly from the table view interface as well as invite team members to their organizations. The authentication system introduced configurable email confirmations for new user registrations, allowing administrators to toggle confirmations and edit email templates directly from the dashboard. Community contributors added TypeScript support to Supabase, implemented UMD support in supabase-js, and began building Python client libraries starting with postgrest-py. Supabase is preparing to transition from Alpha to Beta by stabilizing its Auth API, improving dashboard performance, and benchmarking platform systems.


### [Supabase Auth](https://yomu.fyi/post/supabase-auth.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Aug 5, 2020
- Type: Announcement

Following an early community launch on Hacker News, the open-source Firebase alternative Supabase faced heavy developer demand for a dedicated authentication and authorization solution. In response, the team evaluated existing open-source tools such as KeyCloak and Kratos before landing on an architecture combining a forked Netlify GoTrue server with PostgreSQL and PostgREST. The resulting Supabase Auth system provides client methods like signUp and signIn alongside authorization powered by PostgreSQL Row Level Security and policies. By leveraging SQL policies to enforce access rules directly in the database, developers avoid repetitive middleware filtering while maintaining granular access controls. Future roadmap plans include adding third-party OAuth providers, custom email templates, and simplified policy management interfaces for non-technical users.


### [Continuous PostgreSQL Backups using WAL-G](https://yomu.fyi/post/continuous-postgresql-backups-using-wal-g.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Angelico de los Reyes
- Published: Aug 2, 2020
- Type: Tutorial

Continuous PostgreSQL backups require coordinating physical base snapshots with ongoing write-ahead log archiving to support point-in-time recovery. The open-source WAL-G utility simplifies this workflow by managing backup pushes and fetches to cloud storage services such as Amazon S3. In a typical setup on Ubuntu with PostgreSQL 12, envdir supplies AWS credentials and storage prefixes to WAL-G commands embedded directly in the PostgreSQL archive configuration. Regular base backups can be automated via cron jobs, which minimizes the volume of WAL archives that must be replayed during a restore. When recovering to a new instance, administrators fetch the latest base backup, define a restore command with targeted recovery timestamps in the configuration, and initiate replay using a recovery signal file.


### [Supabase Alpha May 2020](https://yomu.fyi/post/supabase-alpha-may-2020.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Jun 1, 2020
- Type: Announcement

Supabase announced its May 2020 alpha development updates following acceptance into a fully remote Y Combinator cohort and unexpected attention from Hacker News. Operating under a ship early and often philosophy, the team continued deploying functional changes despite ongoing alpha bugs and platform instability. This milestone includes a revamped user interface configured to support incoming product capabilities alongside an initial, experimental Table View designed to rival tools like Airtable. Furthermore, Supabase rolled out realtime listeners that allow developers to subscribe to database changes comparable to Firebase functionality. The platform also added instant RESTful APIs and auto-updating documentation generated dynamically by introspecting modifications to the underlying Postgres schema.


### [Supabase Alpha April 2020](https://yomu.fyi/post/supabase-alpha-april-2020.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: May 1, 2020
- Type: Announcement

The April 2020 alpha release of Supabase introduces developer tools designed to make Postgres as straightforward to use as Firebase. Users can configure a Postgres instance in less than two minutes and execute database queries directly from the dashboard interface. The platform provides auto-generated RESTful APIs along with auto-updating documentation created through database schema introspection. Development work also encompasses realtime listeners for database change subscriptions, query performance analysis tools, and connectors that route realtime updates to queues or webhooks. Supabase is not yet production ready, storing only GitHub usernames and emails while running for free under Digital Ocean's startup program.


[Newer posts](https://yomu.fyi/topic/open-source/page/27.md) · [Older posts](https://yomu.fyi/topic/open-source/page/29.md)