Loading…
Latest reads
The engineering internet, summarised so you can actually read it.
Supabase ·
Community Day
Supabase kicked off its Launch Week by showcasing integrations, ecosystem updates, and community contributions across its open-source stack. The platform announced that all hosted projects are upgrading to PostgREST 9, introducing inner joins, functions with unnamed parameters, and PostgreSQL 14 compatibility. Through community contributions, Supabase expanded authentication options to include Slack, Spotify, and MessageBird alongside existing providers. Updates to the Python ecosystem include package renaming to supabase, synchronous operation support in postgrest-py, and a rewritten gotrue-py matching JavaScript feature parity. Additionally, Supabase partnered with GitGuardian to automate secret detection and remediation across development lifecycles, protecting sensitive database service role keys.
Thor Schaeffhuggingface.co ·
Introducing the Data Measurements Tool: an Interactive Tool for Looking at Datasets
Hugging Face released an open-source Python library and no-code interface named the Data Measurements Tool to calculate metrics for responsible dataset curation. In AI development, scraped data often enters training pipelines without detailed analysis of representation or downstream model influence, partly because querying raw datasets typically requires complex coding. Built using Hugging Face Dataset and Spaces Hubs alongside Streamlit, the tool provides descriptive, distributional, and comparison statistics. It computes missing values, vocabulary sizes, duplicate counts, label distributions, and Zipf's law adherence to identify unnatural language patterns or markup artifacts. Additionally, it offers embedding clusters via a Sentence-Transformer model and normalized pointwise mutual information metrics across gender and sexual orientation identity terms.
Sasha Luccioni, Yacine Jernite, Margaret MitchellSupabase ·
New in PostgreSQL 14: What every developer should know
PostgreSQL 14 introduces several enhancements to improve application performance, data manipulation, query troubleshooting, and database security. Developers can traverse nested JSON data structures using subscripting and store noncontiguous values using multirange data types in single operations. Applications relying on libpq can leverage client-side query pipelining to send multiple queries across high-latency networks without waiting for individual responses, while parallel execution now supports foreign data wrappers, parallel sequential scans, and materialized view refreshes. For observability, query IDs are integrated into live execution tracking via pg_stat_activity, EXPLAIN VERBOSE outputs, and log files. Security upgrades include predefined read-only roles such as pg_read_all_data alongside SCRAM-SHA-256 enabled as the default authentication mechanism.
Gurjeet SinghSupabase ·
PostgREST 9
PostgREST 9 introduces several core updates to the tool that turns PostgreSQL databases into RESTful APIs. Developers can now perform inner joins when embedding tables using the !inner keyword, allowing top-level table rows to be filtered directly by embedded table attributes across multiple operators. The release also adds support for POST requests to functions containing a single unnamed parameter, which simplifies handling raw JSON payloads sent by external webhooks. To ensure compatibility with PostgreSQL 14, custom authentication functions accessing HTTP context headers and JWT claims must be updated to parse JSON properties from consolidated settings. Additional enhancements included in this version provide support for partitioned tables alongside documentation improvements and bug fixes.
Steve ChavezSupabase ·
How we launch at Supabase
Following their participation in YCombinator, Supabase sought a structured method to maintain product momentum and recreate the urgency of an accelerator internally. The team established Launch Week, a quarterly shipping sprint using a fixed-timeline and flexible-scope model where a major feature or announcement releases each day for a week. Each cycle begins with a broad distributed planning session that maps high-level goals and generates ideas, followed by execution and finely scheduled launch days covering coordinated social media, blog posts, and community engagement. Following the launch, team members transition into maintenance mode and downtime to handle support, bug fixes, and open-source contributions. Finally, a structured retrospective evaluates successes, failures, and process relevance before carrying lessons into the next planning meeting.
Ant WilsonSupabase ·
Supabase Launch Week III: Holiday Special
Supabase organized Launch Week III: Holiday Special from November 29 to December 3, 2021, delivering daily product releases, tooling improvements, and ecosystem announcements. The week began with Community Day to showcase external contributions and open source tooling utilized by the team. On Tuesday, Supabase released Supabase Studio, allowing engineers to utilize the official platform management dashboard in self-hosted configurations and local development workflows. Midweek releases included the rollout of Realtime Row Level Security on Wednesday, followed immediately by Thursday's announcement detailing the acquisition of Logflare. The five-day launch series concluded on Friday with a final batch of five distinct feature announcements across the Supabase platform.
Paul CopplestoneGrab ·
Using real-world patterns to improve matching in theory and practice
Continuous ride-hailing assignment relies on solving the minimum weight bipartite matching problem between passengers and driver-partners. While traditional implementations assume a precalculated cost matrix, computing shortest-path travel times across large road networks dominates total execution time. Researchers introduced an Incremental Kuhn-Munkres algorithm that leverages the spatial locality of optimal matches to compute edge costs on demand. The approach integrates priority queues and lower-bounding techniques with refinement rules to avoid evaluating distant pairs while guaranteeing the same optimal assignment. Evaluated on Singapore road network data and real Grab production workloads, the incremental techniques reduced exact cost calculations and decreased assignment running times by over an order of magnitude.
Tenindra Abeywickramahuggingface.co ·
Accelerating PyTorch distributed fine-tuning with Intel technologies
Transfer learning makes CPU-based clusters a viable, cost-effective option for shorter deep learning fine-tuning jobs. To accelerate PyTorch workloads, clusters can leverage Intel Ice Lake architecture featuring AVX-512 and Vector Neural Network Instructions alongside specialized software optimizations. Implementing the Intel Extension for PyTorch and the oneAPI Collective Communications Library (oneCCL) addresses communication bottlenecks across distributed nodes. In demonstrated tests fine-tuning transformer models on GLUE tasks across Amazon EC2 c6i.16xlarge instances, two-node jobs delivered roughly a 1.7x speedup, while four-node setups achieved a 3x speedup. These performance libraries and multi-node configurations maintain consistent scaling across both smaller datasets like MRPC and larger benchmarks like QQP.
Julien Simonhuggingface.co ·
Fine-Tune XLSR-Wav2Vec2 for low-resource ASR with 🤗 Transformers
Fine-tuning the multilingual XLS-R speech model adapts pretrained cross-lingual audio representations to automatic speech recognition tasks with limited labeled data. The demonstration configures a Wav2Vec2-XLS-R-300M checkpoint using Connectionist Temporal Classification on roughly four hours of validated Turkish audio from Common Voice. Building the pipeline requires pairing a Wav2Vec2FeatureExtractor for audio signals with a custom Wav2Vec2CTCTokenizer derived from dataset transcriptions. Training with Hugging Face Trainer over 3,200 steps decreases the validation word error rate from 0.7000 down to 0.3195. While the resulting transcription demonstrates recognizable phonetic alignment, output quality can be further improved by extending training schedules, refining preprocessing, and adding language model decoding.
Patrick von PlatenSupabase ·
Supabase Beta October 2021
Supabase announced its October 2021 beta release, introducing three new authentication providers for Slack, Spotify, and MessageBird phone logins. The platform rolled out database multi-schema support, enabling users to browse different database schemas directly within the Dashboard interface and query them through the PostgREST API after enabling schema access. Documentation and instructional resources were expanded to cover PostgreSQL functions, authentication server-side functions, RedwoodJS quickstarts, self-hosting architectures, and production readiness checklists. In addition to ecosystem releases such as supabase-py v0.0.3, nuxt-supabase v2.2.1, and vue-supabase v2.2.3, the project reached over 21,268 GitHub stars. The team also announced Launch Week III: The Trilogy, set to begin on November 29th on Discord.
Paul Copplestonehuggingface.co ·
Scaling up BERT-like model Inference on modern CPU - Part 2
Scaling transformer inference on modern CPUs requires coordinating hardware capabilities with software optimizations across memory allocation, parallel execution, and mathematical kernel libraries. Intel Ice Lake Xeon processors deliver up to seventy-five percent faster inference across natural language processing tasks compared to Cascade Lake by combining Sunny Cove architecture improvements with dedicated extensions like Intel Extension for PyTorch. Fine-tuning software knobs such as alternative allocators, threading runtimes like Intel OpenMP, and core allocations can involve thousands of possible configuration combinations. Rather than evaluating these combinations through exhaustive brute force, Bayesian optimization via Intel SigOpt achieves comparable latency within roughly twenty trials and reveals experiment-dependent parameter importance. These software tuning strategies enable low-latency production transformer serving without dedicated accelerators.
Ella Charlaix, Jeff Boudier, Morgan Funtowicz, Michael BenayounSupabase ·
Supabase $30m Series A
Supabase announced a $30 million Series A funding round led by Coatue, bringing its total funding to $36 million. The company aims to provide an open source alternative to Firebase while addressing the scalability limitations typical of abstracted database platforms. Rather than building a compatibility layer, Supabase provisions a full PostgreSQL database for every project alongside integrated authentication, storage, RESTful APIs, and realtime data streaming. Platform features rely directly on core PostgreSQL primitives, including Row Level Security policies, database triggers, and logical replication streams. Funding will support long-term development of a cloud-native PostgreSQL platform offering database branching, distributed read-write nodes, scalable storage, and ephemeral compute that scales down to zero.
Paul Copplestonehuggingface.co ·
Course Launch Community Event
Hugging Face scheduled the release of Part 2 of its machine learning course for November 15th alongside a collaborative community event. While Part 1 covered text classification and Model Hub uploads, Part 2 expands into token classification, causal and masked language modeling, translation, summarization, question answering, and ecosystem tools like Datasets and Tokenizers. The event includes two days of technical talks from industry practitioners covering transfer learning, ethics in machine learning development, and workflows across Keras, PyTorch, and TensorFlow. Participants collaborate on team projects to fine-tune natural language processing models and develop interactive portfolio demos, with AWS supplying complimentary compute through Amazon SageMaker. Attendees who successfully complete and demonstrate a project receive a certificate of completion.
Sylvain Guggerhuggingface.co ·
Large Language Models: A New Moore's Law?
Recent advancements in generative artificial intelligence have spurred the rapid growth of large language models like Megatron-Turing NLG 530B, which require vast financial investments and substantial energy consumption. Training these massive parameter architectures demands hundreds of multi-GPU servers and generates significant carbon footprints for relatively modest benchmark gains. Rather than relying on brute-force scaling, engineering teams can adopt smaller, more frugal architectures through knowledge distillation and transfer learning techniques. Practical alternatives include fine-tuning existing pretrained models, deploying on energy-efficient cloud infrastructure, and applying optimizations like pruning, layer fusion, and quantization. These methods deliver low-latency inference and high task accuracy while significantly decreasing hardware requirements, development time, and environmental impact.
Julien Simonhuggingface.co ·
Train a Sentence Embedding Model with 1B Training Pairs
Sentence embeddings map sentences into dense vectors to capture semantic meaning for downstream tasks such as clustering, asymmetric question answering, and text search. To train general-purpose models at scale, the team utilized a dataset compiled from multiple sources containing up to one billion sentence pairs. Training leveraged contrastive learning with Multiple Negative Ranking Loss and in-batch negatives, running across seven TPU v3-8 accelerators with JAX and Flax. Batches combined samples from at least two datasets to balance hard negatives within topics against broader cross-topic semantic structure. The project resulted in twenty trained models—including variants of RoBERTa, DistilBERT, and MPNet—alongside eight specialized evaluation datasets and an interactive demo covering sentence similarity, search, and gender bias.
Antoine SIMOULINGrab ·
Designing products and services based on Jobs to be Done
The Jobs to be Done (JTBD) framework helps organizations understand consumer goals by defining a job as the progress a consumer seeks to achieve in a specific context. To prioritize new features, GrabFood applied this framework and force diagrams to investigate consumer motivations and friction points. Interviews revealed that working parents ordering family meals experienced anxiety from navigating excessive individual choices during checkout. To address this obstacle, Grab developed an algorithm within the GrabMerchant application that identified complementary food items and automatically suggested ready-made bundles. Following the release, thousands of merchants adopted one-tap bundle creation, streamlining ordering for families while supporting restaurant growth.
Tim Langehuggingface.co ·
The Age of Machine Learning As Code Has Arrived
Recent findings from the 2021 State of AI Report and Kaggle State of Machine Learning and Data Science Survey indicate that machine learning is expanding into critical infrastructure while Transformers become general-purpose architectures across text, vision, and audio. In response, organizations face questions about scaling infrastructure, team composition, and engineering maturity. Rather than treating machine learning as isolated sandbox experiments or hiring solely data scientists, teams benefit from adopting established software engineering and DevOps principles like versioning, testing, automation, and continuous deployment. Furthermore, the rise of pre-trained Transformer architectures enables practitioners to fine-tune existing off-the-shelf models rather than training from scratch, reducing compute costs and training duration. Tools from platforms such as Hugging Face streamline model deployment, latency optimization, and infrastructure abstraction.
Julien SimonSupabase ·
Replenysh uses Supabase to implement OTP in less than 24-hours
Replenysh builds infrastructure connecting brands with local communities to track, monetize, and recover used packaging and materials. The engineering team previously maintained their own backend infrastructure, pairing custom Twilio and password authentication with a Haskell server and a Postgres database on Heroku. Seeking to accelerate development velocity and minimize ongoing DevOps maintenance during a codebase refactor, Replenysh required an alternative backend platform that preserved relational Postgres and Row Level Security. The team selected Supabase over alternatives like Firebase because it provided native Row Level Security alongside built-in authentication tooling. By adopting Supabase Auth, Replenysh successfully implemented an SMS one-time password system in less than 24 hours before deploying their mobile application to production.
Supabase ·
Hacktoberfest Hackathon Winners 2021
The organizers of the second open-source Hacktoberfest Hackathon announced the top ten projects across five distinct categories, selecting a winner and runner-up for each. Awarded projects include Pickle, a privacy-focused developer analytics service featuring realtime event synchronization, and CourseBuddy, a collaborative study platform for students. Other winning entries encompass the Feedback widget for sentiment collection, a utility management Flutter application, and Spookd, a Halloween-themed game built with Framer and connected to Supabase. Runner-up projects feature tools such as og:supa for generating Open Graph preview images, Relm for community management, Party Parrot as a Service, the TLWR path logger for Flutter, and the uwudaily mood-logging website. All winners receive custom gold medal shirts, runners-up receive silver medal shirts, and every participant receives a limited-edition participation tee.
Thor Schaeffhuggingface.co ·
Fine tuning CLIP with Remote Sensing (Satellite) images and captions
Pre-trained vision-language models like CLIP generalize well across everyday imagery but struggle with remote sensing domain characteristics. To enable text-based search across satellite imagery, a distributed team fine-tuned the openai/clip-vit-base-patch32 model on TPU hardware using JAX and Flax during the Hugging Face Flax/JAX Community Week. The training pipeline incorporated satellite image and caption pairs from the RSICD, UCM, and Sydney datasets alongside extensive image transforms and backtranslation-based text augmentation. Contrastive optimization using Adam with a learning rate of 5e-6 achieved top-1 retrieval accuracy of 0.883 compared to 0.572 on the baseline model. An interactive demo uses a NMSLib index for text-to-image matching, image-to-image matching, and patch-level feature detection.
Arto, Dev Vidhani, Goutham, Mayank Bhaskar, Ritobrata Ghosh, Sujit Pal