Loading…
Latest reads
The engineering internet, summarised so you can actually read it.
Supabase ·
Supabase Beta July 2022
Supabase published its July 2022 beta update alongside the announcement of Launch Week 5 scheduled for August 15-19. Key feature releases include a developer preview of Flutter SDK 1.0 with an emphasis on developer experience and Auth Helpers supporting server-side rendering environments like SvelteKit. The team introduced pg_jsonschema, an experimental 10-line Postgres extension created with pgx to enforce structure on json and jsonb columns. Security and identity features were expanded with an hCaptcha integration in authentication settings to block bot attacks. Additionally, email one-time passwords now accommodate lengths between 6 and 10 digits, while the GenerateLink method returns explicit verification metadata.
Ant WilsonSupabase ·
Supabase Flutter SDK 1.0 Developer Preview
Supabase released the Developer Preview of version 1.0 for its Flutter SDK, prioritizing improvements to developer experience across Dart applications. The new version eliminates the need for boilerplate state classes, automating authentication state persistence and deep link processing directly after initialization. To better align with Dart conventions, the SDK now throws exceptions across auth, storage, and Postgrest instead of returning error objects, while also deprecating the .execute() method for database queries. Switching to the app_links library expands deep linking support to macOS and Windows alongside existing mobile and web platforms. Supabase also introduced a companion Auth UI library for prebuilt authentication interfaces and announced plans to integrate the Multiplayer Realtime engine in the stable release.
Tyler ShukertSupabase ·
Implementing "seen by" functionality with Postgres
Tracking unique post views within Postgres presents trade-offs between counter accuracy, row bloat, and concurrent write performance. To evaluate options under an architecture constraint preventing external dependencies, a benchmark suite generated synthetic users, skewed post distributions, and replayed view actions. The benchmark tested four implementations: a naive counter column, an hstore key-value approach, an association table, and HyperLogLog (HLL). Benchmark results demonstrated that simple-hstore achieved the lowest average latency among deduplicating approaches at 2.15 milliseconds, closely followed by HLL at 2.16 milliseconds. Despite hstore's raw performance in prototypes, HLL is recommended for production because it circumvents row bloat as view counts grow while avoiding expensive row counts.
VictorSupabase ·
Revamped Auth Helpers for Supabase (with SvelteKit support)
Supabase has released an updated version of its framework-specific Auth Helpers libraries, introducing official support for SvelteKit alongside existing React and Next.js tooling to simplify server-side rendering authentication. The updated libraries transition from the @supabase/supabase-auth-helpers namespace to @supabase/auth-helpers within a dedicated monorepo managed by Turborepo. Structuring the codebase with Turborepo and changesets enables publishing isolated packages per framework through GitHub Actions, ensuring each helper contains only its own relevant dependencies. Versioning has also been reset to sub-0.x to reflect early framework lifecycles like pre-1.0 SvelteKit and manage expectations for future breaking changes. Meanwhile, the team has documented migration steps from previous packages and announced that Remix Auth Helpers are currently in active development.
Andrew SmithGrab ·
How we automated FAQ responses at Grab
Internal engineering on-call engineers at Grab spent substantial working hours handling repetitive questions in Slack channels, such as how-to inquiries and access permission requests. To resolve this without building an in-house tool, the team conducted an anonymized vendor comparison and selected OneBar through an employee voting process and a phased proof-of-concept. Initial rollouts were restricted by contract to 20 channels, leading the team to prioritize deployment based on Slack message volume and member counts. Populating the knowledge base required roughly a quarter of consistent updates alongside tech talks, while a targeted crowdsourcing campaign among new onboarders expanded the glossary and grew usage to approximately 3,000 users.
Preeti KarkeraNextdoor ·
Typeahead Search at Nextdoor
Nextdoor built a proximity-based autocomplete service to power typeahead search and mention features across its hyperlocal platform for hundreds of millions of entities, including users and businesses. The system shards geographic data using Uber's open-source H3 geohashing library and stores prefix indexes in memory using Redis sorted sets. By adopting a Command Query Responsibility Segregation architecture, ingestion writes are processed on Redis primary nodes and replicated to read-only search nodes with under 10 milliseconds of replication lag. Dedicated APIs handle indexing, typeahead lookups, and ranking before returning hydrated results. Operating since August 2021, the service processes hundreds of millions of monthly typeahead queries while maintaining a P95 search latency below 30 milliseconds.
Jerry TianSupabase ·
Supabase Beta June 2022
Supabase released its Beta June 2022 updates covering developer tooling, authentication, and infrastructure changes. The team decoupled Supabase Auth Helpers into dedicated packages for Next.js and React, while introducing project pause and restore features to enable unlimited free projects. Developers can now initiate project creation directly through the Supabase CLI and toggle realtime updates from the table editor side panel. Authentication updates include exposing client IP addresses in audit logs, introducing an auth.jwt function, bubbling PostgreSQL errors from GoTrue, and deprecating older authentication functions. Additionally, infrastructure migrations are underway to support multiplayer features, accompanied by community-led tutorials, sponsorships, and platform growth reaching fifty thousand GitHub stars.
Ant WilsonSupabase ·
Flutter Tutorial: building a Flutter chat app
Developers can construct cross-platform real-time chat applications for iOS, Android, and web by pairing Flutter with Supabase. The architecture relies on Supabase to manage authentication and provide direct SDK access to a Postgres database without requiring custom backend server code. The database schema defines profiles with regex validation constraints and a messages table configured for live subscriptions through Postgres publication settings. Flutter clients capture user input, insert new records into the messages table, handle database exceptions, and format chat bubbles with dynamic timestamps. While this initial setup implements user registration and message broadcasting in a shared room, securing chat rooms requires subsequent row-level security configuration.
Tyler ShukertGrab ·
Graph Networks - 10X investigation with Graph Visualisations
Fraud detection traditionally required investigators to manually combine large datasets from disparate anti-fraud systems using statistical methods, which proved slow and inefficient. Grab built an interactive Graph Visualisation platform to transform raw records into connected visual maps without requiring manual queries or switching tools. The platform manages over three billion nodes and edges, allowing investigators to selectively expand data points and replay chronological events using temporal filters. Visual relationship mapping helps teams verify account appeals, uncover device-sharing rings, and spot anti-money laundering behavior through transaction density patterns.
Fujiao Liuhuggingface.co ·
Policy Gradient with PyTorch
Policy gradient algorithms optimize reinforcement learning policies directly without learning intermediate action-value functions. Instead of assigning discrete Q-values to actions, these methods parameterize a stochastic policy and apply gradient ascent on an objective score function. This setup eliminates manual exploration tuning, resolves perceptual aliasing in identical states, and naturally accommodates continuous or high-dimensional action spaces. The Reinforce Monte Carlo policy gradient algorithm operationalizes this approach by collecting full episodic trajectories and adjusting parameters along the gradient of the log-action probabilities scaled by return. Practitioners implement this algorithm using PyTorch to evaluate agent robustness across environments like CartPole-v1, PixelCopter, and Pong.
Thomas SimoniniSupabase ·
Visualizing Supabase Data using Metabase
Organizations seeking to analyze database records can bypass code-heavy Python visualization libraries by connecting a Supabase backend directly to Metabase. Deploying the open-source Metabase Docker container exposes an initial setup interface on default port 3000, allowing administrators to input PostgreSQL connection credentials retrieved from Supabase database settings. Once connected, Metabase automatically surfaces table insights and automated x-rays across the public schema, exposing metrics such as inventory count ranges, price distributions, and column-specific statistics without requiring manual queries. For specialized analytics, users can execute custom SQL joins across relational tables like Product and Vendor to render tailored visual dashboard formats, including configurable bar charts mapped to designated axis fields.
Ant Wilsonhuggingface.co ·
Liftoff! How to get started with your first ML project 🚀
Beginners in machine learning frequently face difficulties when selecting a framework and defining the scope of their initial hands-on project. Sentence Transformers provides an accessible starting point by computing dense vector representations for sentences, paragraphs, and images, enabling semantic search through cosine similarity. A structured four-step strategy guides practitioners through listing library capabilities, identifying compelling datasets, selecting a familiar secondary tool, and brainstorming project concepts. Applying this framework, a song lyrics dataset was paired with Gradio Blocks to build a prompt-based playlist generator. Developing this application required evaluating pre-trained models, hosting generated embeddings on Hugging Face Spaces, and utilizing multi-processor support to accelerate embedding generation.
Nima BoscarinoSupabase ·
Partial data dumps using Postgres Row Level Security
Dumping full production databases onto local development machines creates severe security issues once applications hold real user data. To safely generate local seed data, developers can leverage PostgreSQL Row Level Security to restrict data access. The process involves provisioning a dedicated database user with select privileges on the target schema and tables. Next, administrators enable row-level security policies on those tables to filter records using criteria such as specific primary keys, email domain patterns, date intervals, or boolean flags. Finally, running pg_dump with the restricted user credentials and the --enable-row-security flag produces a sanitized seed.sql file containing only approved records.
Paul Copplestonehuggingface.co ·
Accelerate Large Model Training using DeepSpeed
Training large models on hardware with limited GPU memory frequently causes out-of-memory errors when using standard Distributed Data Parallel. To solve this bottleneck, Hugging Face Accelerate integrates DeepSpeed ZeRO data parallelism to shard optimizer states, gradients, and model parameters across workers. For a 900-million-parameter DeBERTa model, ZeRO Stage 2 increased the maximum per-device batch size from eight to forty while achieving a 3.5-fold training speedup over DDP without degrading accuracy or F1 score. Advanced configurations allow sequence-to-sequence chatbot finetuning and full ZeRO Stage 3 CPU offloading for models like the 1.5-billion-parameter GPT-XL, enabling batch size 16 training where DDP fails. Accelerate enables these memory optimizations through simple configuration files and the accelerate launch command with minimal or zero code modifications.
Sourab Mangrulkar, Sylvain Guggerhuggingface.co ·
Announcing Evaluation on the Hub
Hugging Face introduced Evaluation on the Hub, a no-code tool designed to streamline the evaluation of machine learning models across diverse datasets. Traditional evaluation approaches often suffer from reproducibility issues, implementation inconsistencies, and cumbersome workflows across varied metrics and datasets. Powered by AutoTrain and Hugging Face Spaces, the service allows users to configure tasks, map dataset columns, select evaluation metrics, and run tests directly from dataset pages. Completed evaluations automatically open pull requests on the respective model cards to encode standardized verification metadata and update public dataset leaderboards. This system establishes consistent benchmarking pipelines across tasks like image classification, text summarization, and named entity recognition without requiring local code execution.
Lewis Tunstall, Abhishek, Tristan Thrush, Sasha Luccioni, Leandro von Werra, Nazneen Rajani, Aleksandra Piktus, Omar Sanseviero, Douwe Kielahuggingface.co ·
Getting Started With Embeddings
Embeddings represent unstructured information such as text and images as numerical vectors in a shared semantic space. To demonstrate their utility, a simple semantic search engine is built over US Social Security Medicare frequently asked questions. The system generates 384-dimensional vector representations for thirteen FAQ entries by dispatching POST requests to the Hugging Face Inference API using the sentence-transformers/all-MiniLM-L6-v2 model. Incoming user queries are converted into matching vector representations and evaluated against stored dataset vectors using the util.semantic_search function from the Sentence Transformers library. By calculating cosine similarity scores, the system retrieves and ranks the five most semantically relevant questions without requiring custom keyword rules or massive labeled training sets.
Omar Espejelhuggingface.co ·
Convert Transformers to ONNX with Hugging Face Optimum
Exporting Hugging Face Transformers models to the ONNX format can be accomplished through three different abstraction levels. The low-level approach utilizes torch.onnx.export, requiring manual specification of dummy inputs, input names, output names, opset versions, and dynamic axes configurations. At the intermediate level, the transformers.onnx package simplifies the conversion process by relying on FeaturesManager and prebuilt configuration objects to handle dynamic axis definitions automatically. The high-level method uses Hugging Face Optimum classes such as ORTModelForSequenceClassification by setting the from_transformers flag to True inside from_pretrained. This Optimum export leverages transformers.onnx internally and produces a model ready for immediate inference execution or integration into pipelines.
Philipp SchmidSupabase ·
Python data loading with Supabase
Supabase offers a PostgreSQL-based backend-as-a-service platform alongside an open-source Python SDK for automating CRUD operations and data-intensive tasks. After spinning up a new project in the Supabase dashboard, developers can configure relational schemas in the Table Editor by establishing foreign key constraints between tables like Vendor and Product. Authentication for the SDK relies on passing project URL parameters and API keys, which can be securely stored and loaded through environment variables. Programmatic data population utilizes the supabase client library combined with the Faker package to generate realistic synthetic entries and push them directly to database tables using insert and execute calls. Populated records can subsequently be examined in real time through the dashboard Table Editor interface to verify schema integrity and data persistence.
Ant Wilsonhuggingface.co ·
Intel and Hugging Face Partner to Democratize Machine Learning Hardware Acceleration
Intel has joined Hugging Face's Hardware Partner Program to accelerate Transformer training, fine-tuning, and inference on Intel platforms. Large Transformer models introduce latency bottlenecks in production workloads like search and chatbots, where hardware-level optimization typically requires tedious trial and error. To streamline model optimization, the collaboration introduces Optimum Intel, an open-source library integrating the Intel Neural Compressor for automated quantization, pruning, and distillation. A demonstration applies post-training dynamic quantization to a fine-tuned DistilBERT classification model using a CPU-only PyTorch setup. The quantized model converted 38 Linear and 2 Embedding operators to 8-bit integers, reducing evaluation duration by 1.34x while keeping the accuracy drop within a 5% threshold.
Julien Simonhuggingface.co ·
Director of Machine Learning Insights [Part 3: Finance Edition]
Machine learning leaders in financial institutions face significant operational hurdles when integrating automated models into production environments. Practitioners must navigate complex legacy architectures, strict regulatory oversight, and privacy mandates tied to personally identifiable financial records. Integrating models often fails when organizations lack clear communication buffers, deploy models using improper prediction windows, or treat systems as opaque black boxes without verifying underlying mechanics. Despite these challenges, financial applications increasingly rely on machine learning for anti-money laundering compliance, automated fraud screening, and loan underwriting. Success in this regulated sector demands comprehensive evaluation across representative input spaces, robust ongoing model monitoring, and explainable decision outputs.
Britney Muller