Loading…
Latest reads
The engineering internet, summarised so you can actually read it.
Supabase ·
Supabase Beta June 2022
Supabase released its Beta June 2022 updates covering developer tooling, authentication, and infrastructure changes. The team decoupled Supabase Auth Helpers into dedicated packages for Next.js and React, while introducing project pause and restore features to enable unlimited free projects. Developers can now initiate project creation directly through the Supabase CLI and toggle realtime updates from the table editor side panel. Authentication updates include exposing client IP addresses in audit logs, introducing an auth.jwt function, bubbling PostgreSQL errors from GoTrue, and deprecating older authentication functions. Additionally, infrastructure migrations are underway to support multiplayer features, accompanied by community-led tutorials, sponsorships, and platform growth reaching fifty thousand GitHub stars.
Ant WilsonSupabase ·
Flutter Tutorial: building a Flutter chat app
Developers can construct cross-platform real-time chat applications for iOS, Android, and web by pairing Flutter with Supabase. The architecture relies on Supabase to manage authentication and provide direct SDK access to a Postgres database without requiring custom backend server code. The database schema defines profiles with regex validation constraints and a messages table configured for live subscriptions through Postgres publication settings. Flutter clients capture user input, insert new records into the messages table, handle database exceptions, and format chat bubbles with dynamic timestamps. While this initial setup implements user registration and message broadcasting in a shared room, securing chat rooms requires subsequent row-level security configuration.
Tyler ShukertGrab ·
Graph Networks - 10X investigation with Graph Visualisations
Fraud detection traditionally required investigators to manually combine large datasets from disparate anti-fraud systems using statistical methods, which proved slow and inefficient. Grab built an interactive Graph Visualisation platform to transform raw records into connected visual maps without requiring manual queries or switching tools. The platform manages over three billion nodes and edges, allowing investigators to selectively expand data points and replay chronological events using temporal filters. Visual relationship mapping helps teams verify account appeals, uncover device-sharing rings, and spot anti-money laundering behavior through transaction density patterns.
Fujiao Liuhuggingface.co ·
Policy Gradient with PyTorch
Policy gradient algorithms optimize reinforcement learning policies directly without learning intermediate action-value functions. Instead of assigning discrete Q-values to actions, these methods parameterize a stochastic policy and apply gradient ascent on an objective score function. This setup eliminates manual exploration tuning, resolves perceptual aliasing in identical states, and naturally accommodates continuous or high-dimensional action spaces. The Reinforce Monte Carlo policy gradient algorithm operationalizes this approach by collecting full episodic trajectories and adjusting parameters along the gradient of the log-action probabilities scaled by return. Practitioners implement this algorithm using PyTorch to evaluate agent robustness across environments like CartPole-v1, PixelCopter, and Pong.
Thomas SimoniniSupabase ·
Visualizing Supabase Data using Metabase
Organizations seeking to analyze database records can bypass code-heavy Python visualization libraries by connecting a Supabase backend directly to Metabase. Deploying the open-source Metabase Docker container exposes an initial setup interface on default port 3000, allowing administrators to input PostgreSQL connection credentials retrieved from Supabase database settings. Once connected, Metabase automatically surfaces table insights and automated x-rays across the public schema, exposing metrics such as inventory count ranges, price distributions, and column-specific statistics without requiring manual queries. For specialized analytics, users can execute custom SQL joins across relational tables like Product and Vendor to render tailored visual dashboard formats, including configurable bar charts mapped to designated axis fields.
Ant Wilsonhuggingface.co ·
Liftoff! How to get started with your first ML project 🚀
Beginners in machine learning frequently face difficulties when selecting a framework and defining the scope of their initial hands-on project. Sentence Transformers provides an accessible starting point by computing dense vector representations for sentences, paragraphs, and images, enabling semantic search through cosine similarity. A structured four-step strategy guides practitioners through listing library capabilities, identifying compelling datasets, selecting a familiar secondary tool, and brainstorming project concepts. Applying this framework, a song lyrics dataset was paired with Gradio Blocks to build a prompt-based playlist generator. Developing this application required evaluating pre-trained models, hosting generated embeddings on Hugging Face Spaces, and utilizing multi-processor support to accelerate embedding generation.
Nima BoscarinoSupabase ·
Partial data dumps using Postgres Row Level Security
Dumping full production databases onto local development machines creates severe security issues once applications hold real user data. To safely generate local seed data, developers can leverage PostgreSQL Row Level Security to restrict data access. The process involves provisioning a dedicated database user with select privileges on the target schema and tables. Next, administrators enable row-level security policies on those tables to filter records using criteria such as specific primary keys, email domain patterns, date intervals, or boolean flags. Finally, running pg_dump with the restricted user credentials and the --enable-row-security flag produces a sanitized seed.sql file containing only approved records.
Paul Copplestonehuggingface.co ·
Accelerate Large Model Training using DeepSpeed
Training large models on hardware with limited GPU memory frequently causes out-of-memory errors when using standard Distributed Data Parallel. To solve this bottleneck, Hugging Face Accelerate integrates DeepSpeed ZeRO data parallelism to shard optimizer states, gradients, and model parameters across workers. For a 900-million-parameter DeBERTa model, ZeRO Stage 2 increased the maximum per-device batch size from eight to forty while achieving a 3.5-fold training speedup over DDP without degrading accuracy or F1 score. Advanced configurations allow sequence-to-sequence chatbot finetuning and full ZeRO Stage 3 CPU offloading for models like the 1.5-billion-parameter GPT-XL, enabling batch size 16 training where DDP fails. Accelerate enables these memory optimizations through simple configuration files and the accelerate launch command with minimal or zero code modifications.
Sourab Mangrulkar, Sylvain Guggerhuggingface.co ·
Announcing Evaluation on the Hub
Hugging Face introduced Evaluation on the Hub, a no-code tool designed to streamline the evaluation of machine learning models across diverse datasets. Traditional evaluation approaches often suffer from reproducibility issues, implementation inconsistencies, and cumbersome workflows across varied metrics and datasets. Powered by AutoTrain and Hugging Face Spaces, the service allows users to configure tasks, map dataset columns, select evaluation metrics, and run tests directly from dataset pages. Completed evaluations automatically open pull requests on the respective model cards to encode standardized verification metadata and update public dataset leaderboards. This system establishes consistent benchmarking pipelines across tasks like image classification, text summarization, and named entity recognition without requiring local code execution.
Lewis Tunstall, Abhishek, Tristan Thrush, Sasha Luccioni, Leandro von Werra, Nazneen Rajani, Aleksandra Piktus, Omar Sanseviero, Douwe Kielahuggingface.co ·
Getting Started With Embeddings
Embeddings represent unstructured information such as text and images as numerical vectors in a shared semantic space. To demonstrate their utility, a simple semantic search engine is built over US Social Security Medicare frequently asked questions. The system generates 384-dimensional vector representations for thirteen FAQ entries by dispatching POST requests to the Hugging Face Inference API using the sentence-transformers/all-MiniLM-L6-v2 model. Incoming user queries are converted into matching vector representations and evaluated against stored dataset vectors using the util.semantic_search function from the Sentence Transformers library. By calculating cosine similarity scores, the system retrieves and ranks the five most semantically relevant questions without requiring custom keyword rules or massive labeled training sets.
Omar Espejelhuggingface.co ·
Convert Transformers to ONNX with Hugging Face Optimum
Exporting Hugging Face Transformers models to the ONNX format can be accomplished through three different abstraction levels. The low-level approach utilizes torch.onnx.export, requiring manual specification of dummy inputs, input names, output names, opset versions, and dynamic axes configurations. At the intermediate level, the transformers.onnx package simplifies the conversion process by relying on FeaturesManager and prebuilt configuration objects to handle dynamic axis definitions automatically. The high-level method uses Hugging Face Optimum classes such as ORTModelForSequenceClassification by setting the from_transformers flag to True inside from_pretrained. This Optimum export leverages transformers.onnx internally and produces a model ready for immediate inference execution or integration into pipelines.
Philipp SchmidSupabase ·
Python data loading with Supabase
Supabase offers a PostgreSQL-based backend-as-a-service platform alongside an open-source Python SDK for automating CRUD operations and data-intensive tasks. After spinning up a new project in the Supabase dashboard, developers can configure relational schemas in the Table Editor by establishing foreign key constraints between tables like Vendor and Product. Authentication for the SDK relies on passing project URL parameters and API keys, which can be securely stored and loaded through environment variables. Programmatic data population utilizes the supabase client library combined with the Faker package to generate realistic synthetic entries and push them directly to database tables using insert and execute calls. Populated records can subsequently be examined in real time through the dashboard Table Editor interface to verify schema integrity and data persistence.
Ant Wilsonhuggingface.co ·
Intel and Hugging Face Partner to Democratize Machine Learning Hardware Acceleration
Intel has joined Hugging Face's Hardware Partner Program to accelerate Transformer training, fine-tuning, and inference on Intel platforms. Large Transformer models introduce latency bottlenecks in production workloads like search and chatbots, where hardware-level optimization typically requires tedious trial and error. To streamline model optimization, the collaboration introduces Optimum Intel, an open-source library integrating the Intel Neural Compressor for automated quantization, pruning, and distillation. A demonstration applies post-training dynamic quantization to a fine-tuned DistilBERT classification model using a CPU-only PyTorch setup. The quantized model converted 38 Linear and 2 Embedding operators to 8-bit integers, reducing evaluation duration by 1.34x while keeping the accuracy drop within a 5% threshold.
Julien Simonhuggingface.co ·
Director of Machine Learning Insights [Part 3: Finance Edition]
Machine learning leaders in financial institutions face significant operational hurdles when integrating automated models into production environments. Practitioners must navigate complex legacy architectures, strict regulatory oversight, and privacy mandates tied to personally identifiable financial records. Integrating models often fails when organizations lack clear communication buffers, deploy models using improper prediction windows, or treat systems as opaque black boxes without verifying underlying mechanics. Despite these challenges, financial applications increasingly rely on machine learning for anti-money laundering compliance, automated fraud screening, and loan underwriting. Success in this regulated sector demands comprehensive evaluation across representative input spaces, robust ongoing model monitoring, and explainable decision outputs.
Britney MullerGrab ·
How facial recognition technology keeps you safe
Grab utilizes facial recognition technology across its platform for driver authentication, passenger verification, and digital electronic Know Your Customer (e-KYC) processes. The core pipeline consists of image preprocessing through face detection and alignment, anti-spoofing checks, feature extraction into high-dimensional vector embeddings, and downstream verification or search. To counter spoof attacks like screen replays, synthetic moiré patterns are generated and cropped face patches are used during training and inference to focus on local structures rather than global semantic noise. Face verification challenges involving shallow ID datasets and masked faces are resolved using semi-Siamese training architectures and masked data augmentations.
Kai Feng Teehuggingface.co ·
The Annotated Diffusion Model
Denoising Diffusion Probabilistic Models generate data samples by learning to reverse a discrete-time forward diffusion process that incrementally adds Gaussian noise to inputs. The forward process adds noise across a predefined schedule of variance parameters up to a fixed number of steps, transforming data into an isotropic Gaussian distribution. Because the true reverse conditional probability distribution is intractable, a neural network optimizes the variational lower bound to approximate the distribution mean at each step while keeping the variance fixed. Sampling subsequently proceeds by iteratively denoising pure Gaussian noise through successive network evaluations to reconstruct a clean data sample. Although diffusion models demonstrate high generation quality, their primary drawback remains the requirement of multiple forward passes during inference compared to models like generative adversarial networks.
Niels Rogge, Kashif Rasulhuggingface.co ·
Deep Q-Learning with Space Invaders
Tabular Q-learning fails to scale to complex environments like Atari games due to massive observation spaces that make maintaining discrete state-action tables impractical. Deep Q-learning addresses this limitation by using a neural network to approximate Q-values for each possible action from input states. To process visual inputs effectively and capture temporal motion, raw game screens are grayscaled, resized to 84x84 pixels, and stacked in groups of four consecutive frames before passing through convolutional and fully connected layers. The training process stabilizes learning through experience replay buffers, fixed target networks that update periodically, and Double DQN architectures that decouple action selection from target evaluation. Agents can then be trained on environments like Space Invaders using frameworks such as RL-Zoo without tabular memory constraints.
Thomas SimoniniGrab ·
Graph concepts and applications
Real-world systems generate dynamic, non-random connections that traditional statistical approaches fail to characterize or forecast. Graph models represent these structures through vertices and edges, abstracting complex networks into mathematically tractable relationships. Common data representation formats include the Resource Description Framework (RDF), which models subject-predicate-object triples with IRIs, literals, and blank nodes, and Labeled Property Graphs (LPGs), which store arbitrary key-value properties directly on nodes and edges. Graph databases, derived from the LPG model, treat relationships with equal weight to entities, delivering responsive traversals for highly interconnected systems. While they offer agility and explicit relationship modeling, graph databases lack a standardized query language and remain poorly suited for standard transaction-focused workloads compared to relational databases.
Wenxiang LuSupabase ·
Supabase Beta May 2022
Supabase announced several platform updates and performance improvements across its database, hosting, and developer tooling ecosystem for May 2022. Developers managing preview deployments on Jamstack platforms like Vercel and Netlify can now configure wildcard redirect domains such as *.mydomain.com/welcome. For serverless workflows, Edge Functions support a -no-verify-jwt CLI deployment flag to facilitate direct invocation via external webhooks without mandatory token validation. In addition, Supabase exposed a Prometheus-compatible metrics endpoint to all users for real-time project monitoring and alerting. Database security policies also gained a dedicated Target roles configuration, replacing older role check expressions and cutting query execution time from 46 seconds to 7 milliseconds in one customer deployment.
Ant WilsonGrab ·
Automated Experiment Analysis - Making experimental analysis scalable
Manual ad-hoc analysis of online controlled experiments at Grab introduced operational inefficiencies, inconsistent quality control, and scalability barriers across teams. To resolve these issues, Grab extended its GrabX experimentation platform with an Automated Experiment Analysis system that standardises metrics and automates statistical evaluations. The architecture stores experiment configurations and metric definitions from Cosmos DB into Azure Data Lake as bronze datasets, uses Spark on Databricks via Azure Data Factory to process subjects into silver datasets, and applies an internal Python Decision Engine to generate final gold results. These gold datasets are stored in star-schema fact and dimension tables and presented directly in the GrabX interface using embedded Power BI visualisations. The automation eliminates repetitive data pipeline construction for analysts, ensures reproducible findings aligned with initial hypotheses, and accelerates product launch decisions.
Albert Cheng