Loading…
Latest reads
The engineering internet, summarised so you can actually read it.
Grab ·
Debugging High Latency Due to Context Leaks
Market-Store, Grab's feature store for real-time machine learning features, experienced latency spikes from under 200 milliseconds to 2 seconds as traffic grew. Metrics and logs showed no direct correlation to API issues, but heap profiling with PPROF revealed continuously increasing memory held by child contexts. Further analysis tracked the leak to an update in Grab's open-source Async Library, which switched background contexts to uncancelled task contexts for worker runners. Because parent contexts maintained references to these uncancelled child contexts, the garbage collector could not reclaim their memory. This progressive memory exhaustion directly degraded API latency.
Sourabh Sumanhuggingface.co ·
Sentence Transformers in the Hugging Face Hub
Sentence Transformers has collaborated with the Hugging Face Hub as part of its v2 release to make sentence, paragraph, and image embedding models readily accessible. The Hub now hosts over 90 pretrained models supporting more than 100 languages for use cases like semantic search and multilingual zero-shot classification. Alongside direct model loading in minimal code, the platform provides interactive Hub widgets for feature extraction and sentence similarity computation. Developers can execute inference programmatically through Inference API endpoints. Additionally, the library supports direct model uploading to the Hub via save_to_hub, which automatically generates a model card detailing layer architecture and dual usage with Sentence Transformers and Transformers.
Omar Sanseviero, Nils Reimershuggingface.co ·
Few-shot learning in practice: GPT-Neo and the 🤗 Accelerated Inference API
Limited labeled data often prevents machine learning models from achieving high performance during traditional fine-tuning. Few-shot learning addresses this hurdle in natural language processing by supplying a short task description and a few demonstration examples at inference time. EleutherAI developed GPT-Neo, an open transformer-based model trained on the Pile dataset, which can execute few-shot tasks via the Hugging Face Accelerated Inference API. Compared to larger models like GPT-3, the 2.7B parameter version of GPT-Neo requires three to four examples alongside tuned hyperparameters like temperature and stopping sequences to generate accurate completions. Because few-shot learning relies heavily on pre-trained associations, careful output monitoring and human feedback mechanisms remain necessary to prevent automated bias.
Philipp SchmidSupabase ·
Supabase Beta May 2021
Supabase published its May 2021 beta updates covering authentication, regional infrastructure, and tooling. Developers can now implement Apple and Twitter authentication via Supabase Auth, addressing App Store requirements and expanding third-party login options. The platform introduced a Policy Editor with templates to configure Row Level Security for Storage, alongside CSV data retrieval capabilities. Infrastructure expanded with a new Tokyo region to support developers in Japan and China, while Supabase Grid was publicly released as part of an effort to open-source the dashboard. Additionally, community contributors advanced client libraries, completing postgrest-go and progressing development on Swift libraries including gotrue-swift, realtime-swift, and storage-swift.
Paul CopplestoneSupabase ·
Supabase Beta June 2021
Supabase announced a series of platform updates, documentation additions, and tooling enhancements for June 2021 ahead of its scheduled Launch Week. Developers can now provision a Postgres database directly through a new Vercel integration and enable authentication using Discord logins. Supabase Storage added support for upsert operations alongside public buckets that allow unauthenticated file access. The web dashboard introduced a restart button for debugging and a table policy editor with templates to configure Row Level Security. Additional backend improvements include timestamps for PostgREST error logs, an OpenAPI mode patch ignoring anonymous privileges for Prisma users, and an XSS fix in the Auth0 Next.js library.
Paul Copplestonehuggingface.co ·
Using & Mixing Hugging Face Models with Gradio 2.0
The Hugging Face Model Hub hosts more than 10,000 user-submitted machine learning models spanning natural language processing, image classification, and audio processing. Machine learning developers increasingly need to demo models to interdisciplinary collaborators or non-programmers to uncover biases and failure points. The Gradio 2.0 library enables users to create graphical user interfaces and load almost any Hugging Face model using a single line of code. By default, these interfaces query Hugging Face's hosted Inference API without requiring an API key, though users can also run computations locally via the transformers library. Gradio interfaces can be customized through the Interface class and composed to load multiple models in parallel or chain them in series.
Abubakar AbidGrab ·
Building a Hyper Self-Service, Distributed Tracing and Feedback System for Rule & Machine Learning (ML) Predictions
Grab's Trust, Identity, Safety, and Security team processes billions of daily rule and machine learning decisions for fraud detection, safety, and identity checks. Earlier logging approaches using plain text Kibana logs and the ActionTrace library lacked structured formats, dynamic entity customization, and fine-grained access controls. To resolve these limitations, the team built Archivist, a centralized tracing, statistics, and feedback system. Archivist ingests events through an SDK into Kafka streams, buffers and routes data into Elasticsearch indices and Amazon S3, and provides a role-based user portal. The platform handles 80 million daily logs across roughly 50 business scenarios, reducing scenario onboarding times from days to minutes.
Warren ZhouGrab ·
Our Journey to Continuous Delivery at Grab (Part 2)
Conveyor, an in-house continuous delivery system at Grab, introduces hermetic deployments by tracking application code alongside static and dynamic configuration parameters. This hermeticity guarantees that production releases use combinations of versioned artifacts previously verified in staging, preventing rollback incompatibilities. Conveyor replaces single multi-environment pipelines with decoupled pipelines, while automating cluster locking, deployment slot scheduling, release note generation, and canary monitoring with automated rollbacks. These automations reduced production deployment failure rates from 1.5% to an average of 0.3% over a three-month period. Grab also doubled the volume of production changes between 2018 and 2020 while saving more than 5,000 man-days of engineering effort in 2020.
Sylvain BougerelSupabase ·
Supabase Beta April 2021
Supabase focused on platform stability, security enhancements, and community support during its April 2021 beta update cycle. To address visual accessibility needs for developers with astigmatisms or those operating in brightly lit environments, the team delivered Light Mode within the dashboard settings. The release also introduced OpenAPI-powered Storage API documentation, community translations for the main repository spanning nineteen distinct languages, and an experimental synchronization server connecting Postgres with Stripe. Dashboard components are systematically migrating into an open-source UI library, establishing necessary foundations for local development tooling and self-hosted environments. Additionally, the team reported substantial contributor growth across its realtime server and UI repositories while continuing engineering on a Workflows engine aimed at a July release.
Paul Copplestonehuggingface.co ·
Scaling-up BERT Inference on CPU (Part 1)
Scaling BERT-like model inference on CPUs presents practical deployment challenges as transformer architectures proliferate in production environments. Using a reproducible Hydra-based benchmarking framework, inference workloads were evaluated on an Amazon Web Services c5.metal instance featuring an Intel Xeon Platinum 8275 processor with AVX512 and VNNI support. Out-of-the-box evaluations showed PyTorch 1.8.1 delivering lower latency than Google TensorFlow 2.4.0, likely due to PyTorch relying on OpenMP and Intel MKL/oneDNN rather than Eigen. Implementing multiple inference streams by pinning parallel model instances to dedicated CPU cores yielded near-linear throughput scaling. Matching instance core allocations to specific problem sizes also demonstrated that smaller workloads can achieve optimal latency on fewer cores, enabling up to a sixfold reduction in cloud compute costs.
Morgan FuntowiczGrab ·
How We Improved Agent Chat Efficiency with Machine Learning
Agent typing time represented a large portion of Grab's chat support journey, and 85% of messages were still free typed because agents customized static templates to fit their personal style. To accelerate typing across multilingual markets without robotic templates, Grab built SmartChat, a machine learning feature that provides contextual sentence completion. The team opted for a lightweight seq2seq architecture using single-layered GRU encoder-decoders in TensorFlow instead of bulky attention models to keep model latency under 100ms. The user interface was implemented in React using a content-editable div with inline typeahead suggestions activated via keyboard shortcuts.
Suman Anandhuggingface.co ·
Introducing 🤗 Accelerate
🤗 Accelerate enables PyTorch developers to execute raw training loops across CPUs, multi-GPU setups, and TPUs with minimal boilerplate modifications. Standard distributed training typically demands manual device assignments, DistributedSampler configurations, and divergent code paths that break single-device portability. By wrapping models, optimizers, and dataloaders through a unified prepare call, Accelerate automatically manages hardware placement, mixed precision operations, and sampler batch slicing without requiring custom distributed samplers. Distributed evaluation is streamlined using a gather utility that consolidates prediction tensors across active worker processes. Furthermore, a dedicated command-line interface provides an interactive questionnaire to store runtime configuration defaults and orchestrate local or AWS SageMaker runs.
Sylvain Guggerhuggingface.co ·
Distributed Training: Train BART/T5 for Summarization using 🤗 Transformers and Amazon SageMaker
Hugging Face and Amazon SageMaker introduced optimized Deep Learning Containers alongside a SageMaker Python SDK estimator to streamline distributed transformer model training. This workflow demonstrates fine-tuning the 400-million-parameter facebook/bart-large-cnn model for dialogue summarization on the 16,000-conversation samsum dataset. Using SageMaker Data Parallelism enabled in the distribution configuration, training executes across two ml.p3dn.24xlarge compute instances equipped with sixteen total GPUs and a cumulative batch size of 64. The HuggingFace estimator pulls the run_summarization.py script directly from GitHub, enables mixed precision with fp16, and tracks evaluation performance through ROUGE scores. Following completion, the unzipped model artifacts and a generated model card are pushed to the Hugging Face Hub using the huggingface_hub library for hosted inference.
Philipp SchmidSupabase ·
Supabase Beta March 2021
Supabase announced a series of platform updates for March 2021, headlined by the launch of Supabase Storage and new local development capabilities. The new storage offering integrates an open-source Fastify and TypeScript API backed by S3, while direct database connections now bundle PgBouncer for connection pooling. Developers can run the platform locally via the Supabase CLI, utilize Docker tooling for self-hosting, and access the open-sourced Supabase UI component library. Community contributions brought OAuth scopes to Supabase Auth for requesting elevated third-party provider permissions, alongside newly published free and pro pricing tiers. In addition, development is underway on an Elixir-based workflow engine compatible with the Amazon States Language for automated event handling.
Paul CopplestoneSupabase ·
Supabase Dot Com
Supabase announced a migration of its primary website from supabase.io to supabase.com over the upcoming week. User API endpoints will remain unchanged on supabase.co, requiring no configuration adjustments for active applications. The company originally selected supabase.io as a meme-inspired placeholder after releasing a Postgres Realtime engine on GitHub in 2019 and failing to find an available dot com name. Following advice from Y Combinator to secure a dot com address, the founders bypassed domain brokers charging percentage fees and negotiated directly with the domain owner, whose business address was located one hundred meters from the founder's home. The migration updates public website destinations while preserving existing developer infrastructure unaffected.
Paul CopplestoneSupabase ·
PgBouncer is now available in Supabase
Serverless JavaScript frameworks and developer tools frequently establish separate database connections during bursty traffic, rapidly exhausting PostgreSQL connection limits in the absence of traditional middleware. To resolve connection surges, Supabase integrated the open-source connection pooler PgBouncer directly onto the PostgreSQL server across all newly created projects. Rather than increasing the total number of connections PostgreSQL can open, PgBouncer recycles open connections and queues excess requests until active connections become available. Developers can manage pooling through the dashboard across Session, Transaction, and Statement modes, with Transaction mode recommended for serverless functions despite disabling session features like prepared statements. The default pool size is initially configured to 15 connections.
Angelico de los ReyesSupabase ·
Workflows are coming to Supabase
Supabase is developing Workflows, an Elixir-based orchestration engine designed to coordinate complex serverless tasks and event-driven logic. Standard serverless and database-triggered functions often struggle with delayed execution and queuing without external cron processes. To solve this with native Postgres integration, the new engine adopts the open-source Amazon States Language specification to orchestrate functions across platforms including AWS, GCP, Azure, OpenFaaS, and Postgres itself. Execution states, jobs, queues, and logs are deeply integrated with Postgres using the Oban job processing library, though transient in-memory workers are also supported. The engine responds directly to HTTP calls or database change events delivered through Supabase Realtime.
Francesco CecconSupabase ·
Supabase Launches NFT Marketplace
Non-fungible tokens frequently suffer from the copy-paste problem, where public URLs allow unauthorized users to download underlying media directly from web hosts or IPFS. To address this issue, BuyMeth proposes combining public blurhash thumbnails with full image files encrypted under the active owner's cryptographic key and hosted on IPFS. Completed sales trigger Metamask to re-encrypt the file with the buyer's public key, followed by a one-week escrow challenge window where automated verification matches decrypted image hashes against public thumbnails. Ongoing royalty distributions to previous owners provide a financial mechanism to disincentivize leaking unencrypted original image files. The announced platform serves as an April Fools joke rather than an active Supabase product release.
Ant Wilsonhuggingface.co ·
Understanding BigBird's Block Sparse Attention
Transformer-based models face severe computational bottlenecks due to quadratic time and memory complexity, making sequences longer than 512 tokens impractical. BigBird addresses this limitation by approximating full attention with block sparse attention, scaling context lengths up to 4096 tokens at a lower computational cost. Instead of attending to all tokens, BigBird combines sliding attention for local context, global tokens for long-range relationships, and random tokens to accelerate information transfer. The model is integrated into HuggingFace Transformers, enabling fine-tuning for tasks like long-document summarization and extractive question answering. When sequences are shorter than 1024 tokens, using original full attention remains recommended.
Vasudev GuptaSupabase ·
Storage is now available in Supabase
Supabase launched Storage, adding a scalable object store to its existing Postgres, authentication, and API services. Existing open-source storage servers like Ceph, Swift, Minio, and Zenko were evaluated but rejected due to auth incompatibilities and external dependencies like etcd, MongoDB, and Kafka. Consequently, the team implemented a custom Storage API server built with Fastify and TypeScript behind the Kong gateway. Object metadata and access control reside directly in Postgres, leveraging Row Level Security policies written in SQL rather than a proprietary domain-specific language. Objects stream directly to managed backends like AWS S3 using Node streams with minimal in-memory buffering, accompanied by a default one-hour Cache-Control header.
Inian Parameshwaran