How Apache DataFusion Runs SQL In-Process
Apache DataFusion is an embeddable query engine: SQL goes in, runs as columnar Arrow batches inside your own process, and results come out without a separate service.
Loading…
Topic
Rust
66 posts about Rust. Every summary links to the original.
- 66 posts
- 17 companies
- 2 explainers
See the idea
Rust explainers
Axum at the Trust Boundary: From Request to Protected Response
Axum is a Rust web framework for composing routes, middleware, and extractors. This explainer shows how different login flows become verified identity before a handler responds.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
How to Retrieve a Phone Number's SMS Pumping Score in Rust
SMS Pumping Fraud exploits phone number input fields to generate revenue from SMS messages sent to numbers controlled by a specific mobile network operator. Twilio’s Lookup v2 API provides an SMS Pumping Risk Score from 0 to 100, based on a proprietary model using network data including Verify Fraud Guard signals and unusual traffic patterns. This Rust tutorial uses the rustlio crate, dotenvy, serde, and tokio to call the API and retrieve the score along with carrier risk category and block status. Canadian numbers do not return carrier information, and Twilio advises against using the score for US or Canadian numbers. Guidelines recommend sending messages for scores 0-60, adding friction for 60-75, treating as suspicious for 75-90, and blocking for 90-100.
The results of the 2026 Developer Survey are here!
The 2026 Developer Survey gathered responses from over 30,000 developers over seven weeks, covering work patterns, technology usage, and AI adoption. Only 18% of developers work in an office daily, and the share of solo developers rose from 4% to 10%, possibly reflecting AI-enabled solopreneurship. In technology rankings, SQL surpassed HTML/CSS for the second spot, while Rust climbed to 12th most used language, potentially influenced by memory safety advocacy. AI usage is widespread: 73% of AI tool users employ them daily, with Claude Code (66%) and GitHub Copilot (59%) as leading agents, primarily for code generation (67%) and debugging (61%). Developers prioritize quality over cost (75% vs 38%) when using AI, and organizations implement cost controls like usage limits (38%) and model routing (27%), though 21% report increased spending.
RustRover vs VS Code + rust-analyzer: What Changes in Your Rust Workflow
This post compares RustRover and VS Code with the rust-analyzer extension for Rust development, detailing their architectural differences. VS Code, a general-purpose editor, relies on the rust-analyzer language server to provide completion, diagnostics, inlay hints, navigation, quick-fixes, and refactoring. RustRover is a dedicated professional IDE using JetBrains’ own analysis engine that builds a model of crates, dependencies, targets, and code structure from the Cargo workspace, with built-in debugging, testing, and version control. Key differences include RustRover’s integrated debugger versus VS Code’s need for an extension like CodeLLDB, and RustRover’s project-wide refactoring with affected-usage review. AI assistant capabilities also differ, with RustRover offering agent-driven tasks through Air and JetBrains AI while VS Code supports inline suggestions and agent providers like Copilot. The post offers a five-step migration guide for VS Code users and concludes the choice depends on whether one prefers a modular editor or an integrated IDE experience.
Supporting native Rust in Workers with the new Emscripten target for wasm-bindgen
Cloudflare announced an experimental Emscripten target for wasm-bindgen, enabling native Rust code compiled with wasm-bindgen to run directly on Workers with full platform feature virtualization. This resolves the long-standing incompatibility between Emscripten and wasm-bindgen by having Emscripten drive the build while wasm-bindgen produces portable bindings included in Emscripten’s library system. The integration virtualizes timers, file system operations, and sockets atop Workers’ Node.js APIs, bringing significantly improved library compatibility for many previously unsupported Rust crates after minor patches to libraries like libc and socket2. Tokio async runtime support is achieved through WebAssembly JavaScript Promise Integration, which suspends blocking calls and returns control to the event loop without stalling. In testing, a Rust Minecraft server (Pumpkin) was deployed inside a Durable Object with TCP ingress using real Tokio sockets, persisting world state through Durable Object storage and supporting multiplayer connections.
100 Exercises to Learn Rust, Updated
The post announces a major update to the free JetBrains Academy course ‘100 Exercises to Learn Rust,’ which runs inside RustRover. The course teaches Rust through hands-on exercises building a ticket management system across eight chapters from ownership to futures, assuming fluency in another language. The update adds a rebuilt introduction, an FAQ, a final challenge turning the system into an async REST API by editing Cargo.toml and verifying with a requests.http file, visible test assertions, and a dev/release profile experiment. A ‘fast track’ lets experienced users skip to the final challenge to quickly see which areas need revisiting. The post advises using AI for explanations, not code generation, and turning off AI code completion during exercises to maximize learning from getting stuck and working out solutions.
tokenizers v1: encode, decode and scaling, measured
The tokenizers library is being reworked for version 1 with a heavy focus on performance, as tokenization can become a bottleneck that starves models of data when workloads scale. The work draws on ideas from other open-source tokenizers like gigatoken, tiktoken, and kitoken. Key optimizations include using SIMD instructions for UTF-8 text, ensuring merges never cross pre-token boundaries, and reusing scratch buffers to avoid allocations. The release candidate is implemented in the Rust pre-release on crates.io, with the crate split into tk-encode, tk-serialize, tk-convert, and tk-train. Batched model calls and faster decoding with parallel batch processing are also part of the improvements.
Saving another 100TB of RAM with math (and Rust)
Cloudflare describes how excessive memory use in pingora-ketama, the consistent-hashing component of its Pingora Backend Router, was investigated as a resource-efficiency problem at global scale. Consistent hashing maps servers and cache keys onto a circular hash space, but a single hash per server creates uneven ranges, while multiple hashes improve balance; with 160 points per server, the coefficient of variation falls to about 8% in the 100-server example. The work produced a v2 ring with compacted storage, faster sorting, and a configurable base number of hashes per node, while retaining the existing v1 ring and supporting simultaneous use during migration. After the old ring was decommissioned, measured memory usage for PBR fell by 100TB globally, and the changes are available in pingora-ketama behind an unadvertised Cargo feature.
ClickHouse is now available on the dbt platform
The ClickHouse adapter for dbt v2, dbt’s Rust rewrite, entered public beta on September 16, 2026, while ClickHouse became a natively supported data warehouse on the dbt platform in private beta. The v2 adapter ships inside the open-source dbt binary, connects through an ADBC driver using Apache Arrow, and supports both open-source ClickHouse and ClickHouse Cloud. Existing models, tests, and profiles carry over, although some features remain incomplete and dbt plans to support the v1 and v2 adapters in parallel. The platform integration provides ClickHouse connections for Studio, environments, scheduled jobs, and Catalog, while the planned path to general availability includes parity items such as ON CLUSTER DDL and distributed materializations, followed by Fusion capabilities.
Translating CUDA Tile Operations from Python to Rust Using Agentic AI
cuTile Rust (cutile-rs) extends Rust’s ownership model to tile-based GPU kernels, and this post describes an agentic workflow for translating cuTile Python and Triton-TileIR implementations. Because all three front ends target CUDA Tile IR and share the tileiras compiler, the workflow compares reference and translated IR while making Python’s implicit specializations explicit through Rust const generics, fixed dtype symbols, and FFI checks. A bounded multi-agent pipeline covers analysis, device kernels, host and FFI code, correctness, and benchmarking, with machine-checkable verdicts controlling progress. It ported all 24 public TileGym operators, roughly 40 kernels, and reported 99.5% average cuTile Python performance across 347 paired configurations on NVIDIA DGX B200. The skill ships with TileGym and uses acceptance checks requiring correctness, backend execution, and a geomean speedup of at least 0.95.
Rust AI in Practice: Building LLM Applications With Rig
Rig is an open-source Rust library that provides a common interface for LLM providers including OpenAI, Anthropic, and Gemini, avoiding application rewrites when switching services. It structures applications around a provider client, completion model, agent, preamble, and registered tools, with async APIs handling provider network requests. In the Rat Code demo, built with Ratatui, separate tools read and write files and run shell commands, while Rig’s streaming API updates the terminal as responses arrive. The library also supports RAG through vector store abstractions, local inference via Ollama, llama.cpp, or Candle, and cassette-based integration tests that replay recorded HTTP interactions in CI; live model quality requires separate scheduled testing.
The Evolution of WSL Support in JetBrains IDEs
JetBrains IDEs have supported WSL through several architectures, but the 2026.2 release makes Native mode the recommended entry point for projects stored there. In Native mode, the Windows IDE delegates filesystem and process operations to the Rust-based IJent agent inside WSL, with EelApi abstracting local and remote environments for IDE and plugin code. Earlier approaches included 9P filesystem access, WSLg, and Remote Development, with problems involving symlink handling, latency, GUI limitations, installation overhead, and client–backend communication costs. Performance tests on a cold first open of spring-framework measured Native mode at 11.5 seconds to become ready versus 18.5 seconds for 9P, while project-tree scanning took 3.7 versus 8.1 seconds. Remote Development remains available, but direct project opening is now preferred in IntelliJ IDEA, WebStorm, and PhpStorm.
Introducing CUDA Rust: Two Tracks for Writing GPU Kernels
NVIDIA’s CUDA Rust effort targets the GPU-kernel gap in Rust-based systems, where host code can be Rust but kernels often require another language. It presents two tracks: cuda-oxide compiles #[kernel] functions through Rust MIR, Pliron, and LLVM IR to PTX for SIMT programming, while cutile-rs expresses computations as tiles and JIT-compiles embedded ASTs through CUDA Tile IR. The SIMT example uses DisjointSlice, typed thread indices, and launch contracts to constrain per-thread output access and validate launch configuration; Tile leaves thread mapping and shared-memory management to its compiler. The post recommends starting with Tile because the compiler decides tile mapping per architecture, and using SIMT when explicit thread or memory control is needed. Both projects are early-stage, with cuda-oxide in early alpha and cutile-rs further along but still incomplete and subject to API changes.
Developing provably correct Rust code with Verus
Rust’s type system prevents many memory-safety bugs, but it does not prove that programs compute intended results, avoid leaks, or correctly handle every unsafe and concurrent operation. Verus is an open-source automated program verifier that checks Rust implementations against formal specifications for all possible inputs, using solvers to discharge proof obligations while developers provide guidance such as loop invariants. Specifications and proofs use Rust-like requires and ensures annotations written alongside the code, while ordinary Rust compilers ignore them and Verus typically returns feedback in under a second. The tool can prove properties of unsafe code, lock invariants, and concurrent systems, and Amazon has used it for key Nitro Isolation Engine primitives and other critical infrastructure.
Type-Safe Authentication in Rust with Auth0 and Axum
CLI applications cannot use a standard browser redirect OAuth login flow when no browser or redirect URL is available, so the tutorial uses OAuth 2.0 Device Authorization Flow. It builds an end-to-end Rust workspace containing a CLI, a shared auth-lib crate, and an Axum API configured with Auth0. The CLI requests a device code, displays a verification URL and user code, polls for an ID token and access token, then sends the access token as a Bearer credential. The API middleware fetches JWKS, selects the key matching the JWT’s kid, validates RS256 signatures plus issuer and audience claims, and deserializes strongly typed UserClaims. With valid claims, a protected route returns a message and subject; otherwise the extractor returns an unauthorized response.
How we saved 100 terabytes of memory by optimizing 1.1.1.1’s DNS cache
Cloudflare's Big Pineapple platform, which powers 1.1.1.1 and other DNS services, stores over 250 billion cache entries simultaneously. To decrease fleet-wide memory consumption, engineers implemented five changes to in-memory cache data structures. Key modifications included swapping growable types like Vec and String for Box slices to eliminate capacity overhead, combining distinct record sections into a single list indexed by offsets, omitting redundant owner domain names, and serializing record bytes into contiguous buffers. In benchmarks, these optimizations decreased per-entry memory footprints by 56% while improving cache lookup latency by 19% and insert throughput by 43%. Across production instances, the rollout reduced working-set memory by approximately 100 terabytes.
How Ubuntu Is Using Rust to Rebuild Core System Tools
Canonical is selectively adopting Rust to rewrite core system utilities across Ubuntu releases to improve memory safety, security, and long-term resilience. Starting in Ubuntu 26.04 LTS, uutils coreutils shipped as the default implementation targeting complete GNU compatibility, alongside sudo-rs, which introduced intentional behavior changes like asterisk display on password entry. Canonical is also funding ntpd-rs to consolidate NTP, NTS, and PTP into a single utility, while collaborating on UPKI to bring certificate revocation to system utilities. To manage Rust packaging at distribution scale, Canonical vendors dependencies per package rather than creating individual Debian packages for crates and embeds software bills of materials into binaries using cargo auditable. Future plans include exploring Rust implementations of compression libraries like bzip2-rs, zlib-rs, and zstd-rs.
How to Build Passwordless Auth With Twilio Verify in Rust
Passwordless authentication replaces passwords with one-time codes delivered by SMS, and this tutorial builds a Rust app around that flow. The app uses Twilio Verify for OTP generation and validation, with GET and POST routes at "/" for collecting a phone number and at "/verify" for entering the received code. It sets up a Cargo project with Axum, Handlebars templates, sessions, dotenvy configuration, and the rustlio Twilio client, while environment variables hold the Account SID, Auth Token, and Verify Service SID. The implementation also serves HTML forms and static assets, with verification status rendered as a success or error message. Running cargo run starts the server on port 8080, where users can submit a number, receive an SMS, and see whether the verification code is valid.
Device Authorization Flow for a Rust CLI and an Axum API
Command-line tools lack browser redirect capabilities, preventing standard OAuth login flows. To resolve this limitation, the OAuth 2.0 Device Authorization Flow issues a temporary device code, presents a verification URL to the user, and waits for authentication in an external browser. A Rust workspace demonstrates this end-to-end pattern across a CLI, an Axum web API, and a shared authentication library. The shared library relies on jsonwebtoken and reqwest to fetch JSON Web Key Sets and validate token signatures, issuers, and audiences against typed claims. Axum secures protected routes through custom FromRequestParts extractors that decode Bearer headers, enabling verified CLI access.
Secure all your internal vibe-coded applications — in one click
AI-assisted development enables employees to build applications rapidly, but unmanaged deployments can accidentally expose internal company data to the public Internet. Cloudflare introduced direct Cloudflare Access integration for Cloudflare Workers, allowing organizations to enforce authentication policies directly at the account or individual Worker level rather than per hostname. When enabled, incoming requests are authenticated before reaching application code across custom domains, routes, workers.dev subdomains, and preview environments. Developers can access authenticated identity data such as emails and groups directly via the Worker context object without manually validating JSON Web Tokens. This capability was implemented on Cloudflare's Rust-based FL2 proxy, which separated Worker routing from Worker execution so routing runs safely prior to Access policy enforcement.
Don’t stop early: Case-folding source code at memory speed
GitHub's code search engine, Blackbird, must case-fold over 480TB of source code across 180 million repositories during indexing and query matching. To accelerate this operation on source code that is overwhelmingly ASCII, the engineering team replaced early-exit branching with an unconditional branch-free loop. The implementation tests uppercase ASCII ranges using wrapping arithmetic, modifies bits in place, and detects non-ASCII bytes with an accumulator register tested only after the loop completes. Eliminating data-dependent exits allowed LLVM to generate SIMD instructions and achieve throughput exceeding 45 GiB/s on an Apple M4 processor. The optimized implementation was released as the open-source Rust crate casefold.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.