How to Move Independent Work Through an asyncio Pipeline
A practical asyncio workflow for moving independent work through typed events, queues, concurrent tasks, and awaited API calls. Learn where validation and fallback belong.
Loading…
Topic
Python
250 posts about Python. Every summary links to the original.
- 250 posts
- 28 companies
- 1 explainer
See the idea
Python explainers
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Bridging the Domain Gap: AI Race Coach built with Antigravity and Gemini
At Sonoma Raceway, Google Developer Experts built an AI-powered race coach to turn vehicle telemetry into split-second driving advice, extending an earlier pilot that handled basic telemetry. The system uses Antigravity for stateful orchestration and ingestion, Python and Jetpack Compose for mobile processing and display, Agent Development Kit for agent coordination, and hybrid Gemma 4 edge reasoning plus Gemini API cloud analysis. A custom USB interface connected a Pixel 10 directly to the vehicle telemetry network, providing a 10 Hz stream from hundreds of sensors, while an activated on-device TPU reached 40 tokens per second for local coaching alerts delivered through text-to-speech. The test identified a throttle application zone in Turn 2 associated with a 0.1-second gain, and the initiative is scheduled to continue at Interlagos, Brazil.
Why we built ADK 2.0
ADK 2.0 addresses the structural challenge of moving AI agents from prototypes into production, where infinite loops, hallucinated bypasses, and failures without clean exceptions make model-led orchestration unreliable. Because language models are slow, costly, and variable for routing, scheduling, and error handling, the release separates execution routing from language processing and reserves LLMs for ambiguous or cognitive steps. Building on ADK v1’s support for Python, Java, Go, TypeScript, and Kotlin, it adds a structured workflow runtime and task-collaboration model, with Workflows available in Python since March and newly launched for Go. Deterministic graphs combine tool calls, Human-in-the-Loop steps, and specialized agents with explicit branching and error handling. Dynamic workflows extend this model with native Python and asyncio control flows for retries, branching, and reusable sub-workflows.
10 Years of Meta’s Commitment to Python
Meta has marked ten consecutive years as a sponsor of the Python Software Foundation (PSF) to ensure the language remains robust and sustainable. Python represents the most widely used programming language across Meta's infrastructure, powering applications like Instagram and Threads as well as internal AI research. Beyond financial backing, Meta engineers actively contribute core language features, author Python Enhancement Proposals (PEPs), and build open-source tools such as the Pyrefly type checker. Company sponsorship directly funds the PSF's Developer-in-Residence program, core infrastructure, and security enhancements for the Python Package Index (PyPI). Continued support of the PSF functions as a strategic investment in maintaining the stability of the broader open-source ecosystem.
How I hunt for vulnerabilities with AI
The author describes using AI-assisted code review to hunt for vulnerabilities in ClickHouse, a large C++ codebase, despite having limited bug-bounty experience. GitHub Copilot, Claude, ChatGPT, and Gemini models were used to inspect code, generate vulnerability hypotheses, explain unfamiliar subsystems, and identify older or newly introduced attack surfaces. Promising ideas were recorded, then investigated with Python scripts against a locally running ClickHouse Docker container, followed by manual review and reproduction to distinguish genuine findings from hallucinations and misconfigurations. The work produced a few real vulnerabilities that were reported to ClickHouse’s security team and bug bounty program, while the broader conclusion is that AI can accelerate exploration and validation but still requires persistent questioning and human verification.
From test bench to lakehouse: how AVL modernizes measurement data analytics with Impulse
AVL’s Lakehouse for Measurement Data addresses the scale, reproducibility, and governance limitations of desktop tools and isolated scripts used for automotive measurement analysis. Built on Databricks, the platform ingests ASAM MDF4 and other files into a Medallion Architecture, applies configurable DQX quality rules in a hierarchical Silver model, and uses Impulse to compile declarative Python TSAL expressions into distributed Spark execution. Engineers can select channels, create virtual signals with alias resolution, unit conversion, time alignment, and interpolation, define events, and compute duration- or distance-weighted aggregations in about 10 lines of Python. Impulse supports Gold-layer reporting, ad-hoc Spark DataFrames, and ML feature matrices, with Unity Catalog governance and Workflow orchestration. AVL reports reducing analysis time from days to minutes, processing many recordings per run, lowering infrastructure costs versus on-premises solutions, and enabling self-service, reproducible, standardized analysis.
Shipping huggingface_hub every week with AI, open tools, and a human in the loop
huggingface_hub, the Python client underlying libraries such as transformers, datasets, diffusers, and sentence-transformers, moved from releases every four to six weeks to a weekly cadence. A single manually triggered GitHub Actions workflow now computes versions, manages branches, tags and publishes both huggingface_hub and the hf CLI, creates downstream RC test branches, and handles post-release tasks. An open-weights model drafts structured release notes and Slack announcements using PR metadata and documentation diffs, while OpenCode, HF Inference Providers, PyPI Trusted Publishing, and pinned, checksum-verified tooling provide the stack. Deterministic manifest validation checks missing or extra PR references, then a human reviews and publishes the drafts. The reported result is weekly shipping at about $0.25 per full release, with more consistent notes, earlier downstream breakage detection, and shorter contributor feedback loops.
PP-OCRv6 on Hugging Face: 50-Language OCR from 1.5M to 34.5M Parameters
PP-OCRv6 is PaddleOCR’s latest universal OCR model family, targeting text detection and recognition across documents, screenshots, displays, labels, and scene text. It offers tiny, small, and medium tiers spanning 1.5M to 34.5M parameters, while the small and medium tiers support 50 languages. Its PPLCNetV4 backbone is shared across detection and recognition, with RepLKFPN for lightweight multi-scale detection and EncoderWithLightSVTR for recognition using local context and global attention. On PaddleOCR’s official in-house multi-scenario benchmarks, PP-OCRv6_medium reaches 86.2% detection Hmean and 83.2% recognition accuracy, improving over PP-OCRv5_server by 4.6 percentage points and 5.1 percentage points. PaddleOCR provides Paddle Inference, Transformers, and ONNX Runtime paths, and results can be saved as visualization images or structured JSON for downstream workflows.
Build Cross-Language Multi-Agent Team with Google’s Agent Development Kit and A2A
The post presents a contract-compliance pipeline in which a Python agent uses Gemini to extract contract terms and a Go service applies deterministic corporate-policy checks. It connects the services with the Agent2Agent (A2A) protocol and uses Google's Agent Development Kit (ADK), including RemoteA2aAgent, to expose the remote validator as a local sub-agent in a SequentialAgent. A2A supplies discovery through Agent Cards at /.well-known/agent.json, JSON-RPC 2.0 messaging through message/send, typed Message Parts, and task states such as submitted, working, completed, and failed. Pipeline state is shared through ADK's ToolContext.state, with failures transitioning to MANUAL_REVIEW for human legal review when the Go service is unavailable. The accompanying open-source application demonstrates live handoffs, policy verdicts, and simulated network faults, while the conclusion emphasizes language independence, decoupling, resilience, and deterministic auditing.
From the Hugging Face Hub to robot hardware with Strands Agents and LeRobot
Strands Robots integrates Hugging Face’s LeRobot stack with Strands Agents to connect dataset recording, policy execution, simulation, hardware deployment, and fleet coordination in one agent loop. Its integration keeps LeRobot responsible for hardware drivers, calibration, recording, and dataset handling, while Strands supplies AgentTools, a shared Robot abstraction, policy providers, and a Zenoh peer mesh; simulation and hardware use the same LeRobotDataset format. The walkthrough records demonstrations in MuJoCo, optionally pushes them to the Hub, runs policies through a common interface, and switches from simulation to a physical SO-101 with mode="real". The companion script and notebook run the default simulation workflow on a laptop without hardware, a GPU, or Hugging Face credentials, although the default Mock policy produces structurally valid but behaviorally unhelpful random demonstrations.
Build Your Own Transaction Foundation Model for Financial Intelligence
Production financial intelligence systems frequently rely on hand-engineered tabular features and rule sets that are brittle, expensive to maintain, and blind to historical sequential patterns. NVIDIA demonstrates an accelerated reference pipeline that replaces standard BPE tokenization with a GPU-based domain tokenizer, converting raw transactions into semantic tokens with an 8,192-token context window. Using the NeMo AutoModel library, a compact 29-million parameter decoder-only transformer is pretrained from scratch on unlabeled transaction sequences using causal language modeling. Learned sequence embeddings are extracted, compressed using PCA, and concatenated with raw tabular features to train a downstream GPU-accelerated XGBoost fraud detection model. On the IBM TabFormer benchmark dataset, this combined approach achieves a 41.76% lift in Average Precision over the baseline model relying solely on raw tabular features.
Profiling in PyTorch (Part 2): From nn.Linear to a Fused MLP
Part 2 of the series examines how PyTorch profiler traces change from a single bias-enabled nn.Linear to a three-layer MLP with an intervening activation, using scripts run on an NVIDIA A100-SXM4-80GB GPU. It explains that nn.Linear represents y = x @ w.T + b, dispatches aten::addmm, and folds bias addition into the cuBLAS GEMM epilogue, while aten::t only rewrites weight strides and launches no GPU kernel. For the MLP, eager execution uses three GEMMs plus separate activation and multiplication kernels, whereas torch.compile fuses the latter operations and keeps the intermediate out of HBM, while leaving the GEMMs unchanged. A hand-written Liger Triton kernel provides the same fusion without Dynamo guards or compile latency, trading Inductor’s shape-specialized speed for generic execution across changing shapes.
Metric Semantic Layer: How Lyft Governs and Scales Key Data Definitions
As Lyft scaled, different teams developed conflicting definitions for key business metrics due to the lack of centralized version control and shared standards. To resolve this, Lyft implemented an internal Metric Semantic Layer as a Python package that stores authoritative metric definitions in YAML files with Jinja SQL templates. The system restricts onboarding to Golden Metrics used across multiple applications, requiring team-based approval from both Business and Operational Owners for any definition changes. Standardized definitions are exposed through Python APIs, integrated into the Amundsen data catalog and self-service user interfaces, and surfaced to AI tools via a Model Context Protocol.
Works on my machine: how we use AI to reproduce reported bugs
Sentry maintains 159 open-source SDK packages across multiple languages, leading to high triage volume and significant boilerplate setup overhead when reproducing reported bugs. To streamline this workflow, the team developed a Claude skill that parses GitHub issue URLs to extract language, framework, and SDK versions. The skill generates a reproduction branch, sets up minimal boilerplate using ecosystem package managers like uv or npm, attempts execution, and opens a pull request with explicit run instructions. By allowing the model to exit early when encountering overly complex environments, the tool reliably prepares reproducible playgrounds without stumbling on edge cases. This workflow saved several hours in testing scenarios like Django HTTPS proxy bugs while avoiding excessive bot noise through scoped human interaction.
Introducing the Google Colab CLI
Google announces the Google Colab Command-Line Interface (CLI), a tool that connects a local terminal with remote Colab runtimes for developers and AI agents. It can provision GPUs or TPUs with commands such as colab --gpu A100, execute local Python scripts and ML pipelines remotely with colab exec, recover artifacts and replayable .ipynb logs, and provide interactive access through colab repl or colab console. The source demonstrates an agent-driven QLoRA fine-tuning workflow for google/gemma-3-1b-it on a Text-to-SQL dataset: an Antigravity agent provisions a T4, installs transformers, datasets, peft, trl, bitsandbytes, and accelerate, runs finetune_run.py, saves a notebook log, downloads model artifacts, and stops the runtime. The resulting safetensors adapter and configuration files can be loaded locally, while the CLI is presented as programmable and usable by terminal-based agents.
Custom stores, custom tools, and auto-review for the Cursor SDK
The Cursor SDK’s TypeScript and Python releases add custom tools, auto-review, and nested subagents. Custom tools let developers pass function definitions through local.customTools, exposed via a built-in MCP server, making them available to all subagents. Auto-review routes local agent tool calls through a classifier; instructions in permissions.json control which calls run automatically and which are held for review. The SDK now supports a JSONL store for agent metadata as an alternative to SQLite, and developers can implement the LocalAgentStore interface for custom backends. Subagents can spawn their own subagents without additional configuration, and the release includes reliability and platform fixes.
Data Science vs Data Analytics: Compare Careers, Skills, and Degrees
Data analytics and data science are compared as data-focused career paths, with analytics interpreting existing, usually structured data for trends and business decisions, while science builds models and automated systems to predict future outcomes. The guide contrasts typical questions, outputs, tools, education paths, and roles: analysts use SQL, Excel, Tableau, or Power BI for dashboards and reports, whereas data scientists use Python, R, Apache Spark, and MLflow for predictive models and algorithms. It describes analytics types from descriptive through prescriptive, and a data science workflow spanning collection, feature engineering, training, validation, and deployment, including unstructured data such as text, images, and sensor streams. It also explains collaboration, including analysts defining problems and baselines before scientists build models, and offers portfolio suggestions and questions for choosing a path.
Executable UDFs are now in public beta on ClickHouse Cloud
Executable UDFs are available in public beta on ClickHouse Cloud, allowing Python functions packaged as zip files to be called from SQL like built-in functions. ClickHouse runs them in a pool of long-lived sandboxed processes, keeping model code near the data and supporting use in ad-hoc queries, joins, and materialized views. The demonstration wraps a roughly 270K-parameter PyTorch autoencoder in an embed_trade UDF that consumes 14 features, produces a 32-dimensional embedding and reconstruction error, and scores trades as they enter through a materialized view. Training uses streamed Parquet chunks, incremental StandardScaler fitting, early stopping, and saved model artifacts; the deployed pool keeps the 2MB model hot, so inference occurs once per trade and downstream queries read the resulting table.
Profiling in PyTorch (Part 1): A Beginner's Guide to torch.profiler
The post introduces torch.profiler through a beginner-oriented walkthrough of profiling a matrix multiplication and addition, explaining how to configure the profiler and interpret its outputs. It uses a small function with torch.matmul and torch.add, annotates it with record_function, captures CPU and CUDA activities after warm-up iterations, and exports a profiler table plus a Chrome trace. The table summarizes event counts and CPU or GPU time, while the trace shows execution order, call hierarchy, CPU-GPU gaps, kernel launches, and overlap. Comparing 64x64 and 4096x4096 matrices demonstrates a shift from overhead-bound behavior to compute-bound behavior; the guide also covers dispatch chains, hidden device-to-device copies, profiler buffer activity, and torch.compile's runtime and caching behavior.
You don’t need to pick one: how Sentry and OpenTelemetry work together
Integrating Sentry into systems already instrumented with OpenTelemetry does not require replacing existing backend SDKs or rewriting service instrumentation. A hybrid architecture pairs the Sentry SDK on the frontend for browser tracing, Session Replay, and logs with existing OpenTelemetry configurations across backend services. Frontend requests propagate W3C traceparent headers to backend endpoints, which then export traces and logs over OTLP directly to Sentry endpoints or through an intermediate OpenTelemetry Collector. Ingested telemetry attaches backend spans and standard Python logs to the initial frontend user actions, establishing a unified distributed trace across the entire request path. Although Sentry's OTLP ingest currently supports logs and traces rather than metrics, dedicated backend Sentry SDKs remain an optional addition later for backend exception tracking and profiling.
Introducing Arrow UDFs in PySpark: A Faster, Leaner Replacement for Pandas UDFs
Apache Spark’s Pandas UDFs improve Python UDF throughput through Arrow-based serialization and batching, but Pandas/Arrow conversion can add copies, limit zero-copy behavior, and weaken support for complex datatypes. Native Arrow UDFs, introduced in Databricks Runtime 18.0, operate directly on Arrow data without converting inputs to Pandas or NumPy objects, preserving columnar layout and enabling vectorized processing through Arrow’s compute and memory model. The new @arrow_udf decorator supports scalar, aggregate, and table functions, while type-hinted @udf definitions, mapInArrow, and applyInArrow extend the interface to DataFrame and grouped operations; iterator-based modes can amortize initialization costs. In a comparison, an Arrow UDF ran about 10% faster than a Pandas UDF and used about 40% less memory, with better complex-datatype support.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.