Loading…
Global technology leader developing innovative software, cloud services, and AI solutions across mobile, web, and enterprise platforms
- 64 posts summarised
- latest
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
From their blog
Latest articles
Boost Training Goodput: How Continuous Checkpointing Optimizes Reliability in Orbax and MaxText
Orbax and MaxText introduce continuous checkpointing to balance training reliability and performance when hardware failures or preemption events interrupt long-running jobs. Fixed checkpoint intervals create a trade-off: infrequent saves risk substantial lost work, while frequent saves can block or bottleneck training, particularly over unstable networks. MaxText enables asynchronous checkpointing that attempts a new save only after the previous background request is fulfilled; its continuous mode overrides checkpoint_period, and max_num_checkpoints_to_keep can limit storage use. A llama-3.1-70B benchmark on two v5p-128 slices found smaller P50 checkpoint intervals, but average training step time increased because of more frequent device-to-host transfers; the analysis estimated resource conservation against mean-time-between-failure assumptions. Orbax also supports minimum intervals and customizable preservation policies, while co-locating storage with training is recommended because cross-metro bandwidth can reduce checkpointing speed and reliability.
MediaTek NPU and LiteRT: Powering the next generation of on-device AI
MediaTek’s diverse NPU hardware and specialized compilation requirements have made on-device AI deployment difficult, especially for sophisticated generative models. The LiteRT NeuroPilot Accelerator, described as a ground-up successor to the TFLite NeuroPilot delegate, provides a unified API over MediaTek NPUs and supports both offline Ahead-of-Time (AOT) and on-device compilation. Its workflow can compile models for target SoCs, package Android assets and runtimes for Google Play’s Play for On-device AI, and invoke NPU inference through LiteRT with GPU or CPU fallback options. A new C++ API and Native Hardware Buffer Interoperability enable zero-copy transfers from AHardwareBuffer and OpenGL/OpenCL buffers, supporting real-time camera and video pipelines. On supported MediaTek chipsets, listed Gemma and Qwen models receive NPU-specific optimizations; the post reports up to 12x CPU and 10x GPU acceleration, including over 1600 tokens/sec prefill and 28 tokens/sec decode for Gemma 3n E2B.
Unlocking Peak Performance on Qualcomm NPU with LiteRT
Google introduces the LiteRT Qualcomm AI Engine Direct (QNN) Accelerator, developed with Qualcomm to replace the previous TFLite QNN delegate and simplify NPU deployment across supported Qualcomm SoCs. It integrates vendor compilers and runtimes behind a unified workflow, supports ahead-of-time and on-device compilation, and can fall back to GPU or CPU when needed. Across 72 canonical vision, audio, and NLP models, the accelerator supports 90 LiteRT ops, enables full NPU delegation for 64, and reports up to 100x CPU and 10x GPU speedups. On Snapdragon 8 Elite Gen 5, FastVLM-0.5B uses int8 weights, int16 activations, and specialized transformer kernels to reach 0.12 second time-to-first-token, over 11,000 prefill tokens per second, and over 100 decode tokens per second.
Unleashing new AI capabilities for popular frameworks in Firebase Studio
Firebase Studio announced updates to its cloud-based AI workspace that combine Gemini with Agent modes, Model Context Protocol support, and Gemini CLI across Flutter, Angular, React, Next.js, and Web projects. AI-optimized templates default to autonomous Agent mode and include airules.md files that give Gemini framework-specific guidance on coding standards, error handling, dependencies, and development practices. Developers can prompt Gemini to recommend Firebase backend services, import libraries or packages, update code, and explain additional setup steps. Workspace forking preserves an identical copy and the original’s entire agent chat history, supporting backups, experimentation, debugging, and collaboration, while enhanced prompts and a 100MB project upload limit broaden the workflow. The editor’s note says Firebase Studio will sunset on March 22, 2027, with lessons from its preview moving to Google AI Studio and Google Antigravity.
Topics they write about
All topics →Weekly digest
The best engineering reads, once a week.