---
title: "Transcribing on Fly GPU Machines"
description: "Fly GPU Machines are used to run Whisper Webservice for audio transcription, with the post noting that Fly GPUs are deprecated and unavailable after August 1. The setup uses a GPU-friendly Docker image, an a100-pcie-40gb Machine, and a Fly Volume mounted at /root/.cache/whisper so model files download and persist outside the image; ASR_MODEL=large and ASR_ENGINE=faster_whisper configure the service. It shows flyctl commands for creating the app, GPU-hosted volume, Machine, public IPs, and HTTPS routes, then verifies GPU availability with nvidia-smi and exposes the web UI and OpenAPI specification. A second path automates volume and Machine creation through the Machines API, requiring cpu_kind plus valid cpus and memory_mb values. It also demonstrates a custom Ubuntu 22.04 image with NVIDIA CUDA libraries, FFmpeg, Python, and Whisper, concluding that GPU Machines can support composable AI workloads."
---

# Transcribing on Fly GPU Machines

[Fly.io](https://yomu.fyi/company/fly-io) · Nov 13, 2023

**Type:** Tutorial

## Summary

Fly GPU Machines are used to run Whisper Webservice for audio transcription, with the post noting that Fly GPUs are deprecated and unavailable after August 1. The setup uses a GPU-friendly Docker image, an a100-pcie-40gb Machine, and a Fly Volume mounted at /root/.cache/whisper so model files download and persist outside the image; ASR\_MODEL=large and ASR\_ENGINE=faster\_whisper configure the service. It shows flyctl commands for creating the app, GPU-hosted volume, Machine, public IPs, and HTTPS routes, then verifies GPU availability with nvidia-smi and exposes the web UI and OpenAPI specification. A second path automates volume and Machine creation through the Machines API, requiring cpu\_kind plus valid cpus and memory\_mb values. It also demonstrates a custom Ubuntu 22.04 image with NVIDIA CUDA libraries, FFmpeg, Python, and Whisper, concluding that GPU Machines can support composable AI workloads.

## Context

Running Whisper outside OpenAI’s platform requires Python, several gigabytes of storage, and preferably a GPU. The post presents Whisper Webservice as a packaged option for self-hosted audio transcription and uses it to demonstrate Fly GPU Machines.

## Approach / What changed

Deploy the GPU-enabled Whisper Webservice Docker image on an a100-pcie-40gb Fly Machine, mount a Fly Volume at /root/.cache/whisper for model caching, configure the service for the large faster-whisper model, and expose it over HTTP and HTTPS. The post also covers Machines API automation and building a custom Ubuntu-based GPU image.

## Takeaways

- Fly GPU Machines offer a100-pcie-40gb and a100-sxm4-80gb options; the stated distinction is 40 GB versus 80 GB of GPU memory.
- Creating the volume with --vm-gpu-kind places it on a GPU host, and using the same region and GPU kind for the Machine can place both on the same host.
- Machines API requests need matching cpu\_kind and gpu\_kind settings; Machine creation also requires valid cpus and memory\_mb values for performance Machines.

**Tags:** [Docker](https://yomu.fyi/topic/docker), [Fly](https://yomu.fyi/topic/fly), [GPU](https://yomu.fyi/topic/gpu), [Python](https://yomu.fyi/topic/python)

- Source: [Fly.io](https://fly.io/blog/transcribing-on-fly-gpu-machines)
- Source URL: https://fly.io/blog/transcribing-on-fly-gpu-machines
- Ingested by Yomu: 2026-09-02T12:00:29.130Z

[Read original post](https://fly.io/blog/transcribing-on-fly-gpu-machines)