---
title: "Ollama"
description: "6 posts about Ollama, summarised, each linking to the original."
---

# Ollama
> 6 posts about Ollama, summarised, each linking to the original.

## Articles

### [Use a local and open source code assistant](https://yomu.fyi/post/use-a-local-and-open-source-code-assistant.md)
- Company: [Red Hat](https://yomu.fyi/company/red-hat.md)
- Author: Seth Kenlon
- Published: Sep 9, 2026
- Type: Tutorial

Closed or cloud-based coding assistants can expose work to external services, so the article presents a local, open-source alternative built around OpenCode. OpenCode is an AI coding agent—not an LLM—that runs as a terminal interface, desktop application, or IDE extension and connects to models through the Model Context Protocol (MCP). The setup installs OpenCode, selects a local model runtime such as Ollama for AMD/NVIDIA GPUs or OpenVINO Model Server with INT4 Qwen3-8B for Intel integrated graphics, then configures opencode.jsonc and an editor plug-in. Examples cover Qwen3:14b with Ollama, Podman-based OpenVINO serving, and NetBeans integration; the resulting assistant can answer queries and edit files after permission is granted, although local inference may be slow and generated advice can be unreliable.


### [NVIDIA PAIR Virtual Inference Router Expands Available Compute on Your Local Network](https://yomu.fyi/post/nvidia-pair-virtual-inference-router-expands-available-compute-on-your.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Tanya Lenz
- Published: Sep 3, 2026
- Type: Explainer

NVIDIA Personal AI Router (PAIR) addresses the GPU bottleneck created when multi-agent workflows and concurrent local sessions generate many independent inference requests. It acts as a virtual inference router rather than an inference engine, proxying compatible Ollama and LM Studio interfaces, discovering and securely pairing local systems, and scheduling each request onto one eligible node based on readiness, engine state, model availability, workload, and GPU utilization. The agent keeps one familiar connection, while PAIR streams responses back and exposes placement through Jobs and metrics views; it does not merge GPUs, pool VRAM, or split a single request across machines. In an unofficial Hermes five-subagent demonstration, a three-device cluster averaged 8 minutes 48 seconds versus 18 minutes on one RTX Spark laptop, with results explicitly dependent on configuration and workload.


### [Picture This: Open Source AI for Image Description](https://yomu.fyi/post/picture-this-open-source-ai-for-image-description.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: May 9, 2024
- Type: Tutorial

Nolan Darilek presents a self-hosted image-description service intended to help blind screen-reader users understand photos, diagrams, menus, and other visual material. Humans often provide inadequate alt text, and the post contrasts that history with increasingly capable LLM-generated descriptions from apps and browser add-ons. The implementation combines Ollama, running the Apache-licensed LLaVA 34B multimodal model, with a PocketBase API over SQLite and a small Python client; collection hooks send images and follow-up questions, retain chat history, and enforce per-user access rules. In testing, a cold request on an A100-40GB Fly Machine took about 45 seconds, while stopping idle GPU Machines reduced resource use at the cost of startup delay. The example produced useful scene descriptions but could not reliably identify leafless tree species, illustrating both the prototype’s value and its limitations.


### [GPUs on Fly.io are available to everyone!](https://yomu.fyi/post/gpus-on-fly-io-are-available-to-everyone.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Feb 12, 2024
- Type: Announcement

Fly.io has made GPU instances available to everyone, adding datacenter-grade accelerators to its globally distributed compute platform. Users can select Ampere A100 variants with 40GB or 80GB of memory, or a Lovelace L40s with 48GB, by setting an app or process’s vm.size to the corresponding GPU kind. The post demonstrates an Ollama deployment using an L40s, a 100GB persistent volume mounted at /root/.ollama, and HTTP service settings that allow machines to stop and start automatically with zero minimum running machines. These instances are positioned for large language model inference, text transcription, and image generation; lower-cost A10 GPUs are planned for the next few weeks, while some organizations may need additional verification based on account age and payment history.


### [Fly.io has GPUs now](https://yomu.fyi/post/fly-io-has-gpus-now.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Dec 13, 2023
- Type: Announcement

Fly.io announces GPU support for its cloud, extending its near-user deployment model to AI and machine-learning workloads at the edge. The offering attaches Nvidia A100 GPUs with 40GB or 80GB of RAM, CUDA support, eight AMD EPYC CPU cores, and volumes up to 500GB; L40s are also listed as coming soon and then noted as available. A sample fly.toml deploys Ollama with a 100GB model volume, while auto-start and auto-stop settings can reduce min\_machines\_running to zero when idle. GPU apps can scale across supported regions, retaining the same public IP address and TLS certificates, with an Amsterdam example using fly scale count 2 --region ams. Listed prices are $2.50 per hour for A100 40GB and $3.50 for A100 80GB, with reserved-instance and dedicated-host discounts available.


### [Scaling Large Language Models to zero with Ollama](https://yomu.fyi/post/scaling-large-language-models-to-zero-with-ollama.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Dec 6, 2023
- Type: Tutorial

Running GPU nodes for language-model inference can be costly because they often sit idle between requests, so this guide configures Ollama on a Fly.io Machine that scales to zero when idle. Ollama is described as a wrapper around llama.cpp, enabling self-hosted models with GPU acceleration, while about 70 lines of Go add authentication to protect the endpoint from unauthorized use. Deployment uses an A100-40GB GPU, a private Flycast address, automatic start and stop settings, and a 100GB persistent volume mounted at /root/.ollama so downloaded models survive stopped Machines. A stopped Machine no longer incurs GPU charges, but scaling to zero suspends it on the same host and may fail to wake during capacity issues. The setup supports CLI and API requests, WireGuard access, custom Modelfiles, and JSON-mode function-calling workflows.