Loading post…
Fly.ioExplainer
How Yoko Li makes towns, tamagoes, and tools for local AI
Summary
Yoko Li’s local AI work spans playful agent experiments and practical tools for running language-model workflows on personal hardware. AI Town uses conversations among AI agents to produce changing relationships and emergent behavior, while AI Tamago feeds game state and user inputs to an unmodified LLaMA 2 7B model and uses Ollama’s JSON mode to generate pet actions. The Local AI Starter Kit packages document ingestion, vector storage, and language-model responses into a private, local “chat with a knowledge base” pipeline, avoiding setup across at least four API providers. The post also describes unreliable JSON generation, hallucinations, and grammar-guided workarounds, concluding that randomness is a liability for factual outputs but useful for game behavior and creative experimentation.
Context
The work addresses both the difficulty of setting up document-search and retrieval-augmented generation pipelines and the unreliable, nondeterministic outputs of large language models, including hallucinations and inconsistently formatted JSON.
Approach / What changed
Li’s projects run models locally and use them in different ways: AI Town and AI Tamago turn emergent, stochastic outputs into game behavior, while the Local AI Starter Kit connects document ingestion, embeddings, vector search, and language-model responses. JSON output is improved with Ollama’s JSON mode, code-generation models, or grammar files in llama.cpp.
Takeaways
AI Tamago uses an unmodified LLaMA 2 7B model with game state and user inputs, then uses Ollama’s JSON mode to generate the virtual pet’s next behavior.
The Local AI Starter Kit supports a private document-questioning workflow on local hardware using user-selected models, without requiring billing and API keys for at least four providers.
llama.cpp grammar files can constrain model output with a context-free grammar, but JSON generation can still fail, including by producing an endless stream of newlines.
Source and freshness
- Source
- fly.io
- Published
- Added to Yomu
- Last updated
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.