The risks of building apps on ChatGPT

Summary

The post argues that applications with critical AI features should avoid relying on proprietary, externally hosted models such as OpenAI’s ChatGPT and consider self-hosted open source alternatives. It identifies single-provider, regulation or policy, financial, and governance risks, illustrating the first with a Personal AI Fitness Trainer that failed during an OpenAI outage on November 8, 2023. When the AI is the app’s primary value proposition, an outage, changed behavior, denied usage, price increase, or provider leadership crisis can make the service unusable or threaten the business; less-central integrations may instead cause only degraded experience. Self-hosting a model can insulate an application from provider interruptions, licensing or usage-fee changes, regulator- or company-driven behavior changes, and leadership turmoil, while introducing GPU capital and operating costs with rented cloud GPU access available as an alternative to owning GPUs.

Takeaways

1. 1

   A Personal AI Fitness Trainer failed during an OpenAI outage on November 8, 2023, demonstrating how a provider interruption can make an AI-centered application unavailable.

2. 2

   Self-hosting can protect against unexpected changes to a provider’s licensing, inference costs, model behavior, usage policies, or leadership.

3. 3

   Dedicated GPUs cost more than renting limited access to a provider, while self-hosting adds capital and operating expenses that can also be addressed through cloud GPU access.