Running AI locally — privacy, control, and cost

May 23, 2026

Cloud AI is excellent, and I use it. But a growing share of the lab is aimed at running models on my own hardware. Here’s the reasoning.

Privacy

Every prompt you send to a cloud model is data leaving your control. For a lot of tasks that’s fine. For others — personal notes, private documents, anything you’d rather not hand to a third party — running the model locally means the prompt never leaves the building.

Control

Local models don’t change underneath you. The weights you downloaded today behave the same next month. No deprecations, no silent updates, no rate limits imposed from outside. You pick the model, the settings, and the trade-offs.

Cost

Cloud inference is priced per token, and agent workloads — long loops that make many calls — add up fast. Local inference trades that variable cost for a fixed one: buy the GPU once, pay for electricity, and the marginal cost of another million tokens is close to zero.

The honest trade

Local isn’t strictly better. The frontier cloud models are still ahead of what fits on a single homelab GPU, and good hardware costs real money. So the lab’s approach is a split: cloud for the hardest reasoning, local for the high-volume, privacy-sensitive, always-on work. Right hardware for the right job.

Starter draft — add the GPU plan, the model routing setup, and benchmarks once the AI host is live.

← all posts