bulaev.ai

Ollama

Open source

Agents · Personal & corporate AI OS

Run open models locally with one command; the private, no-per-token layer of a personal AI OS.

Open ollama.com

What it is

Ollama is an open-source runtime for open-weight language models on your own machine. Install it, run ollama run <model>, and a model is downloaded and served through a local HTTP API that is compatible with the OpenAI client libraries, so most tools that talk to a hosted model can be pointed at localhost instead. The site states that local models are free forever, that the software is open source, and that the runtime plugs into Claude Code, Codex, OpenCode, VS Code and n8n. A newer cloud option hosts larger open models in the US, Europe and Singapore with a Pro tier ($20 a month with usage credit, per the site) and the promise that no provider trains on your data.

The reason to run models locally is not quality; hosted frontier models are better. It is privacy for sensitive documents, zero marginal cost for high-volume routine tasks (classification, extraction, summarizing transcripts), and working offline.

Where it fits in the AI OS

Personal & corporate AI OS, the private compute layer. In a company this is where "can we send this to a model" gets a second answer: the contract, the payroll export and the customer list go to the local model, everything else to the hosted one.

Insider notes

members

How Serge's stack uses it

Local model use is not documented in the project's public notes; the stack's agents run on hosted Claude and OpenAI models.

[SERGE: fill in]

Setup in 5 minutes

  1. Download the installer for macOS, Windows or Linux (or curl -fsSL https://ollama.com/install.sh | sh on Linux).
  2. ollama run llama3.2 (or the current small model on the site's library page); chat in the terminal.
  3. curl http://localhost:11434/v1/chat/completions -d '{"model": "llama3.2","messages":[{"role":"user","content":"hi"}]}'.
  4. Point one existing tool at it: set OPENAI_BASE_URL= http://localhost:11434/v1 and a dummy key.
  5. Pick one routine job (summarize meeting transcripts, tag notes) and move it to the local model for a week; compare quality and cost.

Alternatives

  • LM Studio — desktop GUI for local models with a similar server.
  • llama.cpp / vLLM — lower-level engines for servers and tuning.
  • Hosted open models (Groq, Together, Fireworks) — the speed of open models without local hardware.
  • Apple Foundation Models / on-device APIs — for apps, not for a workstation.