Run an LLM on your VPS with Ollama

You do not need a GPU farm to run a useful language model. Small models answer fast on CPU and keep your data on your own server — this guide sets expectations with real numbers.

1. What fits in your RAM

Be honest with yourself: a 7B model on CPU writes slower than you read. For chat bots and batch processing that is fine; for interactive coding assistants it is not. Keep the model on an 8 GB plan and it works without drama.

2. Install Ollama

curl -fsSL https://ollama.com/install.sh | sh
systemctl enable --now ollama
ollama pull llama3.2:3b
ollama run llama3.2:3b "Say hello in one short sentence."

Models are stored in /usr/share/ollama/.ollama/models — a few gigabytes each, so check free disk with df -h before pulling several.

3. OpenAI-compatible API

Ollama speaks the OpenAI API format on port 11434. Any client that supports a custom base URL works:

curl http://127.0.0.1:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"llama3.2:3b","messages":[{"role":"user","content":"Hello"}]}'

4. Keep it private

Do not expose port 11434 to the world. Either keep the API on localhost, or put it behind Caddy with a token:

apt install -y caddy
cat > /etc/caddy/Caddyfile <<'EOF'
llm.example.com {
    @auth header Authorization "Bearer long-random-token"
    handle @auth {
        reverse_proxy 127.0.0.1:11434
    }
    respond 401
}
EOF
systemctl reload caddy

Then point your client at https://llm.example.com/v1 with that token.

5. Use it in your apps

Summarize support tickets into Telegram. Classify incoming messages. Generate drafts for a bot. Run embeddings for a small search index. All of it stays on your server: no API keys, no per-token bills, no data leaving the machine.

ollama ps     # what is loaded right now
ollama list   # downloaded models
journalctl -u ollama -f

Quick answers

Is CPU-only inference usable?

For small models, yes. A 3B model answers simple prompts in a few seconds on a Ryzen 9 core set, and batch jobs do not care about speed at all. Large 30B+ models need a GPU — rent one when you need it.

Does Ollama need a dedicated GPU VPS?

No. Ollama runs on CPU with quantized models by default. GPUs speed up big models, but everything in the table above runs comfortably on a normal Buran plan.

Which plan should I pick?

8 GB RAM (Buran-3, €26/mo) for 7B models, 4 GB (Buran-2, €13/mo) for 3B models, 2 GB for the tiny 0.5B ones used in tests and routing.

Run models up to 7B on Buran-3 with 8 GB DDR5.