Run a Local LLM on a VPS with Ollama - Sizing & Setup

Install Ollama on a VPS, pull Llama 3.2 or Qwen 2.5, expose an OpenAI-compatible API with systemd. Realistic RAM numbers and what a €12 VPS can actually run.

You do not need a GPU farm to run a useful language model. Small models answer fast on CPU and keep your data on your own server - this guide sets expectations with real numbers.

1. What fits in your RAM

ModelRAM needed (q4)On diskSpeed on CPU (measured)Good for
qwen3:0.6b~1 GB~1 GBvery fastRouting, classification, tests
llama3.2:3b~4 GB~2 GB~10–20 tokens/sChat, summaries, drafts
qwen2.5:7b~8 GB~5 GB~4–8 tokens/sBetter quality, still usable
deepseek-r1:14b~16 GB~9 GBslowerReasoning, step-by-step answers
qwen3:32b~24–32 GB~20 GBmuch slower on CPULocal coding drafts, RAG
llama3.3:70b~64 GB~43 GBGPU class - not for a CPU VPSHeavy workloads on a GPU box

RAM figures are the official Ollama minimums for 4-bit quantized weights; the disk column is what the model file actually occupies. Speeds come from our own CPU measurements - the hardware is documented on the benchmarks page. Be honest with yourself: a 7B model on CPU writes slower than you read. For chat bots and batch processing that is fine; for interactive coding assistants it is not - keep the model on an 8 GB plan and it works without drama.

2. Install Ollama

curl -fsSL https://ollama.com/install.sh | sh
systemctl enable --now ollama
ollama pull llama3.2:3b
ollama run llama3.2:3b "Say hello in one short sentence."

Models are stored in `/usr/share/ollama/.ollama/models` - a few gigabytes each, so check free disk with `df -h` before pulling several.

3. OpenAI-compatible API

Ollama speaks the OpenAI API format on port 11434. Any client that supports a custom base URL works:

curl http://127.0.0.1:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"llama3.2:3b","messages":[{"role":"user","content":"Hello"}]}'

4. Keep it private

Ollama has no built-in authentication: anyone who can reach port 11434 can pull and delete your models and drive the CPU. Do not expose it to the world. Either keep the API on localhost, or put it behind Caddy with a token:

apt install -y caddy
cat > /etc/caddy/Caddyfile <<'EOF'
llm.example.com {
    @auth header Authorization "Bearer long-random-token"
    handle @auth {
        reverse_proxy 127.0.0.1:11434
    }
    respond 401
}
EOF
systemctl reload caddy

Then point your client at `https://llm.example.com/v1` with that token.

5. Use it in your apps

Summarize support tickets into Telegram. Classify incoming messages. Generate drafts for a bot. Run embeddings for a small search index. All of it stays on your server: no API keys, no per-token bills, no data leaving the machine.

ollama ps     # what is loaded right now
ollama list   # downloaded models
journalctl -u ollama -f

6. Updates, context and popular models

The most pulled models of 2026 are DeepSeek-R1, Qwen 3 and gpt-oss - their small distilled variants run on the same plans as the table above. The Ollama library shows today's list. `ollama pull <tag>` downloads a model, running the same pull again updates it to the latest weights, and `ollama rm <tag>` gives the disk back.

Check your build with `ollama --version`; stable releases are published on the Ollama releases page. To upgrade Ollama itself, rerun the install script from section 2 - systemd restarts the service with the new binary.

Context length is the second RAM consumer after the model file: Ollama defaults to 4k tokens and memory scales with context × parallel requests. Agents and coding assistants want 32k+, so raise it only when you need it:

ollama --version            # installed build
ollama pull qwen3:8b        # download or update a model
ollama rm qwen2.5:7b        # free disk space
systemctl edit ollama       # add: Environment="OLLAMA_CONTEXT_LENGTH=16384"

Quick answers

Is CPU-only inference usable?

For small models, yes. A 3B model answers simple prompts in a few seconds on a Ryzen 9 core set, and batch jobs do not care about speed at all. Large 30B+ models need a GPU - rent one when you need it.

Does Ollama need a dedicated GPU VPS?

No. Ollama runs on CPU with quantized models by default. GPUs speed up big models, but everything in the table above runs comfortably on a normal Buran plan.

Which plan should I pick?

A 7B q4 model needs about 8 GB of RAM - Core VPS 4 at €12/mo runs it comfortably. 13–14B models want 16 GB (Core VPS 6, €16/mo), 32B wants 24–32 GB (Core VPS 8, €30/mo). A 70B q4 model needs ~64 GB, which is VDS territory - rent a GPU box when you actually need that class.

How much RAM does the context use?

The model file is only the floor. Ollama keeps the KV cache for the active context in RAM as well - the default 4k is cheap, 32k on a 7B model adds several GB, and parallel requests multiply it. If you expose the API to several users, pick a plan one tier above the model size.

Start small: run a 7B model on Core VPS 4 with 8 GB for €12/mo.

Related

View plansGuidesFAQ