Self-Hosted AI Assistant on a $5 VPS: Docker-Compose Deployment of Ollama + OpenWebUI with Offline LLM (Phi-3, Llama3, Mistral) and Sensitive Data Control

Running a powerful AI assistant without sending your data to the cloud is now possible on extremely cheap hardware. In this guide we’ll deploy Ollama (the local LLM runtime) and OpenWebUI (a polished chat frontend) on a $5/month VPS using Docker‑Compose. You’ll get persistent storage, model selection (Phi‑3, Llama 3, Mistral), and a set of security best practices to keep your data private.

Why this stack?

• Ollama handles model loading, quantization, and GPU/CPU inference with a simple CLI.
• OpenWebUI provides a ChatGPT‑like UI, supports multiple models, and can be locked down behind authentication.
• Docker‑Compose makes the whole thing reproducible and easy to back up.

Prerequisites#

• A VPS with at least 1 GB RAM (the $5 plans from providers like Hetzner, DigitalOcean, or Vultr usually offer 1 GB).
• Ubuntu 22.04 LTS (or any recent Debian‑based distro).
• Root or sudo access.
• Basic familiarity with the terminal.

Note: If you plan to run larger models (e.g., Llama 3 70B) you’ll need more RAM or swap, but for Phi‑3‑mini, Llama 3‑8B, and Mistral‑7B the 1 GB RAM + 2 GB swap combo works fine.

Step 1: Prepare the VPS#

First, update the system and install required packages.

# Update package list
sudo apt update && sudo apt upgrade -y

# Install Docker, Docker Compose plugin, and curl
sudo apt install -y docker.io curl

# Add your user to the docker group (log out/in after)
sudo usermod -aG docker $USER

# Verify Docker works
docker run --rm hello-world

Enable Docker to start on boot:

sudo systemctl enable --now docker

Install the Docker Compose v2 plugin (if not already present):

sudo apt install -y docker-compose-plugin
docker compose version

Step 2: Create a Directory for the Stack#

Choose a location for your persistent data, e.g., /opt/ai-assistant.

sudo mkdir -p /opt/ai-assistant
sudo chown $USER:$USER /opt/ai-assistant
cd /opt/ai-assistant

Step 3: Write the Docker‑Compose File#

Create docker-compose.yml with the following contents. This defines two services: ollama (the LLM backend) and openwebui (the frontend). We also mount a volume for Ollama models so they survive container recreations.

version: "3.8"

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    restart: unless-stopped
    volumes:
      - ollama_data:/root/.ollama
    ports:
      - "11434:11434"   # Ollama API
    # Optional: limit CPU/shares if you want to guarantee resources
    # deploy:
    #   resources:
    #     limits:
    #       cpus: "0.5"
    #       memory: 512M

  openwebui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: openwebui
    restart: unless-stopped
    depends_on:
      - ollama
    ports:
      - "8080:8080"
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
      # Uncomment to enable basic auth (see security section)
      # - WEBUI_AUTH=true
      # - [email protected]
      # - WEBUI_ADMIN_PASSWORD=changeme
    volumes:
      - openwebui_data:/app/backend/data

volumes:
  ollama_data:
  openwebui_data:

Explanation#

• Ollama stores models under /root/.ollama inside the container; we bind that to a named volume ollama_data.
• OpenWebUI needs a persistent volume for its SQLite database and uploaded files (openwebui_data).
• The OLLAMA_BASE_URL env var tells OpenWebUI where to reach the Ollama API.
• Ports 11434 (Ollama) and 8080 (OpenWebUI) are exposed to the host; we’ll later put a reverse proxy in front of OpenWebUI.

Step 4: Start the Stack#

docker compose up -d

Check that both containers are healthy:

docker compose ps

You should see something like:

NAME                IMAGE                         COMMAND                  SERVICE      CREATED          STATUS          PORTS
ollama              ollama/ollama:latest          "ollama serve"           ollama       5 seconds ago    Up 4 seconds    0.0.0.0:11434->11434/tcp
openwebui           ghcr.io/open-webui/open-webui:main   "/init"                openwebui    5 seconds ago    Up 3 seconds    0.0.0.0:8080->8080/tcp

Step 5: Pull Your First LLM Model#

Ollama exposes a simple API; you can pull models via its CLI inside the container or using curl. We'll use the Ollama CLI for convenience.

# Enter the ollama container
docker exec -it ollama /bin/bash

# Inside the container, pull a model. Choose one:
# Phi‑3‑mini (3.8B) – very lightweight, good for 1 GB RAM
ollama pull phi3

# Llama 3‑8B (8B) – stronger reasoning, needs ~4‑5 GB RAM with swap
# ollama pull llama3

# Mistral‑7B (7B) – balanced performance
# ollama pull mistral

exit

Tip: Start with phi3 to verify everything works, then experiment with larger models as you feel comfortable.

Step 6: Access OpenWebUI#

Open your browser and go to http://<your-vps-ip>:8080. You should see the OpenWebUI login screen.

If you left authentication disabled (default), you can start chatting immediately. To create an admin user, set the environment variables in the compose file (see the commented lines) and restart:

docker compose down
docker compose up -d

Then use the credentials you set to log in.

Step 7: Persistent Storage & Backups#

All data lives in the two Docker volumes. To back them up:

# Stop the stack (optional but safer)
docker compose stop

# Create a tarball of the volumes
sudo tar czvf ai-assistant-backup-$(date +%F).tar.gz \
  /var/lib/docker/volumes/ollama-assistant_ollama_data/_data \
  /var/lib/docker/volumes/ollama-assistant_openwebui_data/_data

# Restart
docker compose start

Store the tarball somewhere safe (e.g., encrypted cloud storage). To restore, replace the volume contents and restart the stack.

Step 8: Security Best Practices#

Running a public-facing service on a cheap VPS demands hardening. Below are essential steps.

8.1 Firewall (UFW)#

Allow only SSH, HTTP, and HTTPS (if you add a reverse proxy later).

sudo ufw allow OpenSSH
sudo ufw allow 8080/tcp   # OpenWebUI (temporarily)
sudo ufw enable
sudo ufw status

8.2 Reverse Proxy with Nginx + TLS#

Using a domain name (even a free subdomain from DuckDNS) lets you add HTTPS via Let’s Encrypt.

1. Point a domain A record to your VPS IP.
2. Install Nginx and Certbot:

sudo apt install -y nginx certbot python3-certbot-nginx

3. Create an Nginx site config /etc/nginx/sites-available/ai-assistant:

server {
    listen 80;
    server_name yourdomain.com;

    location / {
        proxy_pass http://localhost:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

4. Enable the site and test:

sudo ln -s /etc/nginx/sites-available/ai-assistant /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx

5. Obtain a certificate:

sudo certbot --nginx -d yourdomain.com

Certbot will automatically edit the Nginx config to listen on 443 and redirect HTTP → HTTPS.

After this, you can close port 8080 in the firewall (since traffic now goes through Nginx on 443):

sudo ufw delete allow 8080/tcp
sudo ufw allow 443/tcp
sudo ufw status

8.3 Basic Auth in OpenWebUI (Optional)#

If you prefer not to rely on Nginx auth, enable OpenWebUI’s built‑in basic auth by uncommenting the env vars in docker-compose.yml and setting a strong password. Then restart the stack.

8.4 Fail2Ban (Brute‑Force Protection)#

Install fail2ban to block repeated failed login attempts:

sudo apt install -y fail2ban
sudo systemctl enable --now fail2ban

Create a jail for Nginx (/etc/fail2ban/jail.d/nginx.conf):

[nginx-http-auth]
enabled = true
filter = nginx-http-auth
logpath = /var/log/nginx/*error.log
maxretry = 3
bantime = 3600

Reload fail2ban:

sudo systemctl restart fail2ban

8.5 Regular Updates#

Keep Docker images and the host OS patched:

# Update host
sudo apt update && sudo apt upgrade -y

# Update containers
docker compose pull
docker compose up -d

Consider setting up a weekly cron job for these steps.

Step 9: Model Comparison – Which One to Choose?#

Recommendation for a $5 VPS: Start with Phi‑3‑mini to confirm the pipeline works. If you have enabled swap (≥2 GB) and notice spare RAM, try Mistral‑7B for a better quality‑to‑resource ratio. Only move to Llama 3‑8B if you consistently have >4 GB free RAM (or upgrade the VPS).

You can have multiple models loaded simultaneously; Ollama will keep them in memory until you explicitly remove them (ollama rm <model>). Switching models in OpenWebUI is as easy as selecting from the model dropdown.

Step 10: Using the Assistant#

Once logged into OpenWebUI:

1. Click the model selector (top‑left) and choose the model you pulled.
2. Start a conversation. Your data never leaves the VPS; all inference happens locally via Ollama.
3. To start a new chat, click the + New Chat button.
4. You can also upload files (if you enabled the feature) for context‑aware queries.

Example Prompt#

Explain the difference between supervised and unsupervised learning in two paragraphs.

You should see a concise, on‑point answer generated within a couple of seconds.

Maintenance Checklist#

• Monthly: Run docker compose pull && docker compose up -d to get latest images.
• Monthly: Verify backups exist and are restorable.
• Quarterly: Review firewall rules and fail2ban logs.
• As needed: Add swap if you notice OOM kills (sudo swapon --show).
• As needed: Rotate passwords for any auth you enabled.

Conclusion#

Deploying a private AI assistant on a $5 VPS is not only feasible—it’s practical, private, and surprisingly performant. By combining Ollama’s lightweight model server with OpenWebUI’s polished UI, you gain full control over your data, avoid subscription fees, and can experiment with state‑of‑the‑art LLMs like Phi‑3, Llama 3, and Mistral without ever touching a public API.

If you found this guide useful, explore more self‑hosting tutorials and DevOps insights at mahbuburriad.com.

Happy hacking, and enjoy your AI‑assistant that truly belongs to you.