AI Devcontainer — Architecture & Setup

Overview

This devcontainer provides a self-contained, GPU-accelerated AI development environment running inside a DevPod workspace. It serves a full local LLM stack accessible publicly via a Cloudflare tunnel at https://ai.uclab.dev.

Internet
   │
   ▼
Cloudflare (ai.uclab.dev)
   │  Zero Trust Tunnel
   ▼
cloudflared (container)
   │  http://localhost:8080
   ▼
Open WebUI  ──────────────────►  Ollama API (localhost:11434)
(port 8080)                           │
                                       ▼
                                 GPU: RTX 5060 (8 GB VRAM)
                                 Models: llama3.1:8b, qwen2.5:7b,
                                         deepseek-coder-v2:16b-lite-instruct-q4_K_M,
                                         mistral-nemo

Host Requirements

File Structure

.devcontainer/
├── Dockerfile              # Image definition
├── devcontainer.json       # DevPod/VS Code container config
├── pull-models             # Script to pull Ollama models (idempotent)
├── supervisord.conf        # Reference config (not active — see note below)
├── supervisord/
│   ├── ollama.conf         # Supervisor program: Ollama
│   ├── open-webui.conf     # Supervisor program: Open WebUI
│   └── cloudflared.conf    # Supervisor program: Cloudflare tunnel
└── scripts/
    ├── setup               # Trusts and installs mise tools for workspace
    └── setup_project       # One-time project setup (commitizen, pre-commit)

Docker Image

Base: nvidia/cuda:12.8.0-runtime-ubuntu24.04

The image is built from .devcontainer/Dockerfile with the repo root as context.

Build stages (in order)

Developer tools (via mise)

Defined in /workspaces/ai/mise.toml:

[tools]
node = "lts"
python = "3.13"
uv = "latest"

Tools are installed into /home/ubuntu/.local/share/mise/ and shimmed onto PATH via /etc/profile.d/mise.sh.

Container Configuration (devcontainer.json)

GPU passthrough

"runArgs": [
  "--device", "nvidia.com/gpu=all",
  "--security-opt", "label=disable"
]

Requires nvidia-container-toolkit on the host with CDI configured. The label=disable flag is needed for SELinux compatibility.

Environment variables

Set at the container level (available to all processes):

API keys passed from the host:

Persistent mounts

Both directories live on the host and are bind-mounted into the container, so they survive full container rebuilds:

Port forwarding

Port 8080 is also bound on the host via -p 8080:8080 in runArgs.

Startup sequence (postStartCommand)

sudo supervisord -c /etc/supervisor/supervisord.conf && \
{ until curl -sf http://localhost:11434 >/dev/null 2>&1; do sleep 2; done && pull-models; } &

1. supervisord starts as a daemon — launches Ollama, Open WebUI, and cloudflared
2. A background job polls Ollama until it responds on port 11434
3. Once Ollama is ready, pull-models runs and downloads any missing models
4. postStartCommand returns immediately so the IDE is not blocked

Process Management (Supervisor)

The system supervisor (/etc/supervisor/supervisord.conf) is the system default Debian config. It includes all files from /etc/supervisor/conf.d/*.conf, which are baked into the image from .devcontainer/supervisord/.

All three programs run as user ubuntu with HOME="/home/ubuntu" explicitly set (supervisor does not inherit HOME when switching users).

Priority order

priority=1  ollama       ← starts first
priority=2  open-webui   ← waits 10s (startsecs) for Ollama to be ready
priority=3  cloudflared  ← tunnel up last

ollama

command=/usr/local/bin/ollama serve
user=ubuntu
environment=
    HOME="/home/ubuntu",
    OLLAMA_HOST="0.0.0.0:11434",
    OLLAMA_KEEP_ALIVE="10m",
    OLLAMA_MAX_LOADED_MODELS="1",
    OLLAMA_NUM_PARALLEL="1",
    NVIDIA_VISIBLE_DEVICES="all",
    NVIDIA_DRIVER_CAPABILITIES="all"

OLLAMA_HOST=0.0.0.0:11434 makes Ollama listen on all interfaces (required for DevPod port forwarding to work).

open-webui

command=/usr/local/bin/open-webui serve --host 0.0.0.0 --port 8080
user=ubuntu
environment=
    HOME="/home/ubuntu",
    DATA_DIR="/home/ubuntu/.local/share/open-webui",
    OLLAMA_BASE_URL="http://127.0.0.1:11434",
    WEBUI_AUTH="true",
    ENABLE_MEMORY="true",
    MEMORY_RETRIEVAL_TOP_K="5",
    RAG_EMBEDDING_ENGINE="ollama",
    RAG_EMBEDDING_MODEL="nomic-embed-text",
    ENABLE_RAG_WEB_SEARCH="true",
    RAG_WEB_SEARCH_ENGINE="duckduckgo",
    DO_NOT_TRACK="true",
    ANONYMIZED_TELEMETRY="false"

DATA_DIR must point to a writable path — open-webui was installed as root so its package directory is not writable by ubuntu.

RAG uses nomic-embed-text via Ollama for embeddings. Web search uses DuckDuckGo (no API key required).

cloudflared

command=/usr/local/bin/cloudflared tunnel run 2571f532-7790-4275-8bb4-33e0c20c64d6
user=ubuntu
environment=HOME="/home/ubuntu"

The tunnel ID and credentials are static. Configuration is read from the bind-mounted /home/ubuntu/.cloudflared/config.yml:

tunnel: 2571f532-7790-4275-8bb4-33e0c20c64d6
credentials-file: /home/ubuntu/.cloudflared/2571f532-7790-4275-8bb4-33e0c20c64d6.json

ingress:
  - service: http://localhost:8080

All traffic on ai.uclab.dev is forwarded to Open WebUI on port 8080.

Model Management (pull-models)

Location in image: /usr/local/bin/pull-models

The script is idempotent — it checks ollama list before pulling and skips models already present. Since ~/.ollama is a persistent host mount, models are only downloaded once regardless of how many times the container is rebuilt.

Configured models

Total: ~27 GB. Stored on a 1.8 TB volume (currently 10% used).

Note: The RTX 5060 has 8 GB VRAM. Only one model loads at a time (OLLAMA_MAX_LOADED_MODELS=1). Models larger than VRAM will run partially on CPU.

Logs

All service logs are written to /var/log/ inside the container:

Quick check:

sudo supervisorctl status
sudo supervisorctl tail -f open-webui stderr

Rebuild Checklist

Everything needed to fully recreate the environment is either in the image or on the host:

Warning: Open WebUI user accounts and chat history live inside the container at /home/ubuntu/.local/share/open-webui. To persist this across rebuilds, add a host mount for that path in devcontainer.json.