Setup guide

Pick a path. Both end in the same place.

With the script Without the script

You need

  • Linux with Docker and Compose v2
  • git and openssl
  • About 3 GB free for models
  • An NVIDIA GPU is optional

No GPU? Remove the deploy: block from the ollama service in docker-compose.yaml.

Path 1

With the script

1Run it

bash
curl -fsSL https://YOUR-SITE/install.sh | bash

It is plain bash. Read it first.

2Answer two questions

Access: Cloudflare, Tailscale, or local only.

Model server: Ollama, or vLLM.

Cloudflare asks for a tunnel token. Tailscale asks for an auth key. Leave either blank and fill it into .env later.

3Open it

Go to http://127.0.0.1:3000. Create the first account. It becomes the admin.

Then turn off sign-ups.

Path 2

Without the script

1Install Docker

Use Docker's official repository, not Snap or Docker Desktop: docs.docker.com/engine/install. Then add yourself to the docker group:

bash
sudo usermod -aG docker $USER

Log out and back in so the group applies. Then check:

bash
docker run --rm hello-world
docker compose version

NVIDIA GPU? Install the NVIDIA Container Toolkit too.

2Get the project

bash
git clone https://github.com/razajahangir/inhouse-ai.git in-house-ai
cd in-house-ai
cp .env.example .env

3Make secrets

bash
sed -i "s|^WEBUI_SECRET_KEY=.*|WEBUI_SECRET_KEY=$(openssl rand -hex 32)|" .env
sed -i "s|^POSTGRES_PASSWORD=.*|POSTGRES_PASSWORD=$(openssl rand -hex 32)|" .env
sed -i "s|^LITELLM_MASTER_KEY=.*|LITELLM_MASTER_KEY=sk-$(openssl rand -hex 32)|" .env

Linux sed. On macOS use sed -i ''. Or edit .env by hand. Keys that need values: WEBUI_SECRET_KEY, POSTGRES_PASSWORD, LITELLM_MASTER_KEY (must start with sk-).

4Choose access

Set COMPOSE_PROFILES in .env. The template ships with tailscale,cloudflare, so change it. Changing it later? Run docker compose --profile "*" down first, or the old tunnel keeps running. If you exported COMPOSE_PROFILES in your shell, it overrides .env: run unset COMPOSE_PROFILES.

Cloudflare Tunnel

.env
COMPOSE_PROFILES=cloudflare
CLOUDFLARE_TUNNEL_TOKEN=eyJ...
  1. Cloudflare Zero Trust → Networks → Tunnels → Create a tunnel → Docker.
  2. Copy only the token (starts with eyJ). Do not run its command.
  3. Add a public hostname. Service: HTTP, URL open-webui:8080.
  4. Zero Trust → Access → Applications → add a policy for your email, so the login page isn't public.

Cloudflare can read this traffic at its edge.

Tailscale

.env
COMPOSE_PROFILES=tailscale
TS_AUTHKEY=tskey-...
TS_HOSTNAME=in-house-ai
  1. Tailscale admin → Settings → Keys → generate an auth key.
  2. Admin → DNS → turn on MagicDNS and HTTPS certificates.
  3. Open https://in-house-ai.<your-tailnet>.ts.net.

End to end encrypted. Only your tailnet can reach it.

Local only

.env
COMPOSE_PROFILES=

Open http://127.0.0.1:3000 on this machine.

5Choose a model server

Ollama is the default. Nothing to change.

vLLM is for bigger NVIDIA GPUs. Add vllm to the profiles, set the model, and point LiteLLM at it:

.env
COMPOSE_PROFILES=tailscale,vllm
VLLM_MODEL=Qwen/Qwen2.5-0.5B-Instruct
litellm/config.yaml
- model_name: local-chat
  litellm_params:
    model: hosted_vllm/local-chat
    api_base: http://vllm:8000/v1

After editing litellm/config.yaml, run docker compose restart litellm. Embeddings stay on Ollama, so Ollama and vLLM share the GPU: keep the model small. Gated Hugging Face models also need HF_TOKEN in .env.

6Start it

bash
docker compose up -d
docker compose logs -f ollama-init

Wait for the models to download. ollama-init then exits.

bash
docker compose ps

Everything should be running, except ollama-init (exited 0).

7Open it

Create the first account at your address. It becomes the admin.

Then lock it down

In docker-compose.yaml, under open-webui, set ENABLE_SIGNUP: "false". Apply it:

bash
docker compose up -d

Only the admin adds users now: Admin Panel → Users. To try document answers: Workspace → Knowledge → upload a file, then type # in a chat.

Stuck? See troubleshooting in the README.