A private ChatGPT-style assistant on your own hardware.

Chat with your documents. Everything runs on your machine.

Install
curl -fsSL https://YOUR-SITE/install.sh | bash

Needs Docker. Read the script first, or follow the setup guide.

Built on

  • Open WebUI
  • LiteLLM
  • Ollama
  • vLLM
  • Postgres pgvector
  • Docker
  • Cloudflare
  • Tailscale

What you get

One Compose file, three things that work.

  • Chat for a team Everyone gets their own login.
  • Ask your documents Answers come with citations.
  • Swap the models Change the model without changing anything else.

Privacy

What stays local, what doesn't.

Stays on your machine

  • Your prompts and documents
  • Your chats and search index
  • The models

Leaves your machine

  • Setup downloads
  • Cloudflare tunnel traffic, which Cloudflare can read
  • Tailscale traffic, which is end-to-end encrypted

Architecture

One private network.

Pick how you reach it. No inbound ports are opened.

in-house-net Browser Open WebUI LiteLLM Ollama / vLLM Postgres Cloudflare Tunnel Tailscale This machine only

Setup

Pick your setup.

Open the setup guide → With the script, or by hand.

Pick access

Cloudflare
Public link with a login in front.
Tailscale
Private. Your devices only.
Local only
No remote access.

Pick a model server

Ollama
Laptops and small GPUs.
vLLM
Big NVIDIA GPUs.

Hardware

Runs on a laptop.

Measured on a GTX 1650 with 4 GB. No GPU? It still runs, just slower.

Short prompt

1.1 s

26 tokens · 28.5 tok/s

0.30 s to first word

Medium prompt

9.2 s

41 tokens · 26.7 tok/s

0.42 s to first word

Long prompt

56 s

3,576 tokens · 11.9 tok/s

30.7 s to first word

Test your own machine with benchmarks/bench.py.