Windows 11 Pro + Docker + Ollama + Open WebUI + SearXNG + ComfyUI + OpenCode + Hermes + Remote Access
This is an all-text guide to setup a LLM homelab with local & remote access. It assumes some leve of technical expertise. It was created as a "LLM Install Notes" file on how to install everytihng since I didn't know what I was doing, and thinks kept breaking. These instructions are the end result of Google searches, Reddit posts, ChatGPT/Claude, official documentation, my own LLM server, and head banging…against a wall. :)
Disclaimer: I am using an nVidia GPU. I used a mix of Claude/my LLM server to clean/format this guide, and I'm using Google Analytics to measure traffic.
This guide was made with the following software:
Open PowerShell then run:
# Install WSL
wsl --install
# Reboot, then ensure WSL is up to date
wsl --updatelocalhost:11434 to verify the app is running# Find available models at: https://ollama.com/search
# Larger models require more VRAM and might impact CUI performance.
# This is an example list. The "abliterated" model is uncensored.
# ollama pull <model_name>
ollama pull qwen3.5:9b
ollama pull qwen3.6:27b
ollama pull qwen3.6:35b
ollama pull qwen3-coder-next
ollama pull llama4:latest
ollama pull qwen3.5:122b
ollama pull richardyoung/qwen3.6-27b-abliterated:Q8_0DD must be running. Open PowerShell, and make sure to copy/paste the entire line for which GPU version is needed:
# Pull the OWUI image
docker pull ghcr.io/open-webui/open-webui:main
# AMD GPU
docker run -d -p 3000:8080 -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:main
# nVidia GPU
docker run -d -p 3000:8080 --gpus all -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:cudaCtrl+C then run:docker system prune -f
docker pull ghcr.io/open-webui/open-webui:cuda
Click Allow on the Windows Firewall dialogue box.
Security Warning: Docker's-pflag binds to all network interfaces by default, so OWUI becomes reachable by any device on your local network at<host_ip>:3000as soon as this container starts — independent of whether Cloudflare Tunnel (section 12) is ever set up. This is fine if you want LAN access for multiple users, but it means the Cloudflare Zero Trust policy in section 12 only protects the tunnel hostname — it does nothing to stop someone on your LAN from hitting this port directly. Even if LAN-wide access is exactly what you want, it's worth scoping the Windows Firewall rule to the Private network profile rather than clicking Allow broadly. This does not restrict access to just this machine — devices on your LAN can still reach it. What it does is stop the port from being exposed if this machine ever joins a network Windows doesn't consider trusted (a public Wi-Fi profile, for example), which the broad "Allow" click otherwise permits by default:
Open Powershell and paste the following command. This opens opens port 3000 for inbound requests on a private network.
New-NetFirewallRule -DisplayName "OpenWebUI" -Direction Inbound -Protocol TCP -LocalPort 3000 -Action Allow -Profile PrivateOpen PowerShell and paste the following code:
# Note that $HOME refers to C:\Users\User_Name
$base = "$HOME\searxng"
# Create base, config, and data folders
New-Item -ItemType Directory -Force -Path "$base\config","$base\data"
cd $base
# Pull latest SXNG image
docker pull docker.io/searxng/searxng:latest
# Start the service — This supposedly creates the settings.yml config file, but that has never happened for me
# Setting up the .yml file is covered next
docker run --name searxng -p 8080:8080 docker.io/searxng/searxngNote: I kept getting errors when installing SearXNG regarding the "wikidata: engine init was not successful". It's not a deal-breaker as SearXNG still works so I'm ignoring it for now. Hit Control + C to stop the command then start SearXNG in Docker manually. There are a few different reasons this error could happen, and I haven't narrowed it down yet.
C:\Users\User_Name\searxng and look for settings.ymluse_default_settings: true as the very first line, above the general: sectionformats: and add - json above the - html entrysettings.yml in Notepadultrasecretkey and replace it with a randomly generated 64 character keyC:\Users\User_Namedocker-compose.ymlservices:
open-webui:
image: ghcr.io/open-webui/open-webui:main
container_name: open-webui
depends_on:
- searxng
environment:
- OLLAMA_BASE_URL=http://host.docker.internal:11434
volumes:
- open_webui_data:/app/backend/data
ports:
- "3000:8080"
restart: unless-stopped
searxng:
image: searxng/searxng:latest
container_name: searxng
volumes:
- ./searxng:/etc/searxng
ports:
- "8080:8080"
restart: unless-stopped
volumes:
open_webui_data:C:\Users\User_Name to where the docker-compose.yml resides# Stops all containers, applies docker-compose.yml
# Restarts services in one container
docker compose down
docker compose up -d
# Both services should appear in the same container
# If there are name conflict errors or the services are not nested in the same container
# Remove/reload the services
docker rm -f searxng
docker rm -f open-webui
docker compose up -d
New-NetFirewallRule -DisplayName "SearXNG" -Direction Inbound -Protocol TCP -LocalPort 8080 -Action Allow -Profile PrivateSecurity note: Same as OWUI above — SearXNG's 8080:8080 mapping is reachable from any device on the network by default, with no authentication in front of it. If you only need it to serve OWUI's web search feature (not for standalone browsing by other users), scope it the same way.
Go to Settings → Models. For each model you want available to users:
5020- Do not apologize for not finding informationsearxnghttp://searxng:8080/search?q=<query>&format=json
# Alternatively, use the server's direct IP address:
# http://host_ip:8080/search?q=<query>&format=jsonHigher numbers for Search Result Count and Concurrent Requests don't guarantee better results. Recommended: 5–7 results, 2–3 concurrent requests.
10 and Concurrent Requests to 5. Because who needs rules? :)Note: LLMs and CUI share the same VRAM pool so running them together increases demand on it. Exceeding available VRAM either offloads to system RAM (slower) or causes an out-of-memory error (I've had this happen and crash CUI). Size the LLM model(s) and CUI workflow(s) appropriately.
C:\Users\User_Name\Documents\CUI_Portable) and unzip the downloaded file to itREADME_VERY_IMPORTANT.txt filerun_nvidia_gpu.bat) and create a shortcut to itrun_nvidia_gpu.bat in Notepad and add --listen 0.0.0.0 to the end of the first line then save and closeNew-NetFirewallRule -DisplayName "ComfyUI_Portable" -Direction Inbound -Protocol TCP -LocalPort 8188 -Action Allow -Profile PublicCUI Portable can be accessed over a network viahttp://host_ip:8188or locally viahttp://localhost:8188.
shell:startup — add a shortcut to run_nvidia_gpu.bat from the CUI folder or copy/paste the Desktop shortcutA terminal window will appear and any startup errors will show here. Minimize this if all goes well.
Z-Image-Turbo)C:\Users\User_Name\Documents\CUI_Portable\ComfyUI\models\Folder_Name)Note: that this file moving process will have to be done manually with any missing model components
Disclaimer: Node pack installation issues may require manual config.ini edits (see "If Installing Node Packs Fail" at the end of this section).
$path = "C:\Users\User_Name\AppData\Roaming\Comfy Desktop\installations.json"
$json = Get-Content $path -Raw | ConvertFrom-Json
$json[0].launchArgs = "--enable-manager --listen 0.0.0.0"
$json | ConvertTo-Json -Depth 10 | Set-Content $pathCUI Desktop can be accessed over a network viahttp://host_ip:8188or locally viahttp://localhost:8188.
C:\Users\User_Name\AppData\Roaming\Comfy Desktop, locate installations.json, and edit this line:"launchArgs": "--enable-manager",So it looks like this:
"launchArgs": "--enable-manager --listen 0.0.0.0",New-NetFirewallRule -DisplayName "ComfyUI_Desktop" -Direction Inbound -Protocol TCP -LocalPort 8188 -Action Allow -Profile Publicshell:startupNote: Opening CUI Desktop alone will not enable remote image generation. Click on "ComfyUI" after starting CUI Desktop then minimize.
I had issues with Python defaulting to the integrated GPU instead of the discrete GPU. Disabling the integrated GPU in Device Manager and rebooting (probably not necessary, but done anyway) fixed this issue.
[Alternative] In theory the below instructions would fix the GPU selection issue. However, it did not work for me which is why I recommend the disabling of the integrated GPU in Device Manager.
python.exe:C:\Users\User_Name\AppData\Local\Comfy-Desktop\ComfyUI-Installs\ComfyUI\ComfyUI\.venv\Scripts\python.exeComfyUI Desktop.exeNode packs add custom features, tools, etc. in a workflow that expands what CUI can do. However, when loading another person's workflow there might be node packs that were not installed by default. The integrated manager is a good tool. But it might fail to attempting to install missing packs (without notification of installation failure) and require manually tweaking the config.ini file.
C:\Users\User_Name\AppData\Local\Comfy-Desktop\ComfyUI-Installs\ComfyUI\ComfyUI\user\__managerconfig.ini and set:security_level = weak
network_mode = personal_cloudThis section will enable OWUI to pass image generation and editing requests onto CUI. The instructions are the same for the Desktop app and Portable version.
http://localhost:port (http://host_ip:port for access over a network)Open the JSON file in a web browser to look for the Node IDs. Each Node ID has two numbers: the Node ID and sub-node number. These IDs are examples:
Text: 67 — search for "CLIPTextEncode"
# Look for the section with a "text" field containing a scene description
Model: 66 — search for "unet_name", don't include the file extension
Width/Height: 68 — search for "width" or "height"
Steps: 70 — search for "steps"
Seed: 70 — search for "seed"http://localhost:port (http://host_ip:port for access over a network)Image: search for "images"
Prompt: same as the "Text" Node ID from above
npm config get prefixPath, click EditC:\Users\User_Name\AppData\Roaming\npm) and name the path NPM. Note: this entry might already exist. If it does then no changes are needed so close the window and move on.dir "$env:APPDATA\npm"npm i -g opencode-ai@latestollama listC:\Users\User_Name\.config\opencode\opencode.json to backup_opencode.json (if it exists)opencode.json and copy in the following code. Replace the "actual_name_of_model_01" from the ollama list command earlier and replace "how_the_model_name_will_be_displayed_01" with the model's display name{
"$schema": "https://opencode.ai/config.json",
"provider": {
"ollama": {
"npm": "@ai-sdk/openai-compatible",
"name": "Ollama (local)",
"options": {
"baseURL": "http://localhost:11434/v1",
"apiKey": "ollama"
},
"models": {
"actual_name_of_model_01": { "name": "how_the_model_name_will_be_displayed_01", "tools": true },
"actual_name_of_model_02": { "name": "how_the_model_name_will_be_displayed_02", "tools": true }
}
}
}
}New-NetFirewallRule -DisplayName "OpenCode" -Direction Inbound -Protocol TCP -LocalPort 4096 -Action Allow -Profile Private OpenCode.bat@echo off
if not "%1"=="min" (
start "" /min cmd /c "%~f0" min
exit /b
)
set OPENCODE_SERVER_USERNAME=CHANGE_USER_NAME
set OPENCODE_SERVER_PASSWORD=CHANGE_PASSWORD
opencode.cmd web --port 4096 --hostname 0.0.0.0
http://localhost:4096 — which will also make it accessible over a network via http://host_ip:4096.
Security Warning: Because OC is a coding agent with file and shell access, exposing it with--hostname 0.0.0.0and no authentication would let anyone on the network run commands on this machine.Setting
OPENCODE_SERVER_USERNAMEandOPENCODE_SERVER_PASSWORDenables HTTP Basic Auth — so a login prompt gates access before anyone reaches a session. Don't skip this line. The command also minimizes the OC terminal after running.Hardcoding the UN/PW is a bad security practice, but OC currently does not support multiple users/PWs/roles/sessions/accounts. ACT ACCORDINGLY.
default.txt file with no data in it in the folder. I had to do that at one point to make the "Default Project" folder findable by OC.Security Warning: When searching for a project folder, the entire folder path is exposed to the user.
shell:startupOpenCode.bat file in the startup folder and close the window# Project 1
cd C:\Users\User_Name\Documents\Project_Folder
opencode web --port 4096
# Project 2
cd C:\Users\User_Name\Documents\Project_Folder
opencode web --port 4097It can be used via a terminal, web interface, or plugged into OWUI, and it can use almost any public or local LLM model.
Security Warning: Hermes can do a lot, but if something goes wrong it can do a lot of damage. Given that risk this guide does not connect it to OWUI. That keeps a clear division of labor — a chatbot with lots of capabilities on one side, and preventing "Oops, what happened?" on the other.
hermes modelhttp://localhost:11434/v1hermes model command anytime to change the model being used
Using a local LLM here is fine, but it will not have RAG abilities the way a cloud model would. NOTE - THIS IS NOT ACCURATE, FIX FIX FIX
New-NetFirewallRule -DisplayName "Hermes_Dashboard" -Direction Inbound -Protocol TCP -LocalPort 9119 -Action Allow -Profile Privatehermes dashboard registerHermes_Dashboard.bat. Paste this code in:
@echo off
if not "%1"=="min" (
start "" /min cmd /c "%~f0" min
exit /b
)
hermes dashboard --host 0.0.0.0 --port 9119shell:startup — create a shortcut to the batch file. This starts the Dashboard listening for network requests on boot. There are a couple other ways to accomplish this, but for the sake of consistency this method is recommendedC:\Users\User_Name\AppData\Local\hermesconfig.yaml in Notepadweb:
search_backend: "searxng"
extract_backend: "firecrawl".env file in Notepad. It has no prefixSEARXNG_URL=http://localhost:8080This is a much longer section as it has more components that need to be pieced together. This first part creates an access control policy.
exampledomain.com)Open WebUI), set Action to Allow, and set a session duration (ex. 2 weeks)Text controls drop down and select "Both Directions Allowed"
This next part creates a tunnel which allows a service to be accessed remotely. This tunnel can be used be multiple services.
openwebui) and select exampledomain.com from the dropdownHTTP and enter localhost:3000 (ex: OWUI port)Add public hostname to create that entry and remove the Private IPs entry by clicking on thes trash can.
ex: openwebui) and select the purchased domain (ex: exampledomain.com)Open WebUI policy from the dropdown → saveAccept all available identity providersonetimepinApply instant authenticationSession Duration click on the drop down and select the duration that matches the access policy duration from earlier in this section
openwebui.exampledomain.com — enter your email to receive an OTP and gain accessSecurity Warning: The Cloudflare Access login page shows the same "check your email" confirmation regardless of whether the entered address is actually authorized. This is intentional — it prevents attackers from using the response to determine which email addresses are valid.
Published application routes
Add a published application route
comfyui) and select exampledomain.com from the dropdownHTTP and enter localhost:8188 (ex: ComfyUI port)Note: From this point go back to Section 12 and repeat the instructions. This creates a new service access policy, an application destination, link the two together, put up the Cloudflare login page with one time pin, and give remote access to the service. Technically only one access policy can be used for all services, but that is a poor security practice.Remote access has been verified to work with these browsers/versions
Firefox v152.0.6 (64-bit)
Chrome v150.0.7871.128 (Official Build) (64-bit)
Safari v26.5.2 (21624.2.5.11.8)
gemini-3-pro-image-preview in the Model fieldhttps://generativelanguage.googleapis.com/v1betaSXNG allows the admin to prioritize or block certain domains via settings.yml. Scroll to the # Configuration of the "Hostnames plugin": section.
A good starting reference for blocked / low-priority / high-priority sites: kagi.com/stats?stat=insights
medium to high (or low)After saving advanced parameters, refresh the Admin Panel page. Going back into the same model without refreshing may still show the old values. Medium reasoning is fine for most use cases.
It's possible to use other search engines with OWUI. Here are a few options which can be found under Admin Panel → Settings → Web Search. Most require an API key.
settings.yml, under the engines section, change the disabled: flag from true to false for Reddit (or other sites) if desired.