Image: reddit.com · Built a local, GPU-accelerated chat assistant for my Proxmox cluster on a Jetson Orin Nano (fully offline, no cloud)
An offline Proxmox chat assistant on a Jetson Orin Nano
Ollama, Open WebUI and ProxmoxMCP-Plus on a Jetson Orin Nano answer questions about a Proxmox cluster locally; the setup was worked out with Claude.
Credits
Earliest dated source:
The full story
The maker wanted a chat window that could answer questions like "any issues with my containers?" from live Proxmox data, entirely locally. They run Ollama, Open WebUI and ProxmoxMCP-Plus on an 8GB Jetson Orin Nano with GPU acceleration.
It works, but getting a small 7B model to call tools reliably turned up several bugs: the wrong Open WebUI connection type, a function-calling mode that silently swallowed responses, memory pressure and a Qwen3 issue causing runaway generation. The write-up was put together working interactively with Claude and tested from a fresh clone before publishing.
How it was done
Tools
Themes
Loading themes…
Sources
From a short post by the maker, so the details are as they tell them.
- Reddit: Built a local, GPU-accelerated chat assistant for my Proxmox cluster on a Jetson Orin Nano (fully ofReddit poster: nroamPosted
The maker's post on r/homelab.
If it disappears: archived copy of Reddit: Built a local, GPU-accelerated chat assistant for my Proxmox cluster on a Jetson Orin Nano (fully of (opens in a new tab)
- The code: github.comRepository account: RekklesNA
Linked in the post.