Notes

System and hardware choices: Debian, Proxmox, fnOS, TrueNAS or Flatcar?

There is no single best home AI server. Compare common systems by hardware support, GPU passthrough, storage, update model, learning curve and backups, then choose in a sensible order.

Updated 2026-11-16

Choose what you can maintain, not what looks strongest

A home server stays with you for years. Start with four questions:

  1. What runs on it? Containers only, virtual machines too, or also a NAS?
  2. Is there a GPU? Passthrough, multiple cards, specific drivers?
  3. How important is the data? Can you reinstall, and where is the backup?
  4. How much maintenance do you want? Weekly updates or twice a year?

There is no best option, only a fit for your current stage.

Comparison

Option Good for Not for Watch out
Debian / Ubuntu Server a clean base and full control people who avoid the shell drivers, Docker, firewall and backups are yours
Proxmox VE several VMs/containers on one box, snapshots and backups a single container GPU passthrough, storage and network planning
fnOS NAS UI plus an app store with less tinkering deep customisation or special kernels app compatibility and data export
TrueNAS SCALE storage first, containers/VMs second treating ZFS like ext4 RAM guidance, disk layout, snapshots
Unraid mixed-size disks, media libraries, gradual growth maximum random I/O or strict RAID parity, cache disks, backups
Flatcar / Fedora CoreOS containers only, immutable host, automatic updates frequent host-level changes ignition/butane, different debugging
Windows + WSL2 Windows-first users who occasionally need Linux long-running unattended services sleep, update reboots, GPU limits

One by one

Debian / Ubuntu Server

The most predictable starting point: broad package support, documentation and community. You decide partitioning, Docker, firewall, unattended upgrades and backups.

Good if you want a transparent system and may change plans later. Ubuntu’s snap behaviour and automatic updates deserve a look in advance; Debian packages can be older, so GPU drivers may need an extra source.

Proxmox VE

Debian-based, with VMs, LXC, snapshots, backups and a web UI. A good fit when one machine must play several roles.

Good for separating NAS, models, monitoring and test environments, and for rolling back with snapshots.

Watch out:

  • GPU passthrough needs IOMMU, VFIO and the right driver; it is not one checkbox.
  • ZFS uses RAM; leave headroom.
  • Backups must land on another disk or network storage.

fnOS

A NAS-oriented experience with a graphical interface and an app store, for people who do not want to live in a shell.

Good for file sharing, photo backup and a small set of applications. Confirm that the container/model runtimes you need are supported, and understand the data layout and export path before you depend on it.

TrueNAS SCALE

ZFS at the centre, with containers and VMs around it. Data integrity, snapshots and replication are the strengths.

For data-first setups that need long-term preservation and checksums. ZFS needs enough RAM; RAID is not a backup; expansion and disk replacement follow a process.

Unraid

Built around mixed-size disks and gradual expansion, common for media libraries.

Good when you add one disk at a time and want a single failed disk not to lose the rest. Parity limits write performance; cache and backup strategy are separate decisions.

Flatcar / Fedora CoreOS

Immutable hosts, applications in containers, updates managed by the system.

For container-only hosts that should change rarely and can be configured as code. Debugging differs from a traditional distribution, ignition/butane is required, and not every driver is included.

Windows + WSL2

Fine when Windows is the main desktop and you occasionally need Linux tools or a small model.

Not for long-running unattended services: update reboots, sleep and WSL resource limits all affect availability.

GPU and container runtime

Whatever the host, GPU containers usually need:

  1. a working driver on the host;
  2. a runtime that can reach the device (NVIDIA Container Toolkit or equivalent);
  3. a CUDA/ROCm version inside the image that matches;
  4. correct permissions and resource limits.

Verify in order:

nvidia-smi
docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi

If the host sees the GPU but the container does not, check the runtime before blaming the model.

Disk layout

A practical default:

  • OS disk: small SSD, separate from data.
  • Data disk: mounted by UUID, not device name.
  • Cache/temp: a separate directory with a cleanup routine.
  • Backup: another disk or network location, restored on a schedule.

Do not put everything in one partition. Models, images, databases and backups grow at different rates.

Network plan

  • Give the server a fixed internal IP or a DHCP reservation.
  • Keep internal and public hostnames separate.
  • Prefer Tunnel/VPN for remote access; do not forward ports without a reason.
  • Record MAC, IP, purpose and owner per machine.

Installation order

  1. Update firmware; confirm disks and RAM are detected.
  2. Decide the OS and data disks; do not import old data yet.
  3. Install the system, set a fixed IP and SSH.
  4. Configure drivers and the container runtime; verify the GPU.
  5. Configure storage, snapshots and backups.
  6. Deploy the first service; confirm it returns after a reboot.
  7. Add monitoring and alerts, then migrate other services gradually.

Common mistakes

  • Making the data disk the OS disk, then formatting it during a reinstall.
  • Running an incompatible container on a NAS and having no way to debug it.
  • Using RAID as a backup; RAID does not prevent deletion or ransomware.
  • Updating without a snapshot and having no rollback.
  • Exposing a test environment and forgetting the debug endpoint.

Next

  • Build the network and security baseline: read security.
  • Understand domains, Pages, Workers and Tunnel: read Cloudflare basics.
  • Want the system, containers and monitoring put together: see packages.
Email is the only channel. Questions, corrections or a scope request: hello@localdeploy.dev. This page also exists as Markdown: /en/systems.md.

Not sure where to start?

Run the assessment first, or send your setup and goal by email. The reply says whether it fits and what happens next.