NFS Ganesha vs. Kernel NFSd: What Makes Sense in TrueNAS and When

NFS is the quiet constant of enterprise storage: in production for decades, understood by every platform, boring in the best sense of the word. Yet TrueNAS administrators regularly face a real decision: stick with the classic kernel NFSd or move to NFS Ganesha in user space. This article walks through the differences, the boundaries, and when a switch actually pays off.

Two architectures, one protocol

The kernel NFSd (nfsd) is part of the Linux or FreeBSD kernel. It shares address space, caches and the I/O path with the file system, which keeps things short: an NFS request enters through the network stack, is handled by the in-kernel NFS server and passed straight to ZFS. No context switch, no copy between kernel and user space.

NFS Ganesha, by contrast, is a complete NFS server that runs entirely in user space. It sits on top of the kernel and talks to the file system through FSAL modules (File System Abstraction Layer). That sounds like a disadvantage at first - every I/O path picks up extra layers. In practice, though, it enables capabilities the kernel NFSd simply does not offer.

Criterion	Kernel NFSd	NFS Ganesha
Runtime environment	Kernel space	User space
Backend integration	Direct VFS/ZFS	FSAL modules (ZFS, GlusterFS, Ceph, VFS)
pNFS support	Limited	Full, including Flex Files
NFSv4.2 features	Basics	Broad, including Server-Side Copy
Debugging	dtrace, kdb, involved	gdb, user-space logs
Restart behaviour	Usually needs a reboot	Process restart while online
Raw throughput on 10GbE	Very high, short paths	Slightly higher CPU overhead
Cluster capability	Not designed for it	Actively used (CTDB, Ceph)

TrueNAS SCALE ships with the kernel NFSd by default, and so does TrueNAS CORE (the FreeBSD variant). Ganesha is not part of the default install on either edition, but can be layered onto a SCALE base in custom deployments - typically together with clustered storage or specific compliance requirements.

Where the kernel NFSd wins

For the vast majority of classic workloads, the kernel NFSd is the right choice. It is mature, well instrumented and ready to use in TrueNAS without extra effort. Its strengths show clearly in these areas:

• Raw throughput on 10/25/100 GbE. Without context switches between kernel and user space, the NFSd can saturate line rate as long as ZFS keeps up. For VM datastores under Proxmox or ESXi, that is the deciding factor.
• Low latency on small I/Os. Metadata-heavy workloads such as home directories, git repositories or CI runners benefit from the short path.
• Low-maintenance operations. The NFSd is part of the system - updates arrive with regular TrueNAS releases, no separate package stack.
• Familiar debugging toolkit. nfsstat, rpcinfo, dtrace or bpftrace deliver reliable metrics right at the kernel path.

If you run a TrueNAS environment primarily as an NFS datastore for virtualisation, do not abandon this path without a real reason. The tight coupling to ZFS - including correct handling of sync=always, ZIL/SLOG and ARC - is far better covered in the kernel than in any user-space implementation.

Where Ganesha plays its trump cards

Ganesha becomes interesting the moment requirements enter the picture that the kernel NFSd either does not serve or only serves partially. Three scenarios come up regularly in our consulting engagements:

1. pNFS with Flex Files. Parallel NFS distributes data access directly between clients and multiple data servers - the metadata server stays out of the data path. For HPC clusters or video rendering farms this can raise effective bandwidth substantially, because a single NFS server is no longer the bottleneck. The Linux kernel supports pNFS very well on the client side today; on the server side, Ganesha is by far the more complete implementation.
2. Cluster storage. Ganesha is the reference implementation for NFS on top of CephFS and GlusterFS. If you run TrueNAS alongside a Ceph cluster or are planning a migration, you will almost certainly meet Ganesha along the way.
3. Granular control and debuggability. Ganesha lets you configure different FSALs, log levels and security flavours per export without a kernel reboot. For environments with a high rate of change - multi-tenant storage in particular - that is a tangible advantage.

An often underestimated point: Ganesha can isolate individual exports when needed. Misbehaviour in one export module does not necessarily take down the entire NFS server, unlike some worst-case scenarios in the kernel path. For environments where NFS carries SLA weight, this isolation is a strong argument.

What this means for TrueNAS

TrueNAS is deliberately opinionated: a storage appliance, not a construction kit. The default is the kernel NFSd, and iX Systems concentrates its engineering resources there. We consider that the right call - for 95 percent of cases the kernel path delivers exactly what customers need.

The remaining five percent are where the Ganesha question comes up. In practice, we usually see one of these patterns:

• Migration from a Ceph or Gluster cluster to TrueNAS, with Ganesha used as a temporary bridge.
• HPC setups with a pNFS requirement, where a single namespace should span multiple TrueNAS nodes. That is not a TrueNAS core feature and requires custom engineering.
• Regulated environments, where every export needs its own audit trail and dedicated Kerberos realm.

In all three cases we recommend not running Ganesha on the TrueNAS appliance itself, but on a separate Linux VM or dedicated node that consumes TrueNAS as backend storage. That keeps the appliance supportable and puts responsibility for the user-space server squarely with the operations team.

Configuration example on 10GbE

Below are typical configurations for both variants. Starting point: TrueNAS SCALE with a mirror pool of NVMe devices, a dedicated SLOG and a 10 GbE uplink to a Proxmox cluster.

For the kernel NFSd, the TrueNAS SCALE UI is enough - exports are maintained in /etc/exports.d/ in the background. From our experience these parameters matter most:

# Excerpt /etc/exports.d/tank-vmstore.exports
/mnt/tank/vmstore \
    10.20.30.0/24(rw,sync,no_subtree_check,no_root_squash,\
                  sec=sys,fsid=42,mountpoint)

A matching ZFS configuration completes the picture:

zfs set sync=always tank/vmstore
zfs set recordsize=64K tank/vmstore   # for VM images
zfs set atime=off tank/vmstore
zfs set compression=lz4 tank/vmstore

sync=always is non-negotiable for VM datastores - a SLOG with power-loss protection is required, otherwise latency will suffer badly. We have seen this pattern repeatedly in our Proxmox consulting projects.

For Ganesha on a separate Linux VM, the configuration is significantly richer:

# /etc/ganesha/ganesha.conf (excerpt)
NFS_CORE_PARAM {
    Protocols = 3,4;
    NSM_Use_Caller_Name = true;
}

EXPORT {
    Export_Id = 42;
    Path = /mnt/tank/vmstore;
    Pseudo = /vmstore;
    Access_Type = RW;
    Squash = No_Root_Squash;
    SecType = sys, krb5, krb5i, krb5p;
    FSAL {
        Name = VFS;
    }
    CLIENT {
        Clients = 10.20.30.0/24;
        Access_Type = RW;
    }
}

LOG {
    Default_Log_Level = EVENT;
    Components {
        FSAL = INFO;
        NFS4 = DEBUG;
    }
}

The value here is granularity: log level per export, a dedicated Kerberos realm, or a different FSAL module - all configurable without kernel reboots. A systemctl reload nfs-ganesha picks up changes without downtime, a feature the kernel NFSd does not offer in this form.

A note on benchmarks: we do not publish numbers we have not reproduced ourselves in a properly documented test environment. Vendor figures for throughput and latency vary sharply by workload, client version and ZFS tuning. If you need reliable numbers for your own environment, run fio against a real client - ideally with your actual workload profile, not synthetic 4K random reads.

How to switch - and when not to

Moving from the kernel NFSd to Ganesha is not a pure configuration exercise. We recommend this sequence:

• Requirements analysis: which Ganesha capability is the real driver? pNFS, cluster, multi-tenant control? Without a clear reason the effort is not worthwhile.
• Isolated setup: deploy Ganesha on a separate Linux VM or bare-metal node, attach TrueNAS as backend. Never run it in parallel to the kernel NFSd on the same appliance.
• Client migration: move NFS mounts step by step, ideally through a new export pseudo-root structure. This keeps a rollback path open at all times.
• Monitoring: keep ganesha_stats and nfsstat active on both sides to confirm or dismiss regressions quickly.

Skipping these steps risks a silent decline: latency creeps up first, then locking issues appear, and in the end the migration becomes more expensive than a clean redesign would have been.

Bottom line

The kernel NFSd remains the right choice for most TrueNAS deployments: fast, unfussy, well supported. NFS Ganesha is not a replacement but a complement for scenarios where pNFS, cluster storage or granular control tip the balance. The decision should never come from gut feeling - it should always follow a concrete requirement.

DATAZONE supports you in choosing the right NFS architecture, in TrueNAS design, and in integration with existing Proxmox or VMware environments. Talk to us - our contact page is the fastest path to a solid assessment for your environment.