Skip to main content

Served drives

A served drive keeps one live copy of its data on one node, the active server, and shares it with the other nodes over NFS. Use it when many nodes need the same files at once (a media library, say) or when the data is too big to copy to every node. The trade-off: reads and writes from other nodes cross the network, and if the server's disk dies the live copy is gone.

If you want every node to hold a full copy and failover to restart a service anywhere, use a replicated drive instead.

How it works​

  1. You list candidate servers in served_by, in priority order. Every candidate exports the drive over NFSv4 with the kernel NFS server (exportfs), restricted to the internal subnet, read-write with no_root_squash (root on a mounting node is root on the exported copy), port 2049.
  2. The active server is the first candidate in the list that is reachable. Only that node's copy is live. The other candidates keep a standby copy, synced from the active server on the drive's sync_interval.
  3. Nodes that are not candidates mount the active server at <data_dir>/volume-mounts/<drive>/<volume> (NFS 4.2 over TCP, soft mount, no file locking). Containers see the volume at the path you mounted.
  4. Every 10 seconds each node checks who the active server is. If it changes, the node unmounts and remounts against the new one. Each mount is health-checked every 30 seconds with a 3 second timeout.
  5. If the active server becomes unreachable, the next candidate in served_by takes over automatically. The standby copy it holds is only as fresh as the last sync.

Moving a served volume between nodes on purpose (a manual move or a rebalance) goes through a migration with these phases: stopping, syncing, verifying (both ends are hashed), switching, starting.

Requirements​

  • Every node that mounts or serves the drive needs root and the nfsv4 kernel module. Without them the drive stays unavailable on that node.
  • The internal network must allow NFS between nodes. See Ports and firewall.
  • Candidate nodes must map the drive to a local folder like any other drive.

Set up a served drive​

Open Settings, Storage (/settings/storage). Add a drive (or open an existing card), set Storage mode to Served, and choose the Served by nodes. The order matters: use move atlas up and move atlas down to set priority. Resync every is hidden for served drives in the UI; the config file still accepts sync_interval for them.

On the drive page (/storage/drives/bulk) the Nodes table has a Priority column and About shows the active server.

/settings/storage
Storage settings. The served drive bulk shows its candidate order (atlas, boreas).

Validation rules for served_by: only served drives can have it (only a served drive has a serving node, set mode: served), every name must be a real node (no node named ...), listed once (... is listed more than once), and must map the drive (... must map media to the directory holding its copy).

Using it​

Mount it exactly like a replicated drive: media/library:/data in a stack. The scheduler places the container on any node that can mount the drive. To import files into it, see Folder import: it works from the active server or from a node that mounts the drive, and refuses a standby candidate (the next sync from the real server would erase what was written there).

Options​

Option
Type / default
Reload
Description
drives.<name>.mode
replicated | serveddefault replicated
live reload
Set to served for an NFS-exported drive.
Watch out: Changing mode rebuilds only that drive.
drives.<name>.served_by
list of node namesdefault []
live reload
Candidate servers in priority order. The first reachable one is the active server.
Watch out: Each node must map the drive. Empty is accepted by validation.
drives.<name>.sync_interval
durationdefault 15m
live reload
How often the active server pushes to standby candidates.
nodes.<name>.drives.<drive>
absolute pathno default
live reload
Folder holding the copy. Required on candidates; not needed on nodes that only mount.

Quirks and gotchas​