> ## Documentation Index > Fetch the complete documentation index at: https://docs.siderolabs.com/llms.txt > Use this file to discover all available pages before exploring further. # Longhorn > Configure and install Longhorn distributed block storage on a Talos Linux cluster export const release = 'v1.13.6'; export const version = 'v1.13'; [Longhorn](https://longhorn.io) is a distributed block storage system for Kubernetes that provisions persistent volumes from the local disks of your nodes. It ships with two data engines, and this guide covers installing both on a Talos Linux cluster. * **V1 Data Engine**: the mature, production-ready default. It builds replicated volumes on a Longhorn-native iSCSI stack and runs on any node that has the required system extensions. * **V2 Data Engine**: a newer, SPDK-based engine that delivers higher throughput and lower latency at the cost of extra node configuration. It additionally requires huge pages and a few kernel modules. Pick the tab for the engine you want to deploy. Both paths cover the required system extensions, disk provisioning with `UserVolumeConfig` or `RawVolumeConfig`, pod security configuration, and the final Helm installation. ## Prerequisites Before you begin, ensure that you have the following: * `talosctl` configured and authenticated against your cluster * `kubectl` configured to access the same cluster * Helm 3 installed * At least one dedicated disk per storage node (NVMe recommended) Longhorn also requires two Talos system extensions on every node: * `siderolabs/iscsi-tools` — provides iscsid and iscsiadm for persistent volume operations * `siderolabs/util-linux-tools` — provides fstrim for filesystem trimming The schematic ID 613e1592b2da41ae5e265e8789429f22e121aab91cb4deb6bc3c0b6262961245 corresponds to exactly the four extensions listed above. **If you are provisioning new nodes**, add these extensions when building your Talos image using the [Talos Image Factory UI](https://factory.talos.dev/) or during the Download installation media step when creating machines with Omni. The V1 Data Engine is Longhorn's mature, production-ready default. It builds replicated volumes on a Longhorn-native iSCSI stack and runs on any node that has the required system extensions, so each storage node needs nothing more than a dedicated disk. ## Step 1: Provision a dedicated disk with UserVolumeConfig Talos Linux v1.10 introduced UserVolumeConfig, which replaces the deprecated `machine.disks` API. This resource automatically mounts selected disks under `/var/mnt/`. In this guide, the volume is named `longhorn` which mounts at `/var/mnt/longhorn`. ### 1.1: Inspect available disks Inspect the disks available on each storage node. ```bash theme={null} talosctl get disks --nodes ``` Use this output to determine which disks should be used for Longhorn storage. ### 1.2: Create the UserVolumeConfig Create a UserVolumeConfig document to tell Talos which disk to provision for Longhorn. Talos will automatically mount the volume at `/var/mnt/longhorn`. Adjust based on your hardware and the disk you want to use. The example below uses the first NVMe disk that is not a system disk. ```bash theme={null} cat < longhorn-user-disk.yaml apiVersion: v1alpha1 kind: UserVolumeConfig name: longhorn provisioning: diskSelector: match: disk.transport == 'nvme' && !system_disk grow: false EOF ``` The `diskSelector.match` field accepts Common Expression Language (CEL) expressions. You can target disks more precisely using expressions such as `disk.size > 50GB` or `disk.model == "Samsung SSD 980"`. Run `talosctl get disks` to see the available fields for your hardware. For more information, refer to the Disk Management documentation. ### 1.3: Apply the UserVolumeConfig configuration to storage nodes Define the IP addresses of the worker nodes that will provide storage to Longhorn. Add one entry per worker node that will provide storage to Longhorn: ```bash theme={null} WORKER_IPS=("" "") ``` Then apply the UserVolumeConfig patch to each node: ```bash theme={null} for ip in "${WORKER_IPS[@]}"; do echo "Applying patch to worker node: $ip" talosctl patch machineconfig --nodes "$ip" --patch @longhorn-user-disk.yaml done ``` ### 1.4: Reboot the nodes Reboot each worker node to apply the configuration changes: ```bash theme={null} for ip in "${WORKER_IPS[@]}"; do echo "Rebooting node: $ip" talosctl reboot --nodes "$ip" done ``` ## Step 2: Enable privileged pod security Longhorn requires privileged containers to manage disks and mount volumes. Talos enables the [Pod Security Admission Policies](https://docs.siderolabs.com/kubernetes-guides/security/pod-security#override-the-pod-security-admission-configuration) by default, which blocks the privileged containers. Create the `longhorn-system` namespace and label it to allow privileged workloads: ```bash theme={null} kubectl create namespace longhorn-system kubectl label namespace longhorn-system pod-security.kubernetes.io/enforce=privileged ``` ## Step 3: Install Longhorn with Helm Add the Longhorn Helm repository: ```bash theme={null} helm repo add longhorn https://charts.longhorn.io helm repo update ``` Install Longhorn: ```bash theme={null} helm install longhorn longhorn/longhorn \ --namespace longhorn-system \ --set defaultSettings.defaultDataPath=/var/mnt/longhorn ``` This configures Longhorn to store replicas at `/var/mnt/longhorn`, which matches the disk path created in **Step 1**. Wait for the deployment to complete: ```bash theme={null} kubectl -n longhorn-system rollout status deploy/longhorn-driver-deployer kubectl get pods -n longhorn-system ``` ## Step 4: Verify the installation Check that the Longhorn nodes are registered: ```bash theme={null} kubectl get nodes.longhorn.io -n longhorn-system ``` You should see one entry per storage node, similar to: ``` NAME READY ALLOWSCHEDULING SCHEDULABLE AGE worker-01 True true True 2m ``` Each storage node should appear with `SCHEDULABLE` set to `true` and at least one disk detected. The V2 Data Engine runs Longhorn on top of [SPDK](https://spdk.io) for significantly higher throughput and lower latency than V1. It uses NVMe-over-TCP instead of the iSCSI stack, so each storage node needs three things V1 does not: a **raw, unformatted block device**, huge pages, and a couple of kernel modules. ## Step 1: Provision a raw block device and enable the V2 prerequisites Unlike V1, the V2 Data Engine stores data on unformatted `block` devices. On Talos you allocate one with RawVolumeConfig, which carves an unformatted partition out of a disk and exposes it at a stable path. You also need huge pages and the SPDK kernel modules, enabled through the machine configuration. ### 1.1: Inspect available disks Inspect the disks available on each storage node. ```bash theme={null} talosctl get disks --nodes ``` Use this output to determine which disks should be used for Longhorn storage. ### 1.2: Create the RawVolumeConfig Create a RawVolumeConfig document to tell Talos which disk to carve the raw partition from. Talos leaves the partition unformatted and exposes it at the stable symlink `/dev/disk/by-partlabel/r-longhorn` (the `r-` prefix plus the volume name form the partition label). Adjust based on your hardware and the disk you want to use. The example below uses the first NVMe disk that is not a system disk. ```bash theme={null} cat < longhorn-raw-disk.yaml apiVersion: v1alpha1 kind: RawVolumeConfig name: longhorn provisioning: diskSelector: match: disk.transport == 'nvme' && !system_disk EOF ``` The `diskSelector.match` field accepts Common Expression Language (CEL) expressions. You can target disks more precisely using expressions such as `disk.size > 50GB` or `disk.model == "Samsung SSD 980"`. Run `talosctl get disks` to see the available fields for your hardware. For more information, refer to the Disk Management documentation. ### 1.3: Enable huge pages and the SPDK kernel modules Create a second machine configuration patch that reserves huge pages and loads the kernel modules required by SPDK. ```bash theme={null} cat < longhorn-v2-engine.yaml machine: sysctls: vm.nr_hugepages: "1024" kernel: modules: - name: nvme_tcp - name: vfio_pci - name: uio_pci_generic - name: ublk_drv EOF ``` ### 1.4: Apply the configuration to storage nodes Define the IP addresses of the worker nodes that will provide storage to Longhorn. Add one entry per worker node that will provide storage to Longhorn: ```bash theme={null} WORKER_IPS=("" "") ``` Then apply both patches to each node: ```bash theme={null} for ip in "${WORKER_IPS[@]}"; do echo "Applying patches to worker node: $ip" talosctl patch machineconfig --nodes "$ip" \ --patch @longhorn-raw-disk.yaml \ --patch @longhorn-v2-engine.yaml done ``` ### 1.5: Reboot the nodes Reboot each worker node to apply the configuration changes: ```bash theme={null} for ip in "${WORKER_IPS[@]}"; do echo "Rebooting node: $ip" talosctl reboot --nodes "$ip" done ``` After the nodes come back up, confirm the huge pages were allocated: ```bash theme={null} talosctl read /proc/meminfo --nodes | grep -i hugepages_total ``` You should see `HugePages_Total: 1024`. Then confirm Kubernetes is exposing the huge pages as a schedulable resource: ```bash theme={null} kubectl describe node | grep "hugepages-2Mi:" ``` You should see ```bash theme={null} hugepages-2Mi: 2Gi hugepages-2Mi: 2Gi ``` Then confirm the raw partition was provisioned (note the `r-` prefix on the volume name): ```bash theme={null} talosctl get volumestatus r-longhorn --nodes ``` The volume should report `PHASE` `ready`. The `LOCATION` column shows the underlying device that the `/dev/disk/by-partlabel/r-longhorn` symlink points to. ## Step 2: Enable privileged pod security Longhorn requires privileged containers to manage disks and mount volumes. Talos enables the [Pod Security Admission Policies](https://docs.siderolabs.com/kubernetes-guides/security/pod-security#override-the-pod-security-admission-configuration) by default, which blocks the privileged containers. Create the `longhorn-system` namespace and label it to allow privileged workloads: ```bash theme={null} kubectl create namespace longhorn-system kubectl label namespace longhorn-system pod-security.kubernetes.io/enforce=privileged ``` ## Step 3: Install Longhorn with Helm Add the Longhorn Helm repository: ```bash theme={null} helm repo add longhorn https://charts.longhorn.io helm repo update ``` Install Longhorn with the V2 Data Engine enabled: ```bash theme={null} helm install longhorn longhorn/longhorn \ --namespace longhorn-system \ --set defaultSettings.v1DataEngine=false \ --set defaultSettings.v2DataEngine=true \ --set defaultSettings.createDefaultDiskLabeledNodes=true \ --set persistence.dataEngine=v2 ``` This turns on the SPDK-based V2 Data Engine. Setting `createDefaultDiskLabeledNodes=true` stops Longhorn from creating a default *filesystem* disk on every node, so the V2 engine schedules only onto the dedicated block device you register in **Step 4**. Wait for the deployment to complete: ```bash theme={null} kubectl -n longhorn-system rollout status deploy/longhorn-driver-deployer kubectl get pods -n longhorn-system ``` ## Step 4: Add a block-type disk and verify Enabling the V2 Data Engine does not automatically register the raw partition with Longhorn. Add it to each Longhorn node as a `block`-type disk, pointing at the stable symlink created in **Step 1**. ### 4.1: Add the raw block device to each node The Longhorn node resource is named after the Kubernetes node. For each storage node, set two things: * a **label** (`node.longhorn.io/create-default-disk=config`) that tells Longhorn to build its default disk from a custom configuration instead of auto-detecting one, and * an **annotation** (`node.longhorn.io/default-disks-config`) that points at the raw block device created in **Step 1**. List your storage nodes, then apply both to each one: ```bash theme={null} WORKER_NODES=("worker-01" "worker-02" "worker-03") for node in "${WORKER_NODES[@]}"; do echo "Adding block-type disk to Longhorn node: $node" kubectl label node "$node" \ node.longhorn.io/create-default-disk="config" kubectl annotate node "$node" \ node.longhorn.io/default-disks-config='[{"name":"longhorn-v2","path":"/dev/disk/by-partlabel/r-longhorn","diskType":"block","allowScheduling":true}]' done ``` Since Longhorn v1.11, block disks that already contain a filesystem or partition table are rejected to prevent data loss. A `RawVolumeConfig` partition is unformatted, so it is accepted as-is, no wiping required. ### 4.2: Verify the disks were accepted Confirm that the V2 instance-manager pods are running: ```bash theme={null} kubectl -n longhorn-system get pods -l longhorn.io/component=instance-manager ``` Check that the Longhorn nodes are registered: ```bash theme={null} kubectl get nodes.longhorn.io -n longhorn-system ``` You should see one entry per storage node, similar to: ``` NAME READY ALLOWSCHEDULING SCHEDULABLE AGE worker-01 True true True 2m ``` Each storage node should appear with `SCHEDULABLE` set to `true`. Inspect a node to confirm the block disk was accepted: ```bash theme={null} kubectl -n longhorn-system get nodes.longhorn.io worker-01 \ -o jsonpath="{.status.diskStatus['longhorn-v2'].conditions}" ``` The disk's `Ready` and `Schedulable` conditions should report `status: "True"`. This can take a few seconds after the node is labeled and annotated, so wait a moment and re-run the command if you see `status: "False"`. ## Step 5: Create a test persistent volume Create a test PersistentVolumeClaim. ```bash theme={null} kubectl apply -f - <