Cluster Templates

Omni parses, validates, and converts cluster templates into Omni resources. It then creates or updates these resources via the Omni API. Omni guarantees backward compatibility for cluster templates, so you can use the same template with any future version of Omni. Store all referenced files in machine configuration patches relative to the current working directory.

Structure

A cluster template is a YAML file consisting of multiple documents. Each document contains a kind field that specifies the document type. Some documents also include a name field that specifies the document ID.

kind: Cluster
name: example
labels:
  my-label: my-value
kubernetes:
  version: v1.26.0
talos:
  version: v1.3.2
features:
  diskEncryption: true
patches:
  - name: kubespan-enabled
    inline:
      machine:
        network:
          kubespan:
            enabled: true
systemExtensions:
  - siderolabs/hello-world-service
kernelArgs:
  - talos.dashboard.disabled=1
---
kind: ControlPlane
machines:
  - 27c16241-96bf-4f17-9579-ea3a6c4a3ca8
  - 4bd92fba-998d-4ef3-ab43-638b806dd3fe
  - 8fdb574a-a252-4d7d-94f0-5cdea73e140a
---
kind: Workers
machines:
  - b885f565-b64f-4c7a-a1ac-d2c8c2781373
  - a54f21dc-6e48-4fc1-96aa-3d7be5e2612b
---
kind: Workers
name: xlarge
machines:
  - 1f721dee-6dbb-4e71-9832-226d73da3841
systemExtensions:
  - siderolabs/hello-world-service
---
kind: Machine
name: 27c16241-96bf-4f17-9579-ea3a6c4a3ca8
---
kind: Machine
name: 4bd92fba-998d-4ef3-ab43-638b806dd3fe
install:
  disk: /dev/vda
---
kind: Machine
name: 8fdb574a-a252-4d7d-94f0-5cdea73e140a
install:
  disk: /dev/vda
---
kind: Machine
name: b885f565-b64f-4c7a-a1ac-d2c8c2781373
install:
  disk: /dev/vda
systemExtensions:
  - siderolabs/hello-world-service
---
kind: Machine
name: a54f21dc-6e48-4fc1-96aa-3d7be5e2612b
locked: true
install:
  disk: /dev/vda
---
kind: Machine
name: 1f721dee-6dbb-4e71-9832-226d73da3841
install:
  disk: /dev/vda
kernelArgs:
  - net.ifnames=0
  - talos.dashboard.disabled=1

Each cluster template must have exactly one document of kind: Cluster, kind: ControlPlane, and any number of kind: Workers with different names. Either a ControlPlane or Workers document must reference every Machine document.

Document types

The following sections describe the specific document kinds available in a cluster template and their configurations.

`Cluster`

The Cluster document specifies the cluster configuration and labels. It defines the cluster name and base component versions.

kind: Cluster
name: example
labels:
  my-label: my-value
annotations:
  my-annotation: my-value
kubernetes:
  version: v1.26.1
talos:
  version: v1.3.3
features:
  enableWorkloadProxy: true
  diskEncryption: true
  backupConfiguration:
    interval: 1h
patches:
  - file: patches/example-patch.yaml
systemExtensions:
  - siderolabs/hello-world-service
kernelArgs:
  - talos.dashboard.disabled=1

Field	Type	Description
`kind`	string	`Cluster`
`name`	string	Cluster name: only letters, digits and `-` and `_` are allowed. The cluster name is used as a key by all other documents, so if the cluster name changes, Omni creates a new cluster.
`labels`	map[string]string	Labels to be applied to the cluster.
`annotations`	map[string]string	Annotations to be applied to the cluster.
`kubernetes.version`	string	Kubernetes version to use, `vA.B.C`.
`talos.version`	string	Talos version to use, `vA.B.C`.
`features.enableWorkloadProxy`	boolean	Whether to enable the workload proxy feature. Defaults to `false`.
`features.useEmbeddedDiscoveryService`	boolean	Whether to use the embedded discovery service that runs inside the Omni instance instead of the public one (`discovery.talos.dev`). Defaults to `false`. It is only valid if the Omni instance has the feature enabled.
`features.diskEncryption`	boolean	Whether to enable disk encryption. Defaults to `false`.
`features.backupConfiguration.interval`	string	Cluster etcd backup interval. Must be a valid Go duration. Zero `0` disables automatic backups.
`patches`	array[Patch]	List of patches to apply to the cluster.
`systemExtensions`	array[string]	The list of system extensions to be installed on every machine in the cluster.
`kernelArgs`	array[string]	The list of kernel args to be set on each machine in this cluster.

`ControlPlane`

The ControlPlane document specifies the control plane configuration. It defines the number of control plane nodes and the list of machines to use. Because control plane machines run an etcd cluster, use a number of machines that can achieve a stable quorum (e.g., 1, 3, 5). Changing the set of machines in the control plane triggers a rolling scale-up or scale-down. Configure the control plane with at least a single machine. For high availability, SideroLabs recommends using at least 3 machines.

kind: ControlPlane
labels:
  my-label: my-value
annotations:
  my-annotation: my-value
machines:
  - 27c16241-96bf-4f17-9579-ea3a6c4a3ca8
  - 4bd92fba-998d-4ef3-ab43-638b806dd3fe
  - 8fdb574a-a252-4d7d-94f0-5cdea73e140a
patches:
  - file: patches/example-controlplane-patch.yaml
systemExtensions:
  - siderolabs/hello-world-service

Field	Type	Description
`kind`	string	`ControlPlane`
`labels`	map[string]string	Labels to be applied to the control plane machine set.
`annotations`	map[string]string	Annotations to be applied to the control plane machine set.
`machines`	array[string]	List of machine IDs to use for control plane nodes (mutually exclusive with `machineClass`).
`bootstrapSpec.clusterUUID`	string	UUID of the cluster to restore from (required when restoring a cluster).
`bootstrapSpec.snapshot`	string	Snapshot file name to restore from (required when restoring a cluster).
`patches`	array[Patch]	List of patches to apply to the machine set.
`machineClass`	MachineClass	Machine Class configuration (mutually exclusive with `machines`).
`systemExtensions`	array[string]	The list of system extensions to be installed on every machine in the machine set.
`kernelArgs`	array[string]	The list of kernel args to be set on each machine in this machine set.

`Workers`

The Workers document specifies the worker configuration. It defines the number of worker nodes and the list of machines to use.

kind: Workers
name: workers
labels:
  my-label: my-value
annotations:
    my-annotation: my-value
machines:
  - b885f565-b64f-4c7a-a1ac-d2c8c2781373
updateStrategy:
  rolling:
    maxParallelism: 3
deleteStrategy:
  type: Rolling
  rolling:
    maxParallelism: 5
patches:
  - file: patches/example-workers-patch.yaml
systemExtensions:
  - siderolabs/hello-world-service

Field	Type	Description
`kind`	string	`Workers`
`name`	string	Worker machine set name: only letters, digits and `-` and `_` are allowed. Defaults to `workers` when omitted. Must be unique and not be `control-planes`.
`labels`	map[string]string	Labels to be applied to the worker machine set.
`annotations`	map[string]string	Annotations to be applied to the worker machine set.
`machines`	array[string]	List of machine IDs to use as worker nodes in the machine set (mutually exclusive with `machineClass`).
`patches`	array[Patch]	List of patches to apply to the machine set.
`machineClass`	MachineClass	Machine Class configuration (mutually exclusive with `machines`).
`updateStrategy`	UpdateStrategy	Update strategy for the machine set. Defaults to `type: Rolling` with `maxParallelism: 1`.
`deleteStrategy`	UpdateStrategy	Delete strategy for the machine set. Defaults to `type: Unset`.
`systemExtensions`	array[string]	The list of system extensions to be installed on every machine in the machine set.
`kernelArgs`	array[string]	The list of kernel args to be set on each machine in this machine set.

`MachineClass`

The MachineClass section of ControlPlane or Workers defines the rule for picking the machines in the machine set.

kind: Workers
name: workers
machineClass:
  name: worker-class
  size: 2

Field	Type	Description
`name`	string	Name of the machine class to use.
`size`	number \| string	Number of machines to pick from the matching machine class, or a keyword (`unlimited`, `infinity`, `∞`).

The size field supports keyword unlimited|infinity, which makes the machine set pick all available machines from the specified machine class.

`UpdateStrategy`

The UpdateStrategy section of Workers defines the update or delete strategy for the machine set.

kind: Workers
name: workers
updateStrategy:
  rolling:
    maxParallelism: 3
deleteStrategy:
  type: Rolling
  rolling:
    maxParallelism: 5

Field	Type	Description
`type`	string	Strategy type. Can be `Rolling` or `Unset`. Defaults to `Rolling` for `updateStrategy` and `Unset` for the `deleteStrategy`. When `Unset`, all updates and/or deletes are applied at once.
`rolling.maxParallelism`	number	Maximum number of machines to update and/or delete in parallel. Only used when the `type` is `Rolling`. Defaults to `1`.

`Machine`

The Machine document specifies the install disk and machine-specific configuration patches. They are optional, but either a ControlPlane or Workers document must reference every Machine document.

kind: Machine
name: 27c16241-96bf-4f17-9579-ea3a6c4a3ca8
labels:
  my-label: my-value
annotations:
  my-annotation: my-value
locked: false
install:
  disk: /dev/vda
patches:
  - file: patches/example-machine-patch.yaml
systemExtensions:
  - siderolabs/hello-world-service

Field	Type	Description
`kind`	string	`Machine`
`name`	string	Machine ID.
`labels`	map[string]string	Labels to be applied to the machine set node.
`annotations`	map[string]string	Annotations to be applied to the machine set node.
`locked`	boolean	Whether the machine should be marked as locked. Can be `true` only if the machine is used as a worker.
`install.disk`	string	Disk to install Talos on. Matters only for Talos running from ISO or iPXE.
`patches`	array[Patch]	List of patches to apply to the machine.
`systemExtensions`	array[string]	The list of system extensions to be installed on the machine.
`kernelArgs`	array[string]	The list of kernel args to be set on this machine.

When Talos is not installed and you do not specify the install disk, Omni tries to pick the install disk automatically. It finds the smallest disk larger than 5GB.

Common fields

The following configuration fields are shared across multiple document types.

`patches`

The patches field lists machine configuration patches to apply to a cluster, a machine set, or an individual machine. Config patches modify the configuration before it is applied to each machine in the cluster. Changing configuration patches modifies the machine configuration, which gets automatically applied to the machine.

patches:
  - file: patches/example-patch.yaml
  - name: kubespan-enabled
    inline:
      machine:
        network:
          kubespan:
            enabled: true
  - idOverride: 950-set-env-vars
    labels:
      my-label: my-value
    annotations:
      my-annotation: my-value
    inline:
      machine:
        env:
          MY_ENV_VAR: my-value

Field	Type	Description
`file`	string	Path to the patch file. Path is relative to the current working directory when executing `omnictl`. File should contain Talos machine configuration strategic patch.
`name`	string	Name of the patch. Required for `inline` patches when `idOverride` is not set, optional for `file` patches (default name will be based on the file path).
`idOverride`	string	Override the config patch ID, so it won’t be generated from the `name` or `file`.
`labels`	map[string]string	Labels to be applied to the config patch.
`annotations`	map[string]string	Annotations to be applied to the config patch.
`inline`	object	Inline patch containing Talos machine configuration strategic patch.

A configuration patch may be either inline or file based. Inline patches are useful for small changes, while file-based patches are useful for more complex changes or changes shared across multiple clusters.

`kernelArgs`

You can define kernel args at the cluster, machine set, or individual machine level. Management of kernel args by cluster templates is opt-in: the templates manage them ONLY IF you specify them and they are not null. Otherwise, their lifecycle remains outside of cluster template management. When defined at multiple levels, the lowest/most specific level takes precedence. For example, a kernel arg defined at the machine level overrides the same kernel arg defined at the cluster or machine set level. These args are only supported for “static” machines. If a machine is created from a machine class, kernel args management is not supported. This means setting them in a ControlPlane or Workers document that uses machineClass, or in a Cluster document that contains machine sets based on machineClass, will result in an error.

kernelArgs:
  - talos.dashboard.disabled=1
  - net.ifnames=0

Overview

Getting Started

Infrastructure and Extensions

Omni Cluster Setup

Cluster Management

Security and Authentication

Reference

Troubleshooting and Support

Structure

Document types

`Cluster`

`ControlPlane`

`Workers`

`MachineClass`

`UpdateStrategy`

`Machine`

Common fields

`patches`

`kernelArgs`

Overview

Getting Started

Infrastructure and Extensions

Omni Cluster Setup

Cluster Management

Security and Authentication

Reference

Troubleshooting and Support

​Structure

​Document types

​Cluster

​ControlPlane

​Workers

​MachineClass

​UpdateStrategy

​Machine

​Common fields

​patches

​kernelArgs

Structure

Document types

`Cluster`

`ControlPlane`

`Workers`

`MachineClass`

`UpdateStrategy`

`Machine`

Common fields

`patches`

`kernelArgs`