Talos
Omni
Kubernetes GuidesThis is an experimental feature. It does not support Talos installations with custom built Linux kernel or custom built extensions.
Prerequisites
Before you begin you will need the following:CLI utilities
You will needomnictl installed and configured to follow this guide.
You might also benefit from having talosctl, to discover Talos nodes and monitor the state of the Talos cluster.
Network access
omnictl needs to be able to reach your Omni instance, Image Factory (Omni uses the default Image Factory if not configured otherwise) and Talos nodes over the network.
Also, your Omni instance needs to be able to reach the Talos nodes over the network. If your nodes are behind a firewall, in a private network, or otherwise not directly reachable, you would need to configure a load balancer to forward TCP port 50000 to reach the nodes for Talos API access.
Authorization
You will need to haveos:admin role for the Talos cluster you want to import.
You will need Omni role Operator or above in Omni to be able to create the necessary resources.
Cluster Import
omnictl cluster import command has several key flags that, if omitted, are automatically inferred from the connected cluster or the local environment. These flags can be seen below. Note that this list doesn’t include all possible flags.
omnictl cluster import with the --dry-run flag:
--dry-run flag:
- Discover the talos cluster state using provided/discovered information and build an import context to be used during the import.
- Validate the cluster state to ensure it is healthy and ready to be imported.
- Save a backup of the current machine configurations to a zip file.
- Connect the Talos nodes to Omni by applying necessary configurations to the nodes.
- Create the necessary resources in Omni to represent the imported cluster.
locked. This is to prevent any changes being done by Omni to the imported cluster. Cluster locking is an Omni feature that prevents any changes from being applied to the Talos cluster.
Image Schematic
With Talos Linux Image Factory, it’s possible to create a talos installer using extensions, extra kernel command line arguments as well as other settings. Such configuration is referred to as a Schematic. Cluster import command tries to detect if a Talos machine was configured using a boot asset from an Image Factory. If so, it will ensure that the same Schematic is used with the Image Factory Omni is connected to. If a Talos machine is not configured using an Image Factory, then the validation step will fail by reporting that the Schematic can’t be found. However, this doesn’t necessarily mean that the cluster can’t be imported to Omni. It’s still possible to continue with cluster import in such a scenario by following one option:- Create a Schematic in Omni’s Image Factory that matches the Talos machine configuration and use the provided image to upgrade the Talos machine.
- Re-run the import command with
--forceflag to skip the validations.
--force option was used, Omni will not be able to ensure that it can successfully offer upgrades for these Talos machines while keeping their existing configuration.
Omni will still be able to offer upgrades for these machines, but it will create and use a new Schematic to do so.
If a Talos machine was not configured using an Image Factory asset, the validation step will fail by reporting that the Schematic can’t be found.
This will not affect machines configured by using a boot asset from Talos releases.
Config Patches
When importing a cluster into Omni, the goal is to preserve the cluster’s existing state and avoid modifying node configurations unless absolutely necessary. Cluster Machine config patches are used to achieve this idea. The import command uses initial talos and kubernetes versions to build the default machine config for each node in the cluster. These configs are then compared to the current machine config on each node to build a config patch. Each config patch includes the changes made to the machine config since the Talos cluster was created. Certain machine config fields are not allowed to be set on Omni managed clusters. These fields are not included in the final config patch. After a cluster is imported to Omni, the final machine config for each node is built by applying the config patch to the default machine config built using the initial talos and kubernetes versions used on cluster import. Redacted version of this machine config can be found under theConfig tab of the Cluster Machine overview page in the Omni UI, while config patches can be found under Patches tab of the same overview page.
Unlocking the Cluster
Since Omni uses its own Kubernetes apiserver endpoint, when the cluster is unlocked, Omni will updatekube-apiserver, kube-scheduler and kube-controller-manager static pod definitions to point to the Omni-managed kubernetes apiserver endpoint.
Aborting an ongoing Import
Import operation can be aborted usingcluster import abort command. This command will delete all the resources created by Omni while importing the cluster and will also temporarily remove Omni connection from the Talos nodes.
Unlike cluster deletion, aborting an import will not reset the machines to their initial state. Cluster will just stop being managed by Omni.
Cluster has to be in locked state to be able to abort an import operation.
To abort an ongoing import, use the command below:
- Using backed-up machine configs with the
talosctl apply-configcommand. Note that if the cluster was unlocked and later further modified by the User, its state might not be represented correctly in the backed-up machine configs. - Using config patches tailored for each node to manually revert the changes made during the import step using
talosctl patch machineconfigcommand. You can find a config patch snippet below, aiming to remove partial documents created during the import process.