IRSA with Talos Linux

This guide explains how to implement IAM Roles for Service Accounts (IRSA) for a Kubernetes cluster on Talos Linux. This allows Pods to access AWS services using temporary IAM credentials, similar to how EKS clusters in AWS function. While this example uses AWS S3 and Terraform, the general principles can be applied to any S3-compatible object store and adapted for other infrastructure-as-code tools or a manual setup. For the purpose of this guide, we will create a generic, read-only IRSA Role to test access to S3 from Talos Linux after the IRSA setup is complete. Administrators will need to create their own IAM Roles that target the specific AWS services they need to access from their Kubernetes Pods.

Prerequisites

Before you begin, you will need:

A Kubernetes cluster running on Talos Linux.
An AWS account with permissions to create S3 buckets, IAM roles, and OIDC providers.
git, ssh-keygen, go, kubectl, helm, and aws-cli installed locally.
Terraform (optional, for the examples provided).

This guide is based on the official instructions for setting up the Amazon EKS Pod Identity Webhook in a self-hosted environment.

Step 1: Set Up Environment Variables

First, set the following environment variables. These will be used throughout the guide.

export AWS_ACCOUNT="<YOUR_AWS_ACCOUNT_ID>"
export AWS_REGION="<YOUR_AWS_REGION>"
export NAMESPACE="default"
export PKCS_KEY="sa-signer-pkcs8.pub"
export PRIV_KEY="sa-signer.key"
export PUB_KEY="sa-signer.key.pub"
export S3_BUCKET="<YOUR_UNIQUE_S3_BUCKET_NAME>"
export ISSUER_HOSTPATH="https://${S3_BUCKET}.s3.amazonaws.com"
export SERVICEACCOUNT_NAME="talos-irsa-s3-readonly"

Below is a description of each environment variable used in this guide.

OIDC Hosting Variables

AWS_ACCOUNT: Your 12-digit AWS account ID. Need help finding it?
AWS_REGION: The AWS region where you will create resources. Need help choosing one?
S3_BUCKET: A globally unique name for the S3 bucket that will host the OIDC discovery documents.
ISSUER_HOSTPATH: The public URL of the OIDC issuer (for example, https://${S3_BUCKET}.s3.amazonaws.com).

Token Signing Key Variables

PKCS_KEY: The PKCS#8-formatted public key file.
PRIV_KEY: The private key file (RSA) used for signing.
PUB_KEY: The public key file (RSA) corresponding to the private key.

IRSA Test Variables

NAMESPACE: The Kubernetes namespace for the ServiceAccount.
SERVICEACCOUNT_NAME: The name of the Kubernetes ServiceAccount.

Step 2: Create a Projected Token Signing Keypair

The projected Service Account tokens require a signing keypair.

Clone the amazon-eks-pod-identity-webhook repository to get a helper script for creating the keys document:

git clone https://github.com/aws/amazon-eks-pod-identity-webhook
cd amazon-eks-pod-identity-webhook

Generate an RSA keypair. These files will be used in later steps.

# Generate the keypair
ssh-keygen -t rsa -b 4096 -f ${PRIV_KEY} -m pem
# Convert the SSH pubkey to PKCS8 format
ssh-keygen -e -m PKCS8 -f ${PUB_KEY} > ${PKCS_KEY}

Step 3: Create OIDC Discovery Endpoint

An OIDC discovery endpoint consists of a public S3 bucket containing the JSON Web Key Set (JWKS) and an OpenID provider configuration document.

Create a Public S3 Bucket

The following Terraform example creates a publicly readable S3 bucket. You will need to provide the bucket name you defined in the $S3_BUCKET environment variable to Terraform (e.g., via a .tfvars file or command-line flag).

variable "s3_bucket_name" {
  description = "The name of the S3 bucket for OIDC discovery."
  type        = string
}

resource "aws_s3_bucket" "talos_irsa" {
  bucket = var.s3_bucket_name
}

resource "aws_s3_bucket_ownership_controls" "talos_irsa" {
  bucket = aws_s3_bucket.talos_irsa.id
  rule {
    object_ownership = "BucketOwnerPreferred"
  }
}

resource "aws_s3_bucket_public_access_block" "talos_irsa" {
  bucket = aws_s3_bucket.talos_irsa.id

  block_public_acls       = false
  block_public_policy     = false
  ignore_public_acls      = false
  restrict_public_buckets = false
}

resource "aws_s3_bucket_acl" "talos_irsa" {
  depends_on = [
    aws_s3_bucket_ownership_controls.talos_irsa,
    aws_s3_bucket_public_access_block.talos_irsa,
  ]

  bucket = aws_s3_bucket.talos_irsa.id
  acl    = "public-read"
}

After creating the S3 bucket, you will create and upload the OIDC discovery documents.

Create and Upload OIDC Documents

Generate the keys.json document using the Go tool from the amazon-eks-pod-identity-webhook repository. This command should be run from within the cloned repository directory.
go run ./hack/self-hosted/main.go -key ${PKCS_KEY} | jq '.keys += [.keys[0]] | .keys[1].kid = ""' > keys.json

Create the OIDC discovery document, named openid-configuration:

cat << EOF > openid-configuration
{
    "issuer": "${ISSUER_HOSTPATH}",
    "jwks_uri": "${ISSUER_HOSTPATH}/keys.json",
    "authorization_endpoint": "urn:kubernetes:programmatic_authorization",
    "response_types_supported": [
        "id_token"
    ],
    "subject_types_supported": [
        "public"
    ],
    "id_token_signing_alg_values_supported": [
        "RS256"
    ],
    "claims_supported": [
        "sub",
        "iss"
    ]
}
EOF

Upload both files to your public S3 bucket:

aws s3 cp keys.json s3://${S3_BUCKET}/keys.json
aws s3 cp openid-configuration s3://${S3_BUCKET}/.well-known/openid-configuration

Step 4: Create OIDC Provider and S3 Read-Only IAM Role in AWS

Next, create the IAM OIDC identity provider in AWS and an IAM role that your Kubernetes Service Account can assume. The following Terraform example creates these resources. It assumes you are passing the environment variables you set earlier as Terraform variables (e.g., s3_bucket_name, aws_account_id, etc.).

variable "s3_bucket_name" {
  type = string
}
variable "aws_account_id" {
  type = string
}
variable "namespace" {
  type = string
}
variable "service_account_name" {
  type = string
}

data "tls_certificate" "open_id_connect_talos" {
  url = "https://${var.s3_bucket_name}/.well-known/openid-configuration"
}

resource "aws_iam_openid_connect_provider" "talos" {
  url             = "https://${var.s3_bucket_name}"
  client_id_list  = ["sts.amazonaws.com"]
  thumbprint_list = [data.tls_certificate.open_id_connect_talos.certificates[0].sha1_fingerprint]
}

resource "aws_iam_role" "talos_irsa_s3_readonly" {
  name = var.service_account_name
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRoleWithWebIdentity"
        Effect = "Allow"
        Sid    = ""
        Principal = {
          Federated = "arn:aws:iam::${var.aws_account_id}:oidc-provider/${var.s3_bucket_name}"
        }
        Condition = {
          StringEquals = {
            "${var.s3_bucket_name}:aud": "sts.amazonaws.com",
            "${var.s3_bucket_name}:sub": "system:serviceaccount:${var.namespace}:${var.service_account_name}"
          }
        }
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "talos_irsa_s3_readonly" {
  role       = aws_iam_role.talos_irsa_s3_readonly.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess"
}

Note: For the purpose of this guide, we are attaching the managed AmazonS3ReadOnlyAccess policy to demonstrate IRSA functionality. In a production environment, you should create and attach a custom IAM policy with the least privileges necessary for your specific application’s needs.

Step 5: Configure Talos `machineconfig`

Patch your Talos machineconfig to use the new Service Account issuer and signing key.

Generate a patch file from a template. First, encode the private key:
ENCODED_KEY=$(cat ${PRIV_KEY} | base64 | tr -d '\n')
Note: The tr -d '\n' is used for portability between GNU and BSD base64.

Create the patch file, machineconfig-patch.yaml:

cat <<EOF > machineconfig-patch.yaml
cluster:
  apiServer:
    extraArgs:
      service-account-issuer: ${ISSUER_HOSTPATH}
  serviceAccount:
    key: ${ENCODED_KEY}
EOF

Apply the patch to your Talos configuration and update your cluster. For example, using talosctl:
talosctl apply-config --nodes <NODE_IP> --file machineconfig-patch.yaml
Export your kubeconfig and talosconfig for the updated cluster.

Step 6: Install Required Kubernetes Components

Two components are required on the cluster: cert-manager and amazon-eks-pod-identity-webhook.

Install `cert-manager`

Add the Jetstack Helm repository:

helm repo add jetstack https://charts.jetstack.io
helm repo update

Install the cert-manager Helm chart:

helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --set crds.enabled=true \
  --create-namespace

Next, install the amazon-eks-pod-identity-webhook.

Install `amazon-eks-pod-identity-webhook`

Add the jkroepke Helm repository:

helm repo add jkroepke https://jkroepke.github.io/helm-charts/
helm repo update

Install the amazon-eks-pod-identity-webhook Helm chart:

helm install amazon-eks-pod-identity-webhook jkroepke/amazon-eks-pod-identity-webhook \
  --namespace kube-system \
  set config.defaultAwsRegion=${AWS_REGION}

Step 7: Test AWS S3 Access

Finally, deploy a Pod with the configured Service Account to test access to AWS S3.

Create a manifest for the ServiceAccount and a test Pod. The command below uses a heredoc to create test-pod.yaml and substitute the shell variables.

cat <<EOF > test-pod.yaml
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: ${SERVICEACCOUNT_NAME}
  namespace: ${NAMESPACE}
  annotations:
    eks.amazonaws.com/role-arn: "arn:aws:iam::${AWS_ACCOUNT}:role/${SERVICEACCOUNT_NAME}"
    eks.amazonaws.com/sts-regional-endpoints: "true"
    eks.amazonaws.com/token-expiration: "86400"
---
apiVersion: v1
kind: Pod
metadata:
  namespace: ${NAMESPACE}
  name: aws-cli
spec:
  serviceAccountName: ${SERVICEACCOUNT_NAME}
  containers:
    - name: aws-cli
      image: amazon/aws-cli:latest
      command: ["sleep", "infinity"]
EOF

Apply the manifest:
kubectl apply -f test-pod.yaml
Exec into the aws-cli Pod.and test access by listing S3 buckets:
kubectl exec -it aws-cli -n ${NAMESPACE} -- aws s3 ls
This command should list the S3 buckets in your AWS account, confirming that IRSA is correctly configured.

Overview

Getting Started

Platform specific installation

Deploying and managing workloads

Networking

Security

Build and extend Talos

Configure your Talos cluster

Advanced guides

Reference

Troubleshooting and support

Learn more

Prerequisites

Step 1: Set Up Environment Variables

OIDC Hosting Variables

Token Signing Key Variables

IRSA Test Variables

Step 2: Create a Projected Token Signing Keypair

Step 3: Create OIDC Discovery Endpoint

Create a Public S3 Bucket

Create and Upload OIDC Documents

Step 4: Create OIDC Provider and S3 Read-Only IAM Role in AWS

Step 5: Configure Talos `machineconfig`

Step 6: Install Required Kubernetes Components

Install `cert-manager`

Install `amazon-eks-pod-identity-webhook`

Step 7: Test AWS S3 Access

​Prerequisites

​Step 1: Set Up Environment Variables

​OIDC Hosting Variables

​Token Signing Key Variables

​IRSA Test Variables

​Step 2: Create a Projected Token Signing Keypair

​Step 3: Create OIDC Discovery Endpoint

​Create a Public S3 Bucket

​Create and Upload OIDC Documents

​Step 4: Create OIDC Provider and S3 Read-Only IAM Role in AWS

​Step 5: Configure Talos machineconfig

​Step 6: Install Required Kubernetes Components

​Install cert-manager

​Install amazon-eks-pod-identity-webhook

​Step 7: Test AWS S3 Access

Prerequisites

Step 1: Set Up Environment Variables

OIDC Hosting Variables

Token Signing Key Variables

IRSA Test Variables

Step 2: Create a Projected Token Signing Keypair

Step 3: Create OIDC Discovery Endpoint

Create a Public S3 Bucket

Create and Upload OIDC Documents

Step 4: Create OIDC Provider and S3 Read-Only IAM Role in AWS

Step 5: Configure Talos `machineconfig`

Step 6: Install Required Kubernetes Components

Install `cert-manager`

Install `amazon-eks-pod-identity-webhook`

Step 7: Test AWS S3 Access