Skip to main content
This guide explains how to implement IAM Roles for Service Accounts (IRSA) for a Kubernetes cluster on Talos Linux. This allows Pods to access AWS services using temporary IAM credentials, similar to how EKS clusters in AWS function. While this example uses AWS S3 and Terraform, the general principles can be applied to any S3-compatible object store and adapted for other infrastructure-as-code tools or a manual setup. For the purpose of this guide, we will create a generic, read-only IRSA Role to test access to S3 from Talos Linux after the IRSA setup is complete. Administrators will need to create their own IAM Roles that target the specific AWS services they need to access from their Kubernetes Pods.

Prerequisites

Before you begin, you will need:
  • A Kubernetes cluster running on Talos Linux.
  • An AWS account with permissions to create S3 buckets, IAM roles, and OIDC providers.
  • Terraform (optional, for the examples provided).
  • kubectl and helm installed locally for testing
This guide is based on the official instructions for setting up the Amazon EKS Pod Identity Webhook in a self-hosted environment. In summary, this is what is needed to accomplish IRSA:
  1. Create a keypair, populate an S3 bucket with publicly-accessible OIDC discovery documents, and register the S3 bucket endpoint as an Identity Provider with AWS IAM.
  2. Pass the private key to the Kubernetes API server. Your Kubernetes API server is now an OIDC issuer!
  3. Run amazon-eks-pod-identity-webhook, which mutates pods based on the annotations to inject environment variables like AWS_WEB_IDENTITY_TOKEN_FILE (and cert-manager, since this webhook per Kubernetes requirements needs to serve HTTPS)
The Kubernetes API server can now sign projected service account tokens using the private key, which gets passed to AWS IAM with the role the pod wishes to assume. IAM then confirms the signature using the registered IdP, ensures the the role assumption is allowed, and returns credentials.

Step 1: Create OIDC serving infrastructure

The following Terraform module creates all the AWS infrastructure needed. You can instantiate this module in your terraform using source, or apply it using a .tfvars file or command-line flags.
variable "bucket_name" {
  type = string
}

variable "cluster_name" {
  type = string
}

variable "secret_prefix" {
  type = string
}

locals {
  issuer_hostpath = "s3.${data.aws_region.current.region}.amazonaws.com/${var.bucket_name}"
  openid_configuration = jsonencode({
    issuer                                = "https://${local.issuer_hostpath}"
    jwks_uri                              = "https://${local.issuer_hostpath}/keys.json"
    authorization_endpoint                = "urn:kubernetes:programmatic_authorization"
    response_types_supported              = ["id_token"]
    subject_types_supported               = ["public"]
    id_token_signing_alg_values_supported = ["RS256"]
    claims_supported                      = ["sub", "iss"]
  })

  # the JWKS format and encodings are defined in the RFC
  # https://datatracker.ietf.org/doc/html/rfc7517
  jwks = jsonencode({
    keys = [
      {
        use = "sig"
        alg = "RS256"
        kty = "RSA"
        kid = data.external.pub_der.result.der
        n   = data.external.modulus.result.modulus
        e   = "AQAB"
    }]
  })
}

data "aws_region" "current" {}
data "aws_caller_identity" "current" {}

# generate a key pair that will be used to sign projected service account tokens
resource "tls_private_key" "key" {
  algorithm = "RSA"
  rsa_bits  = 2048
}

# this needs to go into talos machine configuration under cluster.serviceAccount.key
resource "aws_secretsmanager_secret" "signing_key" {
  name = "${var.secret_prefix}/${var.cluster_name}"
}

resource "aws_secretsmanager_secret_version" "signing_key" {
  secret_id     = aws_secretsmanager_secret.signing_key.id
  secret_string = base64encode(tls_private_key.key.private_key_pem)
}

# bucket that we're using as an OIDC discovery endpoint
resource "aws_s3_bucket" "oidc" {
  bucket = var.bucket_name
}

# registering the public bucket host as an IdP
resource "aws_iam_openid_connect_provider" "oidc" {
  url             = "https://${local.issuer_hostpath}"
  client_id_list  = ["sts.amazonaws.com"]
  thumbprint_list = [data.tls_certificate.oidc.certificates[0].sha1_fingerprint]
}

resource "aws_s3_bucket_ownership_controls" "oidc" {
  bucket = aws_s3_bucket.oidc.id
  rule {
    object_ownership = "BucketOwnerPreferred"
  }
}

# we _want_ this bucket to have publicly accessible files
resource "aws_s3_bucket_public_access_block" "oidc" {
  bucket = aws_s3_bucket.oidc.id

  block_public_acls = false
  block_public_policy = false
  ignore_public_acls = false
  restrict_public_buckets = false
}

# the two files we need to serve
resource "aws_s3_object" "keys_json" {
  bucket  = aws_s3_bucket.oidc.id
  key     = "keys.json"
  content = local.jwks
  acl     = "public-read"

  etag = md5(local.jwks)

  depends_on = [
    aws_s3_bucket_ownership_controls.oidc,
    aws_s3_bucket_public_access_block.oidc,
  ]
}

resource "aws_s3_object" "openid-configuration" {
  bucket  = aws_s3_bucket.oidc.id
  key     = ".well-known/openid-configuration"
  content = local.openid_configuration
  acl     = "public-read"

  etag = md5(local.openid_configuration)

  depends_on = [
    aws_s3_bucket_ownership_controls.oidc,
    aws_s3_bucket_public_access_block.oidc,
  ]
}

data "tls_certificate" "oidc" {
  url = "https://${local.issuer_hostpath}"
}

# This is used for the `kid` Key ID field in the JWKS, which is an arbitrary string that can uniquely
# identify a key.
# This logic comes from https://github.com/kubernetes/kubernetes/pull/78502. It creates unique and
# deterministic outputs across platforms.
# See also https://datatracker.ietf.org/doc/html/rfc4648#section-5 for final base64url encoding
data "external" "pub_der" {
  program = ["bash", "-c", <<EOF
set -euo pipefail
pem=$(jq -r .pem)
der=$(echo "$pem" | openssl pkey -pubin -inform PEM -outform DER | openssl dgst -sha256 -binary | base64 | tr -d '=' | tr '/+' '_-')
jq -n --arg der "$der" '{"der":$der}'
EOF
  ]
  query = { pem = tls_private_key.key.public_key_pem }
}

data "external" "modulus" {
  program = ["bash", "-c", <<EOF
set -euo pipefail
pem=$(jq -r .pem)
modulus=$(echo "$pem" | openssl rsa -inform PEM -modulus -noout | cut -d'=' -f2 | xxd -r -p | base64 | tr -d '=' | tr '/+' '_-')
jq -n --arg modulus "$modulus" '{"modulus":$modulus}'
EOF
  ]
  query = { pem = tls_private_key.key.private_key_pem }
}

Step 2: Configure Talos machineconfig

Patch your Talos machineconfig to use the new Service Account issuer and signing key.
  1. Create the patch file, machineconfig-patch.yaml, fetching BASE64_ENCODED_PRIVATE_KEY from AWS SecretsManager as populated by the above module:
    cat <<EOF > machineconfig-patch.yaml
    cluster:
      apiServer:
        extraArgs:
          service-account-issuer: ${ISSUER_HOSTPATH}
      serviceAccount:
        key: ${BASE64_ENCODED_PRIVATE_KEY}
    EOF
    
  2. Apply the patch to your Talos configuration and update your cluster, and wait for the server to come back up with the new config. For example, using talosctl:
    talosctl apply-config --nodes <NODE_IP> --file machineconfig-patch.yaml
    

Step 3: Install Required Kubernetes Components

Two components are required on the cluster: cert-manager and amazon-eks-pod-identity-webhook.

Install cert-manager

  1. Add the Jetstack Helm repository:
    helm repo add jetstack https://charts.jetstack.io
    helm repo update
    
  2. Install the cert-manager Helm chart:
    helm install cert-manager jetstack/cert-manager \
      --namespace cert-manager \
      --set crds.enabled=true \
      --create-namespace
    
Next, install the amazon-eks-pod-identity-webhook.

Install amazon-eks-pod-identity-webhook

  1. Add the jkroepke Helm repository:
    helm repo add jkroepke https://jkroepke.github.io/helm-charts/
    helm repo update
    
  2. Install the amazon-eks-pod-identity-webhook Helm chart:
    helm install amazon-eks-pod-identity-webhook jkroepke/amazon-eks-pod-identity-webhook \
      --namespace kube-system \
      set config.defaultAwsRegion=${AWS_REGION}
    

Step 4: Test

  1. Apply this terraform (or do manually) to create an AWS role for a test service account to assume that has S3 read access.
    resource "aws_iam_role" "talos_irsa_s3_readonly_example" {
      name = var.service_account_name
      assume_role_policy = jsonencode({
        Version = "2012-10-17"
        Statement = [
          {
            Action = "sts:AssumeRoleWithWebIdentity"
            Effect = "Allow"
            Sid    = ""
            Principal = {
              Federated = "arn:aws:iam::${var.aws_account_id}:oidc-provider/${var.issuer_hostname}"
            }
            Condition = {
              StringEquals = {
                "${var.issuer_hostname}:aud": "sts.amazonaws.com",
                "${var.issuer_hostname}:sub": "system:serviceaccount:${var.namespace}:${var.service_account_name}"
              }
            }
          }
        ]
      })
    }
    
    resource "aws_iam_role_policy_attachment" "talos_irsa_s3_readonly_example" {
      role       = aws_iam_role.talos_irsa_s3_readonly_example.name
      policy_arn = "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess"
    }
    
  2. Create a manifest for the ServiceAccount and a test Pod. The command below uses a heredoc to create test-pod.yaml and substitute the shell variables.
    cat <<EOF > test-pod.yaml
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: ${SERVICEACCOUNT_NAME}
      namespace: ${NAMESPACE}
      annotations:
        eks.amazonaws.com/role-arn: "arn:aws:iam::${AWS_ACCOUNT}:role/${SERVICEACCOUNT_NAME}"
        eks.amazonaws.com/sts-regional-endpoints: "true"
        eks.amazonaws.com/token-expiration: "86400"
    ---
    apiVersion: v1
    kind: Pod
    metadata:
      namespace: ${NAMESPACE}
      name: aws-cli
    spec:
      serviceAccountName: ${SERVICEACCOUNT_NAME}
      containers:
        - name: aws-cli
          image: amazon/aws-cli:latest
          command: ["sleep", "infinity"]
    EOF
    
  3. Apply the manifest:
    kubectl apply -f test-pod.yaml
    
  4. Exec into the aws-cli Pod.and test access by listing S3 buckets:
    kubectl exec -it aws-cli -n ${NAMESPACE} -- aws s3 ls
    
    This command should list the S3 buckets in your AWS account, confirming that IRSA is correctly configured.