Prerequisites
Before you begin, you will need:- A Kubernetes cluster running on Talos Linux.
- An AWS account with permissions to create S3 buckets, IAM roles, and OIDC providers.
- Terraform (optional, for the examples provided).
kubectlandhelminstalled locally for testing
- Create a keypair, populate an S3 bucket with publicly-accessible OIDC discovery documents, and register the S3 bucket endpoint as an Identity Provider with AWS IAM.
- Pass the private key to the Kubernetes API server. Your Kubernetes API server is now an OIDC issuer!
- Run
amazon-eks-pod-identity-webhook, which mutates pods based on the annotations to inject environment variables likeAWS_WEB_IDENTITY_TOKEN_FILE(andcert-manager, since this webhook per Kubernetes requirements needs to serve HTTPS)
Step 1: Create OIDC serving infrastructure
The following Terraform module creates all the AWS infrastructure needed. You can instantiate this module in your terraform usingsource, or apply it using a .tfvars file or command-line flags.
Step 2: Configure Talos machineconfig
Patch your Talos machineconfig to use the new Service Account issuer and signing key.
-
Create the patch file,
machineconfig-patch.yaml, fetchingBASE64_ENCODED_PRIVATE_KEYfrom AWS SecretsManager as populated by the above module: -
Apply the patch to your Talos configuration and update your cluster, and wait for the server to come back up with the new config. For example, using
talosctl:
Step 3: Install required Kubernetes components
Two components are required on the cluster:cert-manager and amazon-eks-pod-identity-webhook.
Install cert-manager
-
Add the Jetstack Helm repository:
-
Install the
cert-managerHelm chart:
amazon-eks-pod-identity-webhook.
Install amazon-eks-pod-identity-webhook
-
Add the
jkroepkeHelm repository: -
Install the
amazon-eks-pod-identity-webhookHelm chart:
Step 4: Test
-
Apply this terraform (or do manually) to create an AWS role for a test service account to assume that has S3 read access.
-
Create a manifest for the
ServiceAccountand a testPod. The command below uses aheredocto createtest-pod.yamland substitute the shell variables. -
Apply the manifest:
-
Exec into the
aws-cliPod.and test access by listing S3 buckets:This command should list the S3 buckets in your AWS account, confirming that IRSA is correctly configured.