Introduction

In today’s cloud-native environments, securely managing sensitive information such as API keys, database passwords, and other credentials is critical. Hardcoding these secrets in Infrastructure as Code (IaC) configurations can lead to significant security risks. In this post, we’ll explore a secure approach to managing Kubernetes secrets using Terraform and AWS Secrets Manager, ensuring sensitive data never resides in plain text or is committed to version control.

The Risks of Hardcoded Secrets

Kubernetes heavily relies on secrets for storing sensitive data, but managing these secrets with IaC tools like Terraform introduces challenges. A common error is hardcoding secret values into Terraform files (.tf files), which can lead to exposure if these files are ever shared or committed to public repositories. This practice opens the door to potential security breaches, especially in large teams or open-source projects.

The Secure Approach: AWS Secrets Manager + Terraform

To mitigate these risks, leveraging AWS Secrets Manager to securely store and manage secrets is a best practice. By integrating AWS Secrets Manager with Terraform, sensitive values are kept out of source control, minimizing the chance of leaks. Terraform’s aws_secretsmanager_secret resource allows you to programmatically manage secrets in AWS, ensuring they remain encrypted and protected.

Use Case 1: Automatically Managing Secrets with Terraform

In this use case, we’ll demonstrate how to automatically create and manage secrets in AWS Secrets Manager and reference them in Kubernetes secrets using Terraform. This approach eliminates manual secret management, providing an automated and secure workflow.

Step 1: Create Secrets in AWS Secrets Manager Using Terraform

To begin, define your sensitive data using Terraform’s aws_secretsmanager_secret resource. For instance, let’s create a secret for a GitHub Personal Access Token:

# Create a secret in AWS Secrets Manager
resource "aws_secretsmanager_secret" "token" {
  name        = "token"
  description = "Personal Access Token"
  recovery_window_in_days = 30
}

Step 2: Retrieve Secret Values Using Terraform

Next, retrieve the secret values using the aws_secretsmanager_secret_version data source:

# Fetch the secret version from AWS Secrets Manager
data "aws_secretsmanager_secret_version" "token" {
  secret_id = aws_secretsmanager_secret.token.id
}

Step 3: Create a Kubernetes Secret Using Terraform

Finally, use the retrieved values to create a Kubernetes secret:

# Define a Kubernetes secret with Terraform
resource "kubernetes_secret" "example" {
  metadata {
    name      = "secret"
    namespace = "default"
  }

  data = {
    token = data.aws_secretsmanager_secret_version.token.secret_string
  }
}

Step 4: Deploy the Secret

After configuring your secrets, deploy them using:

terraform init
terraform apply

This automated workflow ensures that sensitive data is securely managed and deployed without exposing it in plain text.

Use Case 2: Manually Managing Secrets with Terraform

In scenarios where sensitive data is not immediately available or needs manual updates (e.g., credentials for third-party APIs), you can create “placeholder” secrets in Terraform and later update the values manually in AWS Secrets Manager.

Step 1: Create a Secret with a Placeholder Value

Here, we create a secret in AWS Secrets Manager with a dummy value (DUMMY_VALUE) and instruct Terraform to ignore updates to the actual secret value:

# Define a secret with a placeholder value
resource "aws_secretsmanager_secret" "api-key" {
  name        = "api-key"
  description = "API key for third-party service"

  # Placeholder value to be updated manually
  secret_string = jsonencode({
    api_key = "DUMMY_VALUE"
  })

  lifecycle {
    ignore_changes = [secret_string]
  }
}

Step 2: Update the Secret in AWS Secrets Manager Console

Once the secret is created, you can update its value in the AWS Secrets Manager console by replacing the placeholder value with the actual sensitive data.

Step 3: Reference the Manually Updated Secret in Terraform

Even though the value is managed manually, you can still reference it in Terraform using the data source:

# Fetch the updated secret value
data "aws_secretsmanager_secret_version" "api-key" {
  secret_id = aws_secretsmanager_secret.api-key.id
}

Step 4: Create a Kubernetes Secret with the Updated Value

Pass the updated secret value to a Kubernetes secret resource:

# Create a Kubernetes secret using the manually updated value
resource "kubernetes_secret" "api-key-secret" {
  metadata {
    name      = "api-key-secret"
    namespace = "default"
  }

  data = {
    api_key = jsondecode(data.aws_secretsmanager_secret_version.api-key.secret_string)["api_key"]
  }
}

Step 5: Deploy the Secret

Deploy the updated configuration with:

terraform init
terraform apply

Conclusion

Managing Kubernetes secrets securely is vital to maintaining a robust infrastructure. By integrating Terraform with AWS Secrets Manager, you can automate and simplify secret management while ensuring your sensitive data remains protected. Whether you choose full automation or manual updates for greater control, this approach keeps your infrastructure secure and free from hardcoded secrets.

Ready to Secure Your Cloud Infrastructure? Contact NimbusStack for a Free Cloud Security Audit Today!