Tag: Terraform

Sometimes you want to change Terraform code without destroying and restoring resources. Following use case might come up: You create an S3 bucket which is the input bucket for some other party. The other party is supposed to upload data into that bucket to be processed by your application. Following code might be used to create the bucket:

resource "aws_s3_bucket" "bucket" {
  bucket = "4bc627ca-23f7-41ab-a9b0-d800128beb56"
}

After an apply the bucket is created. Let’s see the current state:

$ terraform state list
aws_s3_bucket.bucket

Now that the bucket exist you tell the other party the bucket name so that they know where to upload data to. We ignore policy setup here. After some thinking you conclude it is time to refactor the name bucket into input_bucket as it is more appropriate for it’s purpose. So you change the name and do an apply:

  # aws_s3_bucket.bucket will be destroyed
  - resource "aws_s3_bucket" "bucket" {
    - bucket = "4bc627ca-23f7-41ab-a9b0-d800128beb56" -> null
    ...  
  }

  # aws_s3_bucket.input_bucket will be created
  + resource "aws_s3_bucket" "input_bucket" {
    + bucket = "4bc627ca-23f7-41ab-a9b0-d800128beb56"
    ...
  }

Plan: 1 to add, 0 to change, 1 to destroy.

This change destroys the old bucket and re-creates it. For a brief moment the bucket name will be available again for others to claim because S3 bucket names are a globally shared namespace. We want to avoid that scenario even though it is unlikely.

To reconnect Terraforms with the new variable name we need to move the item in the state:

$ terraform state mv aws_s3_bucket.bucket aws_s3_bucket.input_bucket
Move "aws_s3_bucket.bucket" to "aws_s3_bucket.input_bucket"
Successfully moved 1 object(s).

A listing of the state now shows:

$ terraform state list
aws_s3_bucket.input_bucket

The variable is now moved to the new name. Let’s see what happens when we try an apply:

$ terraform apply
aws_s3_bucket.input_bucket: Refreshing state... [id=4bc627ca-23f7-41ab-a9b0-d800128beb56]

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

We could avoid a re-creation of the resource and a potential loss of the bucket name!

Software project for the cloud often involve multiple accounts. A typical setup associates a root account with multiple secondary accounts. The root account holds the Terraform state for all infrastructure on the secondary accounts. The secondary accounts commonly represent stages for development, testing and production.

Let’s say you are working with AWS and use an S3 bucket to hold the state. The following Terraform code manages this:

terraform { 
    required_providers { 
        aws = { 
            source = "hashicorp/aws" 
            version = "~> 3.29" 
        } 
    } 
 
    backend "s3" { 
        bucket = "mybucket" 
        key = "terraform.tfstate" 
        region = "eu-central-1" 
    } 
} 
 
provider "aws" { 
    profile = var.profile 
    region = "eu-central-1" 
} 

In a basic scenario with just one account Terraform just uses the current AWS profile normally provided via the environment variable AWS_PROFILE. In a multi-account scenario we want to use the S3 bucket on the root account and the infrastructure should be build on the secondary account. Terraform can be told about two accounts like that:

export AWS_PROFILE='root-account-profile'
terraform apply -var 'profile=secondary-account-profile'

The backend block in Terraform does not allow for variables. Therefor we set the AWS_PROFILE to the root account profile which will apply to the backend block. The secondary account profile will be passed in as variable. We have chosen to use a -var parameter on the command line but you could use -var-file to pass multiple variables in one file.

This works fine until you need to build the next secondary account. You need another state file in the S3 bucket to hold the state of the new account. This is what workspaces are for. At the moment we are in the default workspace which always exists. Export the variable AWS_PROFILE if you have not done already. Show the current workspaces:

> terraform workspace list
* default

Let’s create a new workspace and use that for the new secondary account:

> terraform workspace new prod
Created and switched to workspace "prod"! 

You're now on a new, empty workspace. Workspaces isolate their state, 
so if you run "terraform plan" Terraform will not see any existing state 
for this configuration.

Now that we have another separate state we can repeat the apply for a different secondary account:

terraform apply -var 'profile=another-secondary-profile'

To switch back to another workspace use select and the workspace name:

terraform workspace select default

You can now manage a cloud infrastructure setup with root account and secondary account with Terraform and workspaces.