Manage variable sets in HCP Terraform
HCP Terraform variable sets let you reuse variables in an efficient and centralized way, so you can set variable values once and use them in multiple workspaces. When you update the values, the changes automatically affect all associated workspaces, making it easier to update credentials and infrastructure settings. HCP Terraform also lets you override variables defined in variable sets on a per-workspace basis, which lets you modify the workspace's configuration without affecting other workspaces that use the variable set.
In this tutorial, you will manage multiple HCP Terraform variable sets. You will define variable sets for your AWS credentials and DynamoDB configuration. You will also review variable precedence between duplicated variables across variable sets, and between variables in variable sets and workspace-specific variables.
Prerequisites
This tutorial assumes that you are familiar with HCP Terraform and the standard Terraform workflow. If you are new to Terraform, complete the Get Started tutorials first. If you are new to HCP Terraform, complete the HCP Terraform Get Started tutorials first.
For this tutorial, you will need:
- an HCP Terraform account and organization
- Terraform v0.15+ installed locally and configured with your HCP Terraform token
- an AWS account
Clone example configuration
First, clone the example repository. This repository contains example configuration to create AWS DynamoDB tables in two environments.
$ git clone https://github.com/hashicorp-education/learn-terraform-variable-sets
Now, change into the repository directory.
$ cd learn-terraform-variable-sets
This repository contains two subdirectories:
$ tree
.
├── README.md
├── dev
│ ├── main.tf
│ ├── variables.tf
│ └── versions.tf
└── staging
├── main.tf
├── variables.tf
└── versions.tf
The dev and staging directories each contain Terraform configuration that
defines a DynamoDB table and configures it using input variables. The
configuration uses the random_pet resource to ensure a unique name for the
table, and configures the DynamoDB table using the db_read_capacity and
db_write_capacity input variables.
Initialize configuration
Navigate to the dev directory.
$ cd dev
Open versions.tf in your code editor, and replace <ORGANIZATION_NAME> in
the cloud block with your own HCP Terraform organization name.
dev/versions.tf
terraform {
cloud {
organization = "<ORGANIZATION_NAME>"
workspaces {
name = "learn-terraform-variable-sets-dev"
}
}
###
Notice that this configuration uses the
learn-terraform-variable-sets-dev workspace.
Initialize the configuration, which will also create the workspace in HCP Terraform.
$ terraform init
Initializing HCP Terraform...
Initializing provider plugins...
Reusing previous version of hashicorp/random from the dependency lock file
- Reusing previous version of hashicorp/aws from the dependency lock file
- Installing hashicorp/random v3.1.0...
- Installed hashicorp/random v3.1.0 (signed by HashiCorp)
- Installing hashicorp/aws v3.63.0...
- Installed hashicorp/aws v3.63.0 (signed by HashiCorp)
HCP Terraform has been successfully initialized!
You may now begin working with HCP Terraform. Try running "terraform plan" to
see any changes that are required for your infrastructure.
If you ever set or change modules or Terraform Settings, run "terraform init"
again to reinitialize your working directory.
Now, navigate to the staging directory.
$ cd ../staging
Open versions.tf in your code editor, and replace <ORGANIZATION_NAME> in
the cloud block with your own HCP Terraform organization name.
staging/versions.tf
terraform {
cloud {
organization = "<ORGANIZATION_NAME>"
workspaces {
name = "learn-terraform-variable-sets-staging"
}
}
###
This configuration uses a different workspace,
learn-terraform-variable-sets-staging.
Initialize the configuration, which will also create the workspace in HCP Terraform.
$ terraform init
Initializing HCP Terraform...
Initializing provider plugins...
Reusing previous version of hashicorp/random from the dependency lock file
- Reusing previous version of hashicorp/aws from the dependency lock file
- Installing hashicorp/random v3.1.0...
- Installed hashicorp/random v3.1.0 (signed by HashiCorp)
- Installing hashicorp/aws v3.63.0...
- Installed hashicorp/aws v3.63.0 (signed by HashiCorp)
HCP Terraform has been successfully initialized!
You may now begin working with HCP Terraform. Try running "terraform plan" to
see any changes that are required for your infrastructure.
If you ever set or change modules or Terraform Settings, run "terraform init"
again to reinitialize your working directory.
You now have two HCP Terraform workspaces configured for the CLI-driven workflow with remote execution.
Create variable sets
HCP Terraform variable sets are groups of reusable variables created at the organization level. A variable set can have one of three scopes:
- Global: It will apply to all current and future workspaces within an organization.
- Project-specific: It will apply to all current and future workspaces within the selected projects.
- Workspace-specific: It will apply only to the selected workspaces.
Using broader variable set scope enables self-service workflows. For instance, you can create a variable set and apply it to a team-specific project, then grant the team permission to create workspaces within the project. Future workspaces will automatically inherit the variable set without requiring additional work or approval. However, we recommend scoping variable sets that contain credentials as narrowly as possible, to avoid granting access to teams or workspaces that do not need them.
Create a credentials variable set
First, navigate to your organization's settings by clicking Settings in the left navigation. Then, select Variable Sets.
Note
If you already have a variable set for your AWS provider credentials, skip to the next section.
Click Create variable set.
Name this first variable set AWS credentials.
Warning
When possible, apply credential variable sets to specific projects or workspaces. Avoid global access and follow the principle of least privilege.
Scroll down to the Variable set scope section and select Apply to specific
projects and workspaces. Select the learn-terraform-variable-sets-dev
and learn-terraform-variable-sets-staging workspaces.
Click +Add Variable. Define an environment variable named AWS_ACCESS_KEY_ID and set it to your AWS Access Key ID. Mark it as sensitive and click Save variable.
Then, click +Add Variable again. Define another environment variable named
AWS_SECRET_ACCESS_KEY and set it to your AWS Secret access key. Mark it
as sensitive and click Save variable.
Tip
If you have temporary AWS credentials, you must also add your AWS_SESSION_TOKEN as an environment variable.
Finally, click Create variable set.
Create configuration settings variable set
You can also use variable sets to define reusable input variables. In this
scenario, you will provision DynamoDB tables for two environments, dev and staging. Use a
variable set to define the read and write capacities for both.
Create another variable set named Default DynamoDB settings. Once again,
apply it to both the learn-terraform-variable-sets-dev and
learn-terraform-variable-sets-staging workspaces.
Define two Terraform variables in the variable set:
- A Terraform variable named
db_write_capacitywith a value of1. - A Terraform variable named
db_read_capacitywith a value of1.
Save the variable set.
Rather than individually defining the database read and write capacity in both workspaces, you were able to just define them once as a variable set and apply them to the workspaces that need them.
Review workspace variables
You can review and manage which variable sets apply to the workspace from the workspace itself.
Navigate to your learn-terraform-variable-sets-dev workspace, then select the Variables tab.
Under Variable sets, the workspace lists both your AWS credentials
and Default DynamoDB settings variable sets and the variables that they
contain.
Apply configuration
In your terminal, navigate to your learn-terraform-variable-sets/dev
directory.
You already initialized your configuration earlier, so now apply it. Respond
yes when prompted to confirm the operation.
$ terraform apply
Running apply in HCP Terraform. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.
Preparing the remote apply...
To view this run in a browser, visit:
https://app.terraform.io/app/hashicorp-training/learn-terraform-variable-sets-dev/runs/run-fRjkg53BhhhwbEUN
Waiting for the plan to start...
Terraform v1.0.7
on linux_amd64
Configuring remote state backend...
Initializing Terraform configuration...
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
##...
Plan: 2 to add, 0 to change, 0 to destroy.
Do you want to perform these actions in workspace "learn-terraform-variable-sets-dev"?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
random_pet.table_name: Creating...
random_pet.table_name: Creation complete after 0s [id=still-pony]
aws_dynamodb_table.table: Creating...
aws_dynamodb_table.table: Creation complete after 4s [id=dev-still-pony]
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
Go to the learn-terraform-variable-sets-dev workspace to find the
resources it manages.
Now, navigate to your staging directory.
$ cd ../staging
Apply your configuration. Respond yes when prompted to confirm the operation.
$ terraform apply
Running apply in HCP Terraform. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.
Preparing the remote apply...
To view this run in a browser, visit:
https://app.terraform.io/app/hashicorp-training/learn-terraform-variable-sets-staging/runs/run-WMTfEG5hFmwaNArq
Waiting for the plan to start...
Terraform v1.0.7
on linux_amd64
Configuring remote state backend...
Initializing Terraform configuration...
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
##...
Plan: 2 to add, 0 to change, 0 to destroy.
Do you want to perform these actions in workspace "learn-terraform-variable-sets-staging"?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
random_pet.table_name: Creating...
random_pet.table_name: Creation complete after 0s [id=guiding-kite]
aws_dynamodb_table.table: Creating...
aws_dynamodb_table.table: Creation complete after 7s [id=staging-guiding-kite]
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
Explore variable set precedence
If any of the variable sets associated with the workspace contain a variable of the same type (input or environment variables) with the same name, HCP Terraform will use lexical order to determine variable precedence.
In this scenario, you applied a default set of DynamoDB settings to both tables. While you generally want to use consistent settings across both resources, you may want to perform load testing on one of your tables.
Navigate back to the variable sets page in your organization settings, create
a new variable set named Add Capacity - DynamoDB load testing, and apply it to the
learn-terraform-variable-sets-staging workspace.
Create two Terraform variables for this variable set:
- Set
db_write_capacityto10 - Set
db_read_capacityto10
Save your new variable set.
Now, navigate back to your learn-terraform-variable-sets-staging workspace and navigate to the Variables page. It lists 3 variable sets applying to your workspace.
Scroll down to find the Default DynamoDB settings variable set, which shows its values as overwritten.
Both your default and load testing variable sets define variables named
db_write_capacity and db_read_capacity. Since the load testing variable
set name begins with the letter "A", that variable set took precedence over
your default settings. If you want HCP Terraform to use the default DynamoDB
variable set instead, you can:
- change the names of the variable sets so that the default set has lexical precedence over the load testing set, or
- remove the load testing variable set from the workspace.
In your terminal, run terraform apply in your staging directory to update
your table's configuration. Respond yes when prompted to confirm the
operation.
$ terraform apply
Running apply in HCP Terraform. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.
Preparing the remote apply...
To view this run in a browser, visit:
https://app.terraform.io/app/hashicorp-training/learn-terraform-variable-sets-staging/runs/run-1j7aMQm8v4HdMWMR
Waiting for the plan to start...
Terraform v1.0.7
on linux_amd64
Configuring remote state backend...
Initializing Terraform configuration...
random_pet.table_name: Refreshing state... [id=guiding-kite]
aws_dynamodb_table.table: Refreshing state... [id=staging-guiding-kite]
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
~ update in-place
Terraform will perform the following actions:
# aws_dynamodb_table.table will be updated in-place
~ resource "aws_dynamodb_table" "table" {
id = "staging-guiding-kite"
name = "staging-guiding-kite"
~ read_capacity = 1 -> 10
tags = {}
~ write_capacity = 1 -> 10
# (5 unchanged attributes hidden)
# (3 unchanged blocks hidden)
}
Plan: 0 to add, 1 to change, 0 to destroy.
Do you want to perform these actions in workspace "learn-terraform-variable-sets-staging"?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
aws_dynamodb_table.table: Modifying... [id=staging-guiding-kite]
aws_dynamodb_table.table: Modifications complete after 2s [id=staging-guiding-kite]
Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
As expected, Terraform used the more recently applied load testing variable set to your configuration and increased the read and write capacities of your table.
Overwrite a variable in a variable set
You can also overwrite a variable defined in a variable set by creating a workspace-specific variable with the same key. HCP Terraform will always use workspace-specific variables over any variables defined in variable sets applied to the workspace.
In a load testing scenario, you may want to scale up your write capacity to
test how your application's performance responds. In your
learn-terraform-variable-sets-staging workspace, create a
workspace-specific input variable named db_write_capacity and set the value
to 15.
HCP Terraform now shows the db_write_capacity variable in the load testing
variable set as overwritten.
In your terminal, run a terraform apply to further scale your DynamoDB
table's write capacity. Respond yes to the prompt to confirm the operation.
$ terraform apply
Running apply in HCP Terraform. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.
Preparing the remote apply...
To view this run in a browser, visit:
https://app.terraform.io/app/hashicorp-training/learn-terraform-variable-sets-staging/runs/run-H5aAbCPCZ3xKiw9j
Waiting for the plan to start...
Terraform v1.0.7
on linux_amd64
Configuring remote state backend...
Initializing Terraform configuration...
random_pet.table_name: Refreshing state... [id=guiding-kite]
aws_dynamodb_table.table: Refreshing state... [id=staging-guiding-kite]
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
~ update in-place
Terraform will perform the following actions:
# aws_dynamodb_table.table will be updated in-place
~ resource "aws_dynamodb_table" "table" {
id = "staging-guiding-kite"
name = "staging-guiding-kite"
tags = {}
~ write_capacity = 10 -> 15
# (6 unchanged attributes hidden)
# (3 unchanged blocks hidden)
}
Plan: 0 to add, 1 to change, 0 to destroy.
Do you want to perform these actions in workspace "learn-terraform-variable-sets-staging"?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
aws_dynamodb_table.table: Modifying... [id=staging-guiding-kite]
aws_dynamodb_table.table: Modifications complete after 3s [id=staging-guiding-kite]
Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
Since you set a workspace-specific variable for the write capacity, HCP Terraform prioritized that value and scaled your table's write capacity to 15, overwriting the variable set values.
For more information on how HCP Terraform inherits variable sets and overrides values, see Workspace variable precedence.
Clean up resources
In your staging directory, destroy the table you created. Respond yes when prompted to confirm the operation.
$ terraform destroy
Running apply in HCP Terraform. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.
Preparing the remote apply...
To view this run in a browser, visit:
https://app.terraform.io/app/hashicorp-training/learn-terraform-variable-sets-staging/runs/run-W8NaRXye3gmGs1cy
Waiting for the plan to start...
Terraform v1.0.7
on linux_amd64
Configuring remote state backend...
Initializing Terraform configuration...
random_pet.table_name: Refreshing state... [id=guiding-kite]
aws_dynamodb_table.table: Refreshing state... [id=staging-guiding-kite]
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
- destroy
Terraform will perform the following actions:
##...
Plan: 0 to add, 0 to change, 2 to destroy.
Do you really want to destroy all resources in workspace "learn-terraform-variable-sets-staging"?
Terraform will destroy all your managed infrastructure, as shown above.
There is no undo. Only 'yes' will be accepted to confirm.
Enter a value: yes
aws_dynamodb_table.table: Destroying... [id=staging-guiding-kite]
aws_dynamodb_table.table: Destruction complete after 3s
random_pet.table_name: Destroying... [id=guiding-kite]
random_pet.table_name: Destruction complete after 0s
Apply complete! Resources: 0 added, 0 changed, 2 destroyed.
Then, change to your dev directory.
$ cd ../dev
Destroy the infrastructure managed in this directory as well.
$ terraform destroy
Running apply in HCP Terraform. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.
Preparing the remote apply...
To view this run in a browser, visit:
https://app.terraform.io/app/hashicorp-training/learn-terraform-variable-sets-dev/runs/run-v3Nszw3Eybo432AH
Waiting for the plan to start...
Terraform v1.0.7
on linux_amd64
Configuring remote state backend...
Initializing Terraform configuration...
random_pet.table_name: Refreshing state... [id=still-pony]
aws_dynamodb_table.table: Refreshing state... [id=dev-still-pony]
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
- destroy
Terraform will perform the following actions:
##...
Plan: 0 to add, 0 to change, 2 to destroy.
Do you really want to destroy all resources in workspace "learn-terraform-variable-sets-dev"?
Terraform will destroy all your managed infrastructure, as shown above.
There is no undo. Only 'yes' will be accepted to confirm.
Enter a value: yes
aws_dynamodb_table.table: Destroying... [id=dev-still-pony]
aws_dynamodb_table.table: Destruction complete after 3s
random_pet.table_name: Destroying... [id=still-pony]
random_pet.table_name: Destruction complete after 0s
Apply complete! Resources: 0 added, 0 changed, 2 destroyed.
Clean up HCP Terraform resources
Then, navigate to your learn-terraform-variable-sets-dev workspace in
HCP Terraform and delete the workspace.
Now, navigate to your learn-terraform-variable-sets-staging workspace
and delete the workspace.
Finally, navigate to your variable sets list under the organization settings.
Delete your Default DynamoDB settings variable set by clicking on it,
scrolling to the bottom, and clicking Delete variable set.
Delete your load testing variable set, and optionally your AWS credentials variable set as well.
Next steps
In this tutorial, you learned how to create and use variable sets to manage your HCP Terraform workspace's input and environment variables. You also learned about variable precedence between variable sets and how to overwrite variables within a workspace.
Check out the following resources to learn more about HCP Terraform configuration options and features: