Terraform for starters¶

Estimated time to read: 13 minutes

As a Terraform expert, I'm happy to provide you with a handy cheat sheet for commonly used Terraform commands. This should help you create templates more efficiently:

Getting started¶

Initialise the Terraform working directory:

terraform init

Validate the Terraform configuration files:

terraform validate

Preview the actions that Terraform will perform:

terraform plan

Apply the changes described in the Terraform configuration files:

terraform apply

Destroy the resources managed by Terraform:

terraform destroy

Format your Terraform configuration files according to the recommended style:

terraform fmt

Show the current state of the infrastructure managed by Terraform:

terraform show

Import existing infrastructure into the Terraform state:

terraform import [address] [resource_id]

Create a new workspace:

terraform workspace new [workspace_name]

List all available workspaces:

terraform workspace list

Switch to a different workspace:

terraform workspace select [workspace_name]

Output the values of Terraform outputs:

terraform output [output_name]

Refresh the state file against the real resources:

terraform refresh

Force-unlock a locked state file:

terraform force-unlock [lock_id]

Remove items from the Terraform state:

terraform state rm [address]

Remember, this cheat sheet only covers the main Terraform commands. You can always refer to the official Terraform documentation for more information and advanced usage: https://www.terraform.io/docs/cli/index.html

To write a Terraform configuration, you'll need to follow these basic steps:

• Install Terraform: Ensure you have Terraform installed on your machine. You can download it from the official website (https://www.terraform.io/downloads.html) and follow the installation instructions for your specific operating system.

• Choose a text editor: Use a text editor of your choice to write Terraform configuration files. Popular editors include Visual Studio Code, Sublime Text, and Atom. You may also want to install a Terraform extension or plugin for syntax highlighting and autocompletion.

• Understand the Terraform syntax: Terraform uses its domain-specific language, HashiCorp Configuration Language (HCL). Familiarise yourself with HCL's syntax, which includes blocks, arguments, expressions, and variables.

• Create a Terraform configuration file: Terraform configuration files have the .tf extension. Create a new file, such as main.tf, to define your resources and other configuration elements.

• Specify the provider: The provider manages the underlying infrastructure resources. Begin your configuration by defining the provider and its required settings. For example, if you're using AWS, you'll need to configure the AWS provider:

provider "aws" {
  region = "us-west-2"
}

• Define resources: Resources are the infrastructure components you want to create and manage using Terraform. Add one or more resources to your configuration file. Here's an example of defining an AWS EC2 instance:

resource "aws_instance" "example" {
  ami           = "ami-0c94855ba95b798c7"
  instance_type = "t2.micro"

  tags = {
    Name = "example-instance"
  }
}

• Use input variables: Input variables allow you to parameterise your configuration, making it more modular and reusable. Create a separate variables.tf file to define your input variables, and use them in your main.tf:

# In variables.tf
variable "region" {
  default = "us-west-2"
}

# In main.tf
provider "aws" {
  region = var.region
}

• Configure output variables: Output variables are used to display information about your infrastructure, such as IP addresses or DNS names. Add an output block to your configuration to define an output variable:

output "instance_public_ip" {
  value = aws_instance.example.public_ip
}

• Use modules: Modules are reusable, self-contained packages of Terraform configurations that can be shared and versioned. Use modules to organise and encapsulate your infrastructure components, making your configuration more maintainable.

• Test and iterate: Finally, use Terraform commands (init, validate, plan, apply, etc.) to test and iterate your configuration. Validate and review the changes before applying them to your infrastructure.

Once you're comfortable with these concepts and steps, you can create more complex Terraform configurations to manage your infrastructure.

Validation tests¶

You can write validation and testing scripts for your Terraform configurations using tools and frameworks designed specifically for this purpose. Test-Driven Development (TDD) and Behavior-Driven Development (BDD) are two popular approaches. Here are a few tools you can use to test your Terraform code:

1. Terratest: Terratest is a Go library that provides patterns and helper functions for testing your infrastructure code. With Terratest, you can write TDD-style tests in Go, allowing you to create and destroy resources during testing. You can find Terratest on GitHub (https://github.com/gruntwork-io/terratest) and read the documentation to learn how to use it.

2. Kitchen-Terraform: Kitchen-Terraform is a plugin for Test Kitchen, a testing framework that supports BDD-style testing. Kitchen-Terraform allows you to write tests in InSpec, a Ruby-based testing language. You can find Kitchen-Terraform on GitHub (https://github.com/newcontext-oss/kitchen-terraform) and read the documentation to learn how to use it.

3. Terraform Compliance: Terraform Compliance is a lightweight, BDD-style testing framework for Terraform that focuses on validating your infrastructure against security and compliance requirements. You can write tests in Gherkin, a natural language syntax used in Cucumber and other BDD tools. You can find Terraform Compliance on GitHub (https://github.com/terraform-compliance/cli) and read the documentation to learn how to use it.

Each of these tools has its advantages and trade-offs. The choice of tool will depend on your preferences, requirements, and the languages you're comfortable with. It's essential to include tests and validations in your infrastructure-as-code development process to ensure your Terraform configurations' quality, security, and compliance.

Terratest validation example¶

Here's an example of using Terratest to validate your Terraform configuration script for the AWS provider, ensuring you use the correct region and availability zone. This example assumes that you have a basic Terraform configuration that provisions an EC2 instance:

main.tf:

provider "aws" {
  region = var.region
}

resource "aws_instance" "example" {
  ami           = var.ami
  instance_type = var.instance_type
  subnet_id     = var.subnet_id

  tags = {
    Name = "example-instance"
  }
}

variables.tf:

variable "region" {
  default = "us-west-2"
}

variable "ami" {
  default = "ami-0c94855ba95c5a1cd"
}

variable "instance_type" {
  default = "t2.micro"
}

variable "subnet_id" {
  default = "subnet-abcdef123"
}

Now, create a Go test file using Terratest to validate your configuration:

aws_instance_test.go:

package test

import (
    "testing"

    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/ec2"
    "github.com/gruntwork-io/terratest/modules/aws"
    "github.com/gruntwork-io/terratest/modules/terraform"
    "github.com/stretchr/testify/assert"
)

func TestAwsInstance(t *testing.T) {
    t.Parallel()

    expectedRegion := "us-west-2"
    expectedAvailabilityZone := "us-west-2a"

    terraformOptions := &terraform.Options{
        TerraformDir: ".",
    }

    defer terraform.Destroy(t, terraformOptions)

    terraform.InitAndApply(t, terraformOptions)

    instanceID := terraform.Output(t, terraformOptions, "aws_instance_id")

    awsSession, err := session.NewSession(&aws.Config{
        Region: aws.String(expectedRegion),
    })
    if err != nil {
        t.Fatal(err)
    }

    ec2Svc := ec2.New(awsSession)
    instanceInfo, err := aws.GetEC2InstanceById(t, instanceID, ec2Svc)
    if err != nil {
        t.Fatal(err)
    }

    assert.Equal(t, expectedRegion, aws.StringValue(instanceInfo.Placement.Region))
    assert.Equal(t, expectedAvailabilityZone, aws.StringValue(instanceInfo.Placement.AvailabilityZone))
}

In this example, we first define the expected region and availability zone. We then use the Terratest terraform package to initialise and apply our Terraform configuration. After that, we obtain the instance ID from the Terraform output and use the AWS SDK to fetch information about the created instance. Finally, we use the assert package to check if the actual region and availability zone match our expectations.

To run this test, make sure you have Go and Terratest installed, and then execute the following command in your terminal:

go test -v -timeout 30m

This will run the test, create the EC2 instance using your Terraform configuration, validate the region and availability zone, and then destroy the instance.

Remember that this is just one example of how to use Terratest to validate your Terraform configuration. You can apply similar concepts to other tools like Kitchen-Terraform or Terraform Compliance to achieve the same goal.

Terraform Compliance validation example¶

Let's create an example using Terraform Compliance to validate that the Terraform configuration uses the correct region and availability zone for an AWS EC2 instance. This example assumes that you have a basic Terraform configuration that provisions an EC2 instance:

main.tf:

provider "aws" {
  region = var.region
}

resource "aws_instance" "example" {
  ami           = var.ami
  instance_type = var.instance_type
  subnet_id     = var.subnet_id

  tags = {
    Name = "example-instance"
  }
}

variables.tf:

variable "region" {
  default = "us-west-2"
}

variable "ami" {
  default = "ami-0c94855ba95c5a1cd"
}

variable "instance_type" {
  default = "t2.micro"
}

variable "subnet_id" {
  default = "subnet-abcdef123"
}

Now, let's write the Terraform Compliance test using Gherkin syntax:

validate_region_availability_zone.feature:

Feature: Validate region and availability zone
  Scenario: Ensure AWS provider is using the correct region and availability zone
    Given I have aws_instance defined
    When it contains subnet_id
    Then it must contain subnet-abcdef123
    And its region must be us-west-2

This feature file contains a single scenario that checks if the aws_instance resource has the correct subnet_id and is in the us-west-2 region.

Before running the tests, make sure you have Terraform Compliance installed:

pip install terraform-compliance

Now, run terraform init and terraform plan -out=plan.out to create a binary plan file. Finally, execute the following command to run the Terraform Compliance test:

terraform-compliance -f validate_region_availability_zone.feature -p plan.out

If your Terraform configuration meets the specified requirements, you'll see a message indicating that all the tests passed. Otherwise, you'll receive an error message with details about the failed test.

Remember that this is just a basic example of using Terraform Compliance. You can write more complex scenarios and multiple feature files to cover various aspects of your infrastructure code validation.

Kitchen-Terraform validation example¶

Let's create an example using Kitchen-Terraform to validate that the Terraform configuration uses the correct region and availability zone for an AWS EC2 instance. This example assumes that you have a basic Terraform configuration that provisions an EC2 instance:

main.tf:

provider "aws" {
  region = var.region
}

resource "aws_instance" "example" {
  ami           = var.ami
  instance_type = var.instance_type
  subnet_id     = var.subnet_id

  tags = {
    Name = "example-instance"
  }
}

variables.tf:

variable "region" {
  default = "us-west-2"
}

variable "ami" {
  default = "ami-0c94855ba95c5a1cd"
}

variable "instance_type" {
  default = "t2.micro"
}

variable "subnet_id" {
  default = "subnet-abcdef123"
}

Now, create the following files to set up Kitchen-Terraform:

1. Gemfile:

source 'https://rubygems.org/'

gem 'kitchen-terraform'

1. .kitchen.yml:

---
driver:
  name: terraform
  root_module_directory: .

provisioner:
  name: terraform

verifier:
  name: terraform

platforms:
  - name: aws

suites:
  - name: default
    verifier:
      inspec_tests:
        - test/integration/default

1. Create a directory for InSpec tests: mkdir -p test/integration/default

2. test/integration/default/controls/aws_instance.rb:

control 'aws_instance' do
  impact 1.0
  title 'Verify that the EC2 instance is in the correct region and availability zone'

  describe aws_ec2_instance(name: 'example-instance') do
    it { should exist }
    its('region') { should eq 'us-west-2' }
    its('availability_zone') { should eq 'us-west-2a' }
  end
end

In the InSpec test file aws_instance.rb, we define a control that checks if the EC2 instance exists and if it is located in the correct region (us-west-2) and availability zone (us-west-2a).

Before running the tests, make sure you have Ruby, Bundler, Kitchen-Terraform, and InSpec installed:

gem install bundler
bundle install

Now, execute the following command to run the Kitchen-Terraform tests:

bundle exec kitchen test

This will run the tests, create the EC2 instance using your Terraform configuration, validate the region and availability zone, and destroy the instance.

Keep in mind that this is just a basic example of using Kitchen-Terraform. You can write more complex InSpec controls and multiple test files to cover various aspects of your infrastructure code validation.

Writing your own Terraform provider or extending an existing one¶

Writing your own Terraform provider or extending an existing one involves developing a plugin in the Go programming language. This requires understanding Go and the Terraform provider/plugin SDK.

Here are the steps to write your own Terraform provider or add more features to an existing one:

• Set up your Go environment: Make sure you have Go installed and configured on your machine. You can follow the official Go installation guide: https://golang.org/doc/install

• Familiarize yourself with the Terraform provider/plugin SDK: The Terraform provider/plugin SDK is a set of packages and interfaces that simplifies writing a Terraform provider. The SDK documentation and examples can be found here: https://pkg.go.dev/github.com/hashicorp/terraform-plugin-sdk/v2

• Create a new Go module or fork an existing provider repository: If you're writing a new provider from scratch, create a new Go module for your provider. If you're extending an existing provider, fork the provider's repository on GitHub.

• Define your provider schema: A provider schema is a Go struct that defines the configuration settings and resources your provider supports. The schema includes required and optional arguments, default values and validation functions. Here's an example:

package main

import (
  "github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
  "github.com/hashicorp/terraform-plugin-sdk/v2/plugin"
)

func Provider() *schema.Provider {
  return &schema.Provider{
    Schema: map[string]*schema.Schema{
      "api_key": {
        Type:        schema.TypeString,
        Required:    true,
        DefaultFunc: schema.EnvDefaultFunc("MYPROVIDER_API_KEY", nil),
        Description: "The API key for the MyProvider service.",
      },
    },
    ResourcesMap: map[string]*schema.Resource{
      "myprovider_resource": resourceMyProviderResource(),
    },
  }
}

func main() {
  plugin.Serve(&plugin.ServeOpts{
    ProviderFunc: Provider,
  })
}

• Implement resource CRUD functions: For each resource supported by your provider, you'll need to implement Create, Read, Update, and Delete (CRUD) functions. These functions interact with the APIs of the service you're building the provider for. Here's an example of a resource CRUD function:

func resourceMyProviderResource() *schema.Resource {
  return &schema.Resource{
    Create: resourceMyProviderResourceCreate,
    Read:   resourceMyProviderResourceRead,
    Update: resourceMyProviderResourceUpdate,
    Delete: resourceMyProviderResourceDelete,

    Schema: map[string]*schema.Schema{
      "name": {
        Type:     schema.TypeString,
        Required: true,
      },
    },
  }
}

func resourceMyProviderResourceCreate(d *schema.ResourceData, m interface{}) error {
  // Implement the logic to create the resource using the service's API
}

func resourceMyProviderResourceRead(d *schema.ResourceData, m interface{}) error {
  // Implement the logic to read the resource using the service's API
}

func resourceMyProviderResourceUpdate(d *schema.ResourceData, m interface{}) error {
  // Implement the logic to update the resource using the service's API
}

func resourceMyProviderResourceDelete(d *schema.ResourceData, m interface{}) error {
  // Implement the logic to delete the resource using the service's API
}

• Add data sources (optional): If your provider requires data sources to fetch information from the service's API, implement data source functions similar to the resource CRUD functions.

• Test your provider: Write acceptance tests using the Terraform provider testing framework for your provider. This will help ensure that your provider works correctly with the Terraform core and the service APIs. The Terraform provider testing framework documentation can be found here: https://pkg.go.dev/github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource

Here's an example of a basic acceptance test for a resource:

func TestAccMyProviderResource_basic(t *testing.T) {
    resourceName := "myprovider_resource.test"
    resource.Test(t, resource.TestCase{
        PreCheck:          func() { testAccPreCheck(t) },
        ProviderFactories: testAccProviderFactories,
        CheckDestroy:      testAccCheckMyProviderResourceDestroy,
        Steps: []resource.TestStep{
            {
                Config: testAccMyProviderResourceConfig_basic,
                Check: resource.ComposeTestCheckFunc(
                    testAccCheckMyProviderResourceExists(resourceName),
                    resource.TestCheckResourceAttr(resourceName, "name", "test-resource"),
                ),
            },
        },
    })
}

• Build your provider: Compile your provider code into a binary executable using the go build command. Place the compiled binary in the appropriate subdirectory of the %APPDATA%\terraform.d\plugins directory on Windows or ~/.terraform.d/plugins directory on macOS and Linux.

• Use your provider: Once your provider is built and placed in the correct directory, you can use it in your Terraform configurations. Define the provider block in your configuration files and provide any required settings.

• Share your provider (optional): If you're building a public provider, you can share it with the Terraform community by publishing it on the Terraform Registry. Follow the guidelines for publishing a provider: https://www.terraform.io/docs/registry/providers/publishing.html

For more detailed information on writing a Terraform provider, you can follow the official Terraform provider development guide: https://learn.hashicorp.com/tutorials/terraform/provider-development?in=terraform/providers