Agenda
1. Why testIaC-code at all?
2. Tests in Terraform
3. Testing Terraform in the CI
4. Tests in other IaC tools
3.
Agenda
1. Why testIaC-code at all?
2. Tests in Terraform
3. Testing Terraform in the CI
4. Tests in other IaC tools
4.
Why test IaC-codeat all?
QAware | 4
Fail fast Document assumptions
Validate compliance
Motivation / benefits are exactly the same as for tests of application code
As with application code, the cost-benefit ratio needs to be appropriate.
5.
Agenda
1. Why testIaC-code at all?
2. Tests in Terraform
3. Testing Terraform in the CI
4. Tests in other IaC tools
6.
Overview over thedifferent types of testing
QAware | 6
1. Validations (of variables)
2. Pre- / Postconditions (on resources)
3. “Unit tests” - custom tests on terraform plan output
4. “Integration tests” - custom tests on temporary deployed resources
5. “End-to-end tests” - custom tests on the live state
}“Contract tests”
Disclaimer: This depicts the “normal” testing pyramid. In Terraform, the
order is different!
Source: https://www.hashicorp.com/de/blog/testing-hashicorp-terraform
7.
1. Validations
QAware |7
● Validate input given in variables
○ Part of the contract of a
module
● Evaluated on the start of every
plan / apply / test
○ Will fail the command and
prevent broken configuration
to be applied
● Defined for a specific variable,
but can contain any expressions
and can reference other variables
(terraform ~> v1.10)
● Multiple blocks per variable are
possible (all conditions have to be
satisfied)
variable "bucket_name" {
description = "The name of the S3 bucket."
type = string
validation {
condition = (
length(var.bucket_name) >= 3 && length(var.bucket_name) <= 63
)
error_message = "Invalid bucket name. The name must be 3 to 63 characters long."
}
}
8.
2a. Preconditions
QAware |8
● Document assumptions
○ Define contracts between
resources within a module
● Evaluated on every plan / apply /
test (before apply)
○ Will fail the command and
prevent broken configuration
to be applied
● Defined for a specific resource or
output, can reference all objects
available before apply (including
data sources)
Source: https://developer.hashicorp.com/terraform/language/expressions/custom-conditions#preconditions-and-postconditions
data "aws_ami" "example" {
...
}
resource "aws_instance" "example" {
instance_type = "t3.micro"
ami = data.aws_ami.example.id
lifecycle {
# The AMI ID must refer to an AMI that contains an operating system
# for the `x86_64` architecture.
precondition {
condition = data.aws_ami.example.architecture == "x86_64"
error_message = "The selected AMI must be for the x86_64 architecture."
}
}
}
9.
2b. Postconditions
QAware |9
● Codify guarantees
● Evaluated on every plan / apply /
test (“asap”)
○ Will fail the command and
stop downstream processing
(but will not undo any
configuration already
applied).
● Defined for a specific resource or
output, can reference pretty
much everything, including itself
resource "aws_s3_object" "example_object" {
bucket = ver.bucket_name
key = "example_object"
source = "test.txt"
lifecycle {
postcondition {
condition = try(self.version_id != "null", false)
error_message = "The example object must be versioned."
}
}
}
10.
3. “Unit Tests”– terraform test on plan
QAware | 10
● Custom tests on the plan-output
● Only evaluated when explicitly
running terraform test
○ No infrastructure changes!
○ But real data sources read
● Defined for an entire module
● Defined in a separate file ending
in .tftest.hcl
● Provider behaviour can be
mocked (e.g. simulate data
source giving a specific value)
# main.tf
…
resource "aws_s3_bucket" "bucket" {
bucket = "${var.bucket_prefix}-bucket"
}
# valid_string_concat.tftest.hcl
variables {
bucket_prefix = "test"
}
run "valid_string_concat" {
command = plan
assert {
condition = aws_s3_bucket.bucket.bucket == "test-bucket"
error_message = "S3 bucket name did not match expected"
}
}
Source: https://developer.hashicorp.com/terraform/language/tests
11.
4. “Integration Tests”– terraform test on apply
QAware | 11
● Custom tests with real
infrastructure
● Only evaluated when explicitly
running terraform test
○ Real infrastructure created
and then torn down
● Defined for an entire module
● Defined in a separate file ending in
.tftest.hcl
● Provider behaviour can be
mocked (e.g. simulate some
resource already present)
# main.tf
…
resource "aws_s3_bucket" "bucket" {
bucket = "${var.bucket_prefix}-bucket"
}
# valid_string_concat.tftest.hcl
variables {
bucket_prefix = "test"
}
run "valid_string_concat" {
command = apply
assert {
condition = aws_s3_bucket.bucket.bucket == "test-bucket"
error_message = "S3 bucket name did not match expected"
}
}
Source: https://developer.hashicorp.com/terraform/language/tests
12.
5. “End-to-end Tests”– advanced validation of the real apply
QAware | 12
● Validate infrastructure
automatically (but independent
of resource lifecycle)
● Evaluated at the end of every
plan / apply / test
○ Will not block the
command.
● Defined for an entire module
check "website_status_code" {
data "http" "static_website" {
url = local.website_endpoint
}
assert {
condition = data.http.static_website.status_code == 200
error_message = "${data.http.static_website.url} returned an unhealthy status"
}
}
Agenda
1. Why testIaC-code at all?
2. Tests in Terraform
3. Testing Terraform in the CI
4. Tests in other IaC tools
16.
Which tests makesense to run regularly?
● Checked by default with each terraform apply
○ Validations
○ Preconditions
○ Postconditions
○ “End-to-End”-Tests
● Needs to be checked manually by terraform test
○ “Unit”-Tests
○ “Integration”-Tests
● Needs manual intervention
○ Manual Tests
QAware | 16
17.
From IDP-project experience
QAware| 17
When building platforms (IDPs) only running terraform internal testing usually is not sufficient.
● Glue code needs to be tested
○ It’s not only terraform, but also the pipelines etc
● External interfaces need to be tested
○ IDP(-management) interacts with other systems, that needs to be tested
● Advanced tests to verify functionality of deployed infrastructure
○ Testing a real world workload
⇒ A custom End-to-End testing framework needs to be built for advanced use-cases
18.
Agenda
1. Why testIaC-code at all?
2. Tests in Terraform
3. Testing Terraform in the CI
4. Tests in other IaC tools
19.
Tests in OpenTofu
QAware| 19
… basically the same as in terraform.
● Validation, custom conditions and check blocks were already included in Terraform 1.5 and became
part of OpenTofu with the fork.
● terraform test was only introduced in Terraform 1.6, but was included in OpenTofu 1.6.
● Provider mocking was included in 1.8.
● OpenTofu has not yet introduced any (major) features of its own in this area.
● Looking at the details, there are of course some initial minor deviations.
20.
External Terraform Test-Frameworks
⇒Terratest, Kitchen-Terraform
QAware | 20
Now obsolete for most test cases.
Often created before tests were introduced in Terraform in their current form.
● Kitchen Terraform has since been deprecated.
Disadvantage:
● Additional tool
● Often different syntax or programming language (e.g. Go or Ruby)
However, it can be useful in special cases, e.g.
● Very complex test cases that require (a lot of) custom test code
● Testing multiple tools with the same test tool (e.g. Terratest for Terraform & Packer & Kubernetes)
21.
Tests in Pulumi
QAware| 21
Pulumi uses ‘normal’ programming languages
⇒ We can use ‘normal’ test frameworks for these languages directly.
Concepts similar to the test pyramid, see https://www.pulumi.com/docs/iac/concepts/testing/
22.
Tests in Crossplane
QAware| 22
Based on Kubernetes manifests and operators, therefore harder to test.
Possible solutions: (I didn’t look into)
● KUTTL ⇒ Blog article
● Crossplane Uptest ⇒ GitHub