Hello everyone. We are now coming into the weekly tips for the month of June; How exciting! As we always do, we will build on-top of already gathered knowledge that we have gathered together over the last 6 weeks of Terraform weekly tips. Last week was the accumulation post for May, where we build an entire environment and even used Terraform to execute the Remote Desktop Manager application with automatically parsed credentials. I highly recommend everyone to read or at least skip that post first, before continuing reading this one -> https://www.codeterraform.com/post/build-a-demo-environment-with-rdp-connectivity-in-just-3-minutes
Todays post will be all about Terraform functions and which specific functions that I see as very valuable in our day-to-day working basis with IaC. I think the best way to approach the subject, is to firstly introduce each of the functions with a short description in a list, where after we will look at some specific examples:
Flatten() -> 'Folds out' Tupples so that they become much easier to work with inside Terraform runtime
Regexall() Combined with the Length() function -> Fantastic combination of functions to use in things like variable validation
Any encode and decode function such as for YAML, JSON and CSV
There are more, but the above are the ones I use the most and that I have worked a lot with during multiple different projects using Terraform. Apart from the very short introduction to each of the functions, let us deep-dive into some code, giving practical examples of each one in action! Let us do it by order as above, starting with Flatten() and what it can do.
Concepts of the function flatten:
In the below example, imagine we build a simple resource group in Azure using Terraform. Inside of set resource group, we define 5 new Virtual Networks, where each of these also consist of 5 subnets. After the creation of all resources, we would like to have a list of strings, only containing ALL the address-prefixes of the subnets for later use. So... How can flatten be the hero here?
First, sample code to create the resources; It is not important knowing how exactly all resources are created, the most important point will come a little later:
resource "azurerm_resource_group" "rg01" {
name = "rg01"
location = "West Europe"
}
resource "azurerm_virtual_network" "vnet_objects" {
count = 5
name = "vnet-${count.index}"
location = "West Europe"
address_space = ["10.0.${count.index}.0/24"]
resource_group_name = azurerm_resource_group.rg01.name
subnet {
name = "subnet01"
address_prefix = "10.0.${count.index}.0/26"
}
subnet {
name = "subnet02"
address_prefix = "10.0.${count.index}.64/26"
}
subnet {
name = "subnet03"
address_prefix = "10.0.${count.index}.128/26"
}
subnet {
name = "subnet04"
address_prefix = "10.0.${count.index}.192/26"
}
}The above code will create 5 subnets on each Vnet, where the name of each set of 5 subnets will have the same names across all 5 Vnet´s, but the address prefixes wont be the same. After creation, we would like a string list only containing all unique address prefixes. To do this, lets look at some output blocks:
output "vnet_objects_raw" {
value = azurerm_virtual_network.vnet_objects
}Lets start simple by investigating the overall structure of the total return of all 5 Vnets with underlying configuration. A code snippet of the first Vnet in the list can be seen below:
Point 1 defines the start of the tupple containing all 5 Vnet's with their underlying subnets. Point 2 defines the FIRST subnet inside the FIRST Vnet. As we can see the structure is complex, but we can begin to carve into the overall tupple to get towards our end goal.
From the raw Vnet output, lets start by using a splat expression (Shorthand for a 'for loop') to take out every single subnet in each of the Vnet's and gather them inside their own tupple:
output "all_subnets_from_all_vnets" {
value = azurerm_virtual_network.vnet_objects.*.subnet
}Which then gives us:
Point 1 defines the start of the NEW tupple which is directly produced by using the above splat expression to first carve out every single subnet object and cast it into a new tupple. Point 2 shows that every single subnet is packed into an underlying set of its own. This is an issue as because of every single subnet being a 'set' In this case does not allow us to continue to simple carve out the wanted point 3 of each of the address prefix's. This is due to the simple fact of how 'sets' Works in Terraform. It is not possible to index over each element in a set, hence making it impossible to simply do another splat expression to carve further into each subnet.
This is where the flatten expression comes in as a hero - Watch what happens, when we enfold the entire expression from the output above into a flatten function and run the output again.
output "using_flatten_to_remove_sets" {
value = flatten(azurerm_virtual_network.vnet_objects.*.subnet)
}
Point 1 is the start of the new tupple, which is the same as in the last output, the difference comes in point 2; The sets of each subnet is now gone and it has been 'folded out' To now only contain a simple object for each of the subnets in the overall tupple. We now, because of the flatten function, have direct access to run yet another splat expression, to finally cast the address prefixes into a list by themselves. Because of these examples being run in simple outputs and not using local variables to help obfuscate, it looks a little intimidating as the final output line:
output "combine_all_the_above" {
value = flatten(azurerm_virtual_network.vnet_objects.*.subnet).*.address_prefix
}Which gives us the final output that we wanted:
The above is simply a snippet, the list is of course 25 indices long, as we had 25 subnets created, each with their own unique address prefix. Furthermore, this is just simply using an output to the terminal. As we have already learned in earlier posts, we could take advantage of local variables and even outputs to files and so much more.
Now... That was a little hairy, lets talk about a way more simple function concept: The use of regexall with length to validate variables.
The concept of the functions length & regexall:
The most common use of the above function combination is in the validation of input variables. This feature of Terraform is very important as it helps us build more robust scripts, where we can enforce certain standards. Lets take some examples:
All the below are inside the variables.tf:
//Lets first build a standard and simple string variable:
variable "resource_base_name" {
description = "the prefix to be used on all new resource names"
type = string
}Lets expand the above variable to contain a validation block:
variable "resource_base_name" {
description = "the prefix to be used on all new resource names"
type = string
validation {
condition = length(regexall("^(PRD|TST|DEV)-.*$", var.resource_base_name)) > 0
error_message = "the value provided '${var.resource_base_name}' must start with either PRD-, TST- or DEV-"
}
}Let us break down the above code - First of, the validation block is defined within the variable called 'resource_base_name' Inside of this block both the 'condition' And 'error_message' Parameters must be specified. As part of the condition, it simply checks whether the expression that ran inside of it, returns a boolean value of true OR false. This parameter cannot handle any other type of value and as it is already obvious, a return of true will pass the validation, a return of false makes Terraform runtime fail and the 'error_message' Parameter value is returned to the user. Also note that we in the example above do not define a default value for the variable - If we did, it also MUST parse the exact same validation.
Let us run terraform plan where we parse a 'bad' String for the variable 'resource_base_name'
terraform plan -var="resource_base_name=PROD-"
Point 1 shows that Terraform plan failed - An exception was thrown during runtime (Which will stop any kind of deployment, also in pipelines). Point 2 is Terraform throwing a default message as part of the exception. Point 3 shows our 'error_message' As we personally defined it. It is ALWAYS recommended to provide an informatic error message as part of variable validation. To this, because we used the 'regexall' Function to look for a specific prefix, provide it as part of the message so the user knows how to set the value for the variable correctly. Finally, by parsing a value for the variable with the correct prefix, Terraform will behave exactly the same for that specific variable and value as if there was no validation defined at all.
We will talk much more about the concept of variable validation in later posts. Furthermore, both functions 'regexall' And 'length' Is useful in many more scenarios, but the most common one is the validation of values inside variables that are to be parsed as part of a Terraform configuration.
The concept of conversion functions:
Rather than this only being one function, rather its a group of functions, all designed to transform data between Terraform and different output types. With Terraform, we can 'encode' And 'decode' All the following formats:
YAML
CSV
JSON
Each of the formats has their own sets of an encode and decode function respectively. We can use this functionality to parse live data from Terraform runtime, like return objects from our code and to files or even the terminal itself in our favorite format. Lets first define what the concepts of 'encode' And 'decode' Means:
Encode -> We transform direct Terraform objects structures to a single string in a given format. We use this concept when we want to output something from Terraform and into something else outside of it.
Decode -> We import string data (does NOT have to be a single string) in a given format and lets Terraform transform it to objects during runtime, so it can be used in a configuration. We use this concept to input data into Terraform runtime.
Say we wanted to have all our Vnet´s in their raw data form inside a json specific file on our local disc. To do this, we can use the resource provider 'local' And the resource 'local_file' Where we parse all the Vnet's data during runtime. Lets first add the above described resource to the configuration of our Vnets:
resource "local_file" "vnet_objects_to_json" {
filename = "vnets_configuration.json"
content = jsonencode(azurerm_virtual_network.vnet_objects)
}Using the 'local_file' Resource type and providing a filename and parse the content, we can make Terraform create a file called 'vnets_configuration.json' On our local disc. The file will be placed in the same root folder as where the main.tf lays but it can be placed wherever we see fit, simply by defining a relative path to a wanted location. Furthermore, please note that we manually parse the extension to the filename, as otherwise Terraform will not add an extension at all - Hence, it does not automatically determine the correct file extension based on the content. Finally, notice we use the 'jsonencode' Function to transform our raw Terraform data into valid json.
Result:
If we open the file, we will see everything cramped together unto 1 single line. Terraform does this to make configuration files as small as possible in terms of size on the disk or in RAM dependent on how we get this json encoded content out.
Inside the file:
This is not very human friendly, even though it is perfectly syntax correctly defined json. We can use a tool like -> Best JSON Viewer and JSON Beautifier Online (codebeautify.org) to 'Beautify' It and make it way more readable.
This was all for today, thank you so much for reading this blog! I appriciate every single one of my readers and hope that my tips are useful. Until next time!
Remember, I post a new 'Terraform weekly-tips' Every Monday, so stay tuned!
Want to learn more about Terraform? Click here -> terraform (codeterraform.com)
Want to learn more about other cool stuff like Automation or Powershell -> powershell (codeterraform.com) / automation (codeterraform.com)
Комментарии