Code consistency with pre-commit checks
In recent years, I’ve been writing a lot of bash scripts, some Terraform and the odd bit of PowerShell. The main thing I miss from all of these new languages is the ability to format code upon saving the file, code completion (there are some VSCode plugins that have some abilities but not like Eclipse, writing Java, i.m.o) and the ability to see hints and tips from the IDE – I’m so used to Eclipse with Checkstyle, PMD, FindBugs and the likes, that writing in these new languages can at times be deeply frustrating in terms of the lack of good IDE tooling out there.
One thing I have been talking to my colleagues about for the last few years is the ability to check code before we commit it and preferably being blocked from committing code (pre-commit hooks) if it’s not up to basic standards.
A couple of weeks ago, I was watching Terraform at scale: lessons from looking at 100s of IaC setups in which Sören Martius talks about learnings from looking at hundreds of companies working with IaC. In the talk he mentions pre-commit checks using Anton Babenko’s pre-commit-terraform , for automatically formatting, validating and checking over Terraform code prior to committing changes.
I decided to test it out to see whether it would be worth introducing at work. One thing I do prefer is to use a Docker image if one is available as it means I don’t have to install a load of tools I’ll only ever use for one task on my home laptop and luckily there’s a Docker image available for pre-commit-terraform. But before I start running pre-commit-terraform, I need a some basic Terraform code to see what pre-commit-terraform does, so below is a very basic set of Terraform scripts for creating an AWS VPC plus an insecure ingress rule.
provider.tf
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 4.0"
}
}
}
provider "aws" {
region = "us-east-1"
access_key = "mysecretaccesskey"
secret_key = var.awsSecretKey
}
variables.tf
variable "awsAccessKey" {
type = string
sensitive = true
}
variable "awsSecretKey" {
type = string
sensitive = true
}
main.tf
resource "aws_vpc" "example" {
cidr_block = "10.0.0.0/16"
}
resource "aws_security_group_rule" "my-rule" {
type = "ingress"
cidr_blocks = ["0.0.0.0/0"]
}
For those of you who are eagle eyed, you will notice that there is an unused variable “awsAccessKey” as the value has been hardcoded deliberately to test out whether any of the pre-commit checks detect a key being checked in.
Now that I have a few files, it’s time to test out pre-commit-terraform. In the root of your repository that contains Terraform code, run the following:
docker pull ghcr.io/antonbabenko/pre-commit-terraform
echo "repos:
- repo: https://github.com/antonbabenko/pre-commit-terraform
rev: v1.76.0
hooks:
- id: terraform_fmt" > .pre-commit-config.yaml
docker run -v $(pwd):/lint -w /lint ghcr.io/antonbabenko/pre-commit-terraform run -a
If everything went well, you might see output like the following:
I did skip the pulling of the Docker image as I already have it downloaded.
The pre-commit check terraform_fmt ran but it failed and it also modified the three files according to Terraform’s fmt command. Neat 🙂
What happens if I run it again? This time the code isn’t formatted as the formatting has already been applied, so the hook passes:
Time to add in some more hooks. Edit .pre-commit-config.yaml and add the following hooks below terraform_fmt:
- id: terraform_docs
args:
- --args=--config=.terraform-docs.yml
- id: terraform_tflint
- id: terraform_tfsec
- id: terrascan
We’re also going to need the .terraform-docs.yml file, so create that file with the following content:
formatter: "markdown table" # this is required
header-from: main.tf
footer-from: ""
recursive:
enabled: false
path: modules
sections:
hide: []
show: []
content: ""
output:
file: "TFDOC.md"
mode: inject
template: |-
<!-- BEGIN_TF_DOCS -->
{{ .Content }}
<!-- END_TF_DOCS -->
output-values:
enabled: false
from: ""
sort:
enabled: true
by: name
settings:
anchor: true
color: true
default: true
description: false
escape: true
hide-empty: false
html: true
indent: 2
lockfile: true
read-comments: true
required: true
sensitive: true
type: true
Now re-run the Docker container. If all went well, you should have a lot of output on your screen – too much for a single screenshot!
First, terraform_fmt passed as the files haven’t been modified since we last ran terraform_fmt.
Next terraform_docs ran and passed. You should find you have a TFDOC.md file in your repo root, e.g.
terraform_tflint ran 3rd and failed because we need to add in the required_version field in the terraform block and remove the unused variable “awsAccessKey”.
updated copy of providers.tf
terraform {
required_version = "1.3.2"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 4.0"
}
}
}
provider "aws" {
region = "us-east-1"
access_key = "mysecretaccesskey"
secret_key = var.awsSecretKey
}
update copy of variables.tf
variable "awsSecretKey" {
type = string
sensitive = true
}
terraform_tfsec ran 4th and failed because of our rather too permissive ingress rule and because it could do with a description.
Finally, terrascan ran 5th and failed because we didn’t setup flow logging with our VPC.
That’s pretty cool for checking over Terraform code but can we check other files too? In the folder I’ve been using to test, I have a Java Maven project which I was using to test out the Terraform cdk. I’ll also add in a README.md with the following content:
* speling isnt my fortee
Given pre-commit-terraform uses pre-commit behind the scenes, we can add in any hooks (not sure if that’s entirely true but I was able to add in a number of hooks and didn’t have issues!) from the pre-commit website list https://pre-commit.com/hooks.html.
edited .pre-commit-config.yaml
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.3.0
hooks:
- id: end-of-file-fixer
- id: trailing-whitespace
- repo: https://github.com/antonbabenko/pre-commit-terraform
rev: v1.76.0
hooks:
- id: terraform_fmt
- id: terraform_docs
args:
- --args=--config=.terraform-docs.yml
- id: terraform_tflint
- id: terraform_tfsec
- id: terrascan
- repo: https://github.com/crate-ci/typos
rev: v1.12.10
hooks:
- id: typos
args: []
This time, we can see the Docker container pulling three repos and then executing the hooks in order. I took the below screenshot post an initial run with the new config, so “trim trailing whitespace” and “fix end of files” both went green in the screenshot but the first time I ran them they went red.
And the typos hook picked up my bad spelling, although it didn’t detect everything! 🙂
So, this is all great but I’m manually running the docker container ad hoc… how do we get it to be executed every time I try to commit changes? For that you will need a pre-commit script. There are examples in
#!/bin/bash
echo "hello world - this is a pre-commit hook"
docker run -v $(pwd):/lint -w /lint ghcr.io/antonbabenko/pre-commit-terraform run -a
And to test it out, stage a file and test committing it.
Below you can see I have staged one file and tried to commit the file. Upon running git commit, the pre-commit hook has kicked in and started running our pre-commit hooks:
Because I’ve still got one or two breaking hooks, the commit didn’t go ahead. But if I manually fix up (aka remove code 😂) to make the hooks pass, it successfully passes all the checks and commits the code 🙂
There are one or two things in my code that weren’t detected (the hardcoded AWS access key and one more deliberate spelling mistake) – I suspect I just need to find the right pre-commit hook(s). Overall, these pre-commit hooks give the ability to increase code consistency and check for basic errors which should therefore reduce pull request efforts, so they are definitely worthwhile and I’ll be looking to introduce this at work soon.
Please enable the Disqus feature in order to add comments