GitHub Actions
GitHub actions allow you to automate your software development and delivery process right from the place you store your code. It lets you Build, Test, and Deploy your applications. It has a generous free-tier that you most likely won't cross. So, it's free and powerful.
GitHub actions can help you automate different phases of software development and delivery. As mentioned above, theres phases usually include Build, Test, and Deploy.
GitHub Actions have 4 main sections:
- Event (such as Push and Pull Requests)
- Job (a series of steps with a shared purpose)
- Steps (phases of a job that need to happen one after another)
- Actions/Command (code or commands that need to be executed in a particular step)
GitHub actions must be inside your repo, under this folder: .github/workflows.
GitHub Action to make sure Python code is formatted
Python doesn't have an official formatter, so here we're using a popular one named black (it doesn't matter really. you can choose something else). In this action, we want to maker sure that anyone who makes a Pull Request (PR), has already formatted their code with black. If not, the action will fail and let us know that the PR should not be merged:
# any name you want for the action
name: automation
# which events trigger this action
# here we're saying pull requests to the `main` branch only
on:
pull_request:
branches:
- main
# you can have one or more jobs
jobs:
# job name
format:
# an instance to run the job on
runs-on: ubuntu-latest
# steps start here
steps:
# this step uses an action from the community to checkout the repo
# it will download the repo on the instance running the job
- name: GitHub checkout
uses: actions/checkout@v2
# this step runs a command to install `black`
- name: Install black
run: pip install black==22.*
# finally, this step checks to see if all the files
# are properly formatted
- name: Run black
run: black . --check
You can put the above workflow with any name under .github/workflows.
As mentioned above, GitHub Actions can have more than one job in a single workflow. Let's add another one.
GitHub Action to run our tests (Python)
As explained in the other class, we can use pytest to test our Python code. Let's add another job to the workflow above to do that.
...
test:
runs-on: ubuntu-latest
steps:
- name: GitHub checkout
uses: actions/checkout@v2
- name: Install Pytest
run: pip install pytest
- name: Run Tests
run: python -m pytest
GitHub Action to make sure Go code is formatted
Go, fortunately, comes with a formatter baked into the language tools (gofmt). In this action, we're using the tool to make sure all the files are properly formatted.
name: automation
on:
pull_request:
branches:
- main
jobs:
format:
runs-on: ubuntu-latest
steps:
- name: GitHub checkout
uses: actions/checkout@v2
# we need to install Go as it doesn't come pre-installed in the instance
- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1.19
# run a little script to see if any files is not formatted
- name: Run gofmt
run: if [ "$(gofmt -l . | wc -l)" -gt 0 ]; then exit 1; fi
GitHub Action to run our tests (Go)
test:
runs-on: ubuntu-latest
steps:
- name: GitHub checkout
uses: actions/checkout@v2
- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1.19
- name: Run Tests
run: go test -v ./...
GitHub Actions & AWS
So far, everything we've done wasn't related to AWS. In order to make changes on AWS (such as uploading a file to S3 or updating a Lambda function code), you need to give GitHub permission to do that. There are basically two ways to do this:
- Creating an IAM user and giving GitHub the access key id and secret access key
- Creating an IAM role and giving GitHub the role ARN
But regardless of the approach, we're going to use a GitHub action built by AWS to give our action access to do things on our behalf. Here the link to the action.
Using AWS credentials
For this approach, you need to create an IAM user and then generate programming credentials for it to get the access key id and secret access key (the same way you setup aws cli or aws-vault). After generating the keys, you can add this step to your action:
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: <access-key-id>
aws-secret-access-key: <secret-access-key>
aws-region: ca-central-1
Any step after that will have the permissions associated with the IAM user whose credentials are used here. For example, you can have a step like this after that:
- name: Copy files to the test website with the AWS CLI
run: |
aws s3 sync . s3://my-s3-test-website-bucket
Note that you will need to add these permission on the workflow level too so that GitHub can request a token from AWS:
permissions:
id-token: write # This is required for requesting the JWT
contents: read # This is required for actions/checkout
Using IAM roles
For this approach, we need to create an IAM role instead of a user and provide GitHub with the role ARN. This approach is more secure as even if people get their hands on the role ARN, they won't be able to do anything with it, as we'll associate the role to a certain repo name and branch. We're going to use Terraform to create the IAM role. The process of how to create such a role can be found here.
We're going to create a Terraform module for it.
main.tf
# Create an IAM OIDC identity provider that trusts GitHub
resource "aws_iam_openid_connect_provider" "github_actions" {
url = "https://token.actions.githubusercontent.com"
client_id_list = ["sts.amazonaws.com"]
thumbprint_list = [
data.tls_certificate.github.certificates[0].sha1_fingerprint
]
}
# Fetch GitHub's OIDC thumbprint
data "tls_certificate" "github" {
url = "https://token.actions.githubusercontent.com"
}
# Create role for the action
data "aws_iam_policy_document" "assume_role" {
statement {
actions = ["sts:AssumeRoleWithWebIdentity"]
effect = "Allow"
principals {
identifiers = [aws_iam_openid_connect_provider.github_actions.arn]
type = "Federated"
}
condition {
test = "StringEquals"
variable = "token.actions.githubusercontent.com:sub"
# The repos and branches defined in var.allowed_repos_branches
# will be able to assume this IAM role
values = [
for a in var.allowed_repos_branches :
"repo:<YOUR-GITHUB-HANDLE>/${a["repo"]}:ref:refs/heads/${a["branch"]}"
]
}
}
}
# Assign policy to the role
resource "aws_iam_role" "github_actions_role" {
name_prefix = var.role_name_prefix
assume_role_policy = data.aws_iam_policy_document.assume_role.json
}
resource "aws_iam_role_policy" "this" {
role = aws_iam_role.github_actions_role.name
policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:PutObject",
"s3:Get*"
],
"Resource": "*",
"Effect": "Allow"
},
{
"Action": [
"lambda:UpdateFunctionCode"
],
"Resource": "*",
"Effect": "Allow"
}
]
}
POLICY
}
variables.tf
variable "role_name_prefix" {
type = string
}
variable "allowed_repos_branches" {
description = "GitHub repos/branches allowed to assume the IAM role."
type = list(object({
repo = string
branch = string
}))
}
outputs.tf
output "role_arn" {
value = aws_iam_role.github_actions_role.arn
}
After creating the role, you can use the action like this:
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
role-to-assume: <role-arn>
aws-region: ca-central-1
Any step after this step will have the permissions associated with the role.
Using GitHub Action secrets
For sensitive data, such as AWS credentials, we can use Action secrets on GitHub. To create a secret, you need to Settings > Secrets > Actions and create a new one. Then, in your action files, you can reference them like this:
${{ secrets.MY_SECRET }}
This way, you won't need to push your sensitive data into the repository.
Example (Python Lambda)
Here's an example of packaging and updating a Lambda function written in Python:
name: ci/cd
on:
push:
branches:
- main
permissions:
id-token: write
contents: read
jobs:
deploy:
runs-on: ubuntu-latest
env:
BUCKET_NAME: bucket-name
FOLDER_NAME: python-test
steps:
- name: GitHub checkout
uses: actions/checkout@v2
- name: AWS
uses: aws-actions/configure-aws-credentials@master
with:
role-to-assume: ${{ secrets.ROLE_ARN }}
aws-region: ca-central-1
- name: Package
run: |
mkdir ./package && pip install -r requirements.txt -t ./package
cd package && zip -r9 ../artifact.zip .
cd ../ && zip -g artifact.zip *.py
aws s3 cp artifact.zip s3://${{ env.BUCKET_NAME }}/${{ env.FOLDER_NAME }}/artifact.zip
- name: Update
run: aws lambda update-function-code --function-name python-test --s3-key ${{ env.FOLDER_NAME }}/artifact.zip --s3-bucket ${{ env.BUCKET_NAME }}
Example (Go Lambda)
Here's an example of packaging and updating a Lambda function written in Go:
name: ci/cd
on:
push:
branches:
- main
permissions:
id-token: write
contents: read
jobs:
deploy:
runs-on: ubuntu-latest
env:
BUCKET_NAME: bucket-name
FOLDER_NAME: go-test
GOOS: linux
CGO_ENABLED: 0
GOARCH: amd64
steps:
- name: GitHub checkout
uses: actions/checkout@v2
- name: AWS
uses: aws-actions/configure-aws-credentials@master
with:
role-to-assume: ${{ secrets.ROLE_ARN }}
aws-region: ca-central-1
- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1.19
- name: Package
run: |
go build -o main .
zip artifact.zip main
aws s3 cp artifact.zip s3://${{ env.BUCKET_NAME }}/${{ env.FOLDER_NAME }}/artifact.zip
- name: Update
run: aws lambda update-function-code --function-name go-test --s3-key ${{ env.FOLDER_NAME }}/artifact.zip --s3-bucket ${{ env.BUCKET_NAME }}