Multi-Account Cloud Deployment With Terraform And Github Actions

2024-06-26 1233 words 6 minutes

Contents

In this blog post, we will look at how to implement a multi-account deployment pipeline on AWS using GitHub Actions and Terraform.

We will assume that you have access to at least two AWS accounts: one to hold pipeline resources and one target account where resources will be deployed.

Architecture

We will use two accounts: a pipeline account and a target account. The target account is your dev/staging/prod account. Usually, there is more than one target account in a given pipeline, but we will use one for simplicity. The same approach can be extended to an arbitrary number of target accounts.

To avoid managing AWS credentials in GitHub, we will use the GitHub OIDC provider and a role that is assumed by the workflow.

In the pipeline account, we will deploy:

An S3 bucket to hold Terraform state
A DynamoDB table for state locking
A GitHub OIDC provider
A workflow role with access to Terraform backend and the target account role.

In the target account, we will deploy:

A deployment role with access to deploy resources.

We will use a CloudFormation template to define resources to prevent exposing programmatic access to the pipeline and target accounts. Deployed resources are not expected to change much, but feel free to translate them into Terraform files.

Target Account Deployment Role

The deployment role in the target account grants permissions to deploy resources. In our example, it will have admin permissions, but you can limit permissions based on your workflow requirements. The role principal is the pipeline account, assumed by the workflow role.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30


Transform: AWS::Serverless-2016-10-31

Parameters:

  PipelinesAccountId:
    Description: Account id of pipelines account
    Type: String


Resources:

  AssumeAdminPermissionsRole:
    Type: AWS::IAM::Role
    Properties:
      Description: Role to be assumed by pipelines account users to deploy resources.
      RoleName: target-account-deployment-role
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              AWS: !Sub "arn:aws:iam::${PipelinesAccountId}:root"
            Action: "sts:AssumeRole"
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AdministratorAccess

Outputs:
  TargetAccountDeploymentRoleArn:
    Description: Role arn that grants access to the pipelines
    Value: !GetAtt AssumeAdminPermissionsRole.Arn

Pipelines Account Resources

First, we will define the Terraform S3 bucket, DynamoDB table, and GitHub OIDC provider using the CloudFormation template provided below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61


AWSTemplateFormatVersion: "2010-09-09"

Resources:

  ########################################
  # TERRAFORM STATE BUCKET AND LOCK TABLE
  ########################################
  TerraformStateBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketEncryption:
        ServerSideEncryptionConfiguration:
          - ServerSideEncryptionByDefault:
              SSEAlgorithm: AES256
      VersioningConfiguration:
        Status: Enabled


  TerraformLockTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: "tf-state-table"
      BillingMode: PAY_PER_REQUEST
      PointInTimeRecoverySpecification:
        PointInTimeRecoveryEnabled: true
      AttributeDefinitions:
        - AttributeName: LockID
          AttributeType: S
      KeySchema:
        - AttributeName: LockID
          KeyType: HASH
      SSESpecification:
        SSEEnabled: true


  ########################################
  # Github Oidc Provider
  ########################################
  GithubOidc:
    Type: AWS::IAM::OIDCProvider
    Properties:
      Url: https://token.actions.githubusercontent.com
      ClientIdList:
        - sts.amazonaws.com
      ThumbprintList:
        - ffffffffffffffffffffffffffffffffffffffff


Outputs:

  TerraformStateBucketName:
    Value: !Ref TerraformStateBucket
    Description: Name of terraform state S3 bucket

  TerraformLockTableName:
    Value: !Ref TerraformLockTable
    Description: Name of the DynamoDB lock table

  GithubOidcArn:
    Value: !Ref GithubOidc
    Description: Arn of github oidc provider

To deploy the workflow role, pass the information from the previous stack and GitHub details as parameters. The role is restricted to a single repository and the main branch, but you can customize it according to your setup.

If you have more than one target account, provide the account IDs as a comma-separated list.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103


AWSTemplateFormatVersion: "2010-09-09"
Transform: 'AWS::LanguageExtensions'

Parameters:

  GithubOrg:
    Description: Name of GitHub organization/user (case sensitive)
    Type: String

  RepositoryName:
    Description: Name of GitHub repository (case sensitive)
    Type: String

  GithubOidcArn:
    Description: Arn for the GitHub OIDC Provider.
    Type: String

  TerraformStateBucketName:
    Description: Name of terraform state bucket
    Type: String

  TerraformLockTableName:
    Description: Name of terraform lock table
    Type: String

  TargetAccounts:
    Description: List of account target account ids
    Type: CommaDelimitedList

Resources:

  GithubWorkflowRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub ghw-role-${RepositoryName}
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Action: sts:AssumeRoleWithWebIdentity
            Principal:
              Federated: !Ref GithubOidcArn
            Condition:
              StringEquals:
                token.actions.githubusercontent.com:aud: sts.amazonaws.com
              StringLike:
                token.actions.githubusercontent.com:sub: !Sub repo:${GithubOrg}/${RepositoryName}:ref:refs/heads/main
      Policies:
        - PolicyName: terraform-state-bucket-permissions
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: Allow
                Action:
                  - s3:GetObject
                  - s3:PutObject
                  - s3:DeleteObject
                  - s3:ListObjects
                  - s3:ListObjectsV2
                Resource:
                  - !Sub "arn:${AWS::Partition}:s3:::${TerraformStateBucketName}/*"
              - Effect: Allow
                Action:
                  - s3:ListBucket
                Resource:
                  - !Sub "arn:${AWS::Partition}:s3:::${TerraformStateBucketName}"
        - PolicyName: terraform-lock-table-permissions
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: Allow
                Action:
                  - dynamodb:DescribeTable
                  - dynamodb:GetItem
                  - dynamodb:PutItem
                  - dynamodb:DeleteItem
                Resource:
                  - !Sub "arn:${AWS::Partition}:dynamodb:${AWS::Region}:${AWS::AccountId}:table/${TerraformLockTableName}"


  'Fn::ForEach::Policies':
    - AccountId
    - !Ref TargetAccounts
    - 'AssumePolicy${AccountId}':
        Type: 'AWS::IAM::Policy'
        Properties:
          PolicyName: !Sub "assume-role-${AccountId}"
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: Allow
                Action:
                  - sts:AssumeRole
                Resource:
                    - !Sub "arn:aws:iam::${AccountId}:role/target-account-deployment-role"
          Roles:
            - !Ref GithubWorkflowRole


Outputs:

  GithubWorkflowRoleArn:
    Value: !GetAtt GithubWorkflowRole.Arn
    Description: Arn of github workflow role

Github Workflow & Terraform

An example GitHub workflow is shown below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43


name: Example pipeline

on:
  push:
    branches: [ main ]
  workflow_dispatch: {}

env:

  AWS_REGION : "<REGION>"
  AWS_PIPELINES_ROLE_ARN: "<WORKFLOW ROLE ARN>"

# Permission can be added at job level or workflow level
permissions:
      id-token: write   # This is required for requesting the JWT
      contents: read    # This is required for actions/checkout
jobs:
  AssumeRoleAndCallIdentity:
    runs-on: ubuntu-latest
    steps:
      - name: clone the repository
        uses: actions/checkout@v4

      - name: configure aws credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ env.AWS_PIPELINES_ROLE_ARN }}
          aws-region: ${{ env.AWS_REGION }}

      - name: setup terraform
        uses: hashicorp/setup-terraform@v2


      - name: apply terraform configuration
        run: |
          terraform init

          terraform workspace new sandbox || true
          terraform workspace select sandbox

          terraform plan
          terraform apply -auto-approve
          # terraform destroy -auto-approve

To execute Terraform commands, we assume the workflow role. This role grants access to the Terraform S3 backend and to the target account deployment role. The deployment role is used in the AWS provider configuration shown below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


terraform {
  backend "s3" {
    bucket         = "<STATE BUCKET NAME>"
    key            = "test-app/terraform.tfstate"
    region         = "<REGION>"
    dynamodb_table = "<LOCK TABLE NAME>"
  }

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = "<REGION>"
  assume_role {
    role_arn = "<TARGET ACCOUNT ROLE ARN>"
  }
}

Some Considerations Regarding Github Workflows

Since GitHub stores workflows in the same directory as the infrastructure, don’t forget to set up appropriate rule sets to improve security posture. If possible, limit file modifications to workflows, especially if you have a production workflow defined.

Conclusion

In this blog post, we have explored how to create a simple multi-account deployment pipeline on AWS using GitHub Actions and Terraform.

Happy engineering!