How to Use GitHub Actions to Build a Frontend and Backend CI/CD Pipeline

Background

The personal blog project has been launched, but due to the lack of an automated pipeline, I frequently need to manually handle packaging and deployment, which is cumbersome. Since the project code is maintained on GitHub, I am considering using GitHub Actions to introduce a CI/CD pipeline to automate the build and deployment process.

Project Parameters

Backend Framework: Spring Boot
Frontend Framework: React
Backend Cloud Service: AWS EC2 + Elastic Container Registry
Frontend Cloud Service: AWS S3 + CloudFront
Backend Container: Docker
Code Management: GitHub
Backend Package Management Tool: Gradle
Frontend Package Management Tool: npm

Expected Goals

Frontend CI/CD Configuration:
- Triggered when changes are made to the main branch.
- Execute npm run build to generate build files.
- Upload the build files to AWS S3.
- Invalidate CloudFront cache.
Backend CI/CD Configuration:
- Triggered when changes are made to the main branch.
- Execute gradle build to generate the application.
- Build the Docker image and push it to AWS ECR.
- Stop the old image on the EC2 instance and pull the new image for deployment.

Steps

AWS Setup

1. Create a New IAM User in AWS

First, create a new IAM user and configure the necessary permissions:

Log in to the AWS Management Console.
Go to the IAM (Identity and Access Management) console.
In the left navigation panel, select Users and click Add user.
Enter the username (e.g., github-ci-user).
Under permissions, choose Attach policies directly and add the following policies for the user.

2. Configure Permissions for the GitHub User

You need to provide the GitHub user with the following permissions:

S3 Permissions (for uploading files to S3)
To allow the GitHub user to upload frontend build files to AWS S3, assign the following permission:
- AmazonS3FullAccess: This policy grants full access to S3 buckets (including uploading and modifying files).
ECR Permissions (for uploading Docker images to ECR)
To allow the GitHub user to push Docker images to AWS ECR, assign the following permission:
- AmazonEC2ContainerRegistryFullAccess: This policy grants access to ECR, including creating, pushing, and pulling images.
EC2 Permissions (for stopping, starting, and updating Docker containers)
To allow the GitHub user to perform deployment operations on EC2 instances (e.g., pulling new Docker images and starting containers), configure the following permissions:
- AmazonEC2FullAccess: This policy grants full access to EC2 instances, including managing security groups and stopping and starting instances.
- AmazonSSMFullAccess: If you wish to manage EC2 instances through AWS Systems Manager (e.g., using RunCommand to execute remote commands), assign this policy.
CloudFront Permissions (for invalidating CloudFront cache)
To invalidate CloudFront cache, assign the following permission:
- CloudFrontFullAccess: This policy grants the user full access to CloudFront distributions, including invalidating cache.
IAM Permissions (for using AWS keys)
If you need to ensure the user can successfully log in to AWS and perform CLI operations, assign the following permission:
- IAMReadOnlyAccess: This policy grants read-only access to IAM settings if the user is only used for GitHub Actions and AWS interaction.

3. Generate Access Keys for the User

Generate and record the AWS access keys for the GitHub user (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY):

In the IAM user settings page, select the Security credentials tab.
Click Create access key and record the generated Access Key ID and Secret Access Key. Store them in GitHub Secrets for use in the GitHub Actions scripts.

GitHub Setup

1. Set Up GitHub Secrets

To protect sensitive information, we will use GitHub Secrets. The following Secrets need to be configured:

Log in to GitHub and navigate to your repository page.
Click Settings > Secrets And Variables > Actions > New repository secret. Create the following Secrets:
Frontend S3 and CloudFront:
- AWS_ACCESS_KEY_ID: The access key ID you created in AWS.
- AWS_SECRET_ACCESS_KEY: The corresponding secret key.
- AWS_REGION: The AWS region (e.g., us-west-1).
- CLOUDFRONT_DISTRIBUTION_ID: Your CloudFront distribution ID.
Backend ECR and EC2:
- AWS_ACCESS_KEY_ID: Your AWS access key ID.
- AWS_SECRET_ACCESS_KEY: The corresponding secret key.
- AWS_ACCOUNT_ID: Your AWS account ID.
- AWS_REGION: The AWS region (e.g., us-west-1).
- EC2_SSH_KEY: The private key content for the EC2 instance, stored as text.
- EC2_CONTAINER_IP: The public IP of the EC2 instance.
- EC2_CONTAINER_USER: The SSH login username for the EC2 instance.

2. Add CI/CD Scripts in GitHub

For both frontend and backend projects:
- Create a .github/workflows directory in your project folder.
- Create a ci.yml file in that directory and paste the CI/CD script content.
Add the frontend CI/CD script:

name: Frontend CI/CD Pipeline
on:
  push:
    branches:
      - main  # Monitor pushes to the main branch

jobs:
  build-and-delpoy:
    runs-on: ubuntu-latest
    steps:
    - name: Checkout code
      uses: actions/checkout@v3
    - name: Set up Node.js
      uses: actions/setup-node@v3
      with:
        node-version: '18'  # Use Node.js version 18
    - name: Install dependencies
      run: npm install
    - name: Build the project
      run: npm run build
    - name: Upload to S3
      uses: jakejarvis/s3-sync-action@master
      with:
        args: --acl public-read --follow-symlinks
      env:
        AWS_S3_BUCKET: 'my-blog-front-end-bucket'
        AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
        AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        AWS_REGION: 'us-west-1'   # optional: defaults to us-east-1
        SOURCE_DIR: './build'      # optional: defaults to entire repository
    - name: Invalidate CloudFront Cache
      uses: chetan/invalidate-cloudfront-action@v2
      env:
        AWS_REGION: 'us-west-1'
        DISTRIBUTION: 'E2Y5N6ITGS9BEG'  # Replace with your CloudFront distribution ID
        PATHS: '/*'  # Invalidate all files
        AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
        AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

Add the backend CI/CD script:

name: Backend CI/CD Pipeline
on:
  push:
    branches:
      - main  # Monitor pushes to the main branch
env:
  AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
  AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
  AWS_ACCOUNT_ID: ${{ secrets.AWS_ACCOUNT_ID }}
  AWS_REGION: ${{ secrets.AWS_REGION }}
  EC2_SSH_KEY: ${{ secrets.EC2_SSH_KEY }}
  EC2_CONTAINER_IP: ${{ secrets.EC2_CONTAINER_IP }}
  EC2_CONTAINER_USER: ${{ secrets.EC2_CONTAINER_USER }}
jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      - name: Set up JDK 17
        uses: actions/setup-java@v4
        with:
          distribution: 'corretto'
          java-version: '17'  # Set JDK version to 17
      - name: Set up Gradle
        uses: gradle/actions/wrapper-validation@v4
      - name: Build the project
        run: |
          chmod +x ./gradlew  # Add execute permission for gradlew script
          ./gradlew build
      - name: Log in to Amazon ECR
        uses: aws-actions/amazon-ecr-login@v2
        with:
          aws_access_key_id: ${AWS_ACCESS_KEY_ID}
          aws_secret_access_key: ${AWS_SECRET_ACCESS_KEY}
          region: ${AWS_REGION}  # Replace with your AWS region
      - name: Build Docker image
        run: |
          docker build -t blog:latest .
          docker tag blog:latest ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/victor/blog:latest
      - name: Push Docker image to ECR
        run: |
          docker push ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/victor/blog:latest
      - name: Set up SSH key
        run: |
          mkdir -p ~/.ssh
          echo "${EC2_SSH_KEY}" > ~/.ssh/id_rsa.pem
          chmod 600 ~/.ssh/id_rsa.pem
          # Get and add EC2 host's public key to known_hosts file
          ssh-keyscan -H 18.144.49.164 >> ~/.ssh/known_hosts
          # Ensure correct permissions for known_hosts file
          chmod 644 ~/.ssh/known_hosts
      - name: Deploy to EC2
        run: |
          ssh -i ~/.ssh/id_rsa.pem ${EC2_CONTAINER_USER}@${EC2_CONTAINER_IP} << 'EOF'
            sudo -s
            aws ecr get-login-password --region ${{ secrets.AWS_REGION }} | docker login --username AWS --password-stdin ${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.${{ secrets.AWS_REGION }}.amazonaws.com
            docker pull ${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.${{ secrets.AWS_REGION }}.amazonaws.com/victor/blog:latest
            # Stop and remove the existing container
            docker stop my-blog || true
            docker rm my-blog || true
            docker run -d --name my-blog -p 8080:8080 ${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.${{ secrets.AWS_REGION }}.amazonaws.com/victor/blog:latest
          EOF

Note:

GitHub Secrets: Used to store sensitive information, such as AWS keys, EC2 SSH keys, AWS account ID, etc.
- AWS_ACCESS_KEY_ID
- AWS_SECRET_ACCESS_KEY
- AWS_ACCOUNT_ID
- AWS_REGION
- EC2_SSH_KEY (SSH key for the EC2 instance)
Frontend:
- Build the project with npm run build and upload to S3.
- Use invalidate-cloudfront-action to invalidate CloudFront cache, so users can view the latest content promptly.
Backend:
- Build the Spring Boot project with gradle build.
- Build the Docker image and push it to AWS ECR.
- SSH into the EC2 instance, stop and remove the old containers, pull the new image, and start the new container.

3. Debugging

Push changes to the main branch or merge a PR into the main branch to trigger the script tests. The CI/CD scripts for both frontend and backend used in this article have been successfully debugged to achieve the intended goals.

Reference Documents

https://github.com/actions/setup-java
https://github.com/aws-actions/amazon-ecr-login
https://github.com/jakejarvis/s3-sync-action
https://github.com/chetan/invalidate-cloudfront-action