Building a Robust CI/CD Pipeline with GitHub Actions and GCP

Continuous Integration and Continuous Deployment (CI/CD) pipelines are crucial for modern software development. They automate the testing, building, and deployment processes, ensuring rapid and reliable delivery of applications. In this blog, we’ll explore how to create a CI/CD pipeline using GitHub Actions, Google Cloud Platform (GCP) Artifact Registry, and deploying to Google Cloud Run and Cloud Batch across development, staging, and production environments.

Why CI/CD?

CI/CD eliminates manual errors, provides consistency, and allows teams to deploy applications faster and with confidence. Here’s why this setup is beneficial:

• Automation: Reduces repetitive tasks and human error.
• Consistency: Ensures the same process is followed for every deployment.
• Rapid Feedback: Quickly identifies and addresses issues.
• Scalability: Easily handles multiple environments like dev, staging, and production.

The Workflow

We’ll configure our CI/CD pipeline with the following:

1. Development Environment (Dev): For testing new features.
2. Staging Environment (Stg): For integration testing and user acceptance testing (UAT).
3. Production Environment (Prd): For live deployments.

Each environment will have its own configuration and deployment process to ensure smooth transitions from one stage to the next.

Prerequisites

1. Google Cloud Project: Set up a GCP project.
2. Artifact Registry: Create a Docker Artifact Registry repository.
3. Service Accounts: Create service accounts with appropriate permissions for pushing Docker images and deploying to Cloud Run/Batch.
4. Dockerized Application: Ensure your Python application includes a Dockerfile.
5. GitHub Secrets: Store sensitive information such as GCP credentials, project IDs, and registry URLs in GitHub Secrets.

Step 1: Dockerize Your Application

Start with a simple Dockerfile for your Python application:

FROM python:3.9-slim

WORKDIR /app

COPY . /app

RUN pip install --no-cache-dir -r requirements.txt

EXPOSE 8080

CMD ["python", "app.py"]

This Dockerfile creates a lightweight image for your application and exposes port 8080 for external access.

Step 2: Configure GitHub Actions for CI/CD

We’ll create three workflows to handle deployments to dev, stg, and prd environments. Each workflow uses GitHub Actions to automate the process.

Development Workflow

Create a .github/workflows/dev-deploy.yml file:

name: Deploy to Dev

on:
  pull_request:
    branches:
      - feature/*

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v2

    - name: Set up Cloud SDK
      uses: google-github-actions/setup-gcloud@master
      with:
        version: 'latest'
        service_account_key: ${{ secrets.GCP_SERVICE_ACCOUNT_KEY_DEV }}
        project_id: ${{ secrets.GCP_PROJECT_ID }}

    - name: Configure Docker
      run: |
        gcloud auth configure-docker ${{ secrets.GCP_ARTIFACT_REGISTRY }}

    - name: Build Docker image
      run: |
        docker build -t ${{ secrets.GCP_ARTIFACT_REGISTRY }}/python-app-dev:latest .

    - name: Push Docker image to Artifact Registry
      run: |
        docker push ${{ secrets.GCP_ARTIFACT_REGISTRY }}/python-app-dev:latest

    - name: Deploy to Cloud Run
      run: |
        gcloud run deploy ${{ secrets.GCP_CLOUD_RUN_SERVICE_DEV }} \
          --image ${{ secrets.GCP_ARTIFACT_REGISTRY }}/python-app-dev:latest \
          --platform managed \
          --region us-central1 \
          --allow-unauthenticated

This workflow is triggered by pull requests to feature branches and deploys the application to the dev environment.

Staging Workflow

Create a .github/workflows/stg-deploy.yml file:

name: Deploy to Staging

on:
  push:
    branches:
      - main

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v2

    - name: Set up Cloud SDK
      uses: google-github-actions/setup-gcloud@master
      with:
        version: 'latest'
        service_account_key: ${{ secrets.GCP_SERVICE_ACCOUNT_KEY_STG }}
        project_id: ${{ secrets.GCP_PROJECT_ID }}

    - name: Configure Docker
      run: |
        gcloud auth configure-docker ${{ secrets.GCP_ARTIFACT_REGISTRY }}

    - name: Build Docker image
      run: |
        docker build -t ${{ secrets.GCP_ARTIFACT_REGISTRY }}/python-app-stg:latest .

    - name: Push Docker image to Artifact Registry
      run: |
        docker push ${{ secrets.GCP_ARTIFACT_REGISTRY }}/python-app-stg:latest

    - name: Deploy to Cloud Run
      run: |
        gcloud run deploy ${{ secrets.GCP_CLOUD_RUN_SERVICE_STG }} \
          --image ${{ secrets.GCP_ARTIFACT_REGISTRY }}/python-app-stg:latest \
          --platform managed \
          --region us-central1 \
          --allow-unauthenticated

This workflow is triggered by pushes to the main branch.

Production Workflow

Create a .github/workflows/prd-deploy.yml file:

name: Deploy to Production

on:
  push:
    branches:
      - production

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v2

    - name: Set up Cloud SDK
      uses: google-github-actions/setup-gcloud@master
      with:
        version: 'latest'
        service_account_key: ${{ secrets.GCP_SERVICE_ACCOUNT_KEY_PRD }}
        project_id: ${{ secrets.GCP_PROJECT_ID }}

    - name: Configure Docker
      run: |
        gcloud auth configure-docker ${{ secrets.GCP_ARTIFACT_REGISTRY }}

    - name: Build Docker image
      run: |
        docker build -t ${{ secrets.GCP_ARTIFACT_REGISTRY }}/python-app-prd:latest .

    - name: Push Docker image to Artifact Registry
      run: |
        docker push ${{ secrets.GCP_ARTIFACT_REGISTRY }}/python-app-prd:latest

    - name: Deploy to Cloud Run
      run: |
        gcloud run deploy ${{ secrets.GCP_CLOUD_RUN_SERVICE_PRD }} \
          --image ${{ secrets.GCP_ARTIFACT_REGISTRY }}/python-app-prd:latest \
          --platform managed \
          --region us-central1 \
          --allow-unauthenticated

This workflow is triggered by pushes to the production branch.

Interactions Between Environments

1. Development Phase:
   • Developers create feature branches (e.g., feature/add-new-feature).
   • Pull requests trigger the dev workflow for testing.
2. Staging Phase:
   • Approved pull requests are merged into the main branch, triggering the staging workflow.
   • Staging is used for integration testing and UAT.
3. Production Phase:
   • Once tested, a pull request is created to merge main into production.
   • Merging into production triggers the production workflow, deploying the application to live users.

Conclusion

This CI/CD pipeline ensures a smooth and automated process for deploying applications across dev, stg, and prd environments. By leveraging GitHub Actions and GCP, you can achieve fast, reliable, and scalable deployments, allowing your team to focus on delivering high-quality features.