The Squire’s guide to automated deployments with Cloud Build

Audience: (Intermediate level) Targeting readers that have not yet interacted with Google Cloud before, but have engineering continuous integration, package management, beginner-level container and messaging experience. Requiring these individuals to have a pre-existing frontend application and supporting API server locally in place.Technologies: Cloud BuildCloud Build TriggersArtifact RegistryCloud RunPub/SubRequirements before getting started:Functional client-side repository Functional API server repositoryPre-existing GCP project with billing enabledUnix machineA Hero’s Journey – The Quest BeginsIn the initial stages of development, it’s easy to underestimate the grunt work needed to containerize and deploy your application, especially if you are new to the cloud. Could Google Cloud help you complete your project without adding too much bloat to the work? Let’s find out! This blog will take you on a quest to get to the heart of quick automated deployments by leveraging awesome features from the following products:Cloud Build: DevOps automation platformArtifact Registry: Universal package managerCloud Run: Serverless for containerized applicationsPub/Sub: Global real time messaging To help on this learning journey, we’d like to arm you with a realistic example of this flow as you are fashioning your own CI/CD pipeline. This blog will be referencing an open source Github project that models a best practices architecture using Google Cloud serverless patterns, Emblem.  (Note: References will be tagged with Emblem).Note: This blog will showcase the benefits of using Pub/Sub with multiple triggers, as it does in Emblem. If you are looking for a more direct path to building and deploying your containers with one trigger, check out the following quickstarts: ”Deploying to Cloud Run using Cloud Build” and “Deploying to Cloud Run using Cloud Deploy”. Quest goalsThe following goals will lead you to create a lean automated deployment flow for your API service that will be triggered by any change to the main branch of its source Github repository.Manual deployment with Cloud Build and Cloud RunBefore you run off and attempt to automate anything, you will need a solid understanding of what commands you will be adding to your future cloudbuild.yaml files.Build an image with a Cloud Build triggerCreating the first trigger and cloudbuild.yaml file in the Cloud Build product that will react to any new changes to the main branch of your Github project.React to Cloud Build events with Pub/SubUsing a cool in-built feature of Artifact Registry repositories, create a pubsub topic.Deploy with a Cloud Build triggerCreating a new Cloud Build trigger that listens to the above Pub/Sub topic and a new cloudbuild.yaml file that will initiate deployment of newly created container images from Artifact Registry.Before getting startedFor the purposes of this blog, the following is required:gcloud cli installed on Unix machineAn existing REST API server with associated DockerfileGoogle Cloud project with billing enabled (pricing)You will create a new Github project repository epic-quest-projectand adding your existing REST API server code directory (i.e Emblem:  content-api) to create the following project file structure:code_block[StructValue([(u’code’, u’epic-quest-project/rnu251cu2500u2500 ops/ # where build triggers will livernu2514u2500u2500 server-side/ # where your API server code livesrn u251cu2500u2500 main.pyrn u251cu2500u2500 requirements.txtrn u251cu2500u2500 Dockerfilern u2514u2500u2500 u2026′), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3e169be80b10>)])]Now onto the quest!Goal #1: Manual deployment with Cloud Build and Cloud RunYou will be building and deploying your containers using Google Cloud products, Cloud Build and Cloud Run, via the Google Cloud CLI, also known as gcloud. Within an open terminal, you will be setting up the following environment variables that declare the Google Cloud project ID,  and which region you will be basing your project from. You will also need to enable the following product APIs (Cloud Run, Cloud Build & Artifact Registry APIs) within the Google Cloud project.code_block[StructValue([(u’code’, u’# setting environment variablesrnexport PROJECT_ID=<add your project ID>rnexport REGION=<add your region/location>rnrn# enable relevant apisrngcloud services enable run.googleapis.com \rnartifactregistry.googleapis.com compute.googleapis.com cloudbuild.googleapis.com rnrn# update gcloud with project id and regionrngcloud config set project $PROJECT_IDrngcloud config set compute/region $REGION’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3e16ae17e8d0>)])]The container images you create from the server-side/ directory will be stored in an image repository named “epic-quest”, managed by Artifact Registry.code_block[StructValue([(u’code’, u’gcloud artifacts repositories create epic-quest –repository-format=”DOCKER” –location=$REGION’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3e16ae17ebd0>)])]Now that the “epic quest” Artifact Registry repository has been created you can begin pushing container images to it! Use gcloud builds submit to build and tag an image from the server-side/ directory with Artifact Registry repository specific format: <region>-docker.pkg.dev/<project-id>/<repository>code_block[StructValue([(u’code’, u’# root: epic-quest-projectrncd server-side/rnrn# root: create “server-side” imagerngcloud builds submit . –tag $REGION-docker.pkg.dev/$PROJECT_ID/epic-quest/server-side’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3e169901b450>)])]After pushing your server-side container image to the Artifact Registry repository, you’re all set to deploy it with Cloud Run!code_block[StructValue([(u’code’, u’gcloud run deploy –image=$REGION-docker.pkg.dev/$PROJECT_ID/epic-quest/server-side’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3e16994bc950>)])]code_block[StructValue([(u’code’, u’Service name: server-sidernAllow unauthenticated invocations to [server-side] (y/N)? y’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3e16994bc590>)])]Excellent, you’ve created a basic manual CI/CD pipeline! Now, you can explore what it looks like to have this pipeline automated.Goal #2: Build an image with a Cloud Build triggerTo start automating your small pipeline you will need to create a cloudbuild.yaml file that will configure your first Cloud Build trigger.In the ops directory of epic-quest-project, create a new file named api-build.cloudbuild.yaml. This new yaml file will describe the steps Cloud Build will use to build your container image and push it to the Artifact Registry.(Emblem: ops/api-build.cloudbuild.yaml)code_block[StructValue([(u’code’, u”touch ops/api-build.cloudbuild.yaml rnrnrn# api-build.cloudbuild.yaml contentsrnrnsteps:rn # Docker Build rn – name: ‘gcr.io/cloud-builders/docker’rn args: rn – ‘build’rn – ‘-t’rn – ‘${_REGION}-docker.pkg.dev/${PROJECT_ID}/epic-quest/server-side:${_IMAGE_TAG}’rn – ‘server-side/.’rnrn# Store in Artifact Registryrnimages:rn – ‘${_REGION}-docker.pkg.dev/${PROJECT_ID}/epic-quest/server-side:${_IMAGE_TAG}'”), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3e1699426b90>)])]To configure Cloud Build to automatically execute the steps in the above yaml, use the Cloud Console to create a new Cloud Build trigger:Remember to select `Push to a branch` as  the event that will activate the build trigger and, under `Source` connect your “epic-quest-project” Github repository.You may need to authenticate with your GitHub account credentials to connect a repository to your Google Cloud project. Once you have a repository connected, specify the location of the cloud build configuration in that repository:Submitting this configuration will create a new trigger named api-new-build that will be invoked whenever a change is committed and merged into the main branch of the repository with changes to the server-side/ folder.After committing your changes to server-side/ files locally, you can verify this trigger works by merging a new commit into the main branch of your repository. Once merged, you will be able to observe the build trigger at work in the Build History page of the Cloud Console.Excellent, the container build is now automated! How will Cloud Run know when a new build is ready to deploy? Enter Pub/Sub.Goal #3: React to Cloud Build events with Pub/SubBy default, Artifact Registry will publish messages about changes in its repositories to a Pub/Sub topic named gcr if it exists. Let’s take advantage of that feature for your next Cloud Build trigger. First, create a Pub/Sub topic named gcr:code_block[StructValue([(u’code’, u’gcloud pubsub topics create gcr’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3e168de1cc50>)])]Now, every time a new build is pushed to any Artifact Registry repository, a message is published to the gcr topic with a build digest that identifies that build. Next it’s time to configure your second trigger to complete the automated deployment pipeline. Goal #4: Deploy with a Cloud Build triggerNow you’re arriving at the final step, creating the deployment trigger! This Cloud Build trigger is the last link to complete your automated deployment story.Note: Read more about our opinionated way to perform this step here using Cloud Run with checkbox CD and check out the new support for Cloud Run in Cloud Deploy.In the ops directory of the epic-quest-project, create a new file named api-deploy.cloudbuild.yaml. In short, this will perform the deployment action of the new container image on your behalf. ( Emblem: ops/deploy.cloudbuild.yaml) .code_block[StructValue([(u’code’, u’touch ops/api-deploy.cloudbuild.yaml rnrnrn# api-deploy.cloudbuild.yaml contentsrnrnsteps:rn # Print the full Pub/Sub message for debuggingrn – id: “Echo Pub/Sub message” rn name: gcr.io/cloud-builders/gcloudrn entrypoint: /bin/bashrn args:rn – ‘-c’rn – |rn echo ${_BODY}rnrn # Cloud Run Deployrn – id: “Deploy to Cloud Run”rn name: gcr.io/cloud-builders/gcloudrn args:rn – runrn – deployrn – ${_SERVICE}rn – –image=${_IMAGE_NAME}rn – –region=${_REGION}rn – –revision-suffix=${_REVISION}rn – –project=${_PROJECT_ID}rn – –allow-unauthenticatedrn – –tag=${_IMAGE_TAG}’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3e16ad5ba150>)])]The first step in this Cloud Build configuration will print to the build job log the body of the message published by Artifact Registry and the second step will deploy to Cloud Run.Open the console and create another new Cloud Build Trigger with the following configuration:Instead of choosing a repository event like in the api-build trigger, select Pub/Sub message to create a subscription to the desired Pub/Sub topic along with the trigger:Once again, provide the location of the corresponding Cloud Build configuration file in the repository. Additionally, include values for the substitution variables that exist in the configuration file. Those variables are identifiable by the underscore prefix (_). Note that the _BODY,  _IMAGE_NAME and _REVISION variables reference data included in the body of the Pub/Sub message:The Cloud Build service account by default will initiate the deployment to Cloud Run, so it will need to have Cloud Run Developer and Service Account User IAM rolesgranted to it in the project where the Cloud Run services reside.After granting those roles, check that the pipeline is working by creating a commit to the server-side/ directory in your epic-quest-project GitHub repository. It should result in the automatic invocation of the api-new-build trigger followed closely by the api-deploy trigger, and finally with a new revision in the corresponding Cloud Run service.Your final project setup should similar to the following:code_block[StructValue([(u’code’, u’epic-quest-project/rnu2514u2500u2500 ops/rn u251cu2500u2500 api-build.cloudbuild.yamlrn u2514u2500u2500 api-deploy.cloudbuild.yamlrnu2514u2500u2500 server-side/rn u251cu2500u2500 main.pyrn u251cu2500u2500 requirements.txtrn u251cu2500u2500 Dockerfilern u2514u2500u2500 u2026′), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3e16996a9d90>)])]Quest complete!Excellent, you now have a shiny automated pipeline and leveled up your deployment game! After reading today’s post, we hope you have a better understanding of how to manually create and spin up a container using just Cloud Build and Cloud Run, use Cloud Build triggers to react to Github repository actions, writing cloudbuild.yaml files to add additional configuration to your build triggers, and magical benefits of using Artifact Registry repositories.If you want to learn even more, check out the open source serverless project Emblem on Github.
Quelle: Google Cloud Platform