Run node.js database migrations on Google Cloud SQL during Google Cloud Build

Cloud Build runs using a service account and it looks like you need to grant access to Cloud SQL for this account. You can find additional info about setting service account permissions here.

In the case with Cloud SQL and Node.js it would look something like this:

steps:
  # Install Node.js dependencies
  - id: yarn-install
    name: gcr.io/cloud-builders/yarn
    waitFor: ["-"]

  # Install Cloud SQL proxy
  - id: proxy-install
    name: gcr.io/cloud-builders/yarn
    entrypoint: sh
    args:
      - "-c"
      - "wget https://storage.googleapis.com/cloudsql-proxy/v1.20.1/cloud_sql_proxy.linux.amd64 -O cloud_sql_proxy && chmod +x cloud_sql_proxy"
    waitFor: ["-"]

  # Migrate database schema to the latest version
  # https://knexjs.org/#Migrations-CLI
  - id: migrate
    name: gcr.io/cloud-builders/yarn
    entrypoint: sh
    args:
      - "-c"
      - "(./cloud_sql_proxy -dir=/cloudsql -instances=<CLOUD_SQL_CONNECTION> & sleep 2) && yarn run knex migrate:latest"
    timeout: "1200s"
    waitFor: ["yarn-install", "proxy-install"]

timeout: "1200s"

You would launch yarn install and download Cloud SQL Proxy in parallel. Once these two steps are complete, you run launch the proxy, wait 2 seconds and finally run yarn run knex migrate:latest.

For this to work you would need Cloud SQL Admin API enabled in your GCP project.

Where <CLOUD_SQL_INSTANCE> is your Cloud SQL instance connection name that can be found here. The same name will be used in your SQL connection settings, e.g. host=/cloudsql/example:us-central1:pg13.

Also, make sure that the Cloud Build service account has "Cloud SQL Client" role in the GCP project, where the db instance is located.

Use google-appengine/exec-wrapper. It is an image to do exactly this. Usage (see README in link):

steps:
- name: "gcr.io/google-appengine/exec-wrapper"
  args: ["-i", "gcr.io/my-project/appengine/some-long-name",
         "-e", "ENV_VARIABLE_1=value1", "-e", "ENV_2=value2",
         "-s", "my-project:us-central1:my_cloudsql_instance",
         "--", "bundle", "exec", "rake", "db:migrate"]

The -s sets the proxy target.

As of tag 1.16 of gcr.io/cloudsql-docker/gce-proxy, the currently accepted answer no longer works. Here is a different approach that keeps the proxy in the same step as the commands that need it:

  - id: cmd-with-proxy
    name: [YOUR-CONTAINER-HERE]
    timeout: 100s
    entrypoint: sh
    args:
      - -c
      - '(/workspace/cloud_sql_proxy -dir=/workspace -instances=[INSTANCE_CONNECTION_NAME] & sleep 2) && [YOUR-COMMAND-HERE]'

The proxy will automatically exit once the main process exits. Additionally, it'll mark the step as "ERROR" if either the proxy or the command given fails.

This does require the binary is in the /workspace volume, but this can be provided either manually or via a prereq step like this:

  - id: proxy-install
    name: alpine:3.10
    entrypoint: sh
    args:
      - -c
      - 'wget -O /workspace/cloud_sql_proxy https://storage.googleapis.com/cloudsql-proxy/v1.16/cloud_sql_proxy.linux.386 &&  chmod +x /workspace/cloud_sql_proxy'

Additionally, this should work with TCP since the proxy will be in the same container as the command.