berglas

Berglas

[][berglas-godoc]

Berglas is a command line tool and library for storing and retrieving secrets on Google Cloud. Secrets are encrypted with [Cloud KMS][cloud-kms] and stored in [Cloud Storage][cloud-storage]. An interoperable layer also exists with [Secret Manager][secret-manager].

• As a CLI, berglas automates the process of encrypting, decrypting, and storing data on Google Cloud.

• As a library, berglas automates the inclusion of secrets into various Google Cloud runtimes.

Berglas is not an officially supported Google product.

Setup

Prerequisites

1. Install the [Cloud SDK][cloud-sdk] for your operating system. Alternatively, you can run these commands from [Cloud Shell][cloud-shell], which has the SDK and other popular tools pre-installed.

   If you are running from your local machine, you also need Default Application Credentials:

   gcloud auth application-default login
   

   This will open a web browser and prompt for a login to your Google account. On headless devices, you will need to create a service account. For more information, please see the authentication section.

2. Install the berglas CLI using one of the following methods:

   • Install a pre-compiled binary for your operating system from the latest releases.

   • Use an official Docker container:

     docker run -it us-docker.pkg.dev/berglas/berglas/berglas
     

     Note: older Docker container images are available on Container Registry and Artifact Registry, but new versions are not published there.

   • Use Homebrew on macOS:

     brew install berglas
     

     Note: sometimes the Homebrew formula can be several versions behind.

   • Install from source (requires a working Go installation):

     go install github.com/GoogleCloudPlatform/berglas/v2@latest
     

3. Export your project ID as an environment variable. The rest of this setup guide assumes this environment variable is set:

   export PROJECT_ID=my-gcp-project-id
   

   Please note, this is the project ID, not the project name or project number. You can find the project ID by running gcloud projects list or in the web UI.

Secret Manager Storage

1. Enable required services on the project:

   gcloud services enable --project ${PROJECT_ID} \
     secretmanager.googleapis.com
   

Cloud Storage Storage

1. Export your desired Cloud Storage bucket name. The rest of this setup guide assumes this environment variable is set:

   export BUCKET_ID=my-secrets
   

   Replace my-secrets with the name of your bucket. Set only the name, without the gs:// prefix. This bucket should not exist yet!

2. Enable required services on the project:

   gcloud services enable --project ${PROJECT_ID} \
     cloudkms.googleapis.com \
     storage-api.googleapis.com \
     storage-component.googleapis.com
   

3. Bootstrap a Berglas environment. This will create a new Cloud Storage bucket for storing secrets and a Cloud KMS key for encrypting data.

   berglas bootstrap --project $PROJECT_ID --bucket $BUCKET_ID
   

   This command uses the default values. You can customize the storage bucket and KMS key configuration using the optional flags. Run berglas bootstrap -h for more details.

   If you want full control over the creation of the Cloud Storage and Cloud KMS keys, please see the [custom setup documentation][custom-setup].

4. (Optional) Bootstrap a Berglas environment specifying a bucket location. By default the berglas bucket is created in the multi-regional location US. You can specify your location by using the following command. Please see the list of supported locations in the GCP bucket location documentation page

   export BUCKET_LOCATION=europe-west1
   berglas bootstrap \
     --project $PROJECT_ID \
     --bucket $BUCKET_ID \
     --bucket-location $BUCKET_LOCATION
   

   This command uses the default values. You can customize the storage bucket and KMS key configuration using the optional flags. Run berglas bootstrap -h for more details.

   If you want full control over the creation of the Cloud Storage and Cloud KMS keys, please see the [custom setup documentation][custom-setup].

5. (Optional) Enable [Cloud Audit logging][cloud-audit] on the bucket:

   Please note this will enable audit logging on all Cloud KMS keys and all Cloud Storage buckets in the project, which may incur additional costs.

   1. Download the exiting project IAM policy:

      gcloud projects get-iam-policy ${PROJECT_ID} > policy.yaml
      

   2. Add Cloud Audit logging for Cloud KMS and Cloud Storage:

      cat > policy.yaml
      auditConfigs:
      - auditLogConfigs:
        - logType: DATA_READ
        - logType: ADMIN_READ
        - logType: DATA_WRITE
        service: cloudkms.googleapis.com
      - auditLogConfigs:
        - logType: ADMIN_READ
        - logType: DATA_READ
        - logType: DATA_WRITE
        service: storage.googleapis.com
      EOF
      

   3. Submit the new policy:

      gcloud projects set-iam-policy ${PROJECT_ID} policy.yaml
      

   4. Remove the updated policy from local disk:

      rm policy.yaml
      

CLI Usage

1. Create a secret:

   Using Secret Manager storage:

   berglas create sm://${PROJECT_ID}/foo my-secret-data
   

   Using Cloud Storage storage:

   berglas create ${BUCKET_ID}/foo my-secret-data \
     --key projects/${PROJECT_ID}/locations/global/keyRings/berglas/cryptoKeys/berglas-key
   

2. Grant access to a secret:

   Using Secret Manager storage:

   berglas grant sm://${PROJECT_ID}/foo --member user:[email protected]
   

   Using Cloud Storage storage:

   berglas grant ${BUCKET_ID}/foo --member user:[email protected]
   

3. Access a secret's data:

   Using Secret Manager storage:

   berglas access sm://${PROJECT_ID}/foo
   my-secret-data
   

   Using Cloud Storage storage:

   berglas access ${BUCKET_ID}/foo
   my-secret-data
   

4. Spawn a child process with secrets populated in the child's environment:

   berglas exec -- myapp --flag-a --flag-b
   

   This will spawn myapp with an environment parsed by berglas.

5. Access data from a specific version/generation of a secret:

   Using Secret Manager storage:

   berglas access sm://${PROJECT_ID}/foo#1
   my-previous-secret-data
   

   Using Cloud Storage storage:

   berglas access ${BUCKET_ID}/foo#1563925940580201
   my-previous-secret-data
   

6. Revoke access to a secret:

   Using Secret Manager storage:

   berglas revoke sm://${PROJECT_ID}/foo --member user:[email protected]
   my-previous-secret-data
   

   Using Cloud Storage storage:

   berglas revoke ${BUCKET_ID}/foo --member user:[email protected]
   

7. Delete a secret:

   Using Secret Manager storage:

   berglas delete sm://${PROJECT_ID}/foo
   

   Using Cloud Storage storage:

   berglas delete ${BUCKET_ID}/foo
   

In addition to standard Unix exit codes, if the CLI exits with a known error, Berglas will exit with one of the following:

• 60 - API error. Berglas got a bad response when communicating with an upstream API.

• 61 - Misuse error. You gave unexpected input or behavior. Please read the error message. Open an issue if you think this is a mistake.

The only exception is berglas exec, which will exit with the exit status of its child command, if one was provided.

Integrations

• App Engine (Flex) - When invoked via [App Engine Flex][app-engine-flex], Berglas resolves environment variables to their plaintext values using the [`berglas://reference syntax][reference-syntax]. This integration works with any language runtime because berglas serves as the entrypoint to the Docker container.

• App Engine (Standard) - When invoked via [App Engine][app-engine], Berglas resolves environment variables to their plaintext values using the [berglas://reference syntax][reference-syntax]. This integration only works with the Go language runtime because it requires importing the auto/ package.

• Cloud Run - When invoked via [Cloud Run][cloud-run], Berglas resolves environment variables to their plaintext values using the [berglas:// reference syntax][reference-syntax]. This integration works with any language runtime because berglas serves as the entrypoint to the Docker container.

• Cloud Functions - When invoked via [Cloud Functions][cloud-functions], Berglas resolves environment variables to their plaintext values using the [berglas:// reference syntax][reference-syntax]. This integration only works with the Go language runtime because it requires importing the auto/ package.

• Cloud Build - When invoked via [Cloud Build][cloud-build], Berglas resolves environment variables to plaintext values using the [berglas:// reference syntax][reference-syntax]. This integration only works with volume mounts, so all Berglas secrets need to specify the ?destination parameter.

• Kubernetes - Kubernetes pods can consume Berglas secrets by installing a [MutatingWebhook][k8s-mutating]. This webhook mutates incoming pods with the [berglas:// reference syntax][reference-syntax] in environment references to resolve at runtime. This integration works with any container, but all pods requesting berglas secrets must set an command in their Kubernetes manifests.

• Anything - Wrap any process with berglas exec -- and Berglas will parse any local environment variables with the [berglas:// reference syntax][reference-syntax] and spawn your app as a subprocess with the plaintext environment replaced.

Logging

Both the berglas CLI and berglas library support debug-style logging. This logging is off by default because it adds additional overhead and logs information that may be security-sensitive.

The default logging behavior for the berglas CLI is "text" (it can be changed with the --log-format flag). The default logging behavior for the berglas library is structured JSON which integrates well with Cloud Logging (it can be changed to any valid formatter and you can even inject your own logger).

Library Usage

Berglas is also a Go library that can be imported in Go projects:

import (
	_ "github.com/GoogleCloudPlatform/berglas/v2/pkg/auto"
)

When imported, the berglas package will:

1. Download and decrypt any secrets that match the [Berglas environment variable reference syntax][reference-syntax] in the environment.

2. Replace the value for the environment variable with the decrypted secret.

You can also opt out of auto-parsing and call the library yourself instead:

…

For more examples and documentation, please see the [godoc][berglas-godoc].

Authentication

By default, Berglas uses Google Cloud Default Application Credentials. If you have [gcloud][cloud-sdk] installed locally, ensure you have application default credentials:

gcloud auth application-default login

On GCP services (like Cloud Build, Compute, etc), it will use the service account attached to the resource.

To use a specific service account, set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the filepath to the JSON file where your credentials reside on disk: