krane

krane

This project used to be called kubernetes-deploy. Check out our migration guide for more information including details about breaking changes.

krane is a command line tool that helps you ship changes to a Kubernetes namespace and understand the result. At Shopify, we use it within our much-beloved, open-source Shipit deployment app.

Why not just use the standard kubectl apply mechanism to deploy? It is indeed a fantastic tool; krane uses it under the hood! However, it leaves its users with some burning questions: What just happened? Did it work?

Especially in a CI/CD environment, we need a clear, actionable pass/fail result for each deploy. Providing this was the foundational goal of krane, which has grown to support the following core features:

​:eyes: Watches the changes you requested to make sure they roll out successfully.

:interrobang: Provides debug information for changes that failed.

:1234: Predeploys certain types of resources (e.g. ConfigMap, PersistentVolumeClaim) to make sure the latest version will be available when resources that might consume them (e.g. Deployment) are deployed.

:closed_lock_with_key: Creates Kubernetes secrets from encrypted EJSON, which you can safely commit to your repository

​:running: Running tasks at the beginning of a deploy using bare pods (example use case: Rails migrations)

If you need the ability to render dynamic values in templates before deploying, you can use krane render. Alongside that, this repo also includes tools for running tasks and restarting deployments.

---

Table of contents

KRANE DEPLOY

• Prerequisites
• Installation
• Usage
  • Using templates
  • Customizing behaviour with annotations
  • Running tasks at the beginning of a deploy
  • Deploying Kubernetes secrets (from EJSON)
  • Deploying custom resources
• Walk through the steps of a deployment

KRANE GLOBAL DEPLOY

• Usage

KRANE RESTART

• Usage

KRANE RUN

• Prerequisites
• Usage

KRANE RENDER

• Prerequisites
• Usage

CONTRIBUTING

• Contributing
• Code of Conduct
• License

---

Prerequisites

• Ruby 2.7+
• Your cluster must be running Kubernetes v1.24.0 or higher1

Compatibility

1 We run integration tests against these Kubernetes versions. You can find our official compatibility chart below.

Krane provides support for official upstream supported versions Kubernetes, Ruby that are part of the compatibility matrix; Nevertheless, older releases are still likely to work.

Kubernetes version	Currently Tested?	Last officially supported in gem version
1.29	Yes	--
1.30	Yes	--
1.31	Yes	--
1.32	Yes	--
1.33	Yes	--
1.34	Yes	--

Installation

1. Install kubectl (requires v1.28.0 or higher) and make sure it is available in your $PATH
2. Set up your kubeconfig file for access to your cluster(s).
3. gem install krane

Usage

krane deploy <app's namespace> <kube context>

Environment variables:

• $KUBECONFIG: points to one or multiple valid kubeconfig files that include the context you want to deploy to. File names are separated by colon for Linux and Mac, and semi-colon for Windows. If omitted, Krane will use the Kubernetes default of ~/.kube/config.
• $GOOGLE_APPLICATION_CREDENTIALS: points to the credentials for an authenticated service account (required if your kubeconfig user's auth provider is GCP)

Options:

Refer to krane help for the authoritative set of options.

• --filenames / -f [PATHS]: Accepts a list of directories and/or filenames to specify the set of directories/files that will be deployed, use - to specify reading from STDIN.
• --no-prune: Skips pruning of resources that are no longer in your Kubernetes template set. Not recommended, as it allows your namespace to accumulate cruft that is not reflected in your deploy directory.
• --global-timeout=duration: Raise a timeout error if it takes longer than duration for any resource to deploy.
• --selector: Instructs krane to only prune resources which match the specified label selector, such as environment=staging. If you use this option, all resource templates must specify matching labels. See Sharing a namespace below.
• --selector-as-filter: Instructs krane to only deploy resources that are filtered by the specified labels in --selector. The deploy will not fail if not all resources match the labels. This is useful if you only want to deploy a subset of resources within a given YAML file. See Sharing a namespace below.
• --no-verify-result: Skip verification that workloads correctly deployed.
• --protected-namespaces=default kube-system kube-public: Fail validation if a deploy is targeted at a protected namespace.
• --verbose-log-prefix: Add [context][namespace] to the log prefix

NOTICE: Deploy Secret resources at your own risk. Although we will fix any reported leak vectors with urgency, we cannot guarantee that sensitive information will never be logged.

Sharing a namespace

By default, krane will prune any resources in the target namespace which have the kubectl.kubernetes.io/last-applied-configuration annotation and are not a result of the current deployment process, on the assumption that there is a one-to-one relationship between application deployment and namespace, and that a deployment provisions all relevant resources in the namespace.

If you need to, you may specify --no-prune to disable all pruning behaviour, but this is not recommended.

If you need to share a namespace with resources which are managed by other tools or indeed other krane deployments, you can supply the --selector option, such that only resources with labels matching the selector are considered for pruning.

If you need to share a namespace with different set of resources using the same YAML file, you can supply the --selector and --selector-as-filter options, such that only the resources that match with the labels will be deployed. In each run of deploy, you can use different labels in --selector to deploy a different set of resources. Only the deployed resources in each run are considered for pruning.

Using templates

All templates must be YAML formatted. We recommended storing each app's templates in a single directory, {app root}/config/deploy/{env}. However, you may use multiple directories.

If you want dynamic templates, you may render ERB with krane render and then pipe that result to krane deploy -f -.

Customizing behaviour with annotations

• krane.shopify.io/timeout-override: Override the tool's hard timeout for one specific resource. Both full ISO8601 durations and the time portion of ISO8601 durations are valid. Value must be between 1 second and 24 hours.
  • Example values: 45s / 3m / 1h / PT0.25H
  • Compatibility: all resource types
• krane.shopify.io/required-rollout: Modifies how much of the rollout needs to finish before the deployment is considered successful.
  • Compatibility: Deployment
    • full: The deployment is successful when all pods in the new replicaSet are ready.
    • none: The deployment is successful as soon as the new replicaSet is created for the deployment.
    • maxUnavailable: The deploy is successful when minimum availability is reached in the new replicaSet. In other words, the number of new pods that must be ready is equal to spec.replicas - strategy.RollingUpdate.maxUnavailable (converted from percentages by rounding up, if applicable). This option is only valid for deployments that use the RollingUpdate strategy.
    • Percent (e.g. 90%): The deploy is successful when the number of new pods that are ready is equal to spec.replicas * Percent.
  • Compatibility: StatefulSet
    • full: The deployment is successful when all pods are ready.
• krane.shopify.io/predeployed: Causes a Custom Resource to be deployed in the pre-deploy phase.
  • Compatibility: Custom Resource Definition
  • Default: true
  • true: The custom resource will be deployed in the pre-deploy phase.
  • All other values: The custom resource will be deployed in the main deployment phase.
• krane.shopify.io/deploy-method-override: Cause a resource to be deployed by the specified kubectl command, instead of the default apply.
  • Compatibility: Cannot be used for PodDisruptionBudget, since it always uses create/replace-force
  • Accepted values: create, replace, and replace-force
  • Warning: Resources whose deploy method is overridden are no longer subject to pruning on deploy.
  • This feature is experimental and may be removed at any time.
• krane.shopify.io/skip-endpoint-validation: Skip endpoint validation for the service.
  • Compatibility: Service
  • Default: false
  • true: Endpoint validation is not performed during the deployment of the service.

Running tasks at the beginning of a deploy

To run a task in your cluster at the beginning of every deploy, simply include a Pod template in your deploy directory. krane will first deploy any ConfigMap and PersistentVolumeClaim resources present in the provided templates, followed by any such pods. If the command run by one of these pods fails (i.e. exits with a non-zero status), the overall deploy will fail at this step (no other resources will be deployed).

Requirements:

• The pod's name should include <%= deployment_id %> to ensure that a unique name will be used on every deploy (the deploy will fail if a pod with the same name already exists).
• The pod's spec.restartPolicy must be set to Never so that it will be run exactly once. We'll fail the deploy if that run exits with a non-zero status.
• The pod's spec.activeDeadlineSeconds should be set to a reasonable value for the performed task (not required, but highly recommended)

A simple example can be found in the test fixtures: test/fixtures/hello-cloud/unmanaged-pod-1.yml.erb.

The logs of all pods run in this way will be printed inline. If there is only one pod, the logs will be streamed in real-time. If there are multiple, they will be fetched when the pod terminates.

Deploying Kubernetes secrets (from EJSON)

Note: If you're a Shopify employee using our cloud platform, this setup has already been done for you. Please consult the CloudPlatform User Guide for usage instructions.

Since their data is only base64 encoded, Kubernetes secrets should