helm-unittest

# helm unittest Unit test for _helm chart_ in YAML to keep your chart consistent and robust! Features: - write test file in pure YAML - render locally - create **nothing** on your cluster - [wildcard selection for templates](./DOCUMENT.md#test-suite) - [define values and release options](./DOCUMENT.md#test-job) - [snapshot testing](#snapshot-testing) - [test suite code completion and validation](#test-suite-code-completion-and-validation) ## Documentation If you are ready for writing tests, check the [DOCUMENT](./DOCUMENT.md) for the test API in YAML. - [Install](#install) - [Docker Usage](#docker-usage) - [Get Started](#get-started) - [Test Suite File](#test-suite-file) - [Templated Test Suites](#templated-test-suites) - [Usage](#usage) - [Flags](#flags) - [Yaml JsonPath Support](#yaml-jsonpath-support) - [DocumentSelector](#documentselector) - [Example](#example) - [Open Source Community Examples](#open-source-community-examples) - [Snapshot Testing](#snapshot-testing) - [Dependent subchart Testing](#dependent-subchart-testing) - [Tests within subchart](#tests-within-subchart) - [Test suite code completion and validation](#test-suite-code-completion-and-validation) - [Frequently Asked Questions](#frequently-asked-questions) - [Related Projects / Commands](#related-projects--commands) - [Contributing](#contributing) ## Install When not defining any versions, it will install the latest version of binary into helm plugin directory, otherwise it will install the specified version. Using Helm 3: ``` $ helm plugin install https://github.com/helm-unittest/helm-unittest.git ``` Using Helm 4*: ``` $ helm plugin install https://github.com/helm-unittest/helm-unittest.git --verify=false ``` Using OCI download**: ``` $ helm plugin install oci://ghcr.io/helm-unittest/helm-unittest/unittest:latest ``` or using http download***: ``` $ helm plugin install https://github.com/helm-unittest/helm-unittest/releases/download/v${plugin_version}/unittest-${plugin_version}.tgz ``` __Notes:__
* for Helm 4, installation using webhooks GPG verification is not supported, so `--verify=false` is required when installing from git repository.
** when using oci download, please note the following limitations: - the download contains all os and architecture binaries, making the package larger; - the download is only supported since plugin version 1.1.0 and later; - for helm 4 the archive download can perform a GPG verification, when the public-key.asc is imported into the gpg store.
*** when using http download, please note the same limitations as the oci download, including: - the download can only have a fixed version, which needs to be filled twice in the url; - the archive download does not support auto update of the plugin __Importing the public key for GPG verification:__ ``` # Import the public key curl -SsL https://github.com/helm-unittest/helm-unittest/raw/refs/heads/main/public-key.asc | gpg --import # Convert your keyring to the legacy gpg format # See https://helm.sh/docs/topics/provenance/ gpg --export > ~/.gnupg/pubring.gpg ``` ## Docker Usage ``` … ``` The docker container contains the fully installed helm client, including the helm-unittest plugin. ## Get Started Add `tests` in `.helmignore` of your chart, and create the following test file at `$YOUR_CHART/tests/deployment_test.yaml`: ```yaml suite: test deployment templates: - deployment.yaml tests: - it: should work set: image.tag: latest asserts: - isKind: of: Deployment - matchRegex: path: metadata.name pattern: -my-chart$ - equal: path: spec.template.spec.containers[0].image value: nginx:latest ``` and run: ``` $ helm unittest $YOUR_CHART ``` Now there is your first test! ;) ## Test Suite File The test suite file is written in pure YAML, and default placed under the `tests/` directory of the chart with suffix `_test.yaml`. You can also have your own suite files arrangement with `-f, --file` option of cli set as the glob patterns of test suite files related to chart directory, like: ```bash $ helm unittest -f 'my-tests/*.yaml' -f 'more-tests/**/*.yaml' my-chart ``` Check [DOCUMENT](./DOCUMENT.md) for more details about writing tests. ### Templated Test Suites You may find yourself needing to set up a lot of tests that are a parameterization of a single test. For instance, let's say that you deploy to 3 environments `env = dev | staging | prod`. In order to do this, you can actually write your tests as a helm chart as well. If you go this route, you must set the `--chart-tests-path` option. Once you have done so, helm unittest will run a standard helm render against the values.yaml in your specified directory. ``` /my-chart /tests-chart /Chart.yaml /values.yaml /templates /per_env_snapshots.yaml /Chart.yaml /values.yaml /.helmignore /templates /actual_template.yaml ``` In the above example file structure, you would maintain a helm chart that will render out against the Chart.yaml that is provided and the values.yaml. With rendered charts, any test suite that is generated is automatically ran we do not look for a file suffix or glob. **Note:** since you can create multiple suites in a single template file, you must provide the suite name, since we can no longer use the test suite file name meaningfully. **Note 2:** since you can be running against subcharts and multiple charts, you need to make sure that you do not designate your `--chart-tests-path` to be the same folder as your other tests. This is because we will try to render those non-helm test folders and fail during the unit test. **Note 3:** for snapshot tests, you will need to provide a helm ignore that ignores `*/__snapshot__/*`. Otherwise, subsequent runs will try to render those snapshots. The command for the above chart and test configuration would be: ```shell helm unittest --chart-tests-path tests-chart my-chart ``` ## Usage ``` $ helm unittest [flags] CHART [...] ``` This renders your charts locally (without tiller) and runs tests defined in test suite files. ### Flags ``` … ``` ### Yaml JsonPath Support Now JsonPath is supported for mappings and arrays. This makes it possible to find items in an array, based on JsonPath. For more detail on the [`jsonPath`](https://github.com/vmware-labs/yaml-jsonpath#syntax) syntax. Due to the change to JsonPath, the map keys in `path` containing periods (`.`) or special characters (`/`) are now supported with the use of `""`: ```yaml - equal: path: metadata.annotations["kubernetes.io/ingress.class"] value: nginx ``` In the next releases, it will be possible to validate multiple paths when JsonPath results in multiple results. ### DocumentSelector The test job or assertion can also specify a documentSelector rather than a documentIndex. Note that the documentSelector will always override a documentIndex if a match is found. This field is particularly useful when helm produces multiple templates and the order is not always guaranteed. The `path` in the documentSelector has Yaml JsonPath Support, using JsonPath expressions it is possible to filter on multiple fields. The `value` in the documentSelector can validate complete yaml objects and is optional. ```yaml ... tests: - it: should pass values: - ./values/staging.yaml set: image.pullPolicy: Always resources: limits: memory: 128Mi template: deployment.yaml documentSelector: path: metadata.name value: my-service-name asserts: - equal: path: metadata.name value: my-deploy ``` ## Example Check [`test/data/v3/basic/`](./test/data/v3/basic) for some basic use cases of a simple chart. ### Open Source Community Examples > Open-source solutions that uses helm-unittest to improve helm and kubernetes experience - [Traefik: kubernetes ingress](https://github.com/traefik/traefik-helm-chart/tree/master/traefik/tests) - [Prometheus: community charts](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack/unittests) - [Grafana: kubernetes monitoring](https://github.com/grafana/k8s-monitoring-helm/tree/main/charts/k8s-monitoring) - [HiveMQ: mqtt platform](https://github.com/hivemq/helm-charts/tree/develop/charts/hivemq-platform/tests) - [Gitlab runner](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/tree/main/tests?ref_type=heads) - [External DNS: kubernetes-sigs/external-dns](https://github.com/kubernetes-sigs/external-dns/tree/master/charts/external-dns/tests) ## Snapshot Testing Sometimes you may just want to keep the rendered manifest not changed between changes without every details asserted. That's the reason for snapshot testing! Check the tests below: ```yaml templates: - templates/deployment.yaml tests: - it: pod spec should match snapshot asserts: - matchSnapshot: path: spec.template.spec # or you can snapshot the whole manifest - it: manifest should match snapshot asserts: - matchSnapshot: {} - it: manifest should match snapshot and pattern and not match another pattern asserts: - matchSnapshot: matchRegex: pattern: .*app.* notMatchRegex: pattern: .*bcde.* ``` The `matchSnapshot` assertion validates the content rendered the same as cached last time. It fails if the content has changed, and you should check and update the cache with `-u, --update-snapshot` option of cli. ``` $ helm unittest -u my-chart ``` The cache files are stored as `__snapshot__/*_test.yaml.snap` at the directory your test file placed, you should add them in version control with your chart. ## Dependent subchart Testing If you have hard dependency subcharts (installed via `helm dependency`) existing in `charts` directory (they don't need to be extracted), it is possible to unittest these from the root chart. This feature can be helpful to validate if good default values are accidentally overwritten within your default helm chart. ```yaml # $YOUR_CHART/tests/xxx_test.yaml templates: - charts/postgresql/templates/xxx.yaml tests: - it: set: # this time required to prefix with "postgresql." postgresql.somevalue: should_be_scoped asserts: - ... ``` Note 1: if dependent subcharts uses an alias, use the alias name in the templates. Note 2: using the folder structure in templates can also be used to unittest templates which are placed in subfolders or unittest subcharts from the rootchart. Check [`test/data/v3/with-subchart/`](./test/data/v3/with-subchart) as an example. ## Tests within subchart If you have customized hard dependency subcharts (not installed via `helm dependency`, but added manually) existing in `charts` directory, tests inside would also be executed by default. You can disable this behavior by setting `--with-subchart=false` flag in cli, thus only the tests in root chart will be executed. Notice that the values defined in subchart tests will be automatically scoped, you don't have to add dependency scope yourself: ```yaml # with-subchart/charts/child-chart/tests/xxx_test.yaml templates: - templates/xxx.yaml tests: - it: set: # no need to prefix with "child-chart." somevalue: should_be_scoped asserts: - ... ``` Check [`test/data/v3/with-subchart/`](./test/data/v3/with-subchart) as an example. ## Test Suite code completion and validation Most popular IDEs (IntelliJ, Visual Studio Code, etc.) support applying schemas to YAML files using a JSON Schema. This provides comprehensive documentation as well as code completion while editing the test-suite file: In addition, test-suite files can be validated while editing so incorrectly added additional properties or incorrect data types can be detected while editing: ### Visual Studio Code