vals

vals

vals is a tool for managing configuration values and secrets.

It supports various backends including:

• Vault

• AWS SSM Parameter Store

• AWS Secrets Manager

• AWS S3

• GCP Secrets Manager

• GCP KMS

• Google Sheets

• SOPS-encrypted files

• Terraform State

• 1Password

• 1Password Connect

• Doppler

• CredHub(Coming soon)

• Pulumi State

• Kubernetes

• Conjur

• HCP Vault Secrets

• Bitwarden

• Yandex Cloud Lockbox

• Servercore Secrets Manager

• HTTP JSON

• Keychain

• Scaleway

• Delinea SecretServer

• Infisical

• Use vals eval -f refs.yaml to replace all the refs in the file to actual values and secrets.

• Use vals exec -f env.yaml -- to populate envvars and execute the command.

• Use vals env -f env.yaml to render envvars that are consumable by eval or a tool like direnv

ToC:

• Installation
  • Building from source
• Usage
  • CLI
  • Helm
  • Go
• Expression Syntax
• Supported Backends

Installation

Download binary (Linux, macOS, Windows)

Download the latest executable for your platform and put it into a directory included in PATH.

homebrew (macOS, Linux)

brew install vals

Arch Linux

sudo pacman -S vals

Alpine Linux Edge

apk add vals

MacPorts (macOS)

sudo port install vals

Nix / NixOS

nix profile install nixpkgs#vals

Scoop (Windows)

scoop install vals

Building from source

go build -o vals ./cmd/vals

Custom builds

By default, all providers are compiled into the binary. If you only need specific providers, you can use Go build tags to produce a smaller binary:

# Only include Vault, AWS, GCP and Azure providers
go build -tags "custom_providers,vault,aws,gcp,azure" -o vals ./cmd/vals

The custom_providers tag switches to opt-in mode. Without it, all providers are included. Add the tags for the providers you need:

vault, openbao, aws, gcp, azure, terraform, gitlab, sops, onepassword, doppler, k8s, hcpvaultsecrets, conjur, bitwarden, scaleway, infisical, oci, httpjson, pulumi, secretserver, servercore, keychain, yandex

Utility providers (echo, file, envsubst, exec) are always included.

Use all_providers to explicitly include everything: go build -tags all_providers.

Usage

• CLI
• Helm
• Go

CLI

…

vals has a collection of providers that each an be referred with a URI scheme looks ref+.

For this example, use the Vault provider.

Let's start by writing some secret value to Vault:

$ vault kv put secret/foo mykey=myvalue

Now input the template of your YAML and refer to vals' Vault provider by using ref+vault in the URI scheme:

$ VAULT_TOKEN=yourtoken VAULT_ADDR=http://127.0.0.1:8200/ \
  echo "foo: ref+vault://secret/data/foo?proto=http#/mykey" | vals eval -f -

Voila! vals, replacing every reference to your secret value in Vault, produces the output looks like:

foo: myvalue

Which is equivalent to that of the following shell script:

…

aws kms encrypt
--key-id alias/example
--plaintext $(echo -n "hello, world" | base64 -w0)
--query CiphertextBlob
--output text | tr '/+' '_-'

…

echo test | gcloud kms encrypt
--project myproject
--location global
--keyring mykeyring
--key mykey
--plaintext-file -
--ciphertext-file -
| base64 -w0
| tr '/+' '_-'

…

$ tfstate-lookup -s ./terraform.tfstate module.vpc.aws_vpc.this[0].arn arn:aws:ec2:us-east-2:ACCOUNT_ID:vpc/vpc-0cb48a12e4df7ad4c

$ echo 'foo: ref+tfstate://terraform.tfstate/module.vpc.aws_vpc.this[0].arn' | vals eval -f - foo: arn:aws:ec2:us-east-2:ACCOUNT_ID:vpc/vpc-0cb48a12e4df7ad4c

You can also grab a Terraform output by using `output.OUTPUT_NAME` like:

$ tfstate-lookup -s ./terraform.tfstate output.mystack_apply

which is equivalent to the following input for `vals`:

$ echo 'foo: ref+tfstate://terraform.tfstate/output.mystack_apply' | vals eval -f -

Remote backends like S3, GCS and AzureRM blob store are also supported. When a remote backend is used in your terraform workspace, there should be a local file at `./terraform/terraform.tfstate` that contains the reference to the backend:

{ "version": 3, "serial": 1, "lineage": "f1ad69de-68b8-9fe5-7e87-0cb70d8572c8", "backend": { "type": "s3", "config": { "access_key": null, "acl": null, "assume_role_policy": null, "bucket": "yourbucketnname",

Just specify the path to that file, so that `vals` is able to transparently make the remote state contents available for you.

### Terraform in GCS bucket (tfstategs)

- `ref+tfstategs://bucket/path/to/some.tfstate/RESOURCE_NAME`

Examples:

- `ref+tfstategs://bucket/path/to/some.tfstate/google_compute_disk.instance.id`

It allows to use Terraform state stored in GCS bucket with the direct URL to it. You can try to read the state with command:

$ tfstate-lookup -s gs://bucket-with-terraform-state/terraform.tfstate google_compute_disk.instance.source_image_id 5449927740744213880

which is equivalent to the following input for `vals`:

$ echo 'foo: ref+tfstategs://bucket-with-terraform-state/terraform.tfstate/google_compute_disk.instance.source_image_id' | vals eval -f -

### Terraform in S3 bucket (tfstates3)

- `ref+tfstates3://bucket/path/to/some.tfstate/RESOURCE_NAME`

Examples:

- `ref+tfstates3://bucket/path/to/some.tfstate/aws_vpc.main.id`

It allows to use Terraform state stored in AWS S3 bucket with the direct URL to it. You can try to read the state with command:

$ tfstate-lookup -s s3://bucket-with-terraform-state/terraform.tfstate module.vpc.aws_vpc.this[0].arn arn:aws:ec2:us-east-2:ACCOUNT_ID:vpc/vpc-0cb48a12e4df7ad4c

which is equivalent to the following input for `vals`:

$ echo 'foo: ref+tfstates3://bucket-with-terraform-state/terraform.tfstate/module.vpc.aws_vpc.this[0].arn' | vals eval -f -

…

$ tfstate-lookup -s azurerm://my_rg/my_storage_account/terraform-backend/unique.terraform.tfstate output.virtual_network.name

which is equivalent to the following input for `vals`:

$ echo 'foo: ref+tfstateazurerm://my_rg/my_storage_account/terraform-backend/unique.terraform.tfstate/output.virtual_network.name' | vals eval -f -

…

$ tfstate-lookup -s remote://app.terraform.io/myorg/myworkspace output.virtual_network.name

which is equivalent to the following input for `vals`:

$ echo 'foo: ref+tfstateremote://app.terraform.io/myorg/myworkspace/output.virtual_network.name' | vals eval -f -

…

$ tfstate-lookup -s https://username:[email protected]/api/v4/projects/xx/terraform/state/xxx/ output.my_output

which is equivalent to following input for `vals` (with environment variables `GITLAB_USER` and `GITLAB_TOKEN`):

$ echo 'foo: ref+tfstategitlab://my-gitlab.com/api/v4/projects/xx/terraform/state/xxx/output.my_output' | vals eval -f -

Or you can provide credentials via URL parameters:

$ echo 'foo: ref+tfstategitlab://my-gitlab.com/api/v4/projects/xx/terraform/state/xxx/output.my_output?gitlab_user=username&gitlab_token=token' | vals eval -f -

…

sh export YC_TOKEN="$(yc iam create-token)" export YC_LOCKBOX_ENDPOINT="api.yandexcloud.kz:443" vals get 'ref+yclockbox://SECRET_ID'

…

json [ { "name": "chartify" }, { "name": "go-yaml" } ]

To get name="chartify" using https protocol you would use:

ref+httpjson://api.github.com/users/helmfile/repos#///*[1]/name

To get name="go-yaml" using https protocol you would use:

ref+httpjson://api.github.com/users/helmfile/repos#///*[2]/name

To get name="go-yaml" using http protocol you would use:

ref+httpjson://api.github.com/users/helmfile/repos?insecure=true#///*[2]/

#### Fetch integer value

`ref+httpjson:///?[insecure=false&floatAsInt=false]#/`

Let's say you want to fetch the below JSON object from https://api.github.com/users/helmfile/repos:
```json
[
    {
        "id": 251296379
    }
]

# Running the following will return: 2.51296379e+08
ref+httpjson://api.github.com/users/helmfile/repos#///*[1]/id

# Running the following will return: 251296379
ref+httpjson://api.github.com/users/helmfile/repos?floatAsInt=true#///*[1]/id

…

yaml
permissions:
  id-token: write
  contents: read

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Request OIDC token
        run: |
          TOKEN=$(curl -sS -H "Authorization: Bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
            "${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=https://github.com/${{ github.repository_owner }}" \
            | jq -r '.value')
          echo "::add-mask::$TOKEN"
          echo "INFISICAL_AUTH_JWT=$TOKEN" >> "$GITHUB_ENV"

      - name: Resolve secrets with vals
        env:
          INFISICAL_AUTH_METHOD: OIDC_AUTH
          INFISICAL_OIDC_AUTH_IDENTITY_ID: ${{ vars.INFISICAL_OIDC_AUTH_IDENTITY_ID }}
          # Optional: INFISICAL_URL for self-hosted / EU
        run: vals eval -f refs.yaml

…

yaml
myconfigvalue: ref+awsssm://myconfig/value
mysecretvalue: secretref+awssecrets://mysecret/value

To leverage GitOps most by allowing you to review the content of ref+awsssm://myconfig/value only, you run vals eval --exclude-secret to generate the following:

myconfigvalue: MYCONFIG_VALUE
mysecretvalue: secretref+awssecrets://mysecret/value

This is safe to be committed into git because, as you've told to vals, awsssm://myconfig/value is a config value that can be shared publicly.

Non-Goals

Complex String-Interpolation / Template Functions

In the early days of this project, the original author has investigated if it was a good idea to introduce string interpolation like feature to vals:

foo: xx${{ref "ref+vault://127.0.0.1:8200/mykv/foo?proto=http#/mykey" }}
bar:
  baz: yy${{ref "ref+vault://127.0.0.1:8200/mykv/foo?proto=http#/mykey" }}

But the idea had abandoned due to that it seemed to drive the momentum to vals being a full-fledged YAML templating engine. What if some users started wanting to use vals for transforming values with functions? That's not the business of vals.

Instead, use vals solely for composing sets of values that are then input to another templating engine or data manipulation language like Jsonnet and CUE.

Note though, vals does have support for simple string interpolation like usage. See Expression Syntax for more information.

Merge

Merging YAMLs is out of the scope of vals. There're better alternatives like Jsonnet, Sprig, and CUE for the job.