viper

Viper v2 Feedback

Viper is heading towards v2 and we would love to hear what you would like to see in it. Share your thoughts here: https://forms.gle/R6faU74qPRPAzchZ9

Thank you!

Go configuration with fangs!

Many Go projects are built using Viper including:

• Hugo
• EMC RexRay
• Imgur’s Incus
• Nanobox/Nanopack
• Docker Notary
• BloomApi
• doctl
• Clairctl
• Mercure
• Meshery
• Bearer
• Coder
• Vitess

Install

go get github.com/spf13/viper

NOTE Viper uses Go Modules to manage dependencies.

Why use Viper?

Viper is a complete configuration solution for Go applications including 12-Factor apps. It is designed to work within any application, and can handle all types of configuration needs and formats. It supports:

• setting defaults
• setting explicit values
• reading config files
• dynamic discovery of config files across multiple locations
• reading from environment variables
• reading from remote systems (e.g. Etcd or Consul)
• reading from command line flags
• reading from buffers
• live watching and updating configuration
• aliasing configuration keys for easy refactoring

Viper can be thought of as a registry for all of your applications' configuration needs.

Putting Values in Viper

Viper can read from multiple configuration sources and merges them together into one set of configuration keys and values.

Viper uses the following precedence for merging:

• explicit call to Set
• flags
• environment variables
• config files
• external key/value stores
• defaults

NOTE Viper configuration keys are case insensitive.

Reading Config Files

Viper requires minimal configuration to load config files. Viper currently supports:

• JSON
• TOML
• YAML
• INI
• envfile
• Java Propeties

A single Viper instance only supports a single configuration file, but multiple paths may be searched for one.

Here is an example of how to use Viper to search for and read a configuration file. At least one path should be provided where a configuration file is expected.

// Name of the config file without an extension (Viper will intuit the type
// from an extension on the actual file)
viper.SetConfigName("config")

// Add search paths to find the file
viper.AddConfigPath("/etc/appname/")
viper.AddConfigPath("$HOME/.appname")
viper.AddConfigPath(".")

// Find and read the config file
err := viper.ReadInConfig()

// Handle errors
if err != nil {
	panic(fmt.Errorf("fatal error config file: %w", err))
}

You can handle the specific case where no config file is found.

var fileLookupError viper.FileLookupError
if err := viper.ReadInConfig(); err != nil {
    if errors.As(err, &fileLookupError) {
        // Indicates an explicitly set config file is not found (such as with
        // using `viper.SetConfigFile`) or that no config file was found in
        // any search path (such as when using `viper.AddConfigPath`)
    } else {
        // Config file was found but another error was produced
    }
}

// Config file found and successfully parsed

NOTE (since 1.6) You can also have a file without an extension and specify the format programmatically, which is useful for files that naturally have no extension (e.g., .bashrc).

Writing Config Files

At times you may want to store all configuration modifications made during run time.

// Writes current config to the path set by `AddConfigPath` and `SetConfigName`
viper.WriteConfig()
viper.SafeWriteConfig() // Like the above, but will error if the config file exists

// Writes current config to a specific place
viper.WriteConfigAs("/path/to/my/.config")

// Will error since it has already been written
viper.SafeWriteConfigAs("/path/to/my/.config")

viper.SafeWriteConfigAs("/path/to/my/.other_config")

As a rule of the thumb, methods prefixed with Safe won't overwrite any existing file, while other methods will.

Watching and Re-reading Config Files

Gone are the days of needing to restart a server to have a config take effect--Viper powered applications can read an update to a config file while running and not miss a beat.

It's also possible to provide a function for Viper to run each time a change occurs.

// All config paths must be defined prior to calling `WatchConfig()`
viper.AddConfigPath("$HOME/.appname")

viper.OnConfigChange(func(e fsnotify.Event) {
	fmt.Println("Config file changed:", e.Name)
})

viper.WatchConfig()

Reading Config from io.Reader

Viper predefines many configuration sources but you can also implement your own required configuration source.

viper.SetConfigType("yaml")

var yamlExample = []byte(`
hacker: true
hobbies:
- skateboarding
- snowboarding
- go
name: steve
`)

viper.ReadConfig(bytes.NewBuffer(yamlExample))

viper.Get("name") // "steve"

Setting Defaults

A good configuration system will support default values, which are used if a key hasn't been set in some other way.

Examples:

viper.SetDefault("ContentDir", "content")
viper.SetDefault("LayoutDir", "layouts")
viper.SetDefault("Taxonomies", map[string]string{"tag": "tags", "category": "categories"})

Setting Overrides

Viper allows explict setting of configuration, such as from your own application logic.

viper.Set("verbose", true)
viper.Set("host.port", 5899) // Set an embedded key

Registering and Using Aliases

Aliases permit a single value to be referenced by multiple keys

viper.RegisterAlias("loud", "Verbose")

viper.Set("verbose", true) // Same result as next line
viper.Set("loud", true)    // Same result as prior line

viper.GetBool("loud")    // true
viper.GetBool("verbose") // true

Working with Environment Variables

Viper has full support for environment variables.

NOTE Unlike other configuration sources, environment variables are case sensitive.

// Tells Viper to use this prefix when reading environment variables
viper.SetEnvPrefix("spf")

// Viper will look for "SPF_ID", automatically uppercasing the prefix and key
viper.BindEnv("id")

// Alternatively, we can search for any environment variable prefixed and load
// them in
viper.AutomaticEnv()

os.Setenv("SPF_ID", "13")

id := viper.Get("id") // 13

• By default, empty environment variables are considered unset and will fall back to the next configuration source, unless AllowEmptyEnv is used.
• Viper does not "cache" environment variables--the value will be read each time it is accessed.
• SetEnvKeyReplacer and EnvKeyReplacer allow you to rewrite environment variable keys, which is useful to merge SCREAMING_SNAKE_CASE environment variables with kebab-cased configuration values from other sources.

Working with Flags

Viper has the ability to bind to flags. Specifically, Viper supports pflag as used in the Cobra library.

Like environment variables, the value is not set when the binding method is called, but when it is accessed.

For individual flags, the BindPFlag method provides this functionality.

serverCmd.Flags().Int("port", 1138, "Port to run Application server on")

viper.BindPFlag("port", serverCmd.Flags().Lookup("port"))

You can also bind an existing set of pflags.

pflag.Int("flagname", 1234, "help message for flagname")
pflag.Parse()

viper.BindPFlags(pflag.CommandLine)

i := viper.GetInt("flagname") // Retrieve values from viper instead of pflag

The standard library flag package is not directly supported, but may be parsed through pflag.

package main

import (
	"flag"

	"github.com/spf13/pflag"
)

func main() {
	// Using standard library "flag" package
	flag.Int("flagname", 1234, "help message for flagname")

    // Pass standard library flags to pflag
	pflag.CommandLine.AddGoFlagSet(flag.CommandLine)
	pflag.Parse()

    // Viper takes over
	viper.BindPFlags(pflag.CommandLine)
}

Use of pflag may be avoided entirely by implementing the FlagValue and FlagValueSet interfaces.

// Implementing FlagValue

type myFlag struct {}
func (f myFlag) HasChanged() bool { return false }
func (f myFlag) Name() string { return "my-flag-name" }
func (f myFlag) ValueString() string { return "my-flag-value" }
func (f myFlag) ValueType() string { return "string" }

viper.BindFlagValue("my-flag-name", myFlag{})

// Implementing FlagValueSet

type myFlagSet struct {
	flags []myFlag
}
func (f myFlagSet) VisitAll(fn func(FlagValue)) {
	for _, flag := range flags {
		fn(flag)
	}
}

fSet := myFlagSet{
	flags: []myFlag{myFlag{}, myFlag{}},
}
viper.BindFlagValues("my-flags", fSet)

Remote Key/Value Store Support

To enable remote support in Viper, do a blank import of the viper/remote package.

import _ "github.com/spf13/viper/remote"

Viper supports the following remote key/value stores. Examples for each are provided below.

• Etcd and Etcd3
• Consul
• Firestore
• NATS

Viper will read a config string retrieved from a path in a key/value store.

Viper supports multiple hosts separated by ;. For example: http://127.0.0.1:4001;http://127.0.0.1:4002.

Encryption

Viper uses crypt to retrieve configuration from the key/value store, which means that you can store your configuration values encrypted and have them automatically decrypted if you have the correct GPG keyring. Encryption is optional.

Crypt has a command-line helper that you can use to put configurations in your key/value store.

$ go get github.com/sagikazarmark/crypt/bin/crypt
$ crypt set -plaintext /config/hugo.json /Users/hugo/settings/config.json
$ crypt get -plaintext /config/hugo.json

See the Crypt documentation for examples of how to set encrypted values, or how to use Consul.

Remote Key/Value Store Examples (Unencrypted)

etcd

viper.AddRemoteProvider("etcd", "http://127.0.0.1:4001","/config/hugo.json")
viper.SetConfigType("json") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop", "env", "dotenv"
err := viper.ReadRemoteConfig()

etcd3

viper.AddRemoteProvider("etcd3", "http://127.0.0.1:4001","/config/hugo.json")
viper.SetConfigType("json") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop", "env", "dotenv"
err := viper.ReadRemoteConfig()

Consul

Given a Consul key MY_CONSUL_KEY with the value:

{
    "port": 8080,
    "hostname": "myhostname.com"
}

viper.AddRemoteProvider("consul", "localhost:8500", "MY_CONSUL_KEY")
viper.SetConfigType("json") // Need to explicitly set this to json
err := viper.ReadRemoteConfig()

fmt.Println(viper.Get("port")) // 8080

Firestore

viper.AddRemoteProvider("firestore", "google-cloud-project-id", "collection/document")
viper.SetConfigType("json") // Config's format: "json", "toml", "yaml", "yml"
err := viper.ReadRemoteConfig()

Of course, you're allowed to use SecureRemoteProvider also.

NATS

viper.AddRemoteProvider("nats", "nats://127.0.0.1:4222", "myapp.config")
viper.SetConfigType("json")
err := viper.ReadRemoteConfig()

Remote Key/Value Store Examples (Encrypted)

viper.AddSecureRemoteProvider("etcd","http://127.0.0.1:400