Config map-merge keeps both proto field spellings; Scan fails with duplicate field

Author: jnicholsonwasabiCreated Sep 14, 2026Updated Sep 14, 2026

Labelsbug

What happened:

Kratos merges config sources as map[string]any keyed by the literal YAML/JSON name. Scan then dumps that map as JSON and protojson unmarshals it. protojson treats the proto field name and the json name as the same field, so two layers that both set that field with each spelling and become an object with both keys:

proto: (line 1:22): duplicate field "apply_schema"

An image ships configs/config.yaml with proto names (apply_schema), an overlay uses protojson names (applySchema). Either spelling alone is valid. Together they abort the load.

A Go struct is worse. encoding/json binds one tag and drops the other, so Scan succeeds and the overlay looks applied and is not:

Layers	Destination	Result
`apply_schema: false` then `apply_schema: true`	Go struct or proto	ok, field is `true`
`apply_schema: true` then `applySchema: false`	Go struct (`json:"apply_schema"`)	ok, field stays `true` (overlay ignored)
`apply_schema: true` then `applySchema: false`	`proto.Message`	`duplicate field "apply_schema"`

The same split happens with a .env / env source that inserts the key as written: APP_applySchema=false next to YAML apply_schema: true is the proto failure; the other way around is the silent Go-struct miss.

What you expected to happen:

Merge by destination field, not by string key. Decode each layer first (protobuf field number / struct field), then let the later layer overwrite that field regardless of spelling.

Scan(v) already knows the destination. The types and the proto name / json name aliases are available without guessing them from the text. The natural fix is the same shape as #3881: coerce at scan time from the descriptor, or convert each layer to the destination type before mergeMap.

Please do not fix it by rejecting one spelling, or by making YAML keys case-fold. Both names are legal protojson. The bug is merging names before they have been resolved to a field.

Please do not fix it by inferring types from the text alone. Same reason as #3881.

How to reproduce it (as minimally and precisely as possible):

Self-contained, no protoc required:

package main

import (
	"fmt"
	"os"
	"path/filepath"

	"github.com/go-kratos/kratos/v3/config"
	"github.com/go-kratos/kratos/v3/config/file"
	"google.golang.org/protobuf/proto"
	"google.golang.org/protobuf/reflect/protodesc"
	"google.golang.org/protobuf/reflect/protoreflect"
	"google.golang.org/protobuf/types/descriptorpb"
	"google.golang.org/protobuf/types/dynamicpb"
)

// GoBootstrap binds only the proto field name. encoding/json will not see applySchema.
type GoBootstrap struct {
	ApplySchema bool `json:"apply_schema"`
}

func writeLayer(dir, name, body string) {
	_ = os.WriteFile(filepath.Join(dir, name), []byte(body), 0o600)
}

func load(dir string, v any) error {
	c := config.New(config.WithSource(
		file.NewSource(filepath.Join(dir, "config.yaml")),
		file.NewSource(filepath.Join(dir, "override.yaml")),
	))
	defer c.Close()
	if err := c.Load(); err != nil {
		return err
	}
	return c.Scan(v)
}

func bootstrapDesc() protoreflect.MessageDescriptor {
	fd, err := protodesc.NewFile(&descriptorpb.FileDescriptorProto{
		Name:    proto.String("demo.proto"),
		Syntax:  proto.String("proto3"),
		Package: proto.String("demo"),
		MessageType: []*descriptorpb.DescriptorProto{{
			Name: proto.String("Bootstrap"),
			Field: []*descriptorpb.FieldDescriptorProto{{
				Name:     proto.String("apply_schema"),
				JsonName: proto.String("applySchema"),
				Number:   proto.Int32(1),
				Label:    descriptorpb.FieldDescriptorProto_LABEL_OPTIONAL.Enum(),
				Type:     descriptorpb.FieldDescriptorProto_TYPE_BOOL.Enum(),
			}},
		}},
	}, nil)
	if err != nil {
		panic(err)
	}
	return fd.Messages().ByName("Bootstrap")
}

func protoVal(msg *dynamicpb.Message) string {
	return fmt.Sprintf("%v", msg.Get(msg.Descriptor().Fields().ByName("apply_schema")).Bool())
}

func main() {
	desc := bootstrapDesc()
	dir, _ := os.MkdirTemp("", "conf")
	defer os.RemoveAll(dir)

	report := func(name string, err error, got string) {
		if err != nil {
			fmt.Printf("%-40s FAIL  %v\n", name, err)
			return
		}
		fmt.Printf("%-40s ok    apply_schema=%s\n", name, got)
	}

	// Same spelling: map-merge overwrites, Scan succeeds.
	writeLayer(dir, "config.yaml", "apply_schema: false\n")
	writeLayer(dir, "override.yaml", "apply_schema: true\n")
	var gs GoBootstrap
	err := load(dir, &gs)
	report("go struct, same spelling", err, fmt.Sprintf("%v", gs.ApplySchema))

	pm := dynamicpb.NewMessage(desc)
	err = load(dir, pm)
	report("proto, same spelling", err, protoVal(pm))

	// Mixed spelling: map-merge keeps both keys.
	writeLayer(dir, "config.yaml", "apply_schema: true\n")
	writeLayer(dir, "override.yaml", "applySchema: false\n")

	gs = GoBootstrap{}
	err = load(dir, &gs)
	report("go struct, mixed spelling", err, fmt.Sprintf("%v", gs.ApplySchema))

	pm = dynamicpb.NewMessage(desc)
	err = load(dir, pm)
	report("proto, mixed spelling", err, protoVal(pm))
}

Produce output:

bash

$ # copy file above to main.go
$ go mod init kratos.repo
$ go mod tidy
$ go run ./main.go
go struct, same spelling                 ok    apply_schema=true
proto, same spelling                     ok    apply_schema=true
go struct, mixed spelling                ok    apply_schema=true
proto, mixed spelling                    FAIL  proto: (line 1:22): duplicate field "apply_schema"

The Go-struct mixed line is the silent miss: override was applySchema: false and the field stayed true.

Anything else we need to know?:

Where it happens

mergeMap in config/merge.go overwrites only when the string keys match. Scan in config/config.go calls reader.Source() and unmarshalJSON, which for a proto.Message is protojson.Unmarshal with DiscardUnknown: true (config/reader.go). protojson is right to reject the duplicate. The merge already lost the fact that the two keys are one field.

Same class as #3881

#3881 is "types exist only at Scan, and too late." This is "field identity exists only at Scan, and too late." Converting each layer to the destination type before merge would fix both: bools would be bools, and apply_schema / applySchema would be field 1.

proto.Merge after a per-layer unmarshal is the protobuf form of that. A destination-aware merge of the maps (walk the descriptor, resolve ByName or ByJSONName, write one canonical key) is the map form.

WithDecoder is not enough

A custom YAML decoder can rewrite keys in one file. It cannot see the other layers, so it cannot know that this file's applySchema must overwrite the previous file's apply_schema. The merge is what has to become field-aware.

Environment:

Kratos version (use kratos -v): module github.com/go-kratos/kratos/v3 v3.0.0
Go version (use go version): go version go1.26.5 darwin/arm64
OS (e.g: cat /etc/os-release): macOS 26.6.2 (build 25G83), Darwin 25.6.0 arm64

Source: go-kratos/kratos

View original on GitHub View discussion on GitHub