百科.dev
全部条目在线工具AI 编程趋势榜开源项目技术资讯提交条目
登录
< 返回工具列表
G

gostatsd

> 编程语言
开源

Etsy 的 statsd 在 Go 中的实现,支持标签

398 stars0 点赞2 次浏览
访问官网GitHub

工具介绍

Etsy 的 statsd 在 Go 中的实现,支持标签

gostatsd

An implementation of [Etsy's][etsy] [statsd][statsd] in Go, based on original code from @kisielk.

The project provides both a server called "gostatsd" which works much like Etsy's version, but also provides a library for developing customized servers.

Backends are pluggable and only need to support the backend interface.

Being written in Go, it is able to use all cores which makes it easy to scale up the server based on load.

Building the server

[](# Go version needs to be the same in: CI config, README, Dockerfiles, and Makefile) Gostatsd currently targets Go 1.26.7. If you are compiling from source, please ensure you are running this version.

From the gostatsd directory run make build. The binary will be built in build/bin//gostatsd.

You will need to install the Golang build dependencies by running make setup in the gostatsd directory. This must be done before the first build, and again if the dependencies change. A protobuf installation is expected to be found in the tools/ directory. Managing this in a platform agnostic way is difficult, but PRs are welcome. Hopefully it will be sufficient to use the generated protobuf files in the majority of cases.

If you are unable to build gostatsd please check your Go version, and try running make setup again before reporting a bug.

Running the server

gostatsd --help gives a complete description of available options and their defaults. You can use make run to run the server with just the stdout backend to display info on screen.

You can also run through docker by running make run-docker which will use docker-compose to run gostatsd with a graphite backend and a grafana dashboard.

While not generally tested on Windows, it should work. Maximum throughput is likely to be better on a linux system, however.

The server listens for UDP packets by default. You can use unix sockets providing an absolute path to the socket in the metrics-addr configuration option. The socket mode used in this case is SOCK_DGRAM. Note that using unix sockets will only work on linux and that it will ignore conn-per-reader configuration option.

Configuring the server mode

The server can currently run in two modes: standalone and forwarder. It is configured through the top level server-mode configuration setting. The default is standalone.

In standalone mode, raw metrics are processed and aggregated as normal, and aggregated data is submitted to configured backends (see below)

This configuration mode allows the following configuration options:

  • expiry-interval: interval before metrics are expired, see Metric expiry and persistence section. Defaults to 5m. 0 to disable, -1 for immediate.
  • expiry-interval-counter: interval before counters are expired, defaults to the value of expiry-interval.
  • expiry-interval-gauge: interval before gauges are expired, defaults to the value of expiry-interval.
  • expiry-interval-set: interval before sets are expired, defaults to the value of expiry-interval.
  • expiry-interval-timer: interval before timers are expired, defaults to the value of expiry-interval.
  • flush-aligned: whether or not the flush should be aligned. Setting this will flush at an exact time interval. With a 10 second flush-interval, if the service happens to be started at 12:47:13, then flushing will occur at 12:47:20, 12:47:30, etc, rather than 12:47:23, 12:47:33, etc. This removes query time ambiguity in a multi-server environment. Defaults to false.
  • flush-interval: duration for how long to batch metrics before flushing. Should be an order of magnitude less than the upstream flush interval. Defaults to 1s.
  • flush-offset: offset for flush interval when flush alignment is enabled. For example, with an offset of 7s and an interval of 10s, it will flush at 12:47:10+7 = 12:47:17, etc.
  • ignore-host: indicates whether or not an explicit host field will be added to all incoming metrics and events. Defaults to false
  • max-readers: the number of UDP receivers to run. Defaults to 8 or the number of logical cores, whichever is less.
  • max-parsers: the number of workers available to parse metrics. Defaults to the number of logical cores.
  • max-workers: the number of aggregators to process metrics. Defaults to the number of logical cores.
  • max-queue-size: the size of the buffers between parsers and workers. Defaults to 10000, monitored via channel.* metric, with dispatch_aggregator_batch and dispatch_aggregator_map channels.
  • max-concurrent-events: the maximum number of concurrent events to be dispatching. Defaults to 1024, monitored via channel.* metric, with backend_events_sem channel.
  • estimated-tags: provides a hint to the system as to how many tags are expected to be seen on any particular metric, so that memory can be pre-allocated and reducing churn. Defaults to 4. Note: this is only a hint, and it is safe to send more.
  • log-raw-metric: logs raw metrics received from the network. Defaults to false.
  • metrics-addr: the address to listen to metrics on. Defaults to :8125. Using a file path instead of host:port will create a Unix Domain Socket in the specified path instead of using UDP.
  • namespace: a namespace to prefix all metrics with. Defaults to ''.
  • statser-type: configures where internal metrics are sent to. May be internal which sends them to the internal processing pipeline, logging which logs them, null which drops them. Defaults to internal, or null if the NewRelic backend is enabled.
  • percent-threshold: configures the "percentiles" sent on timers. Space separated string. Defaults to 90.
  • heartbeat-enabled: emits a metric named heartbeat every flush interval, tagged by version and commit. Defaults to false.
  • receive-batch-size: the number of datagrams to attempt to read. It is more CPU efficient to read multiple, however it takes extra memory. See [Memory allocation for read buffers] section below for details. Defaults to 50.
  • receive-buffer-size: the size of single buffer that's used to store datagram during read. Number of buffers is defined by receive-batch-size. Default to 65535 what is theoretical maximum UDP packet size.
  • conn-per-reader: attempts to create a connection for every UDP receiver. Not supported by all OS versions. It will be ignored when unix sockets are used for the connection. Defaults to false.
  • bad-lines-per-minute: the number of metrics which fail to parse to log per minute. This is used to prevent a bad client spamming malformed statsd data, while still logging some information to enable troubleshooting. Defaults to 0.
  • hostname: sets the hostname on internal metrics
  • timer-histogram-limit: specifies the maximum number of buckets on histograms. See [Timer histograms] below.

In forwarder mode, raw metrics are collected from a frontend, and instead of being aggregated they are sent via http to another gostatsd server after passing through the processing pipeline (cloud provider, static tags, filtering, etc).

A forwarder server is intended to run on-host and collect metrics, forwarding them on to a central aggregation service. At present the central aggregation service can only scale vertically, but horizontal scaling through clustering is planned.

Aligned flushing is deliberately not supported in forwarder mode, as it would impact the central aggregation server due to all for forwarder nodes transmitting at once, and the expectation that many forwarding flushes will occur per central flush anyway.

Configuring forwarder mode requires a configuration file, with a section named http-transport. The raw version spoken is not configurable per server (see HTTP.md for version guarantees). The configuration section allows the following configuration options:

  • compress: boolean indicating if the payload should be compressed. Defaults to true
  • compression-type: compression algorithm to use, one of zlib or lz4. Defaults to zlib. Note lz4 is non-standard so make sure the downstream consumer can handle Content-Encoding='lz4'.
  • compression-level: compression level to use (0-9). 0 = best speed, 9 = best compression. Defaults to 9.
  • api-endpoint: configures the endpoint to submit raw metrics to. This setting should be just a base URL, for example https://statsd-aggregator.private, with no path. Required, no default
  • max-requests: maximum number of requests in flight. Defaults to 1000 (which is probably too high)
  • concurrent-merge: maximum number of concurrent goroutines allowed to merge metrics before forwarding. Defaults to 1 for backward-compatibility
  • max-request-elapsed-time: duration for the maximum amount of time to try submitting data before giving up. This includes retries. Defaults to 30s (which is probably too high). Setting this value to -1 will disable retries.
  • consolidator-slots: number of slots in the metric consolidator. Memory usage is a function of this. Lower values may cause blocking in the pipeline (back pressure). A UDP only receiver will never use more than the number of configured parsers (--max-parsers option). Defaults to the value of --max-parsers, but may require tuning for HTTP based servers.
  • transport: see TRANSPORT.md for how to configure the transport.
  • custom-headers : a map of strings that are added to each request sent to allow for additional network routing / request inspection. Not required, default is empty. Example: --custom-headers='{"region" : "us-east-1", "service" : "event-producer"}'
  • dynamic-headers : similar with custom-headers, but the header values are extracted from metric tags matching the provided list of string. Tag names are canonicalized by replacing underscores with hyphens. If a tag is specified in both custom-header and dynamic-header, the vaule set by custom-header takes precedence. Not required, default is empty. Example: --dynamic-headers='["region", "service"]'. This is an experimental feature and it may be removed or changed in future versions.
  • fail-fast-on-non-retryable-errors: boolean flag that controls whether to immediately drop messages that encounter non-retryable HTTP errors (e.g., 4xx status codes indicating client errors). When set to true, non-retryable errors immediately drop the message without retry attempts. When set to false, even non-retryable errors will attempt retries via exponential backoff up to the max-request-elapsed-time limit. Note: Some errors (e.g., TLS certificate validation failures) are always treated as non-retryable. Defaults to false.

The following settings from the previous section are also supported:

  • expiry-*
  • ignore-host
  • max-readers
  • max-parsers
  • estimated-tags
  • log-raw-metric
  • metrics-addr
  • namespace
  • statser-type
  • heartbeat-enabled
  • receive-batch-size
  • conn-per-reader
  • bad-lines-per-minute
  • hostname
  • log-raw-metric

Running as a Lambda Extension (experimental feature)

Gostatsd can be run as a lambda extension in forwarder mode. The metrics are flushed at the end of each lambda invocation by default. The flush interval is ignored for your custom metrics, internal metrics are still flushed on a best effort basis using the configured flush interval.

To support flushes based on the runtime function, a lambda telemetry server is started at the reserved lambda hostname sandbox on port 8083. This can be configured by setting the lambda-extension-telemetry-address configuration parameter. This will need to be done if port 8083 is not available within the lambda runtime.

Issues· 50 开放

查看全部 Issues在 GitHub 打开

暂无开放 Issues,或尚未同步最近议题。

> 标签

Gostatsd

暂无评论,来聊聊你的看法吧

> 工具信息

发布日期2026年8月1日
最后更新2026年9月17日
分类编程语言
定价开源

> 相关工具

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言