百科.dev
全部条目在线工具AI 编程趋势榜开源项目技术资讯提交条目
登录
< 返回工具列表
R

refinery

> 编程语言
开源

Refinery 是一个基于尾部的跟踪感知采样代理。它会检查整个跟踪,并智能地对每个跟踪应用采样决策(是保留还是丢弃)。

400 stars0 点赞2 次浏览
访问官网GitHub

工具介绍

Refinery 是一个基于尾部的跟踪感知采样代理。它会检查整个跟踪,并智能地对每个跟踪应用采样决策(是保留还是丢弃)。

Refinery - the Honeycomb Sampling Proxy

Release Information

For a detailed list of linked pull requests merged in each release, see CHANGELOG.md. For more readable information about recent changes, please see RELEASE_NOTES.md.

With the release of v3.0.0, v2.x enters maintenance for support until April 1, 2026. We recommend migrating to Refinery 3.

Purpose

Refinery is a tail-based sampling proxy and operates at the level of an entire trace. Refinery examines whole traces and intelligently applies sampling decisions to each trace. These decisions determine whether to keep or drop the trace data in the sampled data forwarded to Honeycomb.

A tail-based sampling model allows you to inspect an entire trace at one time and make a decision to sample based on its contents. For example, your data may have a root span that contains the HTTP status code to serve for a request, and another span that contains information on whether the data was served from a cache. Using Refinery, you can choose to keep only traces that had a 500 status code and were also served from a cache.

Refinery's tail sampling capabilities

Refinery support several kinds of tail sampling:

  • Dynamic sampling - This sampling type configures a key based on a trace's set of fields and automatically increases or decreases the sampling rate based on how frequently each unique value of that key occurs. For example, using a key based on http.status_code, you can include in your sampled data:
    • one out of every 1,000 traces for requests that return 2xx
    • one out of every 10 traces for requests that return 4xx
    • every request that returns 5xx
  • Rules-based sampling - This sampling type enables you to define sampling rates for well-known conditions. For example, you can keep 100% of traces with an error and then apply dynamic sampling to all other traffic.
  • Throughput-based sampling - This sampling type enables you to sample traces based on a fixed upper-bound for the number of spans per second. The sampler will dynamically sample traces with a goal of keeping the throughput below the specified limit.
  • Deterministic probability sampling - This sampling type consistently applies sampling decisions without considering the contents of the trace other than its trace ID. For example, you can include 1 out of every 12 traces in the sampled data sent to Honeycomb. This kind of sampling can also be done using head sampling, and if you use both, Refinery takes that into account.

Refinery lets you combine all of the above techniques to achieve your desired sampling behavior.

Setting up Refinery

Refinery is designed to sit within your infrastructure where all traces can reach it. Refinery can run standalone or be deployed in a cluster of two or more Refinery processes accessible via a separate load balancer.

Refinery processes must be able to communicate with each other to concentrate traces on single servers.

Within your application (or other Honeycomb event sources), you would configure the API Host to be http(s)://load-balancer/. Everything else remains the same, such as API key, dataset name, and so on since all that lives with the originating client.

Minimum Configuration

Every Refinery instance should have a minimum of:

  • a linux/amd64 or linux/arm64 operating system
  • 2GB RAM for each server used
  • Access to 2 cores for each server used

In many cases, Refinery only needs one node. If experiencing a large volume of traffic, you may need to scale out to multiple nodes, and likely need a small Redis instance to handle scaling.

We recommend increasing the amount of RAM and the number of cores after your initial set-up. Additional RAM and CPU can be used by increasing configuration values; in particular, CacheCapacity is an important configuration value. Refinery's Stress Relief system provides a good indication of how hard Refinery is working, and when invoked, logs (as reason) the name of the Refinery configuration value that should be increased to reduce stress. Use our scaling and troubleshooting documentation to learn more.

Setting up Refinery in Kubernetes

Refinery is available as a Helm chart in the Honeycomb Helm repository.

You can install Refinery with the following command, which uses the default values file:

bash
helm repo add honeycomb https://honeycombio.github.io/helm-charts
helm install refinery honeycomb/refinery

Alternatively, supply your own custom values file:

bash
helm install refinery honeycomb/refinery --values /path/to/refinery-values.yaml

where /path/to/refinery-values.yaml is the file's path.

Peer Management

When operating in a cluster, Refinery expects to gather all of the spans in a trace onto a single instance so that it can make a trace decision. Since each span arrives independently, each Refinery instance needs to be able to communicate with all of its peers in order to distribute traces to the correct instance.

This communication can be managed in two ways: via an explicit list of peers in the configuration file, or by using self-registration via a shared Redis cache. Installations should generally prefer to use Redis. Even in large installations, the load on the Redis server is quite light, with each instance only making a few requests per minute. A single Redis instance with fractional CPU is usually sufficient.

Configuration

Configuration is controlled by Refinery's two configuration files, which is generally referred to as config.yaml for general configuration and rules.yaml for sampling configuration. These files can be loaded from an accessible filesystem, or loaded with an unauthenticated GET request from a URL.

Learn more about config.yaml and all the parameters that control Refinery's operation in our Refinery configuration documentation.

Learn more about rules.yaml and sampler configuration in our Refinery sampling methods documentation.

It is valid to specify more than one configuration source. For example, it would be possible to have a common configuration file, plus a separate file containing only keys. On the command line, specify multiple files by repeating the command line switch. In environment variables, separate multiple config locations with commas.

Running Refinery

Refinery is a typical linux-style command line application, and supports several command line switches.

refinery -h will print an extended help text listing all command line options and supported environment variables.

Environment Variables

Refinery supports the following environment variables; please see the command line help or the online documentation for the full list. Command line switches take precedence over file configuration, and environment variables take precedence over both.

Environment Variable Description
REFINERY_CONFIG Config file or URL to load (comma-separated for multiple)
REFINERY_RULES_CONFIG Rules config file or URL to load (comma-separated for multiple)
REFINERY_HTTP_LISTEN_ADDRESS HTTP listen address for incoming event traffic
REFINERY_PEER_LISTEN_ADDRESS Peer listen address for communication between Refinery instances
REFINERY_GRPC_LISTEN_ADDRESS gRPC listen address for OTLP traffic
REFINERY_REDIS_HOST Redis host address
REFINERY_REDIS_CLUSTER_HOSTS Redis cluster host addresses (comma-separated)
REFINERY_REDIS_USERNAME Redis username
REFINERY_REDIS_PASSWORD Redis password
REFINERY_REDIS_AUTH_CODE Redis AUTH code
REFINERY_HONEYCOMB_API Honeycomb API URL
REFINERY_HONEYCOMB_API_KEY Honeycomb API key (for logger and metrics)
REFINERY_HONEYCOMB_LOGGER_API_KEY Honeycomb logger API key
REFINERY_HONEYCOMB_LOGGER_ADDITIONAL_ATTRIBUTES Additional attributes for Honeycomb logger (comma-separated)
REFINERY_HONEYCOMB_METRICS_API_KEY API key for legacy Honeycomb metrics
REFINERY_TELEMETRY_ENDPOINT Endpoint to send Refinery's internal telemetry to
REFINERY_OTEL_METRICS_API_KEY API key for OTel metrics if being sent to Honeycomb
REFINERY_OTEL_TRACES_API_KEY API key for OTel traces if being sent to Honeycomb
REFINERY_QUERY_AUTH_TOKEN Token for debug/management queries
REFINERY_AVAILABLE_MEMORY Maximum memory available for Refinery to use (ex: 4GiB)
REFINERY_SEND_KEY Honeycomb API key that Refinery uses to send data to Honeycomb

Note: REFINERY_HONEYCOMB_METRICS_API_KEY takes precedence over REFINERY_HONEYCOMB_API_KEY for legacy metrics configuration. Similarly, REFINERY_HONEYCOMB_LOGGER_API_KEY takes precedence over REFINERY_HONEYCOMB_API_KEY for logger configuration.

Embedded Environment Variables

Any string value in the config file (but not in the rules file) can also contain a reference to environment variables. These variables will be replaced with the value of the corresponding environment variable. This happens once the config file is fully loaded, after processing any command lines.

Environment variable references are in the form of ${ENV_VAR_NAME}. Please note that the ${ and } notation is required.

For example, if you wanted to set the ListenAddr to the value of the LISTEN_ADDR environment variable, you would write:

yaml
Network:
  ListenAddr: "${LISTEN_ADDR}"

Managing Keys

Sending data to Honeycomb requires attaching an API key to telemetry. In order to make managing telemetry easier, Refinery support the ReceiveKeys and SendKey config options, along with AcceptOnlyListedKeys and SendKeyMode. In various combinations, they have a lot of expressive power. Please see the configuration documentation for details on how to set these parameters.

A quick start for specific scenarios is below:

A small number of services

  • Set keys in your applications the way you normally would, and leave Refinery set to the defaults.

Large number of services, central key preferred

  • Do not set keys in your applications
  • Set SendKey to a valid Honeycomb Key
  • Set SendKeyMode to all

Applications must set a key, but control the actual key at Refinery

  • Set SendKey to a valid Honeycomb Key
  • Set SendKeyMode to nonblank

Replace most keys but permit ex

Issues· 81 开放

查看全部 Issues在 GitHub 打开

暂无开放 Issues,或尚未同步最近议题。

> 标签

Go

暂无评论,来聊聊你的看法吧

> 工具信息

发布日期2026年8月1日
最后更新2026年9月17日
分类编程语言
定价开源

> 相关工具

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言