buffa

buffa

A pure-Rust Protocol Buffers implementation with first-class protobuf editions support. Written by Claude and friends ❣️

Why buffa?

The Rust ecosystem lacks an actively maintained, pure-Rust library that supports protobuf editions. Buffa fills that gap with a ground-up design that treats editions as the core abstraction. It passes the full protobuf conformance suite — binary, JSON, and text — with zero expected failures.

Features

• Editions-first. Proto2 and proto3 are understood as feature presets within the editions model. One code path, parameterized by resolved features.

• Two-tier owned/borrowed types. Each message generates both MyMessage (owned, heap-allocated) and MyMessageView (zero-copy from the wire). OwnedView wraps a view with its backing Bytes buffer for use across async boundaries.

• MessageField. Optional message fields deref to a default instance when unset -- no Option> unwrapping ceremony.

• EnumValue. Type-safe open enums with proper Rust enum types and preservation of unknown values, instead of raw i32.

• Linear-time serialization. Cached encoded sizes prevent the exponential blowup that affects libraries without a size-caching pass.

• Unknown field preservation. Round-trip fidelity for proxy and middleware use cases.

• Runtime reflection. buffa-descriptor (under the reflect feature) provides DescriptorPool and DynamicMessage for schema-driven encode, decode, and JSON without generated code — plus extensions, custom-option access, Any pack/unpack, and symbol→file lookup for gRPC server reflection. Generated types implement the same ReflectMessage trait directly (vtable mode), so foo.reflect() borrows in place and a CEL evaluator, transcoding gateway, or generic interceptor treats typed and dynamic messages uniformly — without a re-encode round-trip. See Reflection for the cost relative to generated code.

• no_std + alloc. The core runtime works without std, including JSON serialization via serde. Enabling std adds std::io integration, std::time conversions, and thread-local JSON parse options.

Wire formats

buffa supports binary, JSON, and text protobuf encodings:

• Binary wire format -- full support for all scalar types, nested messages, repeated/packed fields, maps, oneofs, groups, and unknown fields.

• Proto3 JSON -- canonical protobuf JSON mapping via optional serde integration. Includes well-known type serialization (Timestamp as RFC 3339, Duration as "1.5s", int64/uint64 as quoted strings, bytes as base64, etc.).

• Text format (textproto) -- the human-readable debug format. Covers Any expansion ([type.googleapis.com/...] { ... }), extension bracket syntax ([pkg.ext] { ... }), and group/DELIMITED fields. no_std-compatible.

Unsupported features

These are intentionally out of scope:

• Proto2 optional-field getter methods — [default = X] on optional fields does not generate fn field_name(&self) -> T unwrap-to-default accessors. Custom defaults are applied only to required fields via impl Default. Optional fields are Option; use pattern matching or .unwrap_or(X).
• Scoped JsonParseOptions in no_std — serde's Deserialize trait has no context parameter, so runtime options must be passed through ambient state. In std builds, with_json_parse_options provides per-closure, per-thread scoping via a thread-local. In no_std builds, set_global_json_parse_options provides process-wide set-once configuration via a global atomic. The two APIs are mutually exclusive. The no_std global supports singular-enum accept-with-default but not repeated/map container filtering (which requires scoped strict-mode override).

Known limitations

These are gaps we intend to address in future releases:

Semver and API stability

Buffa is pre-1.0. We follow the Rust community convention for 0.x crates: breaking changes increment the minor version (0.1.x → 0.2.0), additive changes increment the patch version (0.1.0 → 0.1.1). Pin to a minor version (buffa = "x.y"; the full x.y.z that cargo add buffa writes is equivalent under caret semantics) to avoid surprises.

The generated code API (struct shapes, Message trait, MessageView trait, EnumValue, MessageField) is considered the primary stability surface. Internal helper modules marked #[doc(hidden)] (__private, __buffa_* fields) may change at any time.

Quick start

Using buf generate (recommended)

Install buf, then create a buf.gen.yaml that uses the published buf.build/anthropics/buffa remote plugin — no local plugin install required:

version: v2
plugins:
  - remote: buf.build/anthropics/buffa
    out: src/gen
    opt:
      - file_per_package=true
      - json=true

buf generate

This emits one .rs file per proto package. Wire them into your crate with a small pub mod tree:

// src/gen/mod.rs (hand-written)
pub mod example {
    pub mod v1 {
        include!("example.v1.rs");
    }
}

If you'd rather have the module tree generated for you, install protoc-gen-buffa-packaging locally and add it as a second plugin (drop the file_per_package=true opt):

version: v2
plugins:
  - remote: buf.build/anthropics/buffa
    out: src/gen
    opt:
      - json=true
  - local: protoc-gen-buffa-packaging
    out: src/gen
    strategy: all

See examples/bsr-quickstart/ for a complete, runnable project, or the user guide for the full set of build setups (local plugins, buffa-build/build.rs, BSR-generated SDKs).

Using buffa-build in build.rs

Alternatively, use buffa-build for a build.rs-based workflow (requires protoc on PATH):

// build.rs
fn main() {
    buffa_build::Config::new()
        .files(&["proto/my_service.proto"])
        .includes(&["proto/"])
        .compile()
        .unwrap();
}

Encoding and decoding

use buffa::Message;

// Encode
let msg = MyMessage { id: 42, name: "hello".into(), ..Default::default() };
let bytes = msg.encode_to_vec();

// Decode (owned)
let decoded = MyMessage::decode_from_slice(&bytes).unwrap();

// Decode (zero-copy view)
let view = MyMessageView::decode_view(&bytes).unwrap();
println!("name: {}", view.name); // &str, no allocation

// Decode (owned view — zero-copy + 'static, for async/RPC use)
let owned_view = MyMessageOwnedView::decode(bytes.into()).unwrap();
println!("name: {}", owned_view.name()); // still zero-copy, but 'static + Send

JSON serialization (with json feature)

let json = serde_json::to_string(&msg).unwrap();
let decoded: MyMessage = serde_json::from_str(&json).unwrap();

Documentation

• User Guide — comprehensive guide to buffa's API, generated code shape, encoding/decoding, views, JSON, well-known types, and editions support.
• Migrating from prost — step-by-step migration guide with before/after code examples.
• Migrating from protobuf — migration guide covering both stepancheg v3 and Google official v4.

Workspace layout

Crate	Purpose
buffa	Core runtime: Message trait, wire format codec, no_std support
buffa-types	Well-known types: Timestamp, Duration, Any, Struct, Api, Type, wrappers, etc.
buffa-descriptor	Protobuf descriptor types (FileDescriptorProto, DescriptorProto, ...)
buffa-codegen	Code generation from protobuf descriptors
buffa-build	build.rs helper for invoking codegen via protoc
buffa-remote-derive	Derive macros implementing the pluggable owned-type traits for newtypes over foreign types
buffa-yaml	YAML serialization for buffa messages
protoc-gen-buffa	protoc plugin binary; also published as buf.build/anthropics/buffa
protoc-gen-buffa-packaging	protoc plugin that emits a mod.rs module tree (local-only)

Performance

Throughput comparison across five representative message shapes, measured at buffa v0.8.0 on a quiesced bare-metal Intel Xeon Platinum 8488C (turbo disabled, performance governor, one core per implementation, median of five passes; full methodology in benchmarks/charts/README.md). Higher is better.

Binary decode

Raw data (MiB/s)

Message	buffa	buffa (view)	buffa (lazy)	prost	prost (bytes)	protobuf-v4	Go
ApiResponse	575	872 (+52%)	912 (+59%)	550 (−4%)	546 (−5%)	430 (−25%)	175 (−70%)
LogRecord	572	1,336 (+134%)	1,716 (+200%)	481 (−16%)	477 (−17%)	555 (−3%)	161 (−72%)
AnalyticsEvent	123	225 (+83%)	11,873 (+9538%)	148 (+20%)	129 (+5%)	222 (+80%)	57 (−54%)
GoogleMessage1	601	786 (+31%)	1,452 (+142%)	698 (+16%)	669 (+11%)	373 (−38%)	263 (−56%)
MediaFrame	10,619	41,441 (+290%)	40,678 (+283%)	6,002 (−43%)	18,432 (+74%)	11,005 (+4%)	1,890 (−82%)

Binary encode

Raw data (MiB/s)

Message	buffa	buffa (view)	buffa (lazy)	prost	prost (bytes)	protobuf-v4	Go
ApiResponse	1,972	1,955 (−1%)	1,959 (−1%)	1,964 (−0%)	—	639 (−68%)	384 (−81%)
LogRecord	3,053	3,509 (+15%)	3,587 (+17%)	2,758 (−10%)	—	1,067 (−65%)	186 (−94%)
AnalyticsEvent	404	426 (+6%)	12,960 (+3109%)	238 (−41%)	—	307 (−24%)	105 (−74%)
GoogleMessage1	2,122	2,123 (+0%)	2,965 (+40%)	1,816 (−14%)	—	522 (−75%)	232 (−89%)
MediaFrame	25,727	27,325 (+6%)	27,441 (+7%)	25,402 (−1%)	—	6,671 (−74%)	2,423 (−91%)

Build + binary encode

The build + encode measure starts from raw field values rather than a pre-built message struct, so it counts struct construction. The buffa (view) path constructs a borrowed view directly over the input slices and never allocates an owned message at all, which is why it is consistently faster than building owned structs and then encoding them.

Raw data (MiB/s)

Message	buffa	buffa (view)
ApiResponse	639	1,224 (+91%)
LogRecord	315	2,447 (+678%)
AnalyticsEvent	267	802 (+200%)
GoogleMessage1	673	900 (+34%)
MediaFrame	14,432	33,481 (+132%)

JSON encode

Raw data (MiB/s)

Message	buffa	prost	Go
ApiResponse	521	589 (+13%)	70 (−87%)
LogRecord	697	882 (+27%)	85 (−88%)
AnalyticsEvent	505	533 (+6%)	33 (−94%)
GoogleMessage1	571	674 (+18%)	73 (−87%)
MediaFrame	702	937 (+33%)	235 (−67%)

JSON decode

Raw data (MiB/s)

Message	buffa	prost	Go
ApiResponse	492	205 (−58%)	40 (−92%)
LogRecord	530	424 (−20%)	63 (−88%)
AnalyticsEvent	169	152 (−10%)	25 (−85%)
GoogleMessage1	413	171 (−59%)	41 (−90%)
MediaFrame	1,235	1,218 (−1%)	215 (−83%)

Message types: ApiResponse (~200 B, flat scalars), LogRecord (~1 KB, strings + map + nested message), AnalyticsEvent (~10 KB, deeply nested + repeated sub-messages), GoogleMessage1 (standard protobuf benchmark message), MediaFrame (~10 KB,