go-plugin

Go Plugin System over WebAssembly

go-plugin is a Go (golang) plugin system over WebAssembly (abbreviated Wasm). As a plugin is compiled to Wasm, it can be size-efficient, memory-safe, sandboxed and portable. The plugin system auto-generates Go SDK for plugins from [Protocol Buffers][protobuf] files. While it is powered by Wasm, plugin authors/users don't have to be aware of the Wasm specification since the raw Wasm APIs are capsulated by the SDK.

It uses the same definition as gRPC, but go-plugin communicates with plugins in memory, not over RPC.

It is inspired by [hashicorp/go-plugin][hashicorp-go-plugin].

Features

The Go plugin system supports a number of features:

Auto-generated Go interfaces: The plugin system generates Go code for hosts and plugins from Protocol Buffers files like gRPC. It is easy to learn how to use go-plugin for protobuf/gRPC users.

Plugins are Go interface implementations: Raw Wasm APIs are hidden so that user can write and consume plugins naturally. To a plugin author: you just implement an interface as if it were going to run in the same process. For a plugin user: you just use and call functions on an interface as if it were in the same process. This plugin system handles the communication in between.

Safe: Wasm describes a memory-safe, sandboxed execution environment. Plugins cannot access filesystem and network unless hosts allow those operations. Even 3rd-party plugins can be executed safely. Plugins can't crash the host process as it is sandboxed.

Portable: Wasm is designed as a portable compilation target for programming languages. Plugins compiled to Wasm can be used anywhere. A plugin author doesn't have to distribute multi-arch binaries.

Efficient: The Wasm stack machine is designed to be encoded in a size- and load-time-efficient binary format.

Bidirectional communication: Wasm allows embedding host functions. As Wasm restricts some capabilities such as network access for security, plugins can call host functions that explicitly embedded by a host to extend functionalities.

Stdout/Stderr Syncing: Plugins can use stdout/stderr as usual and the output will get mirrored back to the host process. The host process can control what io.Writer is attached to stdout/stderr of plugins.

Protocol Versioning: A very basic "protocol version" is supported that can be incremented to invalidate any previous plugins. This is useful when interface signatures are changing, protocol level changes are necessary, etc. When a protocol version is incompatible, a human friendly error message is shown to the end user.

Architecture

go-plugin generates Go SDK for a host and plugins. The plugin system works by loading the Wasm file and communicating over exporting/exported methods.

This architecture has a number of benefits:

• Plugins can't crash your host process: a panic in a plugin is handled by the Wasm runtime and doesn't panic the plugin user.
• Plugins are very easy to write: just write a Go application and GOOS=wasip1 GOARCH=wasm go build.
• Plugins are very easy to distribute: just compile the Go source code to the Wasm binary once and distribute it.
• Plugins are very easy to install: just put the Wasm binary in a location where the host will find it.
• Plugins can be secure: the plugin is executed in a sandbox and doesn't have access to the local filesystem and network by default.

Installation

Download a binary [here][releases] and put it in $PATH.

Requirements

• Go 1.24+

Support Policy

go-plugin is based on [Wazero][wazero] runtime and has Support Policy which follows same rules:

wazero follows the same version policy as Go's Release Policy: two versions. wazero will ensure these versions work and bugs are valid if there's an issue with a current Go version.

For example, if current version of Go is go1.25, go-plugin is ensured to work with Go versions:

• go1.24
• go1.25

Usage

To use the plugin system, you must take the following steps. These are high-level steps that must be done. Examples are available in the examples/ directory.

1. Choose the interface(s) you want to expose for plugins.
   • Define [Messages][protobuf-message] and [Services][protobuf-service] in a proto file.
2. Generate SDK for a host and plugin by go-plugin.
3. Implement the Go interface defined in the plugin SDK.
4. Compile your plugin to Wasm.
5. Load the plugin and call the defined methods.

The development flow is as below.

Tutorial

Let's create a hello-world plugin.

Prerequisite

Install the following tools:

• knqyf263/go-plugin (See Installation)
• [protoc][protoc]
• [Go][go-installation]

Choose the interface you want to expose for plugins

Create greeting.proto.

syntax = "proto3";
package greeting;

option go_package = "github.com/knqyf263/go-plugin/examples/helloworld/greeting";

// The greeting service definition.
// go:plugin type=plugin version=1
service Greeter {
  // Sends a greeting
  rpc SayHello(GreetRequest) returns (GreetReply) {}
}

// The request message containing the user's name.
message GreetRequest {
  string name = 1;
}

// The reply message containing the greetings
message GreetReply {
  string message = 1;
}

Most of the definitions are simply as per [the Protocol Buffers specification][protobuf-spec]. The only difference is the line starting with // go:plugin. It defines parameters for go-plugin. type=plugin means the service defines the plugin interface.

Generate SDK

Run the following command.

$ protoc --go-plugin_out=. --go-plugin_opt=paths=source_relative greeting.proto

Then, you will find 4 files generated in the same directory, greet.pb.go, greet_host.pb.go, greet_plugin.pb.go and greet_vtproto.pb.go.

Optional: Skip the go protobuf generation

If the plugin should be used together with other generators like protoc-gen-go, then it may be helpful to skip the standard protobuf generation. This can be done by setting the option disable_pb_gen=true, for example:

$ protoc \
    --go_opt=paths=source_relative --go_out=. \
    --go-plugin_opt=paths=source_relative,disable_pb_gen=true --go-plugin_out=. \
    greeting.proto

Implement a plugin

The Greeter interface is generated as below in the previous step.

type Greeter interface {
	SayHello(context.Context, GreetRequest) (GreetReply, error)
}

A plugin author needs to implement Greeter and registers the struct via RegisterGreeter. In this tutorial, we use plugin.go as a file name, but it doesn't matter.

//go:build wasip1

package main

import (
	"context"

	"github.com/path/to/your/greeting"
)

// main is required for Go to compile to Wasm.
func main() {}

func init() {
	greeting.RegisterGreeter(MyPlugin{})
}

type MyPlugin struct{}

func (m MyPlugin) SayHello(ctx context.Context, request greeting.GreetRequest) (greeting.GreetReply, error) {
	return greeting.GreetReply{
		Message: "Hello, " + request.GetName(),
	}, nil
}

Then, compile it to Wasm by Go.

$ GOOS=wasip1 GOARCH=wasm go build -o plugin.wasm -buildmode=c-shared plugin.go

Implement a host

Load the plugin binary and call SayHello.

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/path/to/your/greeting"
)

func main() {
	ctx := context.Background()
	
	// Initialize a plugin loader
	p, err := greeting.NewGreeterPlugin(ctx)
	if err != nil {...}
	defer p.Close(ctx)

	// Load a plugin
	plugin, err := p.Load(ctx, "path/to/plugin.wasm")
	if err != nil {...}

	// Call SayHello
	reply, err := plugin.SayHello(ctx, greeting.GreetRequest{Name: "go-plugin"})
	if err != nil {...}
	
	// Display the reply
	fmt.Println(reply.GetMessage())
}

Run

$ go run main.go
Hello, go-plugin

That's it! It is easy and intuitive. You can see the hello-world example [here][hello-world].

References

Host functions

Wasm has limited capability as it is secure by design, but those can't be achieved with Wasm itself. To expand the capability, many compilers implement system calls using WebAssembly System Interface ([WASI][wasi]). But it is still draft ([wasi_snapshot_preview1][wasi_snapshot_preview1]) and some functions are not implemented yet in [wazero][wazero] that go-plugin uses for Wasm runtime. For example, sock_recv and sock_send are not supported for now. It means plugins don't have network access.

Host functions can be used for this purpose. A host function is a function expressed outside WebAssembly but passed to a plugin as an import. You can define functions in your host and pass them to plugins so that plugins can call the functions. Even though Wasm itself doesn't have network access, you can embed such function to plugins.

You can define a service for host functions in a proto file. Note that // go:plugin type=host is necessary so that go-plugin recognizes the service is for host functions. The service name is HostFunctions in this example, but it doesn't matter.

// go:plugin type=host
service HostFunctions {
  // Sends a HTTP GET request
  rpc HttpGet(HttpGetRequest) returns (HttpGetResponse) {}
}

NOTE: the service for host functions must be defined in the same file where other plugin services are defined.

Let's say Greeter is defined in the same file as HostFunctions. Then, Load() will be able to take HostFunctions as an argument as mentioned later.

// go:plugin type=plugin version=1
service Greeter {
  rpc SayHello(GreetRequest) returns (GreetReply) {}
}

go-plugin generates the corresponding Go interface as below.

// go:plugin type=host
type HostFunctions interface {
    HttpGet(context.Context, HttpGetRequest) (HttpGetResponse, error)
}

Implement the interface.

// myHostFunctions implements HostFunctions
type myHostFunctions struct{}

// HttpGet is embedded into the plugin and can be called by the plugin.
func (myHostFunctions) HttpGet(ctx context.Context, request greeting.HttpGetRequest) (greeting.HttpGetResponse, error) {
    ...
}

And pass it when loading a plugin. As described above, Load() takes the HostFunctions interface.

greetingPlugin, err := p.Load(ctx, "plugin/plugin.wasm", myHostFunctions{})

Now, plugins can call HttpGet(). You can see an example [here][host-functions-example].

Define an interface version

You can define an interface version in the // go:plugin line.

// go:plugin type=plugin version=2
service Greeter {
  // Sends a greeting
  rpc Greet(GreetRequest) returns (GreetReply) {}
}

This is useful when interface signatures are changing. When an interface version is incompatible, a human friendly error message is shown to the end user like the following.

API version mismatch, host: 2, plugin: 1

Tips

File access

Refer to [this example][wasi-example].

Logging

fmt.Printf can be used in plugins if you attach os.Stdout as below. See [the example][wasi-example] for more details.

mc := wazero.NewModuleConfig().
    WithStdout(os.Stdout). // Attach stdout so that the plugin can write outputs to stdout
    WithStderr(os.Stderr). // Attach stderr so that the plugin can write errors to stderr
    WithFS(f)              // Loaded plugins can access only files that the host allows.
p, err := cat.NewFileCatPlugin(ctx, cat.WazeroModuleConfig(mc))

If you need structured and leveled logging, you can define host functions so that plugins can call those logging functions.

// The host functions embedded into the plugin
// go:plugin type=host
service LoggingFunctions {
  // Debug log
  rpc Debug(LogMessage) returns (google.protobuf.Empty) {}
  // Info log
  rpc Info(LogMessage) returns (google.protobuf.Empty) {}
  // Warn l