strfry

strfry - a nostr relay

strfry is a relay for the nostr protocol

• Supports most applicable NIPs: 1, 2, 4, 9, 11, 28, 40, 42, 45, 70, 77
• No external database required: All data is stored locally on the filesystem in LMDB
• Hot reloading of config file: No server restart needed for many config param changes
• Zero downtime restarts, for upgrading binary without impacting users
• Websocket compression using permessage-deflate with optional sliding window, when supported by clients. Optional on-disk compression using zstd dictionaries.
• Durable writes: The relay never returns an OK until an event has been confirmed as committed to the DB
• Built-in support for real-time streaming (up/down/both) events from remote relays, and bulk import/export of events from/to jsonl files
• negentropy-based set reconcilliation for efficient syncing with clients or between relays, accurate counting of events between relays, and more
• Prometheus metrics endpoint for monitoring relay activity (client/relay messages by verb, events by kind)

If you are using strfry, please join our telegram chat. Hopefully soon we'll migrate this to nostr.

• Setup
  • Compile
• Operating
  • Running a relay
  • Configuration
  • Selecting and Deleting Events
  • Importing data
  • Exporting data
    • Fried Exports
  • Upload
  • Download
  • Sync
• Monitoring
  • Prometheus Metrics
• Advanced
  • DB Upgrade
  • DB Compaction
  • Zero Downtime Restarts
  • Plugins
  • Router
  • Syncing
  • Compression Dictionaries
• Learn More
• Author and Copyright

Setup

Compile

A C++20 compiler is required, along with a few other common dependencies.

On Debian/Ubuntu use these commands:

sudo apt install -y git g++ make libssl-dev zlib1g-dev liblmdb-dev libflatbuffers-dev libsecp256k1-dev libzstd-dev
git clone https://github.com/hoytech/strfry && cd strfry/
git submodule update --init
make setup-golpe
make -j4

FreeBSD has slightly different commands (warning: possibly out of date):

pkg install -y gmake git perl5 openssl lmdb flatbuffers libuv secp256k1 zstd
git clone https://github.com/hoytech/strfry && cd strfry/
git submodule update --init
gmake setup-golpe
gmake -j4

macOS commands (using Homebrew):

brew install pkg-config libtool openssl zlib lmdb flatbuffers secp256k1 zstd libuv perl
git clone https://github.com/hoytech/strfry && cd strfry/
git submodule update --init
make setup-golpe
make -j4

To upgrade strfry, do the following:

git pull
make update-submodules
make -j4

Operating

Running a relay

Here is how to run the relay:

./strfry relay

For dev/testing, the config file ./strfry.conf is used by default. It stores data in the ./strfry-db/ directory.

By default, it listens on port 7777 and only accepts connections from localhost. In production, you'll probably want a systemd unit file and a reverse proxy such as nginx to support SSL and other features.

Configuration

In order to operate, strfry needs a strfry.conf config file. It searches in the following locations, using the first one it finds:

1. The --config option, for example: strfry --config /path/to/strfry.conf relay
2. The STRFRY_CONFIG environment variable
3. /etc/strfry.conf
4. ./strfry.conf

The strfry.conf distributed in the repo shows the default values for all options. All parameters are optional, except for db. In fact, this is a perfectly valid strfry.conf:

db = "/path/to/db/"

However, in addition to db these are commonly used config parameters:

• relay.bind: By default it only listens for connections from localhost. This is fine during development and testing, or if you are putting a reverse proxy in front of strfry, but if you want the outside world to connect directly to your relay, you should change this to 0.0.0.0 (which means any IP can connect).
• relay.port: By default this is 7777, but you may want to listen on a different port.
• relay.auth.serviceUrl: This is the external address of your relay to the outside world. It must be set for NIP-42 AUTH to work, typically something like wss://my-relay.com.
• relay.info.name, relay.info.description, etc: These are shown in the NIP-11 output and the HTML landing page served by strfry. They provide users extra information about who runs this relay, and its policies.

Selecting and Deleting Events

Because strfry uses a custom LMDB schema, there is no SQL interface for managing the DB. Instead, regular nostr filters (as described in NIP-01) can be used for basic tasks.

For example, strfry scan can be used to select all events matching a particular nostr filter:

./strfry scan '{"kinds":[0,1]}'

Each matching event will be printed on its own line (in other words, in JSONL format).

The strfry delete command can be used to delete events from the DB that match a specified nostr filter. For example, to delete all events from a particular pubkey, use the following command:

./strfry delete --filter '{"authors":["4c7a4fa1a6842266f3f8ca4f19516cf6aa8b5ff6063bc3ec5c995e61e5689c39"]}'

Importing data

The strfry import command reads line-delimited JSON (jsonl) from its standard input and imports events that validate into the DB in batches of 10,000 at a time:

cat my-nostr-dump.jsonl | ./strfry import

• By default, it will verify the signatures and other fields of the events. If you know the messages are valid, you can speed up the import a bit by passing the --no-verify flag.

For local testing/development, this repo includes a deterministic seed-data generator:

scripts/generate-seed-data.sh

This wrapper runs scripts/generate-seed-data.pl, which produces line-delimited JSON suitable for strfry import. With defaults it generates around 100k events.

Useful options:

• --seed : deterministic random seed (same seed + same args => same dataset)
• --users : number of distinct authors
• --kind1-notes : number of kind 1 notes
• --output : output file path (- means stdout)
• --deletions, --replaceable, --param-replaceable, --ephemeral, etc: tune event mix by scenario

To ingest generated data:

cat seeds/events.jsonl | ./strfry import --no-verify

To see all generator options:

scripts/generate-seed-data.sh --help

Exporting data

The strfry export command will print events from the DB to standard output in jsonl, ordered by their created_at field (ascending).

Optionally, you can limit the time period exported with the --since and --until flags. Normally exports will be in ascending order by created_at (oldest first). You can reverse this with --reverse.

Fried Exports

If you pass the --fried argument to strfry export, then the outputed JSON lines will include fried elements. This is precomputed data that strfry can use to re-import these events more quickly. To take advantage of this, use the --fried flag on import as well.

This can be especially useful for upgrading strfry to a new, incompatible database version. See the fried exports documentation for more details on the format.

Upload

This command uses the regular nostr websocket protocol to upload events to a remote relay. The events are read from stdin, and are assumed to be in minified JSONL format (one event per line).

For example:

cat my-events.jsonl | ./strfry upload wss://relay.example.com

This command will try to maintain 50 events in-flight at once. This "pipeline depth" can be increased or decreased with the --pipeline argument.

If you want to send events from your own local strfry DB, you can use either strfry export or the strfry scan command if you want to use an arbitrary filter. For example:

./strfry scan '{"kinds":[0]}' | ./strfry upload wss://relay.example.com

Note that strfry upload will exit once it has completed reading the full input stream. If you want a process that stays running and continuously uploads events added to your DB, use strfry router.

Download

This command is the complement of upload. It connects to a remote relay, performs a REQ request to retrieve events matching a filter, and then prints them to stdout.

For example:

./strfry download wss://relay.example.com > my-events.jsonl

The above uses the default "everything" filter ({}). To specify a more narrow filter, use the --filter flag:

./strfry download wss://relay.example.com --filter '{"kinds":[0]}' > my-events.jsonl

Note that many relays will put an implicit limit on the number of events you can retrieve with a single REQ. To retrieve more, typically you would do "pagination" by setting the until field to the oldest event's created_at and then performing another query.

strfry download also has a --range that can be used to conveniently specify since/until filter flags. Some examples:

--range 2h-    ## since 2 hours ago until present
--range 1Y-6M  ## since 1 year ago until 6 months ago

The units are: s=seconds, m=minutes, h=hours, d=days, w=weeks, M=months, Y=years

Note that strfry download will exit once it has retrieved all of the stored events from the remote relay. If you want a process that stays running and continuously downloads new events as they come in, use strfry router.

Sync

This command uses the negentropy protocol and performs a set reconcilliation between the local DB and the specified relay's remote DB.

That is a fancy way of saying that it figures out which events the remote relay has that it doesn't, and vice versa. Assuming that both sides have some events in common, it does this more efficiently than simply transferring the full set of events (or even just their ids). You can read about the algorithm used in our article on Range-Based Set Reconciliation.

In addition to the C++ implementation used by strfry, negentropy has also been implemented in Javascript, Rust, Go, and more.

Here is how to perform a "full DB" set reconcilliation against a remote server:

./strfry sync wss://relay.example.com

This will download all missing events from the remote relay and insert them into your DB. Similar to stream, you can also sync in the up or both directions:

./strfry sync wss://relay.example.com --dir both

both is especially efficient, because performing the set reconcilliation automatically determines the missing members on each side.

Instead of a "full DB" sync, you can also sync the result of a nostr filter (or multiple filters, use a JSON array of them):

./strfry sync wss://relay.example.com --filter '{"authors":["..."]}'

strfry sync also supports the --range flag documented in strfry download.

Warning: Syncing can consume a lot of memory and bandwidth if the DBs are highly divergent (for example if your local DB is empty and your filter matches many events).

By default strfry keeps a precomputed BTree to speed up full-DB syncs. You can also cache BTrees for arbitrary filters, see the syncing section for more details.

Monitoring

Prometheus Metrics

strfry includes built-in Prometheus metrics support for monitoring relay activity. Metrics are exposed via HTTP at the /metrics endpoint on the