一个纯粹的 Go 实现的 Pastry 分布式哈希表
An open source, pure-Go implementation of the [Pastry Distributed Hash Table](http://en.wikipedia.org/wiki/Pastry_(DHT\) "Pastry on Wikipedia").
Beta1: Wendy is still in active development. It should not be used for mission-critical software. It has been beat on a little, but there are probably still bugs we haven't found or fixed.
This implementation of Wendy is written to be compatible with Go 1. It uses nothing outside of the Go standard library. Nodes in the network must be able to communicate using TCP over a configurable port. Nodes also must be able to have long-running processes.
Wendy was developed on OS X 10.8.1, using Go 1.0.3. It has been verified to work as expected running under Ubuntu 12.04 LTS (64-bit), using Go 1.0.3.
The typical go get secondbit.org/wendy will install Wendy.
We took pains to try and follow the guidelines on writing good documentation for godoc. You can view the generated documentation on the excellent godoc.org.
The "Cluster" represents your network of nodes. The first thing you should do in any application that uses Wendy is initialise the cluster.
First, you need to create the local Node—because Wendy is a peer-to-peer algorithm, there's no such thing as a server or client; instead, everything is a "Node", and only Nodes can connect to the Cluster.
hostname, err := os.Hostname()
if err != nil {
panic(err.Error())
}
id, err := wendy.NodeIDFromBytes([]byte(hostname))
if err != nil {
panic(err.Error())
}
node := wendy.NewNode(id, "your_local_ip_address", "your_global_ip_address", "your_region", 8080)NewNode expects five parameters:
0, Wendy will automatically use a randomly chosen open port.Once you have a Node, you can join the Cluster.
cluster := wendy.NewCluster(node, credentials)NewCluster just creates a Cluster object, initialises the state tables and channels used to keep the algorithm concurrency-safe, and returns it. It requires that you specify the current Node and supply Credentials for the Cluster.
Credentials are an interface that Wendy defines to help control access to your clusters. Credentials could be whatever you want them to be: public/private keys, a single word or phrase, a rather large number... anything at all is fair game. The only rules for Credentials are as follows:
Marshal() on any implementation of Credentials must return a slice of bytes.Valid([]byte) on any implementation of Credentials must decide whether the specified slice of bytes should grant access to the Cluster (return true) or not (return false). The recommended way to do that is to attempt to unmarshal the byte slice into your Credentials implementation (returning false on error) and then comparing the resulting instance with your local instance. But there's nothing stopping you from just returning true, granting anyone who cares to connect full access to your Cluster. Like PSN does (Zing!)In the event that Valid([]byte) returns false for any reason, the Node will not be added to the state tables of the current Node. It will not be notified that its attempt failed, but it will not receive any messages from the Cluster.
To participate in the Cluster, you need to listen for messages. You'll either be used to pass messages along to the correct Node, or will receive messages intended for your Node.
cluster.Listen()
defer cluster.Stop()Listen() is a blocking call, so if you need it to be asynchronous, throw it in a goroutine. Note: If you listen twice on the same Cluster in two different goroutines, concurrency-safety is compromised. You should only ever have one goroutine Listen to any given Cluster.
Stop() ends the Listen call on a Cluster. You'll not receive messages, and will stop participating in the Cluster. It is the graceful way for a Node to exit the Cluster.
Wendy offers several callbacks at various points in the process of exchanging messages within your Cluster. You can use these callbacks to register listeners within your application. These callbacks are simply instances of a type that fulfills the wendy.Application interface and are subsequently registered to a cluster.
…The methods will be invoked at the appropriate points in the lifecycle of the cluster. You should consult the documentation for more information.
Finally, to join a Cluster that has already been formed (which you'll want to do, unless this is the first server in the group you're standing up), you're going to need to use the Join method to announce your presence and initialise your state tables. The Join method is simple:
cluster.Join("127.0.0.1", 8080)The first parameter is simply the IP address of a known Node in the Cluster, as a string. The second is just the port, as an int.
When Join() is called, the Node will contact the specified Node and announce its presence. The specified Node will send the joining Node its state tables and route the join message to the other Nodes in the Cluster, who will also send the joining Node their state tables. These state tables will initialise the joining Node's state tables, allowing it to participate in the Cluster.
Sending a message in Wendy is a little weird. Each message has an ID associated with it, which you can generate based on the contents of the message or some other key. Wendy doesn't care what the relationship between the message and the ID is (Wendy is perfectly happy with random message IDs, in fact), but applications built on Wendy sometimes dictate the terms of the message ID. All Wendy requires is that your message ID, like your Node IDs, has at least 16 bytes worth of data in it.
Messages in Wendy aren't sent to something, they're sent towards something--their message ID. When a Node receives a Message, it checks to see if it knows about any Node with a NodeID closer to the MessageID than its own NodeID. If it does, it forwards the Message on to that Node. If it doesn't it considers the Message to be delivered. There are all sorts of algorithms in place to help the Message reach that delivery quicker, but they're not really the important bit. The important bit is that messages aren't sent to Nodes, they're sent towards their MessageID.
Here's an example of routing a Message with a randomly generated ID (based on the crypto/rand package) through a Cluster:
b := make([]byte, 16)
_, err := rand.Read(b)
if err != nil {
panic(err.Error())
}
id, err := wendy.NodeIDFromBytes(b)
if err != nil {
panic(err.Error())
}
purpose := byte(16)
msg := cluster.NewMessage(purpose, id, []byte("This is the body of the message."))
err = c.Send(msg)
if err != nil {
panic(err.Error())
}You'll notice we set purpose in there to byte(16). Purpose is a way of distinguishing between different types of Messages, and is useful when handling them. We only guarantee that bytes with values 16 and above will go unused by Wendy's own messages. To avoid collisions, you should only use bytes with values of 16 and above when defining your messages.
We repeated that because it's kind of important.
We'd love to see Wendy improve. There's a lot that can still be done with it, and we'd love some help figuring out how to automate some more complete tests for it.
To contribute to Wendy:
go fmt before you send the pull request.We'll review it and merge it in if it's appropriate.
We approached this pragmatically, so there are some differences between the Pastry specification (as we understand it) and our implementation. The end result should not be materially changed.
The following people contributed code that found its way into Wendy:
暂无开放 Issues,或尚未同步最近议题。