roshi-server provides a thin, REST-ish HTTP interface to a farm. All the parameters for each conceptual layer are provided as commandline flags, most with sensible defaults. roshi-server expects an existant set of Redis instances (at least 1) to be running somewhere as the data layer. roshi-server is effectively stateless, so you may run as many instances as required to satisfy your load volume.
roshi-server uses vendored dependencies and a "blessed build" process to ensure
stability over time. Users should get and build roshi-server by cloning this
repository and running make in the roshi-server subdirectory. A working Go
toolchain is assumed.
git clone [email protected]:soundcloud/roshi
cd roshi/roshi-server
make
It's also possible to get roshi-server via go get, and/or build it with a
simple go build, with the caveat that it will use your normal GOPATH to
resolve dependencies, and therefore will enforce no constraints on dependency
versions, which could introduce bugs or strange behavior.
As a demo, start an instance of Redis on the standard port, and run
roshi-server -redis.instances=localhost:6379
This will create a farm with a single cluster containing a single Redis instance. All functionality will work as advertised, albeit with effectively zero fault-tolerance.
The server installs one handler on the root path. Operations are
differentiated by their HTTP verb. Keys and members must be base64 encoded.
For demo purposes, we'll use key foo (base64 Zm9v) and members bar
(base64 YmFy) and baz (base64 YmF6).
Note that write operations will claim success and return 200 as long as quorum is achieved, even if the provided score was lower than what has already been persisted and therefore the operation was actually a no-op.
POST to /. Provide a request body with a JSON array of key-score-member
objects. No URL parameters are accepted.
$ cat insert.json
[{"key":"Zm9v", "score":1.05, "member":"YmFy"},
{"key":"Zm9v", "score":1.99, "member":"YmF6"}]
$ curl -Ss [email protected] -XPOST 'http://localhost:6302' | jq .
{
"duration": "377.863us",
"inserted":2
}GET to /. Provide a request body with a JSON-encoded array of key strings.
There are some URL parameters:
- offset, for pagination, default 0
- limit, for pagination, default 10
- coalesce, merge multiple keys into one response, default false
$ cat select.json
["Zm9v"]
$ curl -Ss [email protected] -XGET 'http://localhost:6302' | jq .
{
"records": {
"foo": [
{
"member": "YmF6",
"score": 1.99,
"key": "Zm9v"
},
{
"member": "YmFy",
"score": 1.05,
"key": "Zm9v"
}
]
},
"offset": 0,
"limit": 10,
"keys": [
"Zm9v"
],
"duration": "238.204us"
}DELETE to /. Provide a request body with a JSON array of key-score-member
objects. No URL parameters are accepted.
$ cat delete.json
[{"key":"Zm9v", "score":2.01, "member":"YmF6"}]
$ curl -Ss [email protected] -XDELETE 'http://localhost:6302' | jq .
{
"duration": "429.993us",
"deleted": 1
}Golang clients that wish to make HTTP requests to roshi-server should
import "github.com/soundcloud/roshi/common" and interact with (i.e. serialize
and deserialize) common.KeyScoreMember tuples directly. That type implements
json.Marshaler such that base64 encoding and decoding is transparent to the
user.
Clients in other languages should ensure all key and member strings are properly base64 encoded.
roshi-server expects to interact with a set of independent Redis instances, which operators should deploy, monitor, and manage. Roshi assumes a successful write to Redis will be persisted; Roshi's durability guarantees are largely predicated on the durability of each Redis instance. Please familiarize yourself with Redis persistence.
In general, Redis will use a lot of RAM and comparatively little CPU, and roshi-server will use very little RAM and comparatively large amount of CPU. It may make sense to co-locate a roshi-server instance with every Redis instance. (It's been our experience that a single server-class machine is best utilized when it runs multiple Redis instances.)