Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jeffreypicard/feature/move to herald #37

Merged
merged 3 commits into from
Jul 11, 2022
Merged

Jeffreypicard/feature/move to herald #37

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
22379dd
changes for repo move
jeffreypicard Jun 3, 2022
f780b1b
Go back to protobuf 1.27.1, update go to 1.18
jeffreypicard Jul 7, 2022
d6deb5f
update readme for new rocksdb version
jeffreypicard Jul 8, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions db/db.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@ import (
"os"
"time"

"github.com/lbryio/hub/db/prefixes"
"github.com/lbryio/hub/db/stack"
"github.com/lbryio/hub/internal"
"github.com/lbryio/hub/internal/metrics"
pb "github.com/lbryio/hub/protobuf/go"
"github.com/lbryio/herald/db/prefixes"
"github.com/lbryio/herald/db/stack"
"github.com/lbryio/herald/internal"
"github.com/lbryio/herald/internal/metrics"
pb "github.com/lbryio/herald/protobuf/go"
"github.com/linxGnu/grocksdb"

log "github.com/sirupsen/logrus"
Expand Down
2 changes: 1 addition & 1 deletion db/db_get.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import (
"fmt"
"log"

"github.com/lbryio/hub/db/prefixes"
"github.com/lbryio/herald/db/prefixes"
"github.com/linxGnu/grocksdb"
)

Expand Down
6 changes: 3 additions & 3 deletions db/db_resolve.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ import (
"sort"
"strings"

"github.com/lbryio/hub/db/prefixes"
"github.com/lbryio/hub/internal"
pb "github.com/lbryio/hub/protobuf/go"
"github.com/lbryio/herald/db/prefixes"
"github.com/lbryio/herald/internal"
pb "github.com/lbryio/herald/protobuf/go"
lbryurl "github.com/lbryio/lbry.go/v3/url"
log "github.com/sirupsen/logrus"
)
Expand Down
6 changes: 3 additions & 3 deletions db/db_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ import (
"strings"
"testing"

dbpkg "github.com/lbryio/hub/db"
"github.com/lbryio/hub/db/prefixes"
"github.com/lbryio/hub/internal"
dbpkg "github.com/lbryio/herald/db"
"github.com/lbryio/herald/db/prefixes"
"github.com/lbryio/herald/internal"
"github.com/linxGnu/grocksdb"
)

Expand Down
2 changes: 1 addition & 1 deletion db/iteroptions.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ package db
import (
"bytes"

"github.com/lbryio/hub/db/prefixes"
"github.com/lbryio/herald/db/prefixes"
"github.com/linxGnu/grocksdb"

log "github.com/sirupsen/logrus"
Expand Down
2 changes: 1 addition & 1 deletion db/prefixes/prefixes.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ import (
"sort"
"strings"

"github.com/lbryio/hub/internal"
"github.com/lbryio/herald/internal"
"github.com/lbryio/lbcd/chaincfg/chainhash"
)

Expand Down
4 changes: 2 additions & 2 deletions db/prefixes/prefixes_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ import (
"os"
"testing"

dbpkg "github.com/lbryio/hub/db"
prefixes "github.com/lbryio/hub/db/prefixes"
dbpkg "github.com/lbryio/herald/db"
prefixes "github.com/lbryio/herald/db/prefixes"
"github.com/linxGnu/grocksdb"
)

Expand Down
2 changes: 1 addition & 1 deletion db/stack/stack.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ package stack
import (
"sync"

"github.com/lbryio/hub/internal"
"github.com/lbryio/herald/internal"
)

type SliceBacked struct {
Expand Down
2 changes: 1 addition & 1 deletion db/stack/stack_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import (
"testing"
"time"

"github.com/lbryio/hub/db/stack"
"github.com/lbryio/herald/db/stack"
)

func TestPush(t *testing.T) {
Expand Down
4 changes: 2 additions & 2 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/lbryio/hub
module github.com/lbryio/herald

go 1.17
go 1.18

// replace github.com/lbryio/lbry.go/v3 => /home/loki/dev/lbry/lbry.go

Expand Down
1 change: 0 additions & 1 deletion go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,6 @@ github.com/btcsuite/winsvc v1.0.0/go.mod h1:jsenWakMcC0zFBFurPLEAyrnc/teJEM1O46f
github.com/casbin/casbin/v2 v2.1.2/go.mod h1:YcPU1XXisHhLzuxH9coDNf2FbKpjGlbCg3n9yuLkIJQ=
github.com/cenkalti/backoff v2.2.1+incompatible/go.mod h1:90ReRw6GdpyfrHakVjL/QHaoyV4aDUVVkXQJJJ3NXXM=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/cespare/xxhash v1.1.0 h1:a6HrQnmkObjyL+Gs60czilIUGqrzKutQD6XZog3p+ko=
github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=
github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/cespare/xxhash/v2 v2.1.2 h1:YRXhKfTDauu4ajMg1TPgFO5jnlC2HCbmLXMcTG5cbYE=
Expand Down
6 changes: 3 additions & 3 deletions main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ import (

_ "net/http/pprof"

"github.com/lbryio/hub/internal"
pb "github.com/lbryio/hub/protobuf/go"
"github.com/lbryio/hub/server"
"github.com/lbryio/herald/internal"
pb "github.com/lbryio/herald/protobuf/go"
"github.com/lbryio/herald/server"
log "github.com/sirupsen/logrus"
"google.golang.org/grpc"
)
Expand Down
2 changes: 1 addition & 1 deletion protobuf/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ MIN_VERSION="3.0"
version_gte "$VERSION" "$MIN_VERSION" || { echo >&2 "error: protoc version must be >= $MIN_VERSION (your $PROTOC is $VERSION)"; exit 1; }


hash protoc-gen-go-grpc 2>/dev/null || go get github.com/golang/protobuf/protoc-gen-go google.golang.org/grpc/cmd/protoc-gen-go-grpc
hash protoc-gen-go-grpc 2>/dev/null || go get google.golang.org/protobuf/cmd/protoc-gen-go google.golang.org/grpc/cmd/protoc-gen-go-grpc
hash protoc-gen-go-grpc 2>/dev/null || { echo >&2 'error: Make sure $GOPATH/bin is in your $PATH'; exit 1; }


Expand Down
2 changes: 1 addition & 1 deletion protobuf/definitions/claim.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
syntax = "proto3";

option go_package = "github.com/lbryio/hub/protobuf/go/pb";
option go_package = "github.com/lbryio/herald/protobuf/go/pb";

package pb;

Expand Down
2 changes: 1 addition & 1 deletion protobuf/definitions/hub.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
syntax = "proto3";

option go_package = "github.com/lbryio/hub/protobuf/go/pb";
option go_package = "github.com/lbryio/herald/protobuf/go/pb";
import "result.proto";

package pb;
Expand Down
2 changes: 1 addition & 1 deletion protobuf/definitions/result.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
syntax = "proto3";

option go_package = "github.com/lbryio/hub/protobuf/go/pb";
option go_package = "github.com/lbryio/herald/protobuf/go/pb";

package pb;

Expand Down
4 changes: 2 additions & 2 deletions protobuf/python/claim_pb2.py
View file
Open in desktop

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions protobuf/python/hub_pb2.py
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions protobuf/python/result_pb2.py
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

27 changes: 17 additions & 10 deletions readme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
# LBRY Hub
# LBRY Herald

A hub provides back-end services to LBRY clients. Services include
A herald server provides back-end services to LBRY clients. Services include

- URL resolution
- search
- wallet services (getting and sending transactions, address subscription)
- hub federation and discovery

This project will eventually subsume and replace the
Expand All @@ -26,13 +25,18 @@ Follow the instructions [here](https://lbry.tech/resources/wallet-server).
### Run this hub

```bash
./hub serve
./herald serve
```

```bash
# run with remote services disabled so it can run completely solo
./herald serve --disable-rocksdb-refresh --disable-load-peers --disable-resolve --disable-es --disable-blocking-and-filtering
```

### Search for stuff

```bash
./hub search text goes here
./herald search text goes here
```

## Contributing
Expand All @@ -41,9 +45,9 @@ Contributions to this project are welcome, encouraged, and compensated. Details

### Dev Dependencies

Install Go 1.17+
Install Go 1.18+

- Ubuntu: `sudo add-apt-repository ppa:longsleep/golang-backports && sudo apt install golang-go`
- Ubuntu: `sudo snap install go`
- OSX: `brew install go`
- Windows https://golang.org/doc/install

Expand All @@ -59,12 +63,15 @@ pip install grpcio grpcio-tools github3.py

Lastly the hub needs protobuf version 3.17.1, it may work with newer version but this is what it's built with, on ubuntu systems you'll have to install this from source see the GitHub actions in `.github/workflows` for an example of this.

Install rocksdb and setup your CGO flags, on ubuntu
Install rocksdb and dependencies your CGO flags, on ubuntu. We use v6.29.5 for feature and statis build support.

```
sudo apt-get install libgflags-dev libsnappy-dev zlib1g-dev libbz2-dev libzstd-dev liblz4-dev
export CGO_CFLAGS="-I/path/to/rocksdb/include"
export CGO_LDFLAGS="-L/path/to/rocksdb -lrocksdb -lstdc++ -lm -lz -lsnappy -llz4 -lzstd -lbz2"
wget https://github.com/facebook/rocksdb/archive/refs/tags/v6.29.5.tar.gz
tar xfzv rocksdb-6.29.5.tar.gz
cd rocksdb-6.29.5
make static_lib
sudo make install
```

```
Expand Down
Loading