Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(docs): Add elasticsearch section to Zebra book #6295

Merged
merged 2 commits into from
Mar 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions book/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
- [zk-SNARK Parameters](user/parameters.md)
- [Mining](user/mining.md)
- [Testnet Mining with s-nomp](user/mining-testnet-s-nomp.md)
- [Kibana blockchain explorer](user/elasticsearch.md)
- [Developer Documentation](dev.md)
- [Contribution Guide](CONTRIBUTING.md)
- [Design Overview](dev/overview.md)
Expand Down
Binary file added book/src/user/elastic1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added book/src/user/elastic2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added book/src/user/elastic3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
101 changes: 101 additions & 0 deletions book/src/user/elasticsearch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Kibana blockchain explorer

The goal here is to export block data from Zebra into an [elasticsearch](https://www.elastic.co/) database and visualize it with the [kibana](https://www.elastic.co/kibana/) front end.

**Attention:** This is an experimental feature tested only in the Zcash Testnet.

Elasticsearch support for zebra was introduced to Zebra in [pull request #6274](https://github.com/ZcashFoundation/zebra/pull/6274).

## Download, build and run Elasticsearch

Installing elasticsearch is easy in linux and macOS by following the [.tar.gz installation guide](https://www.elastic.co/guide/en/elasticsearch/reference/current/targz.html).

Make sure you end up with an `elasticsearch` binary. Run it:

```sh
./bin/elasticsearch
```

The first time you run the database, `elastic` password and the enrollment token for Kibana will displayed in the screen (See [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/targz.html#targz-running)). Please save these as you will need them.

Elasticsearch will listen in https://localhost:9200 by default.

## Download, build and run Kibana

Installing kibana is also easy in linux and macOS following the [.tar.gz installation guide](https://www.elastic.co/guide/en/kibana/current/targz.html).


Make sure you end up with a `kibana` binary and run it:

```sh
./bin/kibana
```

The first time you run kibana, it will provide a link for configuration:

```sh
Kibana has not been configured.

Go to http://localhost:5601/?code=405316 to get started.
```

Visit the url to get started, you will need the kibana enrollment token from elasticsearch and the elastic password from the previous step.

Kibana will listen in https://localhost:5601 by default.

You are now ready to start bumping data into elasticsearch with Zebra.

## Download and build zebra with `elasticsearch` feature

Elasticsearch is an optional and experimental feature, we need to build and install with the `elasticsearch` rust feature enabled using the following command:

```sh
cargo install --features elasticsearch --locked --git https://github.com/ZcashFoundation/zebra zebrad
dconnolly marked this conversation as resolved.
Show resolved Hide resolved
```

Zebra binary will be at `~/.cargo/bin/zebrad`.

## Configure Zebra for `elasticsearch`

Generate a configuration file with the default settings:

```sh
zebrad generate -o ~/.config/zebrad.toml
```
The following changes are needed:

### `network` section

Change the `network` field to `Testnet`. The `Mainnet` should work but it is untested. Also ,the preferred p2p port for the testnet is 18233, so optionally change the `listen_addr` field to `0.0.0.0:18233`.

### `state` section

Add your elastic password generated when running the database for the first time into the `elasticsearch_password` field.

## Run Zebra

If the config is in the default path then just running the binary will start the sync.

```sh
zebrad
```

Sync will take time, but you can use kibana to make sure blocks are being inserted during the process.

## Visualize your data

As soon as the first batch of data is inserted from Zebra into elasticsearch database, an index `zcash_testnet` will be created.

To observe data, go to Analytics → Discover and create a new data view with the `zcash_testnet` index. Make sure you select `header.time` field as Timestamp field.

To see the data, use the calendar to get records for the last 10 years, the first blocks that are inserted will be very old blocks following the chain genesis.

![image info](elastic1.png)

After a while the chain will be in sync.

![image info](elastic2.png)

You can now use all the kibana features. For example, creating dashboards for specific data visualization.

![image info](elastic3.png)