Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add rust-libp2p-server to monorepo #4311

Merged
merged 35 commits into from
Aug 21, 2023
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
f6d3e89
feat: add rust-libp2p-server to monorepo
mxinden Aug 10, 2023
4693d04
Set versions
mxinden Aug 10, 2023
cae6986
Set rust-version
mxinden Aug 10, 2023
b8c955b
fix: use pub(crate)
mxinden Aug 10, 2023
0023ca9
refactor: move to clap
mxinden Aug 10, 2023
9e7b85e
fix: move to tokio and hyper
mxinden Aug 10, 2023
dbdffa8
Remove lock file and license
mxinden Aug 10, 2023
0c439eb
Remove sub-gitignore
mxinden Aug 10, 2023
ed93384
New line
mxinden Aug 10, 2023
5d7bab7
Update readme
mxinden Aug 10, 2023
98b728e
Remove filter for ipv6 quic addresses
mxinden Aug 10, 2023
1972aa1
Test CI
mxinden Aug 14, 2023
68211ab
feat(.github/ci): include keygen and server in examples-and-tools
mxinden Aug 14, 2023
3209297
fix(.github/ci): introduce new step instead of extending examples
mxinden Aug 14, 2023
b9477b6
Revert "Test CI"
mxinden Aug 14, 2023
967989e
Leverage swarm-derive event generation
mxinden Aug 14, 2023
677e6ec
Merge branch 'master' of https://github.com/libp2p/rust-libp2p into s…
mxinden Aug 14, 2023
f71cbef
Use rust-version from workspace
mxinden Aug 16, 2023
1312a0c
Use ghcr.io
mxinden Aug 16, 2023
8550b6f
Update Dockerfile path
mxinden Aug 16, 2023
d38adef
Leverage .target/
mxinden Aug 16, 2023
2382f40
Publish crate to crates.io
mxinden Aug 18, 2023
3ae913d
Merge branch 'master' of https://github.com/libp2p/rust-libp2p into s…
mxinden Aug 18, 2023
a29d05b
.github: remove misc/
mxinden Aug 18, 2023
a44092a
Update .github/workflows/docker-image.yml
mxinden Aug 21, 2023
b9d4013
Update .github/workflows/docker-image.yml
mxinden Aug 21, 2023
01265e7
Update .github/workflows/docker-image.yml
mxinden Aug 21, 2023
055b270
Import quic through libp2p
mxinden Aug 21, 2023
bf6b753
Display not debug peer id in println
mxinden Aug 21, 2023
cf79259
Refactor swarm event metric recording
mxinden Aug 21, 2023
5788c04
Use distroless
mxinden Aug 21, 2023
60fec46
Allow libp2p-server to depend on libp2p
mxinden Aug 21, 2023
1547211
Merge branch 'master' of https://github.com/libp2p/rust-libp2p into s…
mxinden Aug 21, 2023
bfc768f
Reference libp2p-server version in root Cargo.toml
mxinden Aug 21, 2023
2b4a96d
Bump version and add changelog entry
mxinden Aug 21, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions .github/workflows/docker-image.yml
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: Docker Image CI
mxinden marked this conversation as resolved.
Show resolved Hide resolved

on:
push:
branches:
- '**'
tags:
- 'libp2p-server-**'

jobs:
build:
mxinden marked this conversation as resolved.
Show resolved Hide resolved
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
mxinden marked this conversation as resolved.
Show resolved Hide resolved

- uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ github.token }}

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

- name: Get branch or tag name
id: ref-name
run: echo ::set-output name=ref::${GITHUB_REF#refs/*/}
mxinden marked this conversation as resolved.
Show resolved Hide resolved

- name: Build and push
uses: docker/build-push-action@v3
with:
context: .
file: ./misc/server/Dockerfile
push: true
tags: ghcr.io/${{ github.repository }}-server:${{ steps.ref-name.outputs.ref }}
21 changes: 21 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ members = [
"misc/quick-protobuf-codec",
"misc/quickcheck-ext",
"misc/rw-stream-sink",
"misc/server",
"muxers/mplex",
"muxers/test-harness",
"muxers/yamux",
Expand Down
43 changes: 43 additions & 0 deletions misc/server/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Changelog
All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.8.0]
### Changed
- Remove mplex support.

## [0.7.0]
### Changed
- Update to libp2p v0.47.0.

## [0.6.0] - [2022-05-05]
### Changed
- Update to libp2p v0.44.0.

## [0.5.4] - [2022-01-11]
### Changed
- Pull latest autonat changes.

## [0.5.3] - [2021-12-25]
### Changed
- Update dependencies.
- Pull in autonat fixes.

## [0.5.2] - [2021-12-20]
### Added
- Add support for libp2p autonat protocol via `--enable-autonat`.

## [0.5.1] - [2021-12-20]
### Fixed
- Update dependencies.
- Fix typo in command line flag `--enable-kademlia`.

## [0.5.0] - 2021-11-18
### Changed
- Disable Kademlia protocol by default.

## [0.4.0] - 2021-11-18
### Fixed
- Update dependencies.
27 changes: 27 additions & 0 deletions misc/server/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
[package]
name = "libp2p-server"
version = "0.12.0"
authors = ["Max Inden <[email protected]>"]
edition = "2021"
rust-version = { workspace = true }
description = "A rust-libp2p server binary."
license = "MIT"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
base64 = "0.21"
clap = { version = "4.3.12", features = ["derive"] }
env_logger = "0.10.0"
futures = "0.3"
futures-timer = "3"
hyper = { version = "0.14", features = ["server", "tcp", "http1"] }
libp2p = { path = "../../libp2p", features = ["autonat", "dns", "tokio", "noise", "tcp", "yamux", "identify", "kad", "ping", "relay", "metrics", "rsa", "macros"] }
libp2p-quic = { workspace = true, features = ["tokio"] }
mxinden marked this conversation as resolved.
Show resolved Hide resolved
log = "0.4"
prometheus-client = "0.21.2"
serde = "1.0.183"
serde_derive = "1.0.125"
serde_json = "1.0"
tokio = { version = "1", features = ["rt-multi-thread", "macros"] }
zeroize = "1"
16 changes: 16 additions & 0 deletions misc/server/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
FROM rust:1.66-bullseye as builder
WORKDIR /usr/src/rust-libp2p-server

# Run with access to the target cache to speed up builds
WORKDIR /workspace
ADD . .
RUN --mount=type=cache,target=./target \
--mount=type=cache,target=/usr/local/cargo/registry \
cargo build --release --package libp2p-server

RUN --mount=type=cache,target=./target \
mv ./target/release/libp2p-server /usr/local/bin/libp2p-server

FROM debian:bullseye-slim
mxinden marked this conversation as resolved.
Show resolved Hide resolved
COPY --from=builder /usr/local/bin/libp2p-server /usr/local/bin/libp2p-server
CMD ["libp2p-server"]
35 changes: 35 additions & 0 deletions misc/server/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Rust libp2p Server

A rust-libp2p based server implementation running:

- the [Kademlia protocol](https://github.com/libp2p/specs/tree/master/kad-dht)

- the [Circuit Relay v2 protocol](https://github.com/libp2p/specs/blob/master/relay/circuit-v2.md)

- the [AutoNAT protocol](https://github.com/libp2p/specs/blob/master/autonat/README.md)

## Usage

```
cargo run -- --help

A rust-libp2p server binary.

Usage: libp2p-server [OPTIONS] --config <CONFIG>

Options:
--config <CONFIG> Path to IPFS config file
--metrics-path <METRICS_PATH> Metric endpoint path [default: /metrics]
--enable-kademlia Whether to run the libp2p Kademlia protocol and join the IPFS DHT
--enable-autonat Whether to run the libp2p Autonat protocol
-h, --help Print help
```


```
cargo run -- --config ~/.ipfs/config

Local peer id: PeerId("12D3KooWSa1YEeQVSwvoqAMhwjKQ6kqZQckhWPb3RWEGV3sZGU6Z")
Listening on "/ip4/127.0.0.1/udp/4001/quic"
[...]
```
78 changes: 78 additions & 0 deletions misc/server/src/behaviour.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
use libp2p::autonat;
use libp2p::identify;
use libp2p::kad::{record::store::MemoryStore, Kademlia, KademliaConfig};
use libp2p::ping;
use libp2p::relay;
use libp2p::swarm::behaviour::toggle::Toggle;
use libp2p::{identity, swarm::NetworkBehaviour, Multiaddr, PeerId};
use std::str::FromStr;
use std::time::Duration;

const BOOTNODES: [&str; 4] = [
"QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN",
"QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa",
"QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
"QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt",
];

#[derive(NetworkBehaviour)]
pub(crate) struct Behaviour {
relay: relay::Behaviour,
ping: ping::Behaviour,
identify: identify::Behaviour,
pub(crate) kademlia: Toggle<Kademlia<MemoryStore>>,
autonat: Toggle<autonat::Behaviour>,
}

impl Behaviour {
pub(crate) fn new(
pub_key: identity::PublicKey,
enable_kademlia: bool,
enable_autonat: bool,
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved
) -> Self {
let kademlia = if enable_kademlia {
let mut kademlia_config = KademliaConfig::default();
// Instantly remove records and provider records.
//
// TODO: Replace hack with option to disable both.
kademlia_config.set_record_ttl(Some(Duration::from_secs(0)));
kademlia_config.set_provider_record_ttl(Some(Duration::from_secs(0)));
let mut kademlia = Kademlia::with_config(
pub_key.to_peer_id(),
MemoryStore::new(pub_key.to_peer_id()),
kademlia_config,
);
let bootaddr = Multiaddr::from_str("/dnsaddr/bootstrap.libp2p.io").unwrap();
for peer in &BOOTNODES {
kademlia.add_address(&PeerId::from_str(peer).unwrap(), bootaddr.clone());
}
kademlia.bootstrap().unwrap();
Some(kademlia)
} else {
None
}
.into();

let autonat = if enable_autonat {
Some(autonat::Behaviour::new(
PeerId::from(pub_key.clone()),
Default::default(),
))
} else {
None
}
.into();

Self {
relay: relay::Behaviour::new(PeerId::from(pub_key.clone()), Default::default()),
ping: ping::Behaviour::new(ping::Config::new()),
identify: identify::Behaviour::new(
identify::Config::new("ipfs/0.1.0".to_string(), pub_key).with_agent_version(
format!("rust-libp2p-server/{}", env!("CARGO_PKG_VERSION")),
),
),
kademlia,
autonat,
}
}
}
39 changes: 39 additions & 0 deletions misc/server/src/config.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
use libp2p::Multiaddr;
use serde_derive::Deserialize;
use std::error::Error;
use std::path::Path;

#[derive(Clone, Deserialize)]
#[serde(rename_all = "PascalCase")]
pub(crate) struct Config {
pub(crate) identity: Identity,
pub(crate) addresses: Addresses,
}

impl Config {
pub(crate) fn from_file(path: &Path) -> Result<Self, Box<dyn Error>> {
Ok(serde_json::from_str(&std::fs::read_to_string(path)?)?)
}
}

#[derive(Clone, Deserialize)]
#[serde(rename_all = "PascalCase")]
pub(crate) struct Identity {
#[serde(rename = "PeerID")]
pub(crate) peer_id: String,
pub(crate) priv_key: String,
}

#[derive(Clone, Deserialize)]
#[serde(rename_all = "PascalCase")]
pub(crate) struct Addresses {
pub(crate) swarm: Vec<Multiaddr>,
pub(crate) append_announce: Vec<Multiaddr>,
}

impl zeroize::Zeroize for Config {
fn zeroize(&mut self) {
self.identity.peer_id.zeroize();
self.identity.priv_key.zeroize();
}
}
Loading