Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: log bandwidth on substream instead of socket level #3180

Merged

Conversation

melekes
Copy link
Contributor

@melekes melekes commented Nov 30, 2022

Description

Previously, the with_bandwidth_logging extension to Transport would track the bytes sent and received on a socket level. This however only works in conjunction with Transport upgrades where a separate multiplexer is negotiated on top of a regular stream.

With QUIC and WebRTC landing, this no longer works as those Transports bring their own multiplexing stack. To still allow for tracking of bandwidth, we refactor the with_bandwidth_logging extension to count the bytes send on all substreams opened through a StreamMuxer. This works, regardless of the underlying transport technology. It does omit certain layers. However, there isn't necessarily a "correct" layer to count bandwidth on because you can always go down another layer (IP, Ethernet, etc).

Closes #3157.

Change checklist

  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • A changelog entry has been made in the appropriate crates

swarm/src/lib.rs Outdated Show resolved Hide resolved
@melekes melekes marked this pull request as ready for review December 1, 2022 07:33
@melekes melekes changed the title log bandwidth at StreamMuxer level count bandwidth at the application level Dec 1, 2022
Copy link
Member

@mxinden mxinden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Direction looks good to me. A couple of comments.

CHANGELOG.md Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
src/bandwidth.rs Outdated Show resolved Hide resolved
src/bandwidth.rs Outdated Show resolved Hide resolved
src/bandwidth.rs Outdated Show resolved Hide resolved
src/bandwidth.rs Outdated Show resolved Hide resolved
melekes and others added 3 commits December 3, 2022 06:41
also
rename BandwidthLogging `new_with_sinks` to `new`
rename BandwidthConnecLogging to InstrumentedStream
src/transport_ext.rs Show resolved Hide resolved
src/transport_ext.rs Outdated Show resolved Hide resolved
src/transport_ext.rs Show resolved Hide resolved
Copy link
Contributor

@thomaseizinger thomaseizinger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, thank you!

One suggestion, otherwise happy with this modulo @mxinden's comments!

src/transport_ext.rs Outdated Show resolved Hide resolved
@thomaseizinger thomaseizinger changed the title count bandwidth at the application level feat: log bandwidth on substream instead of socket level Dec 7, 2022
@thomaseizinger
Copy link
Contributor

I've updated the PR description to prepare this for merging so we have a decent commit message.

@mergify
Copy link
Contributor

mergify bot commented Dec 9, 2022

This pull request has merge conflicts. Could you please resolve them @melekes? 🙏

thomaseizinger
thomaseizinger previously approved these changes Dec 19, 2022
Copy link
Contributor

@thomaseizinger thomaseizinger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@@ -136,6 +136,10 @@ env_logger = "0.10.0"
clap = { version = "4.0.13", features = ["derive"] }
tokio = { version = "1.15", features = ["io-util", "io-std", "macros", "rt", "rt-multi-thread"] }

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why split these with a newline?

@mergify mergify bot dismissed thomaseizinger’s stale review December 19, 2022 06:48

Approvals have been dismissed because the PR was updated after the send-it label was applied.

@mergify mergify bot merged commit 76abab9 into libp2p:master Dec 19, 2022
Comment on lines +139 to +141
libp2p-mplex = { version = "0.38.0", path = "muxers/mplex" }
libp2p-noise = { version = "0.41.0", path = "transports/noise" }
libp2p-tcp = { version = "0.38.0", path = "transports/tcp", features = ["tokio"] }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it a deliberate choice to link these with a version?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be fixed with #3261.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[transports/webrtc] Record total number of bytes sent / received in UDPMuxNewAddr
3 participants