Generic Send Offload (GSO) Support #1024
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
With GSO, we need to ask whether we can send 2 additional datagrams.
Wit GSO activated, encoding cmsgs paniced since not enough space was available. We need a minimum of 88 bytes.
`space_can_send` is now a separate function which returns whether there is sendable data in a certain space.
With GSO we need to be able to check whether multiple datagrams are sendable.
|
Same as #953 Unfortunately I seemed to have closed the old version and the UI doesn''t let me open it again. |
This allows easy access the information later on.
With GSO storing the buffer inside the builder does not work due to lifetime issues - we need to hold the buffer across various packets. This keeps the buffer outside of the builder and just references the start of a packet via an offset. This also removes the lifetime from `PacketBuilder`.
`finish_and_track_packet` will now take information from the builder when finalizing the packet.
This adds a new stat for the number of transmit calls and fixes the metrics for path challenges.
The option only needs to be sent for GSO (multiple datagrams in a buffer).
Ralith
requested changes
Feb 22, 2021
This change defers padding packets as a runtime decision while the packet is populated. For individual packets, we store in `SentFrames` whether a padding is required. If padding is required a `min_datagram_size` variable is increased, which will later on assure that only the last packet in a datagram gets padded.
This change implements an initial version of GSO support [1][2][3] for Linux, which improves the effiency of sending data. The approach taken in this change is to create a buffer which contains multiple datagrams in `Connection::poll_transmit`. This was picked over trying to merge packets in the endpoint task, since packets seem to need to be padded to a common segment size in order to make them sendable via GSO. In order to to this in an efficient fashion, the `poll_transmit` method was restructred. Instead of selecting spaces upfront, it will now loop through all possible packet spaces, check if there is pending data to send and create packets and datagrams out of this. The last packet which was written to a datagram buffer is not finalized until it is clear whether follow-up packets need to be written, since we need to know whether this packet should get padded to MTU length or not. This change doesn't enable GSO yet, since it will only produce a single datagram. I will create a separate change for this. Performance measurements: **No GSO:** ``` Sent 1073741824 bytes on 1 streams in 4.31s (237.66 MiB/s) ``` **With GSO (up to 8 packets):** ``` Sent 1073741824 bytes on 1 streams in 3.02s (339.18 MiB/s) ``` [1] http://vger.kernel.org/lpc_net2018_talks/willemdebruijn-lpc2018-udpgso-paper-DRAFT-1.pdf [2] http://vger.kernel.org/lpc_net2018_talks/willemdebruijn-lpc2018-udpgso-presentation-20181104.pdf [3] https://lwn.net/Articles/752956/
Ralith
approved these changes
Feb 22, 2021
|
Thanks for doing this work, really cool stuff! |
|
Some follow-up tweaks in #1026. |
15 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Generic Send Offload (GSO) Support
This change implements an initial version of GSO support [1][2][3]
for Linux, which improves the effiency of sending data.
The approach taken in this change is to create a buffer which contains
multiple datagrams in
Connection::poll_transmit. This was pickedover trying to merge packets in the endpoint task, since packets
seem to need to be padded to a common segment size in order to
make them sendable via GSO.
In order to to this in an efficient fashion, the
poll_transmitmethod was restructred. Instead of selecting spaces upfront, it
will now loop through all possible packet spaces, check if there is
pending data to send and create packets and datagrams out of this.
The last packet which was written to a datagram buffer is not
finalized until it is clear whether follow-up packets need to be
written, since we need to know whether this packet should get padded
to MTU length or not.
This change doesn't enable GSO yet, since it will only produce a
single datagram. I will create a separate change for this.
Performance measurements:
No GSO:
With GSO (up to 8 packets):
[1] http://vger.kernel.org/lpc_net2018_talks/willemdebruijn-lpc2018-udpgso-paper-DRAFT-1.pdf
[2] http://vger.kernel.org/lpc_net2018_talks/willemdebruijn-lpc2018-udpgso-presentation-20181104.pdf
[3] https://lwn.net/Articles/752956/