Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A few notes about timings #30

Open
drdanz opened this issue Jan 11, 2021 · 1 comment
Open

A few notes about timings #30

drdanz opened this issue Jan 11, 2021 · 1 comment
Assignees

Comments

@drdanz
Copy link
Member

drdanz commented Jan 11, 2021

I made some very quick tests on my system to measure a few timings. The source code is in this gist: https://gist.github.com/drdanz/9407a58608f1d013573cfcbb0b119f69
Feel free to import it in the repository if you believe it is worth
If you want to try it on your machine, compile start yarp server, start build/server, and run build/client
I used an image internally, just to quickly produce some data with a relatively big size so that the comparison is easier. I had similar results with both smaller and bigger data.

The result on my system is this:

● t_copy:          0.000071  # Make a copy of the image
● t_copywrite:     0.002975  # Make a copy and write to file of the image (raw write, no MATIO involved open and close file every time)
● t_send:          0.000248  # Send the image on the port (using fakeOpen + unix_stream)
● t_send_bg1:      0.000013  # Send the image on the port using background write (using fakeOpen + unix_stream)
● t_send_bg2:      0.000278  # Wait until the image is fully is sent (busy wait)
● t_copysend_bg1:  0.000078  # Copy and Send the image on the port using background write (copy)
● t_copysend_bg2:  0.000091  # Send in bg
● t_copysend_bg3:  0.000316  # Wait until the image is fully is sent (busy wait)
● t_send_tcp:      0.000317  # Send the image on the port using tcp
● t_send_fast_tcp: 0.000222  # Send the image on the port using fast_tcp
● t_send_udp:      0.004661  # Send the image on the port using udp
  • One copy is ok
  • 2 copies is still ok (but it's going to use a significant amount of extra memory)
  • 3 copies in unacceptable, the time is roughly the same as the one for sending the data
  • Writing the data is going to impact on the system significantly.
  • I beleve that writing the file with MATIO is very likely to take more time than the t_copywrite test, since there are other libraries involved.
  • Surprisingly fast_tcp is faster than unix_stream (this is just on the sender side), and udp is incredibly slow (and somehow messes up with the other protocols).

After these considerations, I still believe that the file should not be written on the same process that generates the data. From the timing point of view, the optimal option is, in my opinion, to make a copy of the data (on a buffer or something else) and send it to a separate process using a background write (see t_copysend_bg2).

@traversaro
Copy link
Member

Thanks @drdanz ! I think that after we finish working on #29, we can see if writing to a file can be moved to a different process and if that provides any benefit w.r.t. to the user story (see #16). In particular, a typical failure mode with the usual log it that the yarplogger start is not enforced, so if we have any additional process we need to make sure that is properly automatically launched.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants