You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I made some very quick tests on my system to measure a few timings. The source code is in this gist: https://gist.github.com/drdanz/9407a58608f1d013573cfcbb0b119f69
Feel free to import it in the repository if you believe it is worth
If you want to try it on your machine, compile start yarp server, start build/server, and run build/client
I used an image internally, just to quickly produce some data with a relatively big size so that the comparison is easier. I had similar results with both smaller and bigger data.
The result on my system is this:
● t_copy: 0.000071 # Make a copy of the image
● t_copywrite: 0.002975 # Make a copy and write to file of the image (raw write, no MATIO involved open and close file every time)
● t_send: 0.000248 # Send the image on the port (using fakeOpen + unix_stream)
● t_send_bg1: 0.000013 # Send the image on the port using background write (using fakeOpen + unix_stream)
● t_send_bg2: 0.000278 # Wait until the image is fully is sent (busy wait)
● t_copysend_bg1: 0.000078 # Copy and Send the image on the port using background write (copy)
● t_copysend_bg2: 0.000091 # Send in bg
● t_copysend_bg3: 0.000316 # Wait until the image is fully is sent (busy wait)
● t_send_tcp: 0.000317 # Send the image on the port using tcp
● t_send_fast_tcp: 0.000222 # Send the image on the port using fast_tcp
● t_send_udp: 0.004661 # Send the image on the port using udp
One copy is ok
2 copies is still ok (but it's going to use a significant amount of extra memory)
3 copies in unacceptable, the time is roughly the same as the one for sending the data
Writing the data is going to impact on the system significantly.
I beleve that writing the file with MATIO is very likely to take more time than the t_copywrite test, since there are other libraries involved.
Surprisingly fast_tcp is faster than unix_stream (this is just on the sender side), and udp is incredibly slow (and somehow messes up with the other protocols).
After these considerations, I still believe that the file should not be written on the same process that generates the data. From the timing point of view, the optimal option is, in my opinion, to make a copy of the data (on a buffer or something else) and send it to a separate process using a background write (see t_copysend_bg2).
The text was updated successfully, but these errors were encountered:
Thanks @drdanz ! I think that after we finish working on #29, we can see if writing to a file can be moved to a different process and if that provides any benefit w.r.t. to the user story (see #16). In particular, a typical failure mode with the usual log it that the yarplogger start is not enforced, so if we have any additional process we need to make sure that is properly automatically launched.
I made some very quick tests on my system to measure a few timings. The source code is in this gist: https://gist.github.com/drdanz/9407a58608f1d013573cfcbb0b119f69
Feel free to import it in the repository if you believe it is worth
If you want to try it on your machine, compile start
yarp server
, startbuild/server
, and runbuild/client
I used an image internally, just to quickly produce some data with a relatively big size so that the comparison is easier. I had similar results with both smaller and bigger data.
The result on my system is this:
t_copywrite
test, since there are other libraries involved.After these considerations, I still believe that the file should not be written on the same process that generates the data. From the timing point of view, the optimal option is, in my opinion, to make a copy of the data (on a buffer or something else) and send it to a separate process using a background write (see
t_copysend_bg2
).The text was updated successfully, but these errors were encountered: