You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There are two important time related metrics we measure for records:
the time a record spends in Conduit only (from the time it's read till the time it's ack'd) and,
the time a record spends in the whole pipeline, on average
The first is provided by the built-in metrics, and its measured for each record individually. The second one is calculated (by dividing the pipeline run time with the number of records). The first one should be used carefully in test interpretations, since there's some parallelism in Conduit, due to which the the time Conduit spends handling a record is not always (inversely) proportionate with the average pipeline rate.
The "bytes per second" we have currently is using the first time duration, and as such may be confusing and not always indicative of Conduit's performance. We should change it to be "total bytes ÷ pipeline run time".
The text was updated successfully, but these errors were encountered:
Feature description
There are two important time related metrics we measure for records:
The first is provided by the built-in metrics, and its measured for each record individually. The second one is calculated (by dividing the pipeline run time with the number of records). The first one should be used carefully in test interpretations, since there's some parallelism in Conduit, due to which the the time Conduit spends handling a record is not always (inversely) proportionate with the average pipeline rate.
The "bytes per second" we have currently is using the first time duration, and as such may be confusing and not always indicative of Conduit's performance. We should change it to be "total bytes ÷ pipeline run time".
The text was updated successfully, but these errors were encountered: