io_uring announcement references outdated benchmark

[romange]

60% improvement is cited in the original announcement. I wonder if tokio-iouring authors tried to validate this claim.

If you look at the only open issue in that repository you will see that @frevib confirms that the results are probably off since they are based on non-stable and outdated kernel, buggy liburing library and buggy benchmarking code.

[frevib]

Which benchmarks do you refer to? These benchmarks https://github.com/frevib/io_uring-echo-server/blob/master/benchmarks/benchmarks.md use a stable kernel (5.6), a likely stable liburing and likely stable benchmarking code. We already know the main gain from io_uring is less syscalls and batching of requests. Also io_uring has more features like poll mode that could improve performance. Besides that, it performs about the same as epoll. Netty is seeing benefits from both less sys calls and batching: https://twitter.com/normanmaurer/status/1311221843903295488?s=21

[romange]

Hi Hielke,
I am not questioning the advantages of io_uring API - I know it is better than epoll. However, I am sceptical regarding 60% number quoted in the announcement. I myself have been working with io_uring since March 2020 and I have not seen 60% improvement in my tests (or close to it). Imho, most of the CPU for high throughput use-cases is spent on interrupt handling which seems to be unrelated to epoll/io_uring APIs.

This file: https://github.com/frevib/io_uring-echo-server/blob/master/benchmarks/benchmarks.md has been updated last on Feb 2020. 5.6 was released on March 2020 and still had quite a few stability bugs that were fixed in 5.8+ kernels. You are also confirming state of this benchmark in this comment: frevib/io_uring-echo-server#8 (comment) and that's what I referenced in my question here.
In any case, I am very enthusiastic about the prospects of io_uring and I am glad it is being adopted by tokio.

[frevib]

Hi Roman,

It depends. If you can make a test that is syscall heavy, or if you optimize for batching a lot you could likely make it improve more than 60%. You could e.g. trade in latency for increasing batch size. In terms of “raw” IO io_uring should be about equally performant as epoll, for network IO at least. The Netty developers did manage some huge performance improvements, while some other popular programs cancelled io_uring implementation because they saw no improvement.