core/rawdb: avoid unnecessary receipt processing for log filtering

[s1na]

This is a successor for #23058. In this PR, the new lightweight receiptLogs type has been instead moved to rawdb and made private.

[s1na]

To see how these types are used refer here: s1na#5

[holiman]

LGTM

[s1na]

To estimate how effective the triage suggestion #23058 (comment) would be I added 2 metrics in #23520 which show us how often we lookup a block with zero matching logs and it seems to happen in ~1% of cases. That means we can avoid reading block body from db in 1% of cases by filtering deeper in the stack.

[s1na]

I did a hacky implementation of the optimization in s1na@7fd9ac9 and measured the time for a few contracts/events (see script). The script sends many subsequent getLogs requests, each for a range of 2000 blocks.

The right-side chart is the mean time for responding to a request and left-side is the mean time for processing one matching block within a request. I stopped the node twice and changed branches:

• Last numbers are from baseline (master)
• Middle ones is this PR
• In the first one we avoid reading the block body from db when it has zero matching logs

Something interesting I hadn't noticed is why the response time is increasing over time (I used to test with lower number of requests)

[MariusVanDerWijden]

I would prefer ReadLogs to return an error instead of relying on logs to be non nil here

[s1na]

I hear you. just didn't return an error to be consistent with the rest of the accessors

[s1na]

Added a benchmark to compare decoding a stored receipt the normal way (via ReceiptForStorage) against the new type rlpLogs. Results from my own machine:

BenchmarkDecodeRLPLogs
BenchmarkDecodeRLPLogs/ReceiptForStorage
BenchmarkDecodeRLPLogs/ReceiptForStorage-8              1000000000               0.00242 ns/op
BenchmarkDecodeRLPLogs/ReceiptForStorage-8              1000000000               0.00257 ns/op
BenchmarkDecodeRLPLogs/ReceiptForStorage-8              1000000000               0.00273 ns/op
BenchmarkDecodeRLPLogs/ReceiptForStorage-8              1000000000               0.00268 ns/op
BenchmarkDecodeRLPLogs/ReceiptForStorage-8              1000000000               0.00263 ns/op
BenchmarkDecodeRLPLogs/rlpLogs
BenchmarkDecodeRLPLogs/rlpLogs-8                        1000000000               0.000877 ns/op
BenchmarkDecodeRLPLogs/rlpLogs-8                        1000000000               0.000989 ns/op
BenchmarkDecodeRLPLogs/rlpLogs-8                        1000000000               0.000871 ns/op
BenchmarkDecodeRLPLogs/rlpLogs-8                        1000000000               0.000871 ns/op
BenchmarkDecodeRLPLogs/rlpLogs-8                        1000000000               0.000926 ns/op

[holiman]

Benchmark is wrong -- you need to do the work exactly b.N times

[s1na]

Thanks for the catch. Here are the updated result:

BenchmarkDecodeRLPLogs
BenchmarkDecodeRLPLogs/ReceiptForStorage
BenchmarkDecodeRLPLogs/ReceiptForStorage-8                   452           2589949 ns/op          660146 B/op       8021 allocs/op
BenchmarkDecodeRLPLogs/ReceiptForStorage-8                   452           2593112 ns/op          660132 B/op       8020 allocs/op
BenchmarkDecodeRLPLogs/ReceiptForStorage-8                   453           2568350 ns/op          660157 B/op       8021 allocs/op
BenchmarkDecodeRLPLogs/ReceiptForStorage-8                   459           2568151 ns/op          660126 B/op       8020 allocs/op
BenchmarkDecodeRLPLogs/ReceiptForStorage-8                   453           2599324 ns/op          660153 B/op       8021 allocs/op
BenchmarkDecodeRLPLogs/rlpLogs
BenchmarkDecodeRLPLogs/rlpLogs-8                            1237            970215 ns/op          497811 B/op       5870 allocs/op
BenchmarkDecodeRLPLogs/rlpLogs-8                            1215            966026 ns/op          497810 B/op       5870 allocs/op
BenchmarkDecodeRLPLogs/rlpLogs-8                            1234           1019230 ns/op          497813 B/op       5870 allocs/op
BenchmarkDecodeRLPLogs/rlpLogs-8                            1248            974802 ns/op          497814 B/op       5870 allocs/op
BenchmarkDecodeRLPLogs/rlpLogs-8                            1196            965926 ns/op          497813 B/op       5870 allocs/op

[s1na]

• I re-ran the benchmarks with geth disconnected from the network. This clearly removes the creep-up effect.
• Also did a few runs of each variant in semi-random order to make the distinction more visible.
• Finally, the disk read meter shows that optim2 is in fact reading less data on the margin. Optim2 is around 6.48 and optim1 around 6.77
  • The chaindata data read meter has more variance, not sure why

[holiman]

LGTM
"faster is better"

[holiman]

Oh no needs a rebase

[s1na]

@holiman I rebased. Just a reminder that this PR only has Optim1. Didn't want to make it too big by including Optim2 as well