👑 Readonly B-Tree SST index

[kunga]

Description

Implement a readonly B-Tree SST index instead of the current flat one.

This improvement opens us the possibility of not always needing to keep SST's indexes in memory. We will be able to store a large amount of "cold" data without dozens of compute nodes to load needed parts of indexes on demand. Also enhances stability of Shared Cache and removes (actually reduces) passive bytes segment.

As we write a SST during the compaction process, its indexes will be readonly. However, the B-Tree structure minimizes needed disk I/O operations.

Also we will extend the current index format with additional information about data size and erased row count which will allow us to navigate easily and count statistics without the need of fully loading the whole index into memory.

Internal design doc.

Steps

1. Support simple flat index loading in all the usages

Instead of loading all SST's indexes on a tablet start, do it on the first usage. Before that loading keep memory free.

• Add and implement an index IIndexIter interface that loads index on the first usage 66a3bbb
• Support index loading in TRunIt 66a3bbb
• Support index loading from IPages implementations aab08a6 10fe953
• Support index loading inTCharge 55d8ae0
• Support index loading in TKeysLoader 0f2a2ff bac30c7
• Support index loading in BuildStats 287b4bb a18f545
• Support index loading in TForward 5ad1dcc BTreeIndex Bugfix load indexes in scan #657
• Support index loading in TDump cc271cd
• Remove the rest direct usages of index fields from TPart 16b0f8d
• Do not count flat index pages in transaction usage e4592d6
• Do not load index on tablet start bcae19f

2. Implement readonly B-Tree index

Implement B-Tree data structure, build and write them with an SST, implement searches.

• Implement B-Tree node class with serialization and deserialization to binary pages format cf4d301
• Implement B-Tree builder that accepts data pages and writes B-Tree nodes online 523950c
• Integrate B-Tree builder with TPartWriter 1ccc850
• Optimize B-Tree node format for groups where all row records have the same size 4c94e44 ebe7a9b
• Implement search by row id 402da95
• Implement search by key 1eda4dd
• Implement next and prev methods that can handle page faults 2f7d8b6
• Integrate TRunIt with a new B-Tree index iterator (TPartBtreeIndexIt) 2f7d8b6
• Add EnableLocalDBBtreeIndex setting ba6b7d6
• Integrate TKeysLoader with a new B-Tree index iterator 897ada7

3. Implement Precharge over readonly B-Tree index

Implement Precharge that works over tree structured index instead of the old flat one.

• Add ICharge interface and extract ChargeRange methods e7f64cc BTreeIndex Precharge Simplification #682
• Charge main pages within given row id range BTreeIndex Precharge RowId #754
• Charge main pages within given key range BTreeIndex Charge Keys #1102
• Charge groups pages BTreeIndex Charge Groups #1224
• Treat items limit BTreeIndex Charge Items Limit #1338
• Treats bytes limit BTreeIndex Charge Bytes Limit #1489
• Charge history pages BTreeIndex Charge History #1593
• Bugfix partially-loaded tree BTreeIndex Bugfix partially-loaded tree #1674
• Check that Iterator and Charge behaviour are consistent (all needed pages are precharged) BTreeIndex Test Charge Iter consistency #1869

4. Implement Build Stats over readonly B-Tree index

Implement Build Stats that works over tree structured index instead of the old flat one. We expect that it will no longer need to fully load index and only use upper-level B-Tree index nodes.

• Implement basic version BTreeIndex Stats iterator interface #1949 BTreeIndex Stats Load index nodes #2136
• Improve it not to load all index nodes BTreeIndex Stats Visit top nodes only #2177

5. Implement Scan over readonly B-Tree index

Implement Scan that works over over tree structured index.

• Implement Scan over B-Tree index basic version BTreeIndex Scan Implement basic version  #2260
• Forward-load B-Tree index in Scan BTreeIndex Forwad load #3710

6. Make fixes

Manually check suspicious places and fix what is broken with B-Tree index.

• Fix sticky pages logic after table compaction dda2617
• Keep B-Tree index in memory after compactions BTreeIndex Keep B-TreeIndex in cache after compactions #1165
• Fix B-Tree index iterator behaviour with corner pages BTreeIndex Iterator Remove single run optimization #995
• Bugfix TSchemeShard::Clear method BTreeIndex Bugfix SchemeShard Clear #655
• Count flat or B-Tree index to index size depending on EnableLocalDBBtreeIndex setting BTreeIndex Iterator benchmark #653 BTreeIndex Bugfix index size #671
• Add a possibility to turn off Flat index writing Add EnableLocalDBFlatIndex setting #2546
• Remove flat index usage from most of tests Fix all the test with EnableLocalDBFlatIndex = false EnableLocalDBBtreeIndex = true #2547
• Fix Cache line head don't want to do fetch but should verify Forward cache bugfix index pages queue verify #3134
• Limit leaf B-Tree index nodes small enough for skipping them in BuildStats Temporary use bigger index resolution to avoid full B-Tree index loading BTreeIndex Split Flush method, use bigger resolution #3182
• Check B-Tree index format in loader #4969
• 24-1 Prevent from using B-Tree index with incorrect configuration #4925

6. Test

Verify that everything works together not dramatically worse than before. Make tests on a slice.

• Benchmark TRunIt over B-Tree index BTreeIndex Iterator benchmark #653
• Benchmark Charge over B-Tree index BTreeIndex Test Charge Iter consistency #1869
• Check that EnableLocalDBBtreeIndex and EnableLocalDBFlatIndex settings work
• Check large table with a small amount of Shared Cache
• Check YCSB benchmark
• Check TPC-C benchmark
• Check that our changes do not effect Hive balancing (in mem size)

7. Make optimizations

• Preload indexes during part switches on followers Preload indexes on followers #4744
• ❓ Do not allocate in CreateIndexIter Forward cache simplify #2514 (comment)

8. Release

• Wait for a new release tag
• Enable EnableLocalDBBtreeIndex setting on pre-prod clusters (both indexes will be produced) (for some databases first)
• Wait until a new release is installed and won't be rolled back on prod clusters
• Enable EnableLocalDBBtreeIndex setting on prod clusters (for some databases first)
• Enable EnableLocalDBBtreeIndex setting by default
• Forcibly compact idle shards to build B-Tree index  #7718
• Disable EnableLocalDBFlatIndex setting

[kunga]

TPC-C 26-03-24

https://nda.ya.ru/t/n6PUIdLH7595hj B-Tree + Flat, Flat, B-Tree

B-Tree index

2024-03-26 05:41 UTC tpcc_ydb: Loading data done in 1854 seconds
2024-03-26 05:53 UTC tpcc_ydb: Compaction done in 713 seconds
2024-03-26 06:06 UTC tpcc_ydb: Built index in 747 seconds

tpmC: 202416

Flat index

2024-03-26 01:03 UTC tpcc_ydb: Loading data done in 1854 seconds
2024-03-26 01:13 UTC tpcc_ydb: Compaction done in 556 seconds
2024-03-26 01:24 UTC tpcc_ydb: Built index in 683 seconds

tpmC: 202205

YCSB 26-03-24

https://nda.ya.ru/t/jHlSyKq3759KBW B-Tree, Flat

B-Tree index

2024-03-26 12:18 UTC zipfian workload a: 345.4K Op/s, latency 99% 133.119 ms: 414848.7K oks, 0 not_found, 448 errors, 1201089 ms run time
2024-03-26 12:39 UTC zipfian workload b: 501.3K Op/s, latency 99% 18.655 ms: 600000.1K oks, 0 not_found, 0 errors, 1200882 ms run time
2024-03-26 13:00 UTC zipfian workload c: 548.6K Op/s, latency 99% 11.159 ms: 600000.1K oks, 0 not_found, 0 errors, 1099871 ms run time
2024-03-26 13:19 UTC zipfian workload f: 261.8K Op/s, latency 99% 19.631 ms: 471487.2K oks, 0 not_found, 259 errors, 1201088 ms run time
2024-03-26 13:40 UTC latest workload d: 61.9K Op/s, latency 99% 832.511 ms: 74310.4K oks, 9.9K not_found, 1.9K errors, 1201119 ms run time
2024-03-26 14:12 UTC zipfian workload e: 65.3K Op/s, latency 99% 99.007 ms: 39999.9K oks, 0 not_found, 128 errors, 612947 ms run time

Flat index

2024-03-26 14:43 UTC zipfian workload a: 354.8K Op/s, latency 99% 28.479 ms: 426058.9K oks, 0 not_found, 344 errors, 1201099 ms run time
2024-03-26 15:04 UTC zipfian workload b: 494.7K Op/s, latency 99% 22.751 ms: 594172.8K oks, 0 not_found, 12 errors, 1201091 ms run time
2024-03-26 15:25 UTC zipfian workload c: 532.5K Op/s, latency 99% 14.367 ms: 600000.1K oks, 0 not_found, 0 errors, 1132455 ms run time
2024-03-26 15:45 UTC zipfian workload f: 260.9K Op/s, latency 99% 23.839 ms: 469954.6K oks, 0 not_found, 135 errors, 1201086 ms run time
2024-03-26 16:06 UTC latest workload d: 66.8K Op/s, latency 99% 577.023 ms: 80221.9K oks, 2.2K not_found, 601 errors, 1201123 ms run time
2024-03-26 16:37 UTC zipfian workload e: 63.6K Op/s, latency 99% 474.367 ms: 39999.8K oks, 0 not_found, 264 errors, 629655 ms run time

[kunga]

BuildStats

Flat index

https://nda.ya.ru/t/u4oX-oPx759rBA

B-Tree index

https://nda.ya.ru/t/G8iQ0UAh759rCW

[kunga]

TPC-C 30-04-2024, 16K

https://nda.ya.ru/t/KHu9a09t75gmqt

Flat index

2024-04-30 11:38 UTC tpcc_ydb: Loading data done in 1831 seconds
2024-04-30 11:47 UTC tpcc_ydb: Compaction done in 562 seconds
2024-04-30 12:00 UTC tpcc_ydb: Built index in 757 seconds

Result: Total
  Time: 7200 seconds
  Start measure: 1714480513
  Warehouses: 16000
  New orders: 24262986
  tpmC: 202191
  Efficiency: 98.27
  Throughput: 7497
  Goodput: 8721

NewOrder: OK: 24262986, FAILED: 208 (0.0%)
Payment: OK: 23236184, FAILED: 10 (0.0%)
OrderStatus: OK: 2162490, FAILED: 0
Delivery: OK: 2156203, FAILED: 2988 (0.14%)
StockLevel: OK: 2162091, FAILED: 0
NewOrder:
  50%: 320 ms
  90%: 512 ms
  95%: 512 ms
  99%: 1250 ms
  99.9%: 2000 ms
Payment:
  50%: 144 ms
  90%: 224 ms
  95%: 256 ms
  99%: 480 ms
  99.9%: 512 ms
OrderStatus:
  50%: 96 ms
  90%: 192 ms
  95%: 224 ms
  99%: 320 ms
  99.9%: 480 ms
Delivery:
  50%: 1250 ms
  90%: 2500 ms
  95%: 3000 ms
  99%: 4500 ms
  99.9%: 6000 ms
StockLevel:
  50%: 72 ms
  90%: 144 ms
  95%: 192 ms
  99%: 352 ms
  99.9%: 512 ms

B-Tree index

2024-04-30 17:26 UTC tpcc_ydb: Loading data done in 1825 seconds
2024-04-30 17:35 UTC tpcc_ydb: Compaction done in 566 seconds
2024-04-30 17:47 UTC tpcc_ydb: Built index in 725 seconds

Result: Total
  Time: 7200 seconds
  Start measure: 1714501358
  Warehouses: 16000
  New orders: 24303007
  tpmC: 202524
  Efficiency: 98.43
  Throughput: 7508
  Goodput: 8738

NewOrder: OK: 24303007, FAILED: 136 (0.0%)
Payment: OK: 23273075, FAILED: 6 (0.0%)
OrderStatus: OK: 2166365, FAILED: 0
Delivery: OK: 2163922, FAILED: 2020 (0.09%)
StockLevel: OK: 2165109, FAILED: 0
NewOrder:
  50%: 288 ms
  90%: 512 ms
  95%: 512 ms
  99%: 1000 ms
  99.9%: 1500 ms
Payment:
  50%: 128 ms
  90%: 192 ms
  95%: 224 ms
  99%: 416 ms
  99.9%: 512 ms
OrderStatus:
  50%: 88 ms
  90%: 176 ms
  95%: 224 ms
  99%: 288 ms
  99.9%: 384 ms
Delivery:
  50%: 1250 ms
  90%: 2250 ms
  95%: 2750 ms
  99%: 4000 ms
  99.9%: 5500 ms
StockLevel:
  50%: 64 ms
  90%: 128 ms
  95%: 192 ms
  99%: 320 ms
  99.9%: 512 ms

[kunga]

YSCB 30-04-2024

https://nda.ya.ru/t/sMAtUlme75hHrn

Flat index

2024-04-30 22:59 UTC zipfian workload a: 355.1K Op/s, latency 99% 190.079 ms: 426468.1K oks, 0 not_found, 115 errors, 1201085 ms run time

2024-04-30 23:20 UTC zipfian workload b: 490.8K Op/s, latency 99% 18.271 ms: 589394.8K oks, 0 not_found, 0 errors, 1201091 ms run time

2024-04-30 23:41 UTC zipfian workload c: 520.5K Op/s, latency 99% 11.383 ms: 600000.1K oks, 0 not_found, 0 errors, 1153658 ms run time

2024-05-01 00:01 UTC zipfian workload f: 263.6K Op/s, latency 99% 27.871 ms: 474887.7K oks, 0 not_found, 274 errors, 1201085 ms run time

2024-05-01 00:22 UTC latest workload d: 66.6K Op/s, latency 99% 594.431 ms: 80124K oks, 0 not_found, 0 errors, 1204251 ms run time

2024-05-01 00:53 UTC zipfian workload e: 64.4K Op/s, latency 99% 83.263 ms: 40000K oks, 0 not_found, 98 errors, 621939 ms run time

B-Tree index

2024-05-01 04:02 UTC zipfian workload a: 372.8K Op/s, latency 99% 214.527 ms: 447685.1K oks, 0 not_found, 157 errors, 1201088 ms run time
2024-05-01 08:09 UTC zipfian workload a: 363.4K Op/s, latency 99% 202.239 ms: 436449K oks, 0 not_found, 224 errors, 1201078 ms run time

2024-05-01 04:23 UTC zipfian workload b: 499.7K Op/s, latency 99% 16.015 ms: 600000.1K oks, 0 not_found, 0 errors, 1200871 ms run time
2024-05-01 08:30 UTC zipfian workload b: 496.7K Op/s, latency 99% 16.199 ms: 596508.7K oks, 0 not_found, 1 errors, 1201091 ms run time

2024-05-01 04:44 UTC zipfian workload c: 519.7K Op/s, latency 99% 11.415 ms: 600000.1K oks, 0 not_found, 0 errors, 1157306 ms run time
2024-05-01 08:52 UTC zipfian workload c: 517.9K Op/s, latency 99% 11.263 ms: 600000.1K oks, 0 not_found, 0 errors, 1161620 ms run time

2024-05-01 05:04 UTC zipfian workload f: 268.6K Op/s, latency 99% 27.999 ms: 483795.6K oks, 0 not_found, 421 errors, 1201087 ms run time
2024-05-01 09:12 UTC zipfian workload f: 265.3K Op/s, latency 99% 50.847 ms: 477805K oks, 0 not_found, 734 errors, 1201083 ms run time

2024-05-01 05:25 UTC latest workload d: 63.2K Op/s, latency 99% 635.903 ms: 75794.2K oks, 0 not_found, 0 errors, 1201124 ms run time
2024-05-01 09:33 UTC latest workload d: 64.2K Op/s, latency 99% 647.679 ms: 77055.7K oks, 0 not_found, 0 errors, 1201127 ms run time

2024-05-01 05:56 UTC zipfian workload e: 65.2K Op/s, latency 99% 145.407 ms: 40000K oks, 0 not_found, 7 errors, 614401 ms run time
2024-05-01 10:03 UTC zipfian workload e: 64.2K Op/s, latency 99% 77.759 ms: 40000K oks, 0 not_found, 25 errors, 623347 ms run time

[kunga]

TPC-C 01-05-2024, 12K

https://nda.ya.ru/t/E-eHq6GZ75hyB5

Flat index

2024-05-01 13:17 UTC tpcc_ydb: Loading data done in 1405 seconds
2024-05-01 13:24 UTC tpcc_ydb: Compaction done in 411 seconds
2024-05-01 13:33 UTC tpcc_ydb: Built index in 555 seconds

Result: Total
  Time: 7200 seconds
  Start measure: 1714572511
  Warehouses: 12000
  New orders: 18355322
  tpmC: 152959
  Efficiency: 99.12
  Throughput: 5670
  Goodput: 6601

NewOrder: OK: 18355322, FAILED: 230 (0.0%)
Payment: OK: 17574233, FAILED: 1 (0.0%)
OrderStatus: OK: 1635884, FAILED: 0
Delivery: OK: 1636733, FAILED: 54 (0.0%)
StockLevel: OK: 1635779, FAILED: 2 (0.0%)
NewOrder:
  50%: 128 ms
  90%: 288 ms
  95%: 384 ms
  99%: 512 ms
  99.9%: 1000 ms
Payment:
  50%: 48 ms
  90%: 112 ms
  95%: 160 ms
  99%: 256 ms
  99.9%: 416 ms
OrderStatus:
  50%: 40 ms
  90%: 144 ms
  95%: 176 ms
  99%: 256 ms
  99.9%: 384 ms
Delivery:
  50%: 512 ms
  90%: 512 ms
  95%: 1000 ms
  99%: 1500 ms
  99.9%: 2250 ms
StockLevel:
  50%: 32 ms
  90%: 176 ms
  95%: 256 ms
  99%: 448 ms
  99.9%: 512 ms

B-Tree index

2024-05-01 16:41 UTC tpcc_ydb: Loading data done in 1369 seconds
2024-05-01 16:49 UTC tpcc_ydb: Compaction done in 441 seconds
2024-05-01 16:57 UTC tpcc_ydb: Built index in 513 seconds

Result: Total
  Time: 7200 seconds
  Start measure: 1714584764
  Warehouses: 12000
  New orders: 18358584
  tpmC: 152986
  Efficiency: 99.14
  Throughput: 5669
  Goodput: 6600

NewOrder: OK: 18358584, FAILED: 67 (0.0%)
Payment: OK: 17568479, FAILED: 8 (0.0%)
OrderStatus: OK: 1633702, FAILED: 0
Delivery: OK: 1634485, FAILED: 77 (0.0%)
StockLevel: OK: 1635871, FAILED: 0
NewOrder:
  50%: 144 ms
  90%: 288 ms
  95%: 384 ms
  99%: 512 ms
  99.9%: 1000 ms
Payment:
  50%: 48 ms
  90%: 112 ms
  95%: 160 ms
  99%: 256 ms
  99.9%: 448 ms
OrderStatus:
  50%: 40 ms
  90%: 144 ms
  95%: 192 ms
  99%: 256 ms
  99.9%: 416 ms
Delivery:
  50%: 512 ms
  90%: 512 ms
  95%: 1000 ms
  99%: 1500 ms
  99.9%: 2250 ms
StockLevel:
  50%: 32 ms
  90%: 192 ms
  95%: 288 ms
  99%: 480 ms
  99.9%: 512 ms

[kunga]

TPC-C 02-05-2024, 12K, 54GB cache

https://nda.ya.ru/t/oyxKo3hD75iN9U

Flat index

2024-05-02 08:57 UTC tpcc_ydb: Loading data done in 1478 seconds
2024-05-02 09:05 UTC tpcc_ydb: Compaction done in 443 seconds
2024-05-02 09:14 UTC tpcc_ydb: Built index in 566 seconds

Result: Total
  Time: 7200 seconds
  Start measure: 1714643384
  Warehouses: 12000
  New orders: 18178342
  tpmC: 151485
  Efficiency: 98.16
  Throughput: 5616
  Goodput: 6544

NewOrder: OK: 18178342, FAILED: 1617 (0.01%)
Payment: OK: 17405059, FAILED: 25 (0.0%)
OrderStatus: OK: 1621301, FAILED: 15 (0.0%)
Delivery: OK: 1620295, FAILED: 1185 (0.07%)
StockLevel: OK: 1618387, FAILED: 2 (0.0%)
NewOrder:
  50%: 352 ms
  90%: 512 ms
  95%: 1000 ms
  99%: 1500 ms
  99.9%: 3000 ms
Payment:
  50%: 96 ms
  90%: 224 ms
  95%: 288 ms
  99%: 448 ms
  99.9%: 512 ms
OrderStatus:
  50%: 128 ms
  90%: 288 ms
  95%: 352 ms
  99%: 512 ms
  99.9%: 512 ms
Delivery:
  50%: 1250 ms
  90%: 2500 ms
  95%: 3000 ms
  99%: 4000 ms
  99.9%: 5750 ms
StockLevel:
  50%: 224 ms
  90%: 512 ms
  95%: 512 ms
  99%: 1250 ms
  99.9%: 2000 ms

B-Tree index

2024-05-02 13:18 UTC tpcc_ydb: Loading data done in 1431 seconds
2024-05-02 13:26 UTC tpcc_ydb: Compaction done in 455 seconds
2024-05-02 13:34 UTC tpcc_ydb: Built index in 534 seconds

Result: Total
  Time: 7200 seconds
  Start measure: 1714659003
  Warehouses: 12000
  New orders: 18172437
  tpmC: 151435
  Efficiency: 98.13
  Throughput: 5613
  Goodput: 6540

NewOrder: OK: 18172437, FAILED: 564 (0.0%)
Payment: OK: 17402606, FAILED: 18 (0.0%)
OrderStatus: OK: 1620966, FAILED: 0
Delivery: OK: 1616072, FAILED: 1003 (0.06%)
StockLevel: OK: 1618266, FAILED: 0
NewOrder:
  50%: 352 ms
  90%: 512 ms
  95%: 1000 ms
  99%: 1500 ms
  99.9%: 2750 ms
Payment:
  50%: 96 ms
  90%: 224 ms
  95%: 288 ms
  99%: 480 ms
  99.9%: 512 ms
OrderStatus:
  50%: 144 ms
  90%: 320 ms
  95%: 384 ms
  99%: 512 ms
  99.9%: 512 ms
Delivery:
  50%: 1250 ms
  90%: 2500 ms
  95%: 3000 ms
  99%: 4250 ms
  99.9%: 5750 ms
StockLevel:
  50%: 256 ms
  90%: 512 ms
  95%: 1000 ms
  99%: 1250 ms
  99.9%: 2000 ms

[kunga]

TPC-C 02-05-2024, 12K, 18GB cache

https://nda.ya.ru/t/MYwKhca_75iuBR

Flat index

2024-05-02 19:10 UTC tpcc_ydb: Loading data done in 1501 seconds
2024-05-02 19:18 UTC tpcc_ydb: Compaction done in 450 seconds
2024-05-02 19:27 UTC tpcc_ydb: Built index in 555 seconds

Result: Total
  Time: 7200 seconds
  Start measure: 1714680150
  Warehouses: 12000
  New orders: 13898018
  tpmC: 115815
  Efficiency: 75.05
  Throughput: 4328
  Goodput: 5228

NewOrder: OK: 13898018, FAILED: 105094 (0.75%)
Payment: OK: 13374895, FAILED: 48410 (0.36%)
OrderStatus: OK: 1245056, FAILED: 3990 (0.32%)
Delivery: OK: 1215879, FAILED: 32209 (2.58%)
StockLevel: OK: 1244627, FAILED: 3817 (0.31%)
NewOrder:
  50%: 9000 ms
  90%: 15000 ms
  95%: 20000 ms
  99%: 20000 ms
  99.9%: 20000 ms
Payment:
  50%: 2750 ms
  90%: 4000 ms
  95%: 5250 ms
  99%: 9000 ms
  99.9%: 15000 ms
OrderStatus:
  50%: 3000 ms
  90%: 4000 ms
  95%: 5250 ms
  99%: 8000 ms
  99.9%: 15000 ms
Delivery:
  50%: 9000 ms
  90%: 20000 ms
  95%: 20000 ms
  99%: 20000 ms
  99.9%: 20000 ms
StockLevel:
  50%: 3750 ms
  90%: 5500 ms
  95%: 6000 ms
  99%: 9000 ms
  99.9%: 15000 ms

B-Tree index

2024-05-02 23:01 UTC tpcc_ydb: Loading data done in 1492 seconds
2024-05-02 23:09 UTC tpcc_ydb: Compaction done in 451 seconds
2024-05-02 23:17 UTC tpcc_ydb: Built index in 491 seconds

Result: Total
  Time: 7200 seconds
  Start measure: 1714693941
  Warehouses: 12000
  New orders: 14137163
  tpmC: 117808
  Efficiency: 76.34
  Throughput: 4400
  Goodput: 5299

NewOrder: OK: 14137163, FAILED: 90520 (0.64%)
Payment: OK: 13614464, FAILED: 37143 (0.27%)
OrderStatus: OK: 1268181, FAILED: 2875 (0.23%)
Delivery: OK: 1247545, FAILED: 18966 (1.5%)
StockLevel: OK: 1266308, FAILED: 2891 (0.23%)
NewOrder:
  50%: 8000 ms
  90%: 14000 ms
  95%: 20000 ms
  99%: 20000 ms
  99.9%: 20000 ms
Payment:
  50%: 2500 ms
  90%: 4000 ms
  95%: 5250 ms
  99%: 8000 ms
  99.9%: 15000 ms
OrderStatus:
  50%: 2500 ms
  90%: 4000 ms
  95%: 5250 ms
  99%: 8000 ms
  99.9%: 15000 ms
Delivery:
  50%: 8000 ms
  90%: 15000 ms
  95%: 20000 ms
  99%: 20000 ms
  99.9%: 20000 ms
StockLevel:
  50%: 3250 ms
  90%: 4750 ms
  95%: 5750 ms
  99%: 9000 ms
  99.9%: 15000 ms

[kunga]

TPC-C 03-05-2024, 12K, 2GB cache

https://nda.ya.ru/t/lJkYsNo375joSi

Flat index

2024-05-03 10:52 UTC tpcc_ydb: Loading data done in 1526 seconds
2024-05-03 10:59 UTC tpcc_ydb: Compaction done in 434 seconds
2024-05-03 11:08 UTC tpcc_ydb: Built index in 546 seconds

Result: Total
  Time: 7200 seconds
  Start measure: 1714736605
  Warehouses: 12000
  New orders: 8644037
  tpmC: 72032
  Efficiency: 46.68
  Throughput: 3465
  Goodput: 3453

NewOrder: OK: 8644037, FAILED: 2567753 (22.9%)
Payment: OK: 8832887, FAILED: 1915043 (17.82%)
OrderStatus: OK: 826607, FAILED: 174705 (17.45%)
Delivery: OK: 563554, FAILED: 433429 (43.47%)
StockLevel: OK: 830724, FAILED: 170401 (17.02%)
NewOrder:
  50%: 15000 ms
  90%: 20000 ms
  95%: 20000 ms
  99%: 20000 ms
  99.9%: 20000 ms
Payment:
  50%: 6000 ms
  90%: 14000 ms
  95%: 15000 ms
  99%: 15000 ms
  99.9%: 15000 ms
OrderStatus:
  50%: 6000 ms
  90%: 14000 ms
  95%: 15000 ms
  99%: 15000 ms
  99.9%: 15000 ms
Delivery:
  50%: 15000 ms
  90%: 20000 ms
  95%: 20000 ms
  99%: 20000 ms
  99.9%: 20000 ms
StockLevel:
  50%: 7000 ms
  90%: 14000 ms
  95%: 15000 ms
  99%: 15000 ms
  99.9%: 20000 ms

B-Tree index

2024-05-03 17:04 UTC tpcc_ydb: Loading data done in 1546 seconds
2024-05-03 17:12 UTC tpcc_ydb: Compaction done in 452 seconds
2024-05-03 17:21 UTC tpcc_ydb: Built index in 545 seconds

Result: Total
  Time: 7200 seconds
  Start measure: 1714759001
  Warehouses: 12000
  New orders: 9081395
  tpmC: 75677
  Efficiency: 49.04
  Throughput: 3515
  Goodput: 3611

NewOrder: OK: 9081395, FAILED: 2289951 (20.14%)
Payment: OK: 9290304, FAILED: 1616212 (14.82%)
OrderStatus: OK: 867382, FAILED: 147384 (14.52%)
Delivery: OK: 638883, FAILED: 373551 (36.9%)
StockLevel: OK: 870645, FAILED: 144406 (14.23%)
NewOrder:
  50%: 15000 ms
  90%: 20000 ms
  95%: 20000 ms
  99%: 20000 ms
  99.9%: 20000 ms
Payment:
  50%: 6000 ms
  90%: 13000 ms
  95%: 15000 ms
  99%: 15000 ms
  99.9%: 15000 ms
OrderStatus:
  50%: 6000 ms
  90%: 13000 ms
  95%: 15000 ms
  99%: 15000 ms
  99.9%: 15000 ms
Delivery:
  50%: 15000 ms
  90%: 20000 ms
  95%: 20000 ms
  99%: 20000 ms
  99.9%: 20000 ms
StockLevel:
  50%: 6000 ms
  90%: 13000 ms
  95%: 15000 ms
  99%: 15000 ms
  99.9%: 15000 ms

[kunga]

YCSB 04-05-2024, 12K, 2GB cache

https://nda.ya.ru/t/llFWtuzo75kKft

Flat index

2024-05-04 11:33 UTC zipfian workload a: 369.1K Op/s, latency 99% 158.335 ms: 443248.7K oks, 0 not_found, 422 errors, 1201090 ms run time

2024-05-04 11:54 UTC zipfian workload b: 503.5K Op/s, latency 99% 18.815 ms: 600000.1K oks, 0 not_found, 0 errors, 1191820 ms run time

2024-05-04 12:15 UTC zipfian workload c: 527.8K Op/s, latency 99% 11.335 ms: 600000.1K oks, 0 not_found, 0 errors, 1142053 ms run time

2024-05-04 12:35 UTC zipfian workload f: 276K Op/s, latency 99% 32.607 ms: 497156K oks, 0 not_found, 335 errors, 1201087 ms run time

2024-05-04 12:56 UTC latest workload d: 63.6K Op/s, latency 99% 600.063 ms: 76383.2K oks, 0 not_found, 0 errors, 1201122 ms run time

2024-05-04 13:28 UTC zipfian workload e: 65K Op/s, latency 99% 244.735 ms: 40000K oks, 0 not_found, 33 errors, 615966 ms run time

B-Tree index

2024-05-04 15:25 UTC zipfian workload a: 375.7K Op/s, latency 99% 55.167 ms: 451246.9K oks, 0 not_found, 413 errors, 1201087 ms run time

2024-05-04 15:47 UTC zipfian workload b: 500.3K Op/s, latency 99% 17.599 ms: 600000.1K oks, 0 not_found, 2 errors, 1199313 ms run time

2024-05-04 16:08 UTC zipfian workload c: 524.9K Op/s, latency 99% 11.559 ms: 600000.1K oks, 0 not_found, 0 errors, 1145535 ms run time

2024-05-04 16:28 UTC zipfian workload f: 275.4K Op/s, latency 99% 25.727 ms: 496070.9K oks, 0 not_found, 358 errors, 1201091 ms run time

2024-05-04 16:49 UTC latest workload d: 62.5K Op/s, latency 99% 591.359 ms: 75009.4K oks, 0 not_found, 0 errors, 1201149 ms run time

2024-05-04 17:21 UTC zipfian workload e: 62.9K Op/s, latency 99% 85.119 ms: 40000K oks, 0 not_found, 19 errors, 636709 ms run time