feat(cache): introduce tiered cache abstraction

[MrCroxx]

I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.

Changes:

• Introduce TieredCache and related abstractions, to unify underlying tiered cache implementation on various targets (non-linux doesn't support file cache).
• Refactor file cache value, introduce TieredCacheValue trait to support encode/decode only on demand.
• Remove unused Filter in the file cache system.

Checklist

• I have written necessary rustdoc comments
• I have added necessary unit tests and integration tests
• All checks passed in ./risedev check (or alias, ./risedev c)

Refer to a related PR or issue link (optional)

#198
#4050

[codecov]

Codecov Report

Merging #4406 (f5f1069) into main (7a0a99d) will decrease coverage by 0.08%.
The diff coverage is 61.43%.

@@            Coverage Diff             @@
##             main    #4406      +/-   ##
==========================================
- Coverage   74.54%   74.45%   -0.09%     
==========================================
  Files         848      848              
  Lines      124226   124497     +271     
==========================================
+ Hits        92605    92697      +92     
- Misses      31621    31800     +179     

Flag	Coverage Δ
rust	74.45% <61.43%> (-0.09%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/bench/file_cache_bench/bench.rs	0.00% <0.00%> (ø)
...c/compute/src/compute_observer/observer_manager.rs	0.00% <0.00%> (ø)
src/compute/src/rpc/service/exchange_service.rs	0.00% <ø> (ø)
...mpactor/src/compactor_observer/observer_manager.rs	0.00% <0.00%> (ø)
...rc/storage/hummock_sdk/src/filter_key_extractor.rs	93.48% <0.00%> (-3.15%)	⬇️
src/storage/src/hummock/file_cache/meta.rs	97.11% <ø> (ø)
src/storage/src/hummock/file_cache/mod.rs	88.88% <ø> (ø)
src/storage/src/hummock/mod.rs	82.90% <ø> (ø)
src/stream/src/executor/actor.rs	78.37% <ø> (ø)
src/stream/src/executor/barrier_align.rs	94.52% <ø> (ø)
... and 16 more

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

[hzxa21]

What will this enum be used? Will NoneCache only be used in non-linux target? If yes, I wonder whether we can prevent creating a TieredCache for non-linux target instead of introducing an enum, which makes each interface comes with a match.

[MrCroxx]

NoneCache can be used on any target, but FileCache only supports linux target.

Introducing TieredCache enum is good for integrating tiered cache into hummock, or we must add #[cfg(target_os = "target")] everywhere we use file cache in hummock code.

[hzxa21]

I was thinking about introducing a unify cache interface for both memory cache and file cache. In this case, if the target os linux and file cached is enabled, we create a cache instance containing both memory cache and file cache (similar to what we did in HybridObjectStore). Otherwise, we only create a block cache instance. I don't have a strong opinion on this so we can revisit it when we have file cache fully integrated with the memory cache.

[MrCroxx]

I'm not quite sure if unifying memory cache and tiered cache is a good idea, 🤔. Memory cache call are synchronous but tiered cache calls are asynchronous, unifying them may introduce extra cost for memory cache. Besides (weakly), #[async_trait] is not zero-cost yet, we need to use GAT to reduce the boxing cost (which is not that friendly for everyone, but affordable).

[hzxa21]

I'm not quite sure if unifying memory cache and tiered cache is a good idea, 🤔. Memory cache call are synchronous but tiered cache calls are asynchronous, unifying them may introduce extra cost for memory cache. Besides (weakly), #[async_trait] is not zero-cost yet, we need to use GAT to reduce the boxing cost (which is not that friendly for everyone, but affordable).

Ah... I missed that file cache has async interface. Just ignore what i just said.

[MrCroxx]

@wenym1 PTAL

[wenym1]

IMO, a more elegant and rust-style way to express the semantic of batch write will be, we have a StoreBatchWriter

pub struct StoreBatchWriter<'a, K, V>
where
    K: TieredCacheKey,
    V: TieredCacheValue,
{
    keys: Vec<K>,
    buffer: DioBuffer,
    blocs: Vec<BlockLoc>,

    block_size: usize,

    _phantom: PhantomData<(K, V)>,

    store: &'a mut Store
}

and there will be a start_batch_writer method in Store, which returns a StoreBatchWriter.

When finish of StoreBatchWriter is called, the StoreBatchWriter will do the encoding and write to the Store. And in this way, the definition of Store will not have an extra generic type V, and we don't need to add the new batch and insert method in Store.

[MrCroxx]

Nice advice! I'll refactor that.

[wenym1]

Rest LGTM