Is there an explicit way of clearing cache?

[zhiyuanGH]

I want to explore how can parameters affect the performance of lazy-pulling, e.g. ChunkSize. This is because some recent research papers report that lazy-pulling is ineffective when network latency is high, since the multiple http requests caused by on-demand fetching suffer from high latency.

I try to repeat running a same container to collect performance metrics. I remove images in nerdctl and restart stargz-snapshotter after each run, however it still logs hit layer cache:
Apr 24 16:44:35 gh-virtual-machine containerd-stargz-grpc[274521]: {"key":"default/66/extract-491987201-d0tB sha256:2414385fd51d34e07d564ec6041ee66de902424f028528bce52743d92b1bc875","level":"debug","mountpoint":"/var/lib/containerd-stargz-grpc/snapshotter/snapshots/66/fs","msg":"hit layer cache "ghcr.io/stargz-containers/node:17.8.0-esgz/sha256:e0912037615853fecb2a3de7eacb11e30debbc7ae24d64a7ad69bf9fa84ea10a"","parent":"sha256:a3926353a4b2389bed133fe4b9f8bdb8439529ba6a965b37ef0c1a7921043a00","src":"ghcr.io/stargz-containers/node:17.8.0-esgz/sha256:e0912037615853fecb2a3de7eacb11e30debbc7ae24d64a7ad69bf9fa84ea10a","time":"2023-04-24T16:44:35.493692205+08:00"}

I've notice that #309 implements some cache clearing functions when unmount.

Is there any explicit way to clear cache thoroughly? I think i shouldn't profile the system with cache un-cleared.

[zhiyuanGH]

Perhaps I misunderstand the log. In #600 I see: " layer are cached When filesystem.Mount is called for the first layer in an image, stargz-snapshotter resolves all layers in that image in parallel and caches these resolved layers metadata in LRU cache. When filesystem.Mount is called for the neighbouring layers of that image, the cached layer can be used for speeding up the mounts."

So that hit layer cache will be a common phenomenon even when run a container for the first time.

If this is correct, could I change my question to: Is their any factors will affect repeating profiling the stargz-snapshotter? Can I just delete the image and repeat profiling stargz on the same image?

[ktock]

Yes, stargz snapshotter caches resources for a while for future use.
If you want to ensure to clean up resources on the node, you can stop containerd-stargz-grpc process, remove files under stargz-snapshotter's root dir (typically /var/lib/containerd-stargz-grpc/) and restart stargz-snapshotter.

[zhiyuanGH]

Thanks for your reply.

When I tried to change the PrefetchSize, I noticed that eStargz landmark will be respected. Does it mean that we cannot change the prefetchsize of an eStargz image via configuration file? Is there any way of changing the prefetchsize when pulling an existing eStargz image, or we have to re-build it with a different landmark?

[ktock]

Yes, landmark is respected than PrefetchSize and no configuration flag is provided for limiting the prefetching size. Maybe we can introduce MaxPrefetchSize or PrefetchSizeLimit for this purpose. Patches are welcome if you're interested in.