Experimental TSDB: Add support for local filesystem backend

[slim-bean]

What this PR does:

Add support for TSDB blocks to be saved to the local filesystem via the implementation which already exists for this in Thanos.

Used #2083 as a template

Checklist

• Tests updated
• Documentation added
• CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

Signed-off-by: Edward Welch [email protected]

[pstibrany]

Looks good to me. It could be nice to use in quick-start guide.

[pracucci]

I've pushed some little changes to improve the doc and address @pstibrany comment, then I went to try it out locally and a question arised. How is it supposed to be used? What's the use case for having this in Cortex?

[pstibrany]

What's the use case for having this in Cortex?

As Marco points out, one can simply disable shipping to the external storage, which is effectively the same.

[cyriltovena]

LGTM

I think the use case is for single node cortex, because cortex can run on arm arch and not prometheus.

Which is a big plus for Cortex if you think about it.

I think a documentation of this use case would be nice too.

[pracucci]

I talked to Ed offline. Ed's use case is running a single-node Cortex offline and then having a periodic rsync to synch blocks to a storage synched by a secondary Cortex cluster.

Despite this can be done disabling the shipper and having an rsync of blocks (not WAL), from a UX perspective looks more cleaner/easier using the filesystem storage backend and having an rsync with the filesystem dir as source. Moreover the shipper manipulates the meta.json (ie. adds the external labels) and looks the safest approach in this case.

[pracucci]

I think the use case is for single node cortex, because cortex can run on arm arch and not prometheus.

My take is that for a single node Cortex with blocks storage you don't need the filesystem support. You can just run Cortex single binary disabling the shipper.

[cyriltovena]

Yeah I don't have enough knowledge on this !