From f8d487c4ed115d7828adac2070c4c76c7b23fe58 Mon Sep 17 00:00:00 2001 From: Reid Beels Date: Wed, 11 Jan 2023 09:42:24 -0800 Subject: [PATCH 1/3] docs(caching): Restructure and improve caching docs --- docs/docs/installation/cache.mdx | 60 ++++++++++++++++++++++---------- 1 file changed, 41 insertions(+), 19 deletions(-) diff --git a/docs/docs/installation/cache.mdx b/docs/docs/installation/cache.mdx index 4838fc47e61c1..740dac2c7d6e4 100644 --- a/docs/docs/installation/cache.mdx +++ b/docs/docs/installation/cache.mdx @@ -7,19 +7,46 @@ version: 1 ## Caching -Superset uses [Flask-Caching](https://flask-caching.readthedocs.io/) for caching purposes. Configuring caching is as easy as providing a custom cache config in your -`superset_config.py` that complies with [the Flask-Caching specifications](https://flask-caching.readthedocs.io/en/latest/#configuring-flask-caching). -Flask-Caching supports various caching backends, including Redis, Memcached, SimpleCache (in-memory), or the -local filesystem. Custom cache backends are also supported. See [here](https://flask-caching.readthedocs.io/en/latest/#custom-cache-backends) for specifics. -The following cache configurations can be customized: -- Metadata cache (optional): `CACHE_CONFIG` -- Charting data queried from datasets (optional): `DATA_CACHE_CONFIG` -- SQL Lab query results (optional): `RESULTS_BACKEND`. See [Async Queries via Celery](/docs/installation/async-queries-celery) for details +Superset uses [Flask-Caching](https://flask-caching.readthedocs.io/) for caching purposes. Flask-Caching supports various caching backends, including Redis (recommended), Memcached, SimpleCache (in-memory), or the local filesystem. [Custom cache backends](https://flask-caching.readthedocs.io/en/latest/#custom-cache-backends) are also supported. + +Caching can be configured by providing a dictionaries in +`superset_config.py` that comply with [the Flask-Caching config specifications](https://flask-caching.readthedocs.io/en/latest/#configuring-flask-caching). + +The following cache configurations can be customized in this way: - Dashboard filter state (required): `FILTER_STATE_CACHE_CONFIG`. - Explore chart form data (required): `EXPLORE_FORM_DATA_CACHE_CONFIG` +- Metadata cache (optional): `CACHE_CONFIG` +- Charting data queried from datasets (optional): `DATA_CACHE_CONFIG` + +For example, to configure the filter state cache using redis: + +```python +FILTER_STATE_CACHE_CONFIG = { + 'CACHE_TYPE': 'RedisCache', + 'CACHE_DEFAULT_TIMEOUT': 86400, + 'CACHE_KEY_PREFIX': 'superset_filter_cache', + 'CACHE_REDIS_URL': 'redis://localhost:6379/0' +} +``` + +For chart data, Superset goes up a “timeout search path”, from the chart's configuration +to the dataset’s, the database’s, then ultimately falls back to the global default +defined in `DATA_CACHE_CONFIG`. + +### Dependencies + +In order to use dedicated cache stores, additional python libraries must be installed + +- For Redis: we recommend the [redis](https://pypi.python.org/pypi/redis) Python package +- Memcached: we recommend using [pylibmc](https://pypi.org/project/pylibmc/) client library as + `python-memcached` does not handle storing binary data correctly. + +These libraries can be installed using pip. + +### Fallback Metastore Cache + +Note, that some form of Filter State and Explore caching are required. If either of these caches are undefined, Superset falls back to using a built-in cache that stores data in the metadata database. While it is recommended to use a dedicated cache, the built-in cache can also be used to cache other data. -Please note, that Dashboard and Explore caching is required. If these caches are undefined, Superset falls back to using a built-in cache that stores data -in the metadata database. While it is recommended to use a dedicated cache, the built-in cache can also be used to cache other data. For example, to use the built-in cache to store chart data, use the following config: ```python @@ -30,17 +57,12 @@ DATA_CACHE_CONFIG = { } ``` -- Redis (recommended): we recommend the [redis](https://pypi.python.org/pypi/redis) Python package -- Memcached: we recommend using [pylibmc](https://pypi.org/project/pylibmc/) client library as - `python-memcached` does not handle storing binary data correctly. - -Both of these libraries can be installed using pip. +### SQL Lab Query Results -For chart data, Superset goes up a “timeout search path”, from a slice's (chart's) configuration -to the dataset’s, the database’s, then ultimately falls back to the global default -defined in `DATA_CACHE_CONFIG`. +Caching for SQL Lab query results is used when async queries are enabled and is configured using `RESULTS_BACKEND`. -## Celery beat +Note that this configuration does not use a flask-caching dictionary for its configuration, but instead requires a cachelib object. + See [Async Queries via Celery](/docs/installation/async-queries-celery) for details. ### Caching Thumbnails From 4f74f592e8efdab35028190e2a90d11391e3c3f6 Mon Sep 17 00:00:00 2001 From: Reid Beels Date: Thu, 12 Jan 2023 16:16:15 -0800 Subject: [PATCH 2/3] docs(caching): Clarify chart cache timeout overrides --- docs/docs/installation/cache.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/docs/installation/cache.mdx b/docs/docs/installation/cache.mdx index 740dac2c7d6e4..44f6313541200 100644 --- a/docs/docs/installation/cache.mdx +++ b/docs/docs/installation/cache.mdx @@ -29,10 +29,6 @@ FILTER_STATE_CACHE_CONFIG = { } ``` -For chart data, Superset goes up a “timeout search path”, from the chart's configuration -to the dataset’s, the database’s, then ultimately falls back to the global default -defined in `DATA_CACHE_CONFIG`. - ### Dependencies In order to use dedicated cache stores, additional python libraries must be installed @@ -57,6 +53,10 @@ DATA_CACHE_CONFIG = { } ``` +### Chart Cache Timeout + +The cache timeout for charts may be overridden by the settings for an individual chart, dataset, or database. Each of these configurations will be checked in order before falling back to the default value defined in `DATA_CACHE_CONFIG. + ### SQL Lab Query Results Caching for SQL Lab query results is used when async queries are enabled and is configured using `RESULTS_BACKEND`. From 45b590401a1c83d8b3965e2febb079d6e73c25a3 Mon Sep 17 00:00:00 2001 From: Reid Beels Date: Thu, 12 Jan 2023 16:18:18 -0800 Subject: [PATCH 3/3] docs(caching): Hard wrap lines in markdown --- docs/docs/installation/cache.mdx | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/docs/docs/installation/cache.mdx b/docs/docs/installation/cache.mdx index 44f6313541200..58b4bcb2b0b74 100644 --- a/docs/docs/installation/cache.mdx +++ b/docs/docs/installation/cache.mdx @@ -7,10 +7,14 @@ version: 1 ## Caching -Superset uses [Flask-Caching](https://flask-caching.readthedocs.io/) for caching purposes. Flask-Caching supports various caching backends, including Redis (recommended), Memcached, SimpleCache (in-memory), or the local filesystem. [Custom cache backends](https://flask-caching.readthedocs.io/en/latest/#custom-cache-backends) are also supported. +Superset uses [Flask-Caching](https://flask-caching.readthedocs.io/) for caching purposes. +Flask-Caching supports various caching backends, including Redis (recommended), Memcached, +SimpleCache (in-memory), or the local filesystem. +[Custom cache backends](https://flask-caching.readthedocs.io/en/latest/#custom-cache-backends) +are also supported. Caching can be configured by providing a dictionaries in -`superset_config.py` that comply with [the Flask-Caching config specifications](https://flask-caching.readthedocs.io/en/latest/#configuring-flask-caching). +`superset_config.py` that comply with[the Flask-Caching config specifications](https://flask-caching.readthedocs.io/en/latest/#configuring-flask-caching). The following cache configurations can be customized in this way: - Dashboard filter state (required): `FILTER_STATE_CACHE_CONFIG`. @@ -41,7 +45,10 @@ These libraries can be installed using pip. ### Fallback Metastore Cache -Note, that some form of Filter State and Explore caching are required. If either of these caches are undefined, Superset falls back to using a built-in cache that stores data in the metadata database. While it is recommended to use a dedicated cache, the built-in cache can also be used to cache other data. +Note, that some form of Filter State and Explore caching are required. If either of these caches +are undefined, Superset falls back to using a built-in cache that stores data in the metadata +database. While it is recommended to use a dedicated cache, the built-in cache can also be used +to cache other data. For example, to use the built-in cache to store chart data, use the following config: @@ -55,14 +62,19 @@ DATA_CACHE_CONFIG = { ### Chart Cache Timeout -The cache timeout for charts may be overridden by the settings for an individual chart, dataset, or database. Each of these configurations will be checked in order before falling back to the default value defined in `DATA_CACHE_CONFIG. +The cache timeout for charts may be overridden by the settings for an individual chart, dataset, or +database. Each of these configurations will be checked in order before falling back to the default +value defined in `DATA_CACHE_CONFIG. ### SQL Lab Query Results -Caching for SQL Lab query results is used when async queries are enabled and is configured using `RESULTS_BACKEND`. +Caching for SQL Lab query results is used when async queries are enabled and is configured using +`RESULTS_BACKEND`. -Note that this configuration does not use a flask-caching dictionary for its configuration, but instead requires a cachelib object. - See [Async Queries via Celery](/docs/installation/async-queries-celery) for details. +Note that this configuration does not use a flask-caching dictionary for its configuration, but +instead requires a cachelib object. + +See [Async Queries via Celery](/docs/installation/async-queries-celery) for details. ### Caching Thumbnails