Docker Compose fails to start because of slack issue and TypeError

[Zverik]

I run docker-compose up and get many error, from which the most prominent is "ModuleNotFoundError: No module named 'slack'".

Expected results

Docker image successfully built and run.

Actual results

I get these errors:

• ModuleNotFoundError: No module named 'slack' (in /app/superset/tasks/slack_util.py, line from slack import WebClient)
• TypeError: init() got an unexpected keyword argument 'min_inclusive' (in /app/superset/charts/schemas.py, line error=_("confidence_interval must be between 0 and 1 (exclusive)"),)

When I open the UI, I get the last one printed.

Environment

Everything from the docker files, I did no modifications. Superset cloned from commit 807bd65. Before that I was using a June build of Superset, and it worked fine.

Additional context

Full stack traces:

superset_worker          | ERROR:superset.app:Failed to create app
superset_worker          | Traceback (most recent call last):
superset_worker          |   File "/app/superset/app.py", line 62, in create_app
superset_worker          |     app_initializer.init_app()
superset_worker          |   File "/app/superset/app.py", line 500, in init_app
superset_worker          |     self.init_app_in_ctx()
superset_worker          |   File "/app/superset/app.py", line 471, in init_app_in_ctx
superset_worker          |     self.configure_data_sources()
superset_worker          |   File "/app/superset/app.py", line 516, in configure_data_sources
superset_worker          |     ConnectorRegistry.register_sources(module_datasource_map)
superset_worker          |   File "/app/superset/connectors/connector_registry.py", line 39, in register_sources
superset_worker          |     module_obj = __import__(module_name, fromlist=class_names)
superset_worker          |   File "/app/superset/connectors/sqla/__init__.py", line 17, in <module>
superset_worker          |     from . import models, views
superset_worker          |   File "/app/superset/connectors/sqla/views.py", line 34, in <module>
superset_worker          |     from superset.connectors.base.views import DatasourceModelView
superset_worker          |   File "/app/superset/connectors/base/views.py", line 21, in <module>
superset_worker          |     from superset.views.base import SupersetModelView
superset_worker          |   File "/app/superset/views/__init__.py", line 17, in <module>
superset_worker          |     from . import (
superset_worker          |   File "/app/superset/views/alerts.py", line 28, in <module>
superset_worker          |     from superset.tasks.schedules import schedule_alert_query
superset_worker          |   File "/app/superset/tasks/schedules.py", line 65, in <module>
superset_worker          |     from superset.tasks.slack_util import deliver_slack_msg
superset_worker          |   File "/app/superset/tasks/slack_util.py", line 23, in <module>
superset_worker          |     from slack import WebClient
superset_worker          | ModuleNotFoundError: No module named 'slack'

superset_app             | ERROR:superset.app:Failed to create app
superset_app             | Traceback (most recent call last):
superset_app             |   File "/app/superset/app.py", line 62, in create_app
superset_app             |     app_initializer.init_app()
superset_app             |   File "/app/superset/app.py", line 500, in init_app
superset_app             |     self.init_app_in_ctx()
superset_app             |   File "/app/superset/app.py", line 480, in init_app_in_ctx
superset_app             |     self.init_views()
superset_app             |   File "/app/superset/app.py", line 128, in init_views
superset_app             |     from superset.charts.api import ChartRestApi
superset_app             |   File "/app/superset/charts/api.py", line 45, in <module>
superset_app             |     from superset.charts.schemas import (
superset_app             |   File "/app/superset/charts/schemas.py", line 445, in <module>
superset_app             |     class ChartDataProphetOptionsSchema(ChartDataPostProcessingOperationOptionsSchema):
superset_app             |   File "/app/superset/charts/schemas.py", line 472, in ChartDataProphetOptionsSchema
superset_app             |     error=_("`confidence_interval` must be between 0 and 1 (exclusive)"),
superset_app             | TypeError: __init__() got an unexpected keyword argument 'min_inclusive'

[issue-label-bot]

Issue-Label Bot is automatically applying the label #bug to this issue, with a confidence of 0.81. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[willbarrett]

@Zverik thanks for reporting this. Please run docker-compose build to recreate your containers first to be sure that the dependency really isn't being installed. If that does not resolve the issue, please try again after #10766 is merged. We've been running into some issues with cached dependencies recently in the Docker containers.

[Zverik]

Two side notes while rebuilding:

1. I needed to delete preset/superset and incubator-superset_superset images, otherwise I got errors on building.
2. pip install produces one error: "ERROR: apache-superset 0.999.0.dev0 has requirement pyarrow<0.18,>=0.17.0, but you'll have pyarrow 1.0.1 which is incompatible."

(still starting superset)

[Zverik]

Oh well, clean superset is not initializing (empty tables) because of the pyarrow dependency now:

superset_init            | Traceback (most recent call last):
superset_init            |   File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 583, in _build_master
superset_init            |     ws.require(__requires__)
superset_init            |   File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 900, in require
superset_init            |     needed = self.resolve(parse_requirements(requirements))
superset_init            |   File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 791, in resolve
superset_init            |     raise VersionConflict(dist, req).with_context(dependent_req)
superset_init            | pkg_resources.ContextualVersionConflict: (pyarrow 1.0.1 (/usr/local/lib/python3.6/site-packages), Requirement.parse('pyarrow<0.18,>=0.17.0'), {'apache-superset'})
superset_init            | 
superset_init            | During handling of the above exception, another exception occurred:
superset_init            | 
superset_init            | Traceback (most recent call last):
superset_init            |   File "/usr/local/bin/superset", line 6, in <module>
superset_init            |     from pkg_resources import load_entry_point
superset_init            |   File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3251, in <module>
superset_init            |     @_call_aside
superset_init            |   File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3235, in _call_aside
superset_init            |     f(*args, **kwargs)
superset_init            |   File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3264, in _initialize_master_working_set
superset_init            |     working_set = WorkingSet._build_master()
superset_init            |   File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 585, in _build_master
superset_init            |     return cls._build_from_requirements(__requires__)
superset_init            |   File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 598, in _build_from_requirements
superset_init            |     dists = ws.resolve(reqs, Environment())
superset_init            |   File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 786, in resolve
superset_init            |     raise DistributionNotFound(req, requirers)
superset_init            | pkg_resources.DistributionNotFound: The 'pyarrow<0.18,>=0.17.0' distribution was not found and is required by apache-superset
superset_init exited with code 1

I have opened the frontend, but when I click on any chart, dashboard, or datasource, I get this error, or similar ones:

sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedColumn) column sql_metrics.extra does not exist
LINE 1: ...sql_metrics.expression AS sql_metrics_expression, sql_metric...
                                                             ^

[SQL: SELECT sql_metrics.created_on AS sql_metrics_created_on, sql_metrics.changed_on AS sql_metrics_changed_on, sql_metrics.id AS sql_metrics_id, sql_metrics.metric_name AS sql_metrics_metric_name, sql_metrics.verbose_name AS sql_metrics_verbose_name, sql_metrics.metric_type AS sql_metrics_metric_type, sql_metrics.description AS sql_metrics_description, sql_metrics.d3format AS sql_metrics_d3format, sql_metrics.warning_text AS sql_metrics_warning_text, sql_metrics.table_id AS sql_metrics_table_id, sql_metrics.expression AS sql_metrics_expression, sql_metrics.extra AS sql_metrics_extra, sql_metrics.created_by_fk AS sql_metrics_created_by_fk, sql_metrics.changed_by_fk AS sql_metrics_changed_by_fk 
FROM sql_metrics 
WHERE %(param_1)s = sql_metrics.table_id]
[parameters: {'param_1': 6}]
(Background on this error at: http://sqlalche.me/e/13/f405)

[Zverik]

Closing in favour of #10783. Could not reproduce the error.