Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

Commit

Permalink
[sql lab] Fix issue around VARBINARY type in Presto (apache#5121)
Browse files Browse the repository at this point in the history
When receiving a VARBINARY field out of Presto, it shows up as type
`bytes` out of the pyhive driver. Then the pre 3.15 version of
simplejson attempts to convert it to utf8 by default and it craps out.

I bumped to simplejson>=3.25.0 and set `encoding=None` as documented
here
https://simplejson.readthedocs.io/en/latest/#basic-usage so that we can
handle bytes on our own.
  • Loading branch information
mistercrunch authored Jun 20, 2018
1 parent 2a3d297 commit 409ac68
Show file tree
Hide file tree
Showing 4 changed files with 12 additions and 4 deletions.
2 changes: 1 addition & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ python-dateutil==2.6.1
python-geohash==0.8.5
pyyaml==3.12
requests==2.18.4
simplejson==3.13.2
simplejson==3.15.0
six==1.11.0
sqlalchemy==1.2.2
sqlalchemy-utils==0.32.21
Expand Down
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ def get_git_sha():
'python-geohash',
'pyyaml>=3.11',
'requests',
'simplejson',
'simplejson>=3.15.0',
'six',
'sqlalchemy',
'sqlalchemy-utils',
Expand Down
6 changes: 5 additions & 1 deletion superset/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -310,7 +310,6 @@ def datetime_f(dttm):


def base_json_conv(obj):

if isinstance(obj, numpy.int64):
return int(obj)
elif isinstance(obj, numpy.bool_):
Expand All @@ -323,6 +322,11 @@ def base_json_conv(obj):
return str(obj)
elif isinstance(obj, timedelta):
return str(obj)
elif isinstance(obj, bytes):
try:
return '{}'.format(obj)
except Exception:
return '[bytes]'


def json_iso_dttm_ser(obj, pessimistic=False):
Expand Down
6 changes: 5 additions & 1 deletion superset/views/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -2464,7 +2464,11 @@ def sql_json(self):
rendered_query,
return_results=True)
payload = json.dumps(
data, default=utils.pessimistic_json_iso_dttm_ser, ignore_nan=True)
data,
default=utils.pessimistic_json_iso_dttm_ser,
ignore_nan=True,
encoding=None,
)
except Exception as e:
logging.exception(e)
return json_error_response('{}'.format(e))
Expand Down

0 comments on commit 409ac68

Please sign in to comment.