Druid SQL: Mapping the gaps in capability

[yurmix]

Hi,

SQL is the database world's standard API for both querying, updating (DML), structural transformations (DDL) as well as even administration operations.
Druid SQL exists but it still not thoroughly covers all Druid capabilities and APIs.

There is a will in the community to close that gap (@gianm recently noted about it in the dev email list).

I believe it would be beneficial to document all the gaps which currently exist for Druid SQL to be in full parity with other mature, commercial analytical databases. Such documentation could promote effort by many contributors to help to close that gap.

Regarding technicalities, I have some suggestions:

1. The list should be broken down to manageable chunks so multiple contributors could participate.
2. It would be helpful to note why SQL support is missing: Is it just SQL or is this capability isn't supported by Druid core.
3. When starting to work on a specific item, we could line its GitHub Issue (And people can also link their existing Issues).

I might spend some of my own time working on a parity comparison with Vertica (which is a mature analytical database, shares many qualities with Druid and with which I have experience with).
I'm not entirely sure, but a parity comparison with an OLAP database might also be relevant (I have no experience with these types of databases). Perhaps @julianhyde could shed some light on the relevancy of this subtopic.

[stale]

This issue has been marked as stale due to 280 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If this issue is still relevant, please simply write any comment. Even if closed, you can still revive the issue at any time or discuss it on the [email protected] list. Thank you for your contributions.

[stale]

This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.

[stale]

This issue is no longer marked as stale.

[mistercrunch]

We're hoping to deprecate the REST/native Druid connector in Superset and move towards the SQL / SQLAlchemy connector. Understanding the gaps would help us move forward.

@gianm, is there a source of truth for the existing gaps, and is the plan to put the SQL interface forward?

apache/superset#6032

[gianm]

@mistercrunch The most authoritative list is in the docs here: https://druid.apache.org/docs/latest/querying/sql.html#unsupported-features, however it's not exhaustive.

The first two ("Multi-value dimensions" and "Set operations on DataSketches aggregators") were added recently. There are also some (esoteric) things missing, like the Search query and some lesser-used extensions. Give me a bit and I'll put together a PR removing those two and adding a few more missing things explicitly.

The plan is certainly to continue pushing the SQL interface forwards and reach parity ASAP.

[stale]

This issue has been marked as stale due to 280 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If this issue is still relevant, please simply write any comment. Even if closed, you can still revive the issue at any time or discuss it on the [email protected] list. Thank you for your contributions.

[stale]

This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.