Parse map(k1,v1,k2,v2) from hive to trino #99
Conversation
| private void unparseMapOperator(SqlWriter writer, SqlCall call, int leftPrec, int rightPrec) { | ||
| writer.keyword(call.getOperator().getName()); // "MAP" | ||
| final SqlWriter.Frame frame = writer.startList("(", ")"); | ||
| writer.keyword("ARRAY["); | ||
| // Even numbers are keys | ||
| for (int i = 0; i < call.operandCount(); i += 2) { | ||
| writer.sep(","); | ||
| call.operand(i).unparse(writer, leftPrec, rightPrec); | ||
| } | ||
| writer.keyword("], ARRAY["); | ||
| // Odd numbers are values | ||
| for (int i = 1; i < call.operandCount(); i += 2) { | ||
| call.operand(i).unparse(writer, leftPrec, rightPrec); | ||
| // Last comma is redundant | ||
| if (i < call.operandCount() - 1) { | ||
| writer.sep(","); | ||
| } | ||
| } | ||
| writer.keyword("]"); | ||
| writer.endList(frame); | ||
| } |
There was a problem hiding this comment.
Should not we handle the transformation in Calcite2TrinoUDFConverter? I think if the operands need transformation, this will not take care of them.
There was a problem hiding this comment.
Yes, I understand.
So far, it seems operands number must be determined because CalciteTrinoUDFMap.getUDFTransformer(String calciteOpName, int numOperands). However map is variable length parameter, I can't put it in UDF_MAP.
Do we plan to support functions with variable length parameters? I thinks this is meaningful topic.
There was a problem hiding this comment.
Agreed. Maybe we can introduce getUDFTransformer(String calciteOpName) that means the Op takes variable length parameters?
JiajunBernoulli
force-pushed
the
hive2trino
branch
3 times, most recently
from
bdae026 to
7612eec
Compare
|
Because I can't put |
| // Odd numbers are values | ||
| List<RexNode> values = new ArrayList<>(); | ||
| for (int i = 1; i < sourceOperands.size(); i += 2) { | ||
| values.add(sourceOperands.get(i)); |
There was a problem hiding this comment.
If value is also a map, we failed to convert it.
For example, if the view definition is:
SELECT MAP('key1', MAP('a', 'b', 'c', 'd'), 'key2', MAP('a', 'b', 'c', 'd')) AS col_map
it would only be converted to
SELECT MAP (ARRAY['key1', 'key2'], ARRAY[MAP ('a', 'b', 'c', 'd'), MAP ('a', 'b', 'c', 'd')]) AS "col_map"
However, the result should be
SELECT MAP (ARRAY['key1', 'key2'], ARRAY[MAP (ARRAY['a', 'c'], ARRAY['b', 'd']), MAP (ARRAY['a', 'c'], ARRAY['b', 'd'])]) AS "col_map"
I think we can change line 208 to be
List<RexNode> convertedOperands = visitList(call.getOperands(), (boolean[]) null);
and add corresponding unit tests.
There was a problem hiding this comment.
I agree with you and will try to fix it.
coral-trino/src/main/java/com/linkedin/coral/trino/rel2trino/TrinoSqlDialect.java
Outdated
Show resolved
Hide resolved
JiajunBernoulli
force-pushed
the
hive2trino
branch
3 times, most recently
from
73474a0 to
128fc87
Compare
|
I updated this PR, Do you have any suggestions? |
JiajunBernoulli
force-pushed
the
hive2trino
branch
from
128fc87 to
19cfb14
Compare
JiajunBernoulli
force-pushed
the
hive2trino
branch
from
19cfb14 to
d87d4c9
Compare
|
@ljfgem PTAL. |
Thank you for reminding, integration test is running, will merge if there is no regression. |
…de layer (#491) * Initial commit for migrating Cast * add type derivation changes * type derivation enhance and test debug * remove testing files * spotlessapply + comment out failing unit test * update unit tests * fix regression by TypeDerivationUtil on casts with join on simple aliases * generalize case of casting with join on simple aliases * remove extraneous input to dummy select * temporary removal of type util postprocessor for i-test * revert * spotless checks * update calcite version for linkedin-calcite #98/#99 * remove Calcite2TrinoUDFConverter * clean up tests * unused imports * spotless * improve javadoc/documentation * fix javadoc * improve javadoc * grammar * refactor + clean up --------- Co-authored-by: aastha25 <[email protected]>
Hi, everybody!
I'm interested in converting SQL from hive to trino. This is my first PR. Here's a little bit of my work:
For hive,
SELECT MAP(k1, v1, k2, v2)isbut for trino, sql should be
SELECT MAP(ARRAY[k1,k2], ARRAY[v1,v2])Welcome your suggestions.