Add `typeof` function. #123

Yury-Fridlyand · 2022-09-20T23:15:45Z

Signed-off-by: Yury-Fridlyand [email protected]

Description

TYPEOF function is useful for debugging to check types of other functions.
~~I also fixed cast to DATETIME.~~

Usage

opensearchsql> SELECT typeof(CAST('1961-04-12 09:07:00' AS DATETIME)), typeof(CAST('1961-04-12 09:07:00' AS TIMESTAMP)), typeof(CAST('09:07:00' AS TIME)), typeof(CAST('1961-04-12' AS DATE));
Output longer than terminal width
Do you want to display data vertically for better visual effect? [y/N]: y
fetched rows / total rows = 1/1
-[ RECORD 1 ]-------------------------
typeof(CAST('1961-04-12 09:07:00' AS DATETIME))  | DATETIME
typeof(CAST('1961-04-12 09:07:00' AS TIMESTAMP)) | TIMESTAMP
typeof(CAST('09:07:00' AS TIME))                 | TIME
typeof(CAST('1961-04-12' AS DATE))               | DATE

Check List

New functionality includes testing.
- All tests pass, including unit test, integration test and doctest
New functionality has been documented.
- New functionality has javadoc added
- New functionality has user manual doc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…ions (opensearch-project#754) * Add implementation of `now`, `sysdate`, `localtime` and similar functions (#92) Signed-off-by: Yury-Fridlyand <[email protected]> * Rework on `now` function implementation (#113) Signed-off-by: Yury-Fridlyand <[email protected]> * Minor SQL ANTLR clean-up. Signed-off-by: Yury-Fridlyand <[email protected]> Signed-off-by: Yury-Fridlyand <[email protected]>

Signed-off-by: MaxKsyunz <[email protected]>

…n-2.13.4 Update com.fasterxml.jackson to 2.13.4 to match opensearch repo.

MaxKsyunz · 2022-09-26T21:53:00Z

core/src/main/java/org/opensearch/sql/expression/operator/convert/TypeCastOperator.java

@@ -60,6 +62,7 @@ public static void register(BuiltinFunctionRepository repository) {
    repository.register(castToTime());
    repository.register(castToTimestamp());
    repository.register(castToDatetime());
+    repository.register(typeof());


Could you please move typeof into a separate class?

How can I call it? Is Misc OK?

TypeOfOperator seems right to me.

Fixed in 9296c62.

…ch-project#835) * Add datetime functions `FROM_UNIXTIME` and `UNIX_TIMESTAMP` (#114) * Add implementation for `FROM_UNIXTIME` and `UNIX_TIMESTAMP` functions, UT and IT. Signed-off-by: Yury-Fridlyand <[email protected]> * Collent all DateTime formatters into one place. Signed-off-by: Yury-Fridlyand <[email protected]> * Rename `DateFormatters` -> `DateTimeFormatters`. Signed-off-by: Yury-Fridlyand <[email protected]> Signed-off-by: Yury-Fridlyand <[email protected]>

acarbonetto

LGTM - just a few comments

acarbonetto · 2022-09-28T15:49:31Z

core/src/test/java/org/opensearch/sql/expression/operator/convert/TypeCastOperatorTest.java

@@ -361,4 +372,27 @@ void castToDatetime() {
    assertEquals(new ExprDatetimeValue("2012-08-07 00:00:00"), expression.valueOf(null));
  }

+  @Test
+  void typeof() {
+    assertEquals("UNDEFINED", typeofGetValue(ExprNullValue.of()));


is it possible to test UNKNOWN?

Yes, see typeof(NULL) in integration tests

acarbonetto · 2022-09-28T15:52:11Z

core/src/main/java/org/opensearch/sql/expression/operator/convert/TypeOfOperator.java

+  // Auxiliary function useful for debugging
+  private static DefaultFunctionResolver typeof() {
+    return FunctionDSL.define(BuiltinFunctionName.TYPEOF.getName(),
+        impl(TypeOfOperator::exprTypeOf, STRING, UNKNOWN),


nit: would be nice to have these in alphabetic order like the list above in the same file.

acarbonetto · 2022-09-28T15:53:19Z

integ-test/src/test/java/org/opensearch/sql/ppl/DataTypeIT.java

+        + " | fields `datetime`, `timestamp`, `time`, `date`",
+            TEST_INDEX_DATATYPE_NUMERIC));
+    verifyDataRows(response,
+        rows("DATETIME", "TIMESTAMP", "TIME", "DATE"));


please add a test for complex data types, like ARRAY, STRUCT, and INTERVAL.

To test with ARRAY and STRUCT I have to update parser and AstBuilder/AstExpressionBuilder. I can't imagine where it could be used especially as a return value of another function.
typeof(somefunc(...)) -> STRUCT.

Signed-off-by: Yury-Fridlyand <[email protected]>

MitchellGale · 2022-09-29T07:29:39Z

core/src/test/java/org/opensearch/sql/expression/datetime/UnixTimeStampTest.java

+    return Stream.of(
+        Arguments.of(840101d, LocalDateTime.of(1984, 1, 1, 0, 0, 0)),
+        Arguments.of(840101112233d, LocalDateTime.of(1984, 1, 1, 11,22,33)),
+        Arguments.of(840101112233.123456, LocalDateTime.of(1984, 1, 1, 11, 22, 33, 123456000)),


do the argument.of's need a d at the end too?

d for Double, otherwise it is interpreted as int/long and test crashes, because the test method expect double as the first arg.

MitchellGale · 2022-09-29T07:32:34Z

docs/user/dql/functions.rst

+    | value_1                    | value_2                    |
+    |----------------------------+----------------------------|
+    | 2022-08-02 15:39:05.173069 | 2022-08-02 15:39:05.173069 |
+    +----------------------------+----------------------------+


I get this result from mySQL for this query. Should NOW return with .nnnnnn?

mysql> SELECT NOW() as value_1, NOW() as value_2; +---------------------+---------------------+ | value_1 | value_2 | +---------------------+---------------------+ | 2022-09-29 07:30:55 | 2022-09-29 07:30:55 | +---------------------+---------------------+ 1 row in set (0.00 sec)

MitchellGale · 2022-09-29T07:33:11Z

docs/user/dql/functions.rst

+    | CURRENT_TIMESTAMP()        | CURRENT_TIMESTAMP          |
+    |----------------------------+----------------------------|
+    | 2022-08-02 15:54:19.209361 | 2022-08-02 15:54:19.209361 |
+    +----------------------------+----------------------------+


mysql> SELECT CURRENT_TIMESTAMP(), CURRENT_TIMESTAMP; +---------------------+---------------------+ | CURRENT_TIMESTAMP() | CURRENT_TIMESTAMP | +---------------------+---------------------+ | 2022-09-29 07:32:49 | 2022-09-29 07:32:49 | +---------------------+

MySQL return

Add XYPoint Query Processor

Yury-Fridlyand requested review from forestmvey, MaxKsyunz, acarbonetto and MitchellGale September 20, 2022 23:15

This comment was marked as abuse.

Sign in to view

forestmvey approved these changes Sep 23, 2022

View reviewed changes

Yury-Fridlyand and others added 3 commits September 23, 2022 16:29

Update com.fasterxml.jackson to 2.13.4 to match opensearch repo.

705ad13

Signed-off-by: MaxKsyunz <[email protected]>

Merge pull request opensearch-project#858 from Bit-Quill/integ-jackso…

8245943

…n-2.13.4 Update com.fasterxml.jackson to 2.13.4 to match opensearch repo.

MaxKsyunz requested changes Sep 26, 2022

View reviewed changes

Yury-Fridlyand requested a review from MaxKsyunz September 28, 2022 00:08

MaxKsyunz approved these changes Sep 28, 2022

View reviewed changes

acarbonetto approved these changes Sep 28, 2022

View reviewed changes

Add typeof function.

cd9570d

Signed-off-by: Yury-Fridlyand <[email protected]>

Yury-Fridlyand force-pushed the dev-add-typeof branch from 9296c62 to cd9570d Compare September 28, 2022 17:32

Address PR feedback + add a test.

84eb92b

Signed-off-by: Yury-Fridlyand <[email protected]>

Yury-Fridlyand requested a review from MaxKsyunz September 28, 2022 19:45

Yury-Fridlyand merged commit 349bfcb into integ-add-typeof Sep 29, 2022

Yury-Fridlyand deleted the dev-add-typeof branch September 29, 2022 03:49

Yury-Fridlyand mentioned this pull request Sep 29, 2022

Add typeof function. opensearch-project/sql#867

Merged

6 tasks

MitchellGale reviewed Sep 29, 2022

View reviewed changes

andy-k-improving pushed a commit that referenced this pull request Nov 16, 2024

Merge pull request #123 from naveentatikonda/xy_point_query_processor

f5bb2a2

Add XYPoint Query Processor

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `typeof` function. #123

Add `typeof` function. #123

Yury-Fridlyand commented Sep 20, 2022 •

edited

Loading

This comment was marked as abuse.

MaxKsyunz Sep 26, 2022

Yury-Fridlyand Sep 26, 2022

MaxKsyunz Sep 26, 2022

Yury-Fridlyand Sep 28, 2022

acarbonetto left a comment

acarbonetto Sep 28, 2022

Yury-Fridlyand Sep 28, 2022

acarbonetto Sep 28, 2022

acarbonetto Sep 28, 2022

Yury-Fridlyand Sep 28, 2022

MitchellGale Sep 29, 2022

Yury-Fridlyand Oct 3, 2022

MitchellGale Sep 29, 2022

MitchellGale Sep 29, 2022

Add typeof function. #123

Add typeof function. #123

Conversation

Yury-Fridlyand commented Sep 20, 2022 • edited Loading

Description

Usage

Check List

This comment was marked as abuse.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

acarbonetto left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Add `typeof` function. #123

Add `typeof` function. #123

Yury-Fridlyand commented Sep 20, 2022 •

edited

Loading