-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Minor: return NULL for range and generate_series #10275
Conversation
@@ -57,6 +56,7 @@ impl Range { | |||
TypeSignature::Exact(vec![Int64, Int64]), | |||
TypeSignature::Exact(vec![Int64, Int64, Int64]), | |||
TypeSignature::Exact(vec![Date32, Date32, Interval(MonthDayNano)]), | |||
TypeSignature::Any(3), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this mean we need to find a new kind of signature that accepts either one exact type or null 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it would be better to have a new Signature. Maybe I can add more signatures in the following PR along with dealing #10200
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm thinking of something like
TypeSignature::ExactOrNull(valid_types) => {
current_types.iter().map(|t| {
if t != &DataType::Null {
valid_types.clone()
} else {
vec![DataType::Null]
}
})
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, that would be fine. I'll add a ticket to record it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @Lordworms and @jayzhan211 -- looks good to me
FWIW I also verified that returning NULL is consistent with DuckDB:
D select generate_series(DATE '1992-09-01', NULL, INTERVAL '1' YEAR);
┌──────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ generate_series(CAST('1992-09-01' AS DATE), NULL, to_years(CAST(trunc(CAST('1' AS DOUBLE)) AS INTEGER))) │
│ int32 │
├──────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────┘
@@ -166,146 +166,6 @@ impl ScalarUDFImpl for StringToArray { | |||
} | |||
} | |||
|
|||
make_udf_function!( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this just an accidental left over copy?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, there might be some other left things
} | ||
} | ||
|
||
make_udf_function!( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Likewise this looks like an accidental copy too -- here is the other copy:
datafusion/datafusion/functions-array/src/range.rs
Lines 102 to 163 in 0ea9bc6
make_udf_function!( | |
GenSeries, | |
gen_series, | |
start stop step, | |
"create a list of values in the range between start and stop, include upper bound", | |
gen_series_udf | |
); | |
#[derive(Debug)] | |
pub(super) struct GenSeries { | |
signature: Signature, | |
aliases: Vec<String>, | |
} | |
impl GenSeries { | |
pub fn new() -> Self { | |
Self { | |
signature: Signature::one_of( | |
vec![ | |
TypeSignature::Exact(vec![Int64]), | |
TypeSignature::Exact(vec![Int64, Int64]), | |
TypeSignature::Exact(vec![Int64, Int64, Int64]), | |
TypeSignature::Exact(vec![Date32, Date32, Interval(MonthDayNano)]), | |
], | |
Volatility::Immutable, | |
), | |
aliases: vec![String::from("generate_series")], | |
} | |
} | |
} | |
impl ScalarUDFImpl for GenSeries { | |
fn as_any(&self) -> &dyn Any { | |
self | |
} | |
fn name(&self) -> &str { | |
"generate_series" | |
} | |
fn signature(&self) -> &Signature { | |
&self.signature | |
} | |
fn return_type(&self, arg_types: &[DataType]) -> Result<DataType> { | |
Ok(List(Arc::new(Field::new( | |
"item", | |
arg_types[0].clone(), | |
true, | |
)))) | |
} | |
fn invoke(&self, args: &[ColumnarValue]) -> Result<ColumnarValue> { | |
match args[0].data_type() { | |
Int64 => make_scalar_function(|args| gen_range_inner(args, true))(args), | |
Date32 => make_scalar_function(|args| gen_range_date(args, true))(args), | |
_ => { | |
exec_err!("unsupported type for range") | |
} | |
} | |
} | |
fn aliases(&self) -> &[String] { | |
&self.aliases | |
} | |
} |
Co-authored-by: Andrew Lamb <[email protected]>
Co-authored-by: Andrew Lamb <[email protected]>
thanks @Lordworms and @alamb |
Which issue does this PR close?
Closes #10269
Rationale for this change
What changes are included in this PR?
Are these changes tested?
Are there any user-facing changes?