You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
SortExec has the following error check at execution time and this could be moved into the try_new constructor so the error check happens at planning time instead.
{code:java}
if 1 != self.input.output_partitioning().partition_count() {
return Err(DataFusionError::Internal(
"SortExec requires a single input partition".to_owned(),
));
} {code}
The text was updated successfully, but these errors were encountered:
Comment from Hendrik Makait(hendrik.makait) @ 2021-02-14T17:46:12.067+0000:
I'd love to check that out tomorrow.
Comment from Hendrik Makait(hendrik.makait) @ 2021-02-15T20:05:17.782+0000:
Moving this check into the constructor leads to failing tests. As far as I can see, this is because the planner does an optimization step that inserts a merge for children that contain multiple partitions.
{code:java}
match plan.required_child_distribution() {
Distribution::UnspecifiedDistribution => plan.with_new_children(children),
Distribution::SinglePartition => plan.with_new_children(
children
.iter()
.map(|child| {
if child.output_partitioning().partition_count() == 1 {
child.clone()
} else {
Arc::new(MergeExec::new(child.clone()))
}
})
.collect(),
),
}
{code}
What's the reason for moving this check into planning time? How should I proceed?
Comment from Andy Grove(andygrove) @ 2021-02-16T14:44:11.773+0000:
I see. Ok, maybe we can't do this at planning time then.
Note: migrated from original JIRA: https://issues.apache.org/jira/browse/ARROW-11625
SortExec has the following error check at execution time and this could be moved into the try_new constructor so the error check happens at planning time instead.
{code:java}
if 1 != self.input.output_partitioning().partition_count() {
return Err(DataFusionError::Internal(
"SortExec requires a single input partition".to_owned(),
));
} {code}
The text was updated successfully, but these errors were encountered: