Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SortOrderId is null after planFile operation #8864

Closed
rice668 opened this issue Oct 18, 2023 · 2 comments · Fixed by #8873
Closed

SortOrderId is null after planFile operation #8864

rice668 opened this issue Oct 18, 2023 · 2 comments · Fixed by #8873

Comments

@rice668
Copy link
Contributor

rice668 commented Oct 18, 2023

Apache Iceberg version

main (development)

Query engine

Spark

Please describe the bug 🐞

Reproduction Steps (pseudo test code)

 createTable("id bigint NOT NULL, category string");
    Table table = validationCatalog.loadTable(tableIdent);
    table.replaceSortOrder().asc("id", NullOrder.NULLS_FIRST).commit();
    DataFile dataFile = DataFiles.builder(table.spec())
        .withPath(table.location())
        .withFormat(FileFormat.ORC)
        .withRecordCount(100)
        .withFileSizeInBytes(table.newScan().targetSplitSize() * 10)
        .withSortOrder(SortOrder.builderFor(table.schema()).asc("id", NullOrder.NULLS_FIRST).build()).build();
    table.newAppend().appendFile(dataFile).commit();
    for (FileScanTask scanTask : table.newScan().planFiles()) {
          scanTask.file().sortOrderId() is null  // Sort order id information is lost
    }
@rice668 rice668 changed the title SortOrderId is NULL After planFile operation SortOrderId is lost(NULL) After planFile operation Oct 18, 2023
@rice668 rice668 changed the title SortOrderId is lost(NULL) After planFile operation SortOrderId is null After planFile operation Oct 18, 2023
@rice668 rice668 changed the title SortOrderId is null After planFile operation SortOrderId is null after planFile operation Oct 18, 2023
@Fokko
Copy link
Contributor

Fokko commented Oct 18, 2023

That doesn't look right. @zhangminglei are you interested in providing a PR?

@rice668
Copy link
Contributor Author

rice668 commented Oct 18, 2023

Yes. I am interested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants