refactor: try abandon internal parquet2 patches

[dantengsky]

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

• partially fixes:

  • Improvement: abandon internal patches of parquet2  #6064

  All internal patches except the “legacy lz4”, are abandoned.
  Before Fixed LZ4 jorgecarleitao/parquet2#95, there is an implementation of Lz4, which is not compatible with the cpp / java implementations. IMO, it is not reasonable(even if feasible) to ask upstream to keep it.
  Unfortunately, at least, some stateful test data are compressed by using the non-standard "legacy" lz4.
  Thus this PR still uses a patched version (in repo datafuse-extras). I suggest abandon this non-standard "legacy" lz4 patch in another PR (after we have made sure that no such legacy data exist)

• arrow2 upgraded to 0.12

  1.) type ArrayRef = Arc<dyn Array> of former arrow2 has been removed, instead Box<dyn Array> is used.
  so, in databend, type ArrayRef is redefined as Box<dyn Array> (and did the refactors by following rustc).
  Not sure if the name is still suitable, any suggestions are welcome.
  2.) replace deprecated null_count with unset_bits
  3.) serialize_batch now returns Result<_> instead of tuple, related codes are adjusted accordingly.

• parquet2 upgraded to 0.14

Changelog

• Improvement
• Not for changelog (changelog entry is not required)

Related Issues

Fixes #6064

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Updated
databend	⬜️ Ignored (Inspect)		Jul 7, 2022 at 1:35AM (UTC)

[mergify]

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

[dantengsky]

this PR will be postponed for a while, trying to port the supports legacy lz4 patch to upstream

[BohuTANG]

This PR ready for review? @dantengsky

[dantengsky]

This PR ready for review? @dantengsky

not yet. hopefully, it will be ready next week. but if anything is blocked by this, I'd like to split this PR into two

[mergify]

This pull request's title is not fulfill the requirements. @dantengsky please update it 🙏.

The title should contain one if the following tags:

• feat: this PR introduces a new feature to the codebase
• fix: this PR patches a bug in codebase
• refactor: this PR changes the code base without new features or bugfix
• ci|build: this PR changes build/testing/ci steps
• docs|website: this PR changes the documents or websites
• chore: this PR only has small changes that no need to record

[dantengsky]

This PR ready for review? @dantengsky

Oh, sorry just found I mistakenly marked this PR as ready for review.

[Xuanwo]

This PR seems ready now.

@dantengsky Are there other issues that need addressing?

[dantengsky]

This PR seems ready now.

@dantengsky Are there other issues that need addressing?

stateful test not passed yet :-( (due to silly conditional compilation flags that I set up in our internal parquet2 repo)

[dantengsky]

sorry, revert back to draft (there are typos to be corrected)