Bump pyarrow from 3.0.0 to 6.0.0 (for M1 mac installations; duplicates #740)

[TeoZosa]

⚠️ Note

• (hopefully) Working duplicate of Improve Python Package Installation Process on M1 Macs #740 (thanks to remove pyarrow max version requirement #741)

TL;DR

Updates the pyarrow dependency to 6.0.0 for easier flytekit installation on M1 macs.

Type

• Bug Fix
• Feature
• Plugin

Are all requirements met?

• Code completed
• Smoke tested
• Unit tests added
• Code documentation added
• Any pending items have an associated Issue

Complete description

Previously, installation on M1 macs required pyarrow to be built from source since no prebuilt arm64 wheels were available for the 3.0.0 version. This build, in turn, depended on the 3.0.0 version of the Apache Arrow library (pyarrow's build system had breaking changes between 3.0.0 and 4.0.0). Apache Arrow can be installed via brew, but only the 6.0.0 version is currently available. This meant that users had to also build the Apache Arrow library from source (and remember to set the correct LDFLAGS and CFLAGS, etc.) prior to building pyarrow.

Since pyarrow version 6.0.0 includes prebuilt wheels for the arm64 architecture, simply upgrading the pyarrow dependency version to 6.0.0 elides the above steps, yielding a much more pleasant flytekit installation experience.

Tracking Issue

NA

Follow-up issue

NA

[kumare3]

cc @cosmicBboy / @eapolinario

[TeoZosa]

Just kidding, it looks like #741 needs to be in the latest release for compatibility.

Error message (similar to #740 (comment)):

2021-11-09T00:52:36.5537204Z �[1m�[31mE           #11 16.03 The conflict is caused by:
2021-11-09T00:52:36.5537762Z �[0m
2021-11-09T00:52:36.5538445Z �[1m�[31mE           #11 16.03     The user requested pyarrow==6.0.0
2021-11-09T00:52:36.5539035Z �[0m
2021-11-09T00:52:36.5539756Z �[1m�[31mE           #11 16.03     flytekit 0.23.1 depends on pyarrow<4.0.0 and >=2.0.0

[wild-endeavor]

@TeoZosa we're going to cut a release of flytekit v0.24.0 tomorrow. I just released https://github.com/flyteorg/flytekit/releases/tag/v0.24.0b2 though if you want to play around with it before that happens.

[wild-endeavor]

sorry about closing the other one. thanks!

[welcome]

Congrats on merging your first pull request! 🎉

[wild-endeavor]

tests are failing because integration tests are failing because there's a version conflict. but once this pr is merged, which we just did, and cut a release, we can regenerate the mock repo requirements file.

[TeoZosa]

sorry about closing the other one. thanks!

No worries! I think it's awesome that the team is able get to and through these PRs so quickly. I appreciate your and everyone else's help in getting this PR across the finish line. Thank you for all your hard work!