Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI][Conda] Nightly CI jobs aren't maintained #14828

Closed
kou opened this issue Dec 2, 2022 · 6 comments · Fixed by #14832
Closed

[CI][Conda] Nightly CI jobs aren't maintained #14828

kou opened this issue Dec 2, 2022 · 6 comments · Fixed by #14832
Assignees
@h-vetinari
Labels
Component: Continuous Integration Component: Packaging Type: bug
Milestone
11.0.0

Comments

@kou
Copy link
Member

kou commented Dec 2, 2022

Describe the bug, including details regarding any error messages, version, and platform.

Nightly CI jobs for Conda were fixed by @h-vetinari and @xhochy in GH-14102 but most nightly CI jobs are failing again this past month.

Can we maintain them? If we can't maintain them, can we remove them? Are they useful to maintain https://github.com/conda-forge/arrow-cpp-feedstock ?

http://crossbow.voltrondata.com/

Task Name Since Last Successful Build Last Successful Commit Last Successful Build First Failure 9 Days Ago 8 Days Ago 7 Days Ago 6 Days Ago 5 Days Ago 4 Days Ago 3 Days Ago 2 Days Ago Most Recent Failure
conda-win-vs2019-py37-r40 78 days 5e6da78 pending fail fail fail fail fail fail fail fail fail fail
conda-win-vs2019-py38 65 days 60c9383 pending fail fail fail fail fail fail fail fail fail fail
conda-linux-gcc-py310-arm64 29 days 8e3a1e1 pass fail fail fail fail fail fail fail fail fail fail
conda-linux-gcc-py310-cpu 29 days 8e3a1e1 pass fail fail fail fail fail fail fail fail fail fail
conda-linux-gcc-py37-arm64 29 days 8e3a1e1 pass fail fail fail fail fail fail fail fail fail fail
conda-linux-gcc-py37-cpu-r40 29 days 8e3a1e1 pass fail fail fail fail fail fail fail fail fail fail
conda-linux-gcc-py37-cpu-r41 29 days 8e3a1e1 pass fail fail fail fail fail fail fail fail fail fail
conda-linux-gcc-py37-cuda 29 days 8e3a1e1 pass fail fail fail fail fail fail fail fail fail fail
conda-linux-gcc-py37-ppc64le 29 days 8e3a1e1 pass fail fail fail fail fail fail fail fail fail fail
conda-linux-gcc-py38-arm64 29 days 8e3a1e1 pass fail fail fail fail fail fail fail fail fail fail
conda-linux-gcc-py38-cpu 29 days 8e3a1e1 pass fail fail fail fail fail fail fail fail fail fail
conda-linux-gcc-py38-cuda 29 days 8e3a1e1 pass fail fail fail fail fail fail fail fail fail fail
conda-linux-gcc-py38-ppc64le 29 days 8e3a1e1 pass fail fail fail fail fail fail fail fail fail fail
conda-linux-gcc-py39-arm64 29 days 8e3a1e1 pass fail fail fail fail fail fail fail fail fail fail
conda-linux-gcc-py39-cpu 29 days 8e3a1e1 pass fail fail fail fail fail fail fail fail fail fail
conda-linux-gcc-py39-cuda 29 days 8e3a1e1 pass fail fail fail fail fail fail fail fail fail fail
conda-linux-gcc-py39-ppc64le 29 days 8e3a1e1 pass fail fail fail fail fail fail fail fail fail fail
conda-osx-arm64-clang-py38 29 days 8e3a1e1 pass fail fail fail fail fail fail fail fail fail fail
conda-osx-arm64-clang-py39 29 days 8e3a1e1 pass fail fail fail fail fail fail fail fail fail fail
conda-osx-clang-py37-r40 29 days 8e3a1e1 pass fail fail fail fail fail fail fail fail fail fail
conda-osx-clang-py37-r41 29 days 8e3a1e1 pass fail fail fail fail fail fail fail fail fail fail
conda-osx-clang-py38 29 days 8e3a1e1 pass fail fail fail fail fail fail fail fail fail fail
conda-osx-clang-py39 29 days 8e3a1e1 pass fail fail fail fail fail fail fail fail fail fail
conda-linux-gcc-py310-cuda 15 days 501b799 pass fail fail fail fail fail fail fail fail fail fail
conda-linux-gcc-py310-ppc64le 15 days 501b799 pass fail fail fail fail fail fail fail fail fail fail
conda-osx-arm64-clang-py310 15 days 501b799 pass fail fail fail fail fail fail fail fail fail fail
conda-osx-clang-py310 15 days 501b799 pass fail fail fail fail fail fail fail fail fail fail

Component(s)

Continuous Integration, Packaging
@kou kou added the Type: bug label Dec 2, 2022
@h-vetinari
Copy link
Contributor

I've been planning to update them as soon as conda-forge/arrow-cpp-feedstock#875 is merged, but that PR took quite a while for various reasons (needs to cross-compile for aarch/ppc, which didn't work in conjunction with CUDA; there's a path to make this work but I'm not going to block the PR on that; also, bunch of tests failed and there were some segfaults, but I received essentially no response to many pings about what to do with them, so for now I'll ignore them).

After that update the CI should run fine IMO, and it should be easy to update after that as well. The test failures and segfaults I mentioned IMO make running things in CI here worthwhile, as another angle through which to stresstest the code (before the PR linked above, the conda-forge feedstock didn't run the test suite; now it does).

However I can't speak for the jobs that also include R. Haven't looked at those, so far only @xhochy did AFAIU.

@kou
Copy link
Member Author

kou commented Dec 3, 2022

Thanks for sharing the current status.

Sorry for not replying to segfaults... I couldn't follow conda-forge/arrow-cpp-feedstock#875 because it's too large for me.
If you find problems or have questions of Apache Arrow, how about creating an issue to apache/arrow for each problems/questions? It may bother you but it may be easier to resolve each problem/question.

@h-vetinari
Copy link
Contributor

If you find problems or have questions of Apache Arrow, how about creating an issue to apache/arrow for each problems/questions? It may bother you but it may be easier to resolve each problem/question.

It doesn't bother me at all. I didn't want to spam people with issues, and thought it would be easier to ask specific questions (e.g. about the LLVM usage), but if people prefer issues here that's not a problem for me.

@h-vetinari
Copy link
Contributor

h-vetinari commented Dec 4, 2022 •
edited
Loading

While preparing a PR, I just saw that a3ef64b broadly undid several necessary infrastructure changes in tasks.yml (looks like it might have been an inattentive merge conflict resolution that undid most of #14102's changes) - it's unsurprising that builds have been failing since, because they're pointing to non-existent configs.

CC @assignUser, please be more careful next time

h-vetinari added a commit to h-vetinari/arrow that referenced this issue Dec 4, 2022
@h-vetinari
apacheGH-14828: treat patch files as binary (due to CRLF issues)
b43b8c2
h-vetinari added a commit to h-vetinari/arrow that referenced this issue Dec 4, 2022
@h-vetinari
apacheGH-14828: sync with conda-forge recipe
0d31585
h-vetinari added a commit to h-vetinari/arrow that referenced this issue Dec 4, 2022
@h-vetinari
apacheGH-14828: sync dev/tasks/conda-recipes/.ci_support with conda-f…
a123df8 
…orge feedstock
@github-actions github-actions bot mentioned this issue Dec 4, 2022
Merged
@assignUser
Copy link
Member

@h-vetinari Oh 😱 seems like I picked the wrong chunk and it slipped through the review. Sorry

@kou
Copy link
Member Author

kou commented Dec 5, 2022

Oh, sorry. I missed these changes on merging...

kou pushed a commit that referenced this issue Dec 12, 2022
@h-vetinari
GH-14828: [CI][Conda] Sync with conda-forge, fix nightly jobs (#14832)
16d0eb4 
Synching after conda-forge/arrow-cpp-feedstock#875, which does quite a lot of things, see this [summary](conda-forge/arrow-cpp-feedstock#875 (review)). I'm not keeping the commit history here, but it might be instructive to check the commits there to see why certain changes came about.

It also fixes the CI that was broken by a3ef64b (undoing the changes of #14102 in `tasks.yml`).

Finally, it adapts to conda making a long-planned [switch](conda-forge/conda-forge.github.io#1586) w.r.t. to the format / extension of the artefacts it produces.

I'm very likely going to need some help (or at least pointers) for the R-stuff. CC @ xhochy
(for context, I never got a response to conda-forge/r-arrow-feedstock#55, but I'll open a PR to build against libarrow 10).

Once this is done, I can open issues to tackle the tests that shouldn't be failing, resp. the segfaults on PPC resp. in conjunction with `sparse`.
* Closes: #14828

Authored-by: H. Vetinari <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
@kou kou added this to the 11.0.0 milestone Dec 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@h-vetinari h-vetinari

Labels
Component: Continuous Integration Component: Packaging Type: bug
Projects
None yet
Milestone
11.0.0
Development

Successfully merging a pull request may close this issue.

GH-14828: [CI][Conda] Sync with conda-forge, fix nightly jobs h-vetinari/arrow
3 participants
@kou @assignUser @h-vetinari