Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Spurious failures on AppVeyor due to ar.exe errors #40546

Closed
alexcrichton opened this issue Mar 15, 2017 · 20 comments · Fixed by #40548 or #40777
Closed

Spurious failures on AppVeyor due to ar.exe errors #40546

alexcrichton opened this issue Mar 15, 2017 · 20 comments · Fixed by #40548 or #40777
Labels
A-spurious Area: Spurious failures in builds (spuriously == for no apparent reason)

Comments

@alexcrichton
Copy link
Member

I've previously been classifying these errors under #40240 but now I'm thinking that's incorrect, I suspect these are totally separate errors.

https://ci.appveyor.com/project/rust-lang/rust/build/1.0.2397

[ 13%] Linking CXX static library ../../libLLVMMCDisassembler.a
C:/projects/rust/mingw64/bin/ar.exe: ../../libLLVMMCDisassembler.a: Permission denied
make[2]: *** [lib/MC/MCDisassembler/CMakeFiles/LLVMMCDisassembler.dir/build.make:205: lib/libLLVMMCDisassembler.a] Error 1
make[2]: *** Deleting file 'lib/libLLVMMCDisassembler.a'
make[1]: *** [CMakeFiles/Makefile2:2539: lib/MC/MCDisassembler/CMakeFiles/LLVMMCDisassembler.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 13%] Built target LLVMMCParser
make: *** [Makefile:150: all] Error 2
thread 'main' panicked at '
command did not execute successfully, got: exit code: 2
@alexcrichton alexcrichton added the A-spurious Area: Spurious failures in builds (spuriously == for no apparent reason) label Mar 15, 2017
@alexcrichton
Copy link
Member Author

The first hit on Google is quite ominous. My preference for an attempt to fix this would be to switch to ninja as it's likely just flat out more reliable than makefiles on MinGW

@alexcrichton
Copy link
Member Author

Oh also we're going to want to switch the MSVC build to Ninja regardless, so we'll have it available anyway eventually on the bots.

This was referenced Mar 15, 2017
Merged
Closed
@mati865
Copy link
Contributor

mati865 commented Mar 15, 2017

I've met it few times with MSYS2 and (almost?) always it was caused by CL RF line endings or too long paths or file names.
When compiling Rust from git I found there are files that windows explorer cannot remove since they are too long. Unfortunately I don't have list as I used MSYS2 rm -rf * to clean those files.

@alexcrichton alexcrichton mentioned this issue Mar 21, 2017
Closed
alexcrichton added a commit to alexcrichton/rust that referenced this issue Mar 22, 2017
@alexcrichton
appveyor: Use Ninja to build LLVM on MinGW
6ae2234 
I have a suspicion that MinGW's make is the cause of rust-lang#40546 rather than anything
else, but that's purely a suspicion without any facts to back it up. In any case
we'll eventually be moving the MSVC build over to Ninja in order to leverage
sccache regardless, so this commit simply jumpstarts that process by downloading
Ninja for use by MinGW anyway.

I'm not sure if this closes rust-lang#40546 for real, but this is my current best shot at
closing it out, so...

Closes rust-lang#40546
bors added a commit that referenced this issue Mar 22, 2017
@bors
Auto merge of #40548 - alexcrichton:appveyor-ninja, r=brson
5dc595b 
appveyor: Use Ninja to build LLVM on MinGW

I have a suspicion that MinGW's make is the cause of #40546 rather than anything
else, but that's purely a suspicion without any facts to back it up. In any case
we'll eventually be moving the MSVC build over to Ninja in order to leverage
sccache regardless, so this commit simply jumpstarts that process by downloading
Ninja for use by MinGW anyway.

I'm not sure if this closes #40546 for real, but this is my current best shot at
closing it out, so...

Closes #40546
bors added a commit that referenced this issue Mar 22, 2017
@bors
Auto merge of #40548 - alexcrichton:appveyor-ninja, r=brson
8aad2a2 
appveyor: Use Ninja to build LLVM on MinGW

I have a suspicion that MinGW's make is the cause of #40546 rather than anything
else, but that's purely a suspicion without any facts to back it up. In any case
we'll eventually be moving the MSVC build over to Ninja in order to leverage
sccache regardless, so this commit simply jumpstarts that process by downloading
Ninja for use by MinGW anyway.

I'm not sure if this closes #40546 for real, but this is my current best shot at
closing it out, so...

Closes #40546
frewsxcv added a commit to frewsxcv/rust that referenced this issue Mar 22, 2017
@frewsxcv
Rollup merge of rust-lang#40548 - alexcrichton:appveyor-ninja, r=brson
fde6941 
appveyor: Use Ninja to build LLVM on MinGW

I have a suspicion that MinGW's make is the cause of rust-lang#40546 rather than anything
else, but that's purely a suspicion without any facts to back it up. In any case
we'll eventually be moving the MSVC build over to Ninja in order to leverage
sccache regardless, so this commit simply jumpstarts that process by downloading
Ninja for use by MinGW anyway.

I'm not sure if this closes rust-lang#40546 for real, but this is my current best shot at
closing it out, so...

Closes rust-lang#40546
@alexcrichton
Copy link
Member Author

Unfortunately it looks like this is not fixed

@alexcrichton alexcrichton reopened this Mar 23, 2017
@alexcrichton alexcrichton mentioned this issue Mar 23, 2017
Merged
@alexcrichton alexcrichton mentioned this issue Mar 23, 2017
Merged
@alexcrichton
Copy link
Member Author

Another attempt to fix this

alexcrichton added a commit to alexcrichton/rust that referenced this issue Mar 24, 2017
@alexcrichton
appveyor: Upgrade MinGW toolchains we use
e6e8c91 
In debugging rust-lang#40546 I was able to reproduce locally finally using
the literal toolchain that the bots were using. I reproduced the error maybe 4
in 10 builds. I also have the 6.3.0 toolchain installed through `pacman` which
has yet to have a failed build.

When attempting to reproduce the bug with the toolchain that this commit
switches to I was unable to reproduce anything after a few builds. I have no
idea what the original problem was, but I'm hoping that it was just some random
bug fixed somewhere along the way.

I don't currently know of a technical reason to stick to the 4.9.2 toolchains we
were previously using. Historcal 5.3.* toolchains would cause llvm to segfault
(maybe a miscompile?) but this seems to have been fixed recently. To me if it
passes CI then I think we're good.

Closes rust-lang#40546
@alexcrichton
Copy link
Member Author

@arielb1 yes I think that's the best solution here.

@mati865 ah we don't want to rely on the msys packages as well b/c we can't pin versions. They'll change from under our feet which runs the risk of breaking the build which we'd prefer to avoid.

@crlf0710 this isn't related to LLVM, it's related to the gcc/gdb toolchain versions

@alexcrichton alexcrichton mentioned this issue Apr 18, 2017
Merged
This was referenced Apr 18, 2017
Merged
Merged
This was referenced Apr 20, 2017
Merged
Merged
Merged
@TimNN TimNN mentioned this issue Apr 23, 2017
Merged
@Mark-Simulacrum Mark-Simulacrum mentioned this issue Apr 25, 2017
Merged
alexcrichton added a commit to alexcrichton/rust that referenced this issue Apr 25, 2017
@alexcrichton
appveyor: Upgrade to gcc for mingw 6.3.0
3b33913 
This commit sort of brings back rust-lang#40777 by upgrading back to 6.3.0. While
investigating rust-lang#40546 it was discovered that 6.3.0 appears to not spurious
fail in the same way that 6.2.0 does (which we're currently using). The
workaround for rust-lang#40184 contained in rust-lang#40777 did not work so this commit also
contains a different workaround for the gdb issue. We will not download the
6.2.0 version of gdb and use that instead of the default version that comes with
6.3.0.

I'm going to optimistically say...

Closes rust-lang#40546
@alexcrichton alexcrichton mentioned this issue Apr 25, 2017
Merged
arielb1 pushed a commit to arielb1/rust that referenced this issue Apr 25, 2017
Rollup merge of rust-lang#41420 - alexcrichton:mingw-6.3.0, r=petroch…
38fb097 
…enkov

appveyor: Upgrade to gcc for mingw 6.3.0

This commit sort of brings back rust-lang#40777 by upgrading back to 6.3.0. While
investigating rust-lang#40546 it was discovered that 6.3.0 appears to not spurious
fail in the same way that 6.2.0 does (which we're currently using). The
workaround for rust-lang#40184 contained in rust-lang#40777 did not work so this commit also
contains a different workaround for the gdb issue. We will not download the
6.2.0 version of gdb and use that instead of the default version that comes with
6.3.0.

I'm going to optimistically say...

Closes rust-lang#40546
@Mark-Simulacrum Mark-Simulacrum mentioned this issue Apr 25, 2017
Closed
alexcrichton added a commit to alexcrichton/rust that referenced this issue Apr 26, 2017
@alexcrichton
appveyor: Upgrade to gcc for mingw 6.3.0
d1db74b 
This commit sort of brings back rust-lang#40777 by upgrading back to 6.3.0. While
investigating rust-lang#40546 it was discovered that 6.3.0 appears to not spurious
fail in the same way that 6.2.0 does (which we're currently using). The
workaround for rust-lang#40184 contained in rust-lang#40777 did not work so this commit also
contains a different workaround for the gdb issue. We will not download the
6.2.0 version of gdb and use that instead of the default version that comes with
6.3.0.

I'm going to optimistically say...

Closes rust-lang#40546
bors added a commit that referenced this issue Apr 26, 2017
@bors
Auto merge of #41420 - alexcrichton:mingw-6.3.0, r=petrochenkov
0369832 
appveyor: Upgrade to gcc for mingw 6.3.0

This commit sort of brings back #40777 by upgrading back to 6.3.0. While
investigating #40546 it was discovered that 6.3.0 appears to not spurious
fail in the same way that 6.2.0 does (which we're currently using). The
workaround for #40184 contained in #40777 did not work so this commit also
contains a different workaround for the gdb issue. We will not download the
6.2.0 version of gdb and use that instead of the default version that comes with
6.3.0.

I'm going to optimistically say...

Closes #40546
@Mark-Simulacrum Mark-Simulacrum mentioned this issue Apr 29, 2017
Merged
@Mark-Simulacrum
Copy link
Member

This is, unfortunately, still a problem. See #41580 (comment).

@Mark-Simulacrum
Copy link
Member

Possibly a false alarm. #41580 is targeting the beta branch, which presumably does not have the fix for this failure, in which case it's "normal" for it to fail spuriously. Will leave open for someone to confirm, though.

@alexcrichton
Copy link
Member Author

@Mark-Simulacrum ah yeah I think we're ok, #41420, the fix, is only on master not on beta

@Mark-Simulacrum Mark-Simulacrum mentioned this issue May 21, 2017
Merged
@alexcrichton alexcrichton mentioned this issue Jun 5, 2017
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-spurious Area: Spurious failures in builds (spuriously == for no apparent reason)
Projects
None yet
Milestone
No milestone
7 participants
@alexcrichton @carols10cents @crlf0710 @aidanhs @mati865 @arielb1 @Mark-Simulacrum