Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix redhat arm64 #52244

Merged
merged 17 commits into from
Jul 28, 2021
Merged

Fix redhat arm64 #52244

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
fef1a9e
Fix RHEL 8 ARM64
janvorli May 3, 2021
285cada
Fix ARM build to use PC relative addresses only
janvorli May 4, 2021
9970b60
Update docker images used for building runtime
janvorli Jul 14, 2021
3bd6fbf
Disable lld linker for s390x
janvorli Jul 14, 2021
0433807
Do not add the lld option for macOS and FreeBSD
janvorli Jul 15, 2021
1ea2887
Make adding the lld only when compiling with Clang
janvorli Jul 15, 2021
9bfa3a0
Move LLD configuration into configuretools.cmake
janvorli Jul 15, 2021
99aa111
Update arm/arm64 docker images
janvorli Jul 19, 2021
09c5999
Fix ARM reloc issue
janvorli Jul 19, 2021
f1abcd3
Limit the lld usage to Linux, fix arm32 issue
janvorli Jul 20, 2021
e66eb50
Disable lld for linux-musl until we move official builds to Alpine 3.13
janvorli Jul 21, 2021
33bb47c
Move lld setting to the runtime shell build scripts
janvorli Jul 21, 2021
3d1e65d
Fix FreeBSD cross build
janvorli Jul 21, 2021
5f0ff6d
Fix cmake 3.6 build issue
janvorli Jul 21, 2021
bad50b6
Move LLD setting into init-compiler.sh
janvorli Jul 21, 2021
0c4e740
Cleanup of comments and spacing
janvorli Jul 21, 2021
3856661
Use LLD only for builds with clang
janvorli Jul 22, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion eng/common/templates/jobs/source-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ parameters:
# This is the default platform provided by Arcade, intended for use by a managed-only repo.
defaultManagedPlatform:
name: 'Managed'
container: 'mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-3e800f1-20190501005343'
container: 'mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-20210714125435-9b5bbc2'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will likely cause a merge conflict when the next time arcade folder is mirrored https://github.com/dotnet/arcade/blob/dde80826e2fe464ef34182d2f786a6a51121d9a6/eng/common/templates/jobs/source-build.yml#L17.

There was a proposal to send a heads-up comment from bot account on PR that is modifying eng/common and the author is not dotnet-bot. I think it can save the author the hassle of undoing-and-redoing-the-right-way (which has happened a few times in runtime, aspnetcore and other repos).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, thank you for noticing this, I have not noticed this was in the common folder. I'll make the same change in arcade, that should fix the future issue, I believe (I did something like that in the past).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, reading the comment in the file above the change, we can just revert the change in it as it is used for managed stuff only and lld is not needed in there. I've updated all references to the centos image I've found and haven't noticed the purpose.


# Defines the platforms on which to run build jobs. One job is created for each platform, and the
# object in this array is sent to the job template as 'platform'. If no platforms are specified,
Expand Down
6 changes: 4 additions & 2 deletions eng/native/configuretools.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,12 +65,14 @@ endif()

if (NOT CLR_CMAKE_HOST_WIN32)
# detect linker
set(ldVersion ${CMAKE_C_COMPILER};-Wl,--version)
separate_arguments(ldVersion UNIX_COMMAND "${CMAKE_C_COMPILER} ${CMAKE_SHARED_LINKER_FLAGS} -Wl,--version")
execute_process(COMMAND ${ldVersion}
ERROR_QUIET
OUTPUT_VARIABLE ldVersionOutput)

if("${ldVersionOutput}" MATCHES "GNU ld" OR "${ldVersionOutput}" MATCHES "GNU gold" OR "${ldVersionOutput}" MATCHES "GNU linkers")
if("${ldVersionOutput}" MATCHES "LLD")
set(LD_LLVM 1)
elseif("${ldVersionOutput}" MATCHES "GNU ld" OR "${ldVersionOutput}" MATCHES "GNU gold" OR "${ldVersionOutput}" MATCHES "GNU linkers")
set(LD_GNU 1)
elseif("${ldVersionOutput}" MATCHES "Solaris Link")
set(LD_SOLARIS 1)
Expand Down
2 changes: 1 addition & 1 deletion eng/native/functions.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ function(preprocess_files PreprocessedFilesList)
endfunction()

function(set_exports_linker_option exports_filename)
if(LD_GNU OR LD_SOLARIS)
if(LD_GNU OR LD_SOLARIS OR LD_LLVM)
# Add linker exports file option
if(LD_SOLARIS)
set(EXPORTS_LINKER_OPTION -Wl,-M,${exports_filename} PARENT_SCOPE)
Expand Down
12 changes: 11 additions & 1 deletion eng/native/init-compiler.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ minorVersion="$4"
# clear the existing CC and CXX from environment
CC=
CXX=
LDFLAGS=

if [[ "$compiler" == "gcc" ]]; then cxxCompiler="g++"; fi

Expand Down Expand Up @@ -106,6 +107,15 @@ if [[ -z "$CC" ]]; then
exit 1
fi

if [[ "$compiler" == "clang" ]]; then
if command -v "lld$desired_version" > /dev/null; then
# Only lld version >= 9 can be considered stable
if [[ "$majorVersion" -ge 9 ]]; then
LDFLAGS="-fuse-ld=lld"
fi
fi
fi

SCAN_BUILD_COMMAND="$(command -v "scan-build$desired_version")"

export CC CXX SCAN_BUILD_COMMAND
export CC CXX LDFLAGS SCAN_BUILD_COMMAND
14 changes: 7 additions & 7 deletions eng/pipelines/common/platform-matrix.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ jobs:
targetRid: linux-arm
platform: Linux_arm
container:
image: ubuntu-16.04-cross-20200413125008-09ec757
image: ubuntu-16.04-cross-20210719121212-8a8d3be
registry: mcr
jobParameters:
runtimeFlavor: ${{ parameters.runtimeFlavor }}
Expand All @@ -64,7 +64,7 @@ jobs:
targetRid: linux-arm64
platform: Linux_arm64
container:
image: ubuntu-16.04-cross-arm64-20201022204150-b2c2436
image: ubuntu-16.04-cross-arm64-20210719121212-8a8d3be
registry: mcr
jobParameters:
runtimeFlavor: ${{ parameters.runtimeFlavor }}
Expand All @@ -91,7 +91,7 @@ jobs:
targetRid: linux-musl-x64
platform: Linux_musl_x64
container:
image: alpine-3.9-WithNode-20200602002639-0fc54a3
image: alpine-3.9-WithNode-20210714125437-9b5bbc2
registry: mcr
jobParameters:
runtimeFlavor: ${{ parameters.runtimeFlavor }}
Expand All @@ -116,7 +116,7 @@ jobs:
targetRid: linux-musl-arm
platform: Linux_musl_arm
container:
image: ubuntu-16.04-cross-arm-alpine-20210409142327-044d5b9
image: ubuntu-16.04-cross-arm-alpine-20210719121212-044d5b9
registry: mcr
jobParameters:
runtimeFlavor: ${{ parameters.runtimeFlavor }}
Expand All @@ -143,7 +143,7 @@ jobs:
targetRid: linux-musl-arm64
platform: Linux_musl_arm64
container:
image: ubuntu-16.04-cross-arm64-alpine-20200413125008-406629a
image: ubuntu-16.04-cross-arm64-alpine-20210719121212-b2c2436
registry: mcr
jobParameters:
runtimeFlavor: ${{ parameters.runtimeFlavor }}
Expand All @@ -169,7 +169,7 @@ jobs:
targetRid: linux-x64
platform: Linux_x64
container:
image: centos-7-20201227183837-5fe0e50
image: centos-7-20210714125435-9b5bbc2
registry: mcr
jobParameters:
runtimeFlavor: ${{ parameters.runtimeFlavor }}
Expand All @@ -193,7 +193,7 @@ jobs:
targetRid: linux-x64
platform: Linux_x64
container:
image: centos-7-source-build-20210408124356-5d87b80
image: centos-7-source-build-20210714125450-5d87b80
registry: mcr
jobParameters:
runtimeFlavor: ${{ parameters.runtimeFlavor }}
Expand Down
10 changes: 8 additions & 2 deletions src/coreclr/pgosupport.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,10 @@ function(add_pgo TargetName)
else(CLR_CMAKE_HOST_WIN32)
if(UPPERCASE_CMAKE_BUILD_TYPE STREQUAL RELEASE OR UPPERCASE_CMAKE_BUILD_TYPE STREQUAL RELWITHDEBINFO)
target_compile_options(${TargetName} PRIVATE -flto -fprofile-instr-generate)
set_property(TARGET ${TargetName} APPEND_STRING PROPERTY LINK_FLAGS " -flto -fuse-ld=gold -fprofile-instr-generate")
set_property(TARGET ${TargetName} APPEND_STRING PROPERTY LINK_FLAGS " -flto -fprofile-instr-generate")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to confirm, LLD supports the same PGO linker flags as GOLD, correct?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct. I have verified that a release build with -pgoinstrument passes and that the compiler was really passed these options.

if(NOT LD_LLVM)
set_property(TARGET ${TargetName} APPEND_STRING PROPERTY LINK_FLAGS " -fuse-ld=gold")
endif()
endif(UPPERCASE_CMAKE_BUILD_TYPE STREQUAL RELEASE OR UPPERCASE_CMAKE_BUILD_TYPE STREQUAL RELWITHDEBINFO)
endif(CLR_CMAKE_HOST_WIN32)
elseif(CLR_CMAKE_PGO_OPTIMIZE)
Expand Down Expand Up @@ -54,7 +57,10 @@ function(add_pgo TargetName)
if((CMAKE_CXX_COMPILER_ID MATCHES "Clang") AND (NOT CMAKE_CXX_COMPILER_VERSION VERSION_LESS 3.6))
if(HAVE_LTO)
target_compile_options(${TargetName} PRIVATE -flto -fprofile-instr-use=${ProfilePath} -Wno-profile-instr-out-of-date -Wno-profile-instr-unprofiled)
set_property(TARGET ${TargetName} APPEND_STRING PROPERTY LINK_FLAGS " -flto -fuse-ld=gold -fprofile-instr-use=${ProfilePath}")
set_property(TARGET ${TargetName} APPEND_STRING PROPERTY LINK_FLAGS " -flto -fprofile-instr-use=${ProfilePath}")
if(NOT LD_LLVM)
set_property(TARGET ${TargetName} APPEND_STRING PROPERTY LINK_FLAGS " -fuse-ld=gold")
endif()
else(HAVE_LTO)
message(WARNING "LTO is not supported, skipping profile guided optimizations")
endif(HAVE_LTO)
Expand Down
2 changes: 1 addition & 1 deletion src/coreclr/vm/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -855,7 +855,6 @@ elseif(CLR_CMAKE_TARGET_ARCH_ARM64)
set(VM_SOURCES_DAC_AND_WKS_ARCH
${ARCH_SOURCES_DIR}/stubs.cpp
exceptionhandling.cpp
gcinfodecoder.cpp
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this change intentionally part of this PR? It seems unrelated at a glance.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, there were some multiply defined symbols due to double inclusion of this file and LLD complained.

)

set(VM_HEADERS_DAC_AND_WKS_ARCH
Expand All @@ -865,6 +864,7 @@ elseif(CLR_CMAKE_TARGET_ARCH_ARM64)

set(VM_SOURCES_WKS_ARCH
${ARCH_SOURCES_DIR}/profiler.cpp
gcinfodecoder.cpp
)

if(CLR_CMAKE_HOST_UNIX)
Expand Down
9 changes: 6 additions & 3 deletions src/coreclr/vm/arm/asmhelpers.S
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -928,8 +928,9 @@ LOCAL_LABEL(CallCppHelper3):
.endm

.macro JIT_WRITEBARRIER_DESCRIPTOR name
.word \name
.word \name\()_End
1:
.word \name-1b
.word \name\()_End-1b
.word __\name\()__g_lowest_address_offset
.word __\name\()__g_highest_address_offset
.word __\name\()__g_ephemeral_low_offset
Expand Down Expand Up @@ -983,7 +984,9 @@ g_rgWriteBarrierDescriptors:
LEAF_ENTRY JIT_WriteBarrier_Callable

// Branch to the write barrier
ldr r2, =JIT_WriteBarrier_Loc // or R3? See targetarm.h
ldr r2, =JIT_WriteBarrier_Loc-(1f+4) // or R3? See targetarm.h
1:
add r2, pc
ldr pc, [r2]

LEAF_END JIT_WriteBarrier_Callable
Expand Down
15 changes: 9 additions & 6 deletions src/coreclr/vm/arm/pinvokestubs.S
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,15 +84,19 @@

PROLOG_PUSH "{r4, lr}"

ldr r1, =s_gsCookie
ldr r1, =s_gsCookie-(1f+4)
1:
add r1, pc
ldr r1, [r1]
str r1, [r0]
add r4, r0, SIZEOF__GSCookie

// r4 = pFrame

// set first slot to the value of InlinedCallFrame::`vftable' (checked by runtime code)
ldr r1, .L12
ldr r1, =_ZTV16InlinedCallFrame+8-(2f+4)
2:
add r1, pc
str r1, [r4]

mov r1, 0
Expand Down Expand Up @@ -123,9 +127,6 @@

NESTED_END JIT_PInvokeBegin, _TEXT

.L12:
.word _ZTV16InlinedCallFrame+8

// ------------------------------------------------------------------
// IN:
// InlinedCallFrame (r0) = pointer to the InlinedCallFrame data, including the GS cookie slot (GS cookie right
Expand All @@ -145,7 +146,9 @@
str r2, [r1, #Thread_m_fPreemptiveGCDisabled]

// Check return trap
ldr r2, =g_TrapReturningThreads
ldr r2, =g_TrapReturningThreads-(1f+4)
1:
add r2, pc
ldr r2, [r2]
cbnz r2, LOCAL_LABEL(RarePath)

Expand Down
6 changes: 3 additions & 3 deletions src/coreclr/vm/arm/stubs.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -281,8 +281,8 @@ void StubLinkerCPU::Init(void)
// value of the global into a register.
struct WriteBarrierDescriptor
{
BYTE * m_pFuncStart; // Pointer to the start of the barrier function
BYTE * m_pFuncEnd; // Pointer to the end of the barrier function
DWORD m_pFuncStart; // Offset to the start of the barrier function relative to this struct address
DWORD m_pFuncEnd; // Offset to the end of the barrier function relative to this struct address
DWORD m_dw_g_lowest_address_offset; // Offset of the instruction reading g_lowest_address
DWORD m_dw_g_highest_address_offset; // Offset of the instruction reading g_highest_address
DWORD m_dw_g_ephemeral_low_offset; // Offset of the instruction reading g_ephemeral_low
Expand Down Expand Up @@ -440,7 +440,7 @@ void UpdateGCWriteBarriers(bool postGrow = false)
{
// If the write barrier is being currently used (as in copied over to the patchable site)
// then read the patch location from the table and use the offset to patch the target asm code
PBYTE to = FindWBMapping(pDesc->m_pFuncStart);
PBYTE to = FindWBMapping((BYTE *)pDesc + pDesc->m_pFuncStart);
if(to)
{
to = (PBYTE)PCODEToPINSTR((PCODE)GetWriteBarrierCodeLocation(to));
Expand Down