Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[tests] Stand up iOS arm runtime tests and enable tracing/eventpipe tests #79169

Merged
merged 170 commits into from
Apr 18, 2023
Merged

[tests] Stand up iOS arm runtime tests and enable tracing/eventpipe tests #79169

merged 170 commits into from
Apr 18, 2023

Conversation

mdh1418
Copy link
Member

@mdh1418 mdh1418 commented Dec 2, 2022 •
edited by naricc
Loading

Fixes #74891

On iOS, runtime tests were previously only stood up on x64 (Simulator) in #43954.
This PR looks to extend the runtime testing coverage to iOS devices and arm simulators by doing the following:

  • Provide AOT'd assemblies to the AppleAppBuilder in the runtime build flow following the conventions of library tests (previously no assemblies were provided to the AppleAppBuilder)
  • Default RunAOTCompilation and MonoForceInterpreter properties in the runtime build flow for iOS
  • Enable diagnostics_tracing for iOS runtime tests
  • Reenable runtime tracing/eventpipe tests on iOS not involving subprocesses
  • Tweak ProcessInfo and ProcessInfo2 tests to pass on iOS
  • Skip IpcTraceTest.EnsureCleanEnvironment as ProcessManager.GetProcessInfos is not supported on iOS
  • Align AppleAppBuilder parameter order with the order of definition in the task

Fixes: #81233

@mdh1418 mdh1418 added area-Infrastructure-mono runtime-mono specific to the Mono runtime labels Dec 2, 2022
@ghost
Copy link

ghost commented Dec 2, 2022

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes #74891

On iOS, runtime tests were previously only stood up on x64 (Simulator) in #43954.
This PR looks to extend the runtime testing coverage to iOS devices and arm simulators by doing the following:

  • Provide AOT'd assemblies to the AppleAppBuilder in the runtime build flow following the conventions of library tests (previously no assemblies were provided to the AppleAppBuilder)
  • Default RunAOTCompilation and MonoForceInterpreter properties in the runtime build flow for iOS
  • Enable diagnostics_tracing for iOS runtime tests
  • Reenable runtime tracing/eventpipe tests on iOS not involving subprocesses
  • Tweak ProcessInfo and ProcessInfo2 tests to pass on iOS
  • Skip IpcTraceTest.EnsureCleanEnvironment as ProcessManager.GetProcessInfos is not supported on iOS
  • Align AppleAppBuilder parameter order with the order of definition in the task
Author: mdh1418
Assignees: -
Labels:

area-Infrastructure-mono, runtime-mono
Milestone: -

@ghost ghost assigned mdh1418 Dec 2, 2022
naricc
naricc reviewed Dec 16, 2022
View reviewed changes
src/tests/build.proj
@@ -274,6 +274,9 @@
<BuildDir>$(IntermediateOutputPath)\iOSApps\$(Category)</BuildDir>
<FinalPath>$(XUnitTestBinBase)$(CategoryWithSlash)\$(Category).app</FinalPath>
<RuntimeComponents>diagnostics_tracing;marshal-ilgen</RuntimeComponents>
<DiagnosticPorts>127.0.0.1:9000,nosuspend,listen</DiagnosticPorts>
<RunAOTCompilation>True</RunAOTCompilation>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little confused here; what happens if we are running iOSSimulator and both RunAOTCompilation and MonoForceInterpreter are true? Does this change how iOSSimulator tests are run?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For iOSlike devices and arm based simulators, AOT is needed for the tests to build. (At least that was the case for library tests, which motivated this PR #76551) When setting RunAOTCompilation and MonoForceInterpreter to true, what will happen is AOT will be ran, and if anything goes wrong, interp will be the fallback instead of JIT (which is important because JIT does not work on iOSlike platforms).
I believe the RunAOTCompilation is necessary here for arm architectures, but we can exclude x64 if thats preferred.

@mdh1418
Copy link
Member Author

mdh1418 commented Dec 16, 2022

/azp run runtime-ioslike,runtime-ioslikesimulator

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

This was referenced Dec 17, 2022
Closed
Closed
Closed
@mdh1418
Copy link
Member Author

mdh1418 commented Dec 19, 2022

/azp run runtime-ioslike,runtime-ioslikesimulator

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@mdh1418
Copy link
Member Author

mdh1418 commented Dec 19, 2022

/azp run runtime-ioslike,runtime-ioslikesimulator

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@naricc
Copy link
Contributor

naricc commented Dec 19, 2022

/azp run runtime-ioslike,runtime-ioslikesimulator

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@build-analysis build-analysis bot mentioned this pull request Dec 20, 2022
Closed
@naricc
Copy link
Contributor

naricc commented Dec 20, 2022

/azp run runtime-ioslike

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@build-analysis build-analysis bot mentioned this pull request Jan 6, 2023
Closed
@naricc
Copy link
Contributor

naricc commented Jan 6, 2023

/azp run runtime-ioslike,runtime-ioslikesimulator

@azure-pipelines
Copy link

Azure Pipelines failed to run 2 pipeline(s).

@naricc naricc requested a review from vargaz as a code owner January 6, 2023 20:53
@kotlarmilos
Copy link
Member

/azp run runtime-ioslike,runtime-ioslikesimulator

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@kotlarmilos
Copy link
Member

The failure shouldn't be related. It has been tracked in dotnet/arcade#11683.

mdh1418
mdh1418 commented Apr 13, 2023
View reviewed changes
Copy link
Member Author

@mdh1418 mdh1418 left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a bunch of nits and Jeremy's suggestion for the test grouping, but looks good overall to me!

Would approve, but the approve button is greyed out for me because I was the original PR author

src/tests/issues.targets Outdated Show resolved Hide resolved
eng/pipelines/libraries/helix-queues-setup.yml Outdated Show resolved Hide resolved
src/tests/Common/Coreclr.TestWrapper/MobileAppHandler.cs Outdated Show resolved Hide resolved
src/tests/Common/helixpublishwitharcade.proj Outdated Show resolved Hide resolved
src/tests/Common/helixpublishwitharcade.proj Outdated Show resolved Hide resolved
src/tests/issues.targets Outdated Show resolved Hide resolved
src/tests/issues.targets Outdated Show resolved Hide resolved
src/tests/issues.targets Outdated Show resolved Hide resolved
src/tests/issues.targets Outdated Show resolved Hide resolved
src/tests/issues.targets Outdated Show resolved Hide resolved
steveisok
steveisok approved these changes Apr 14, 2023
View reviewed changes
Copy link
Member

@steveisok steveisok left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@kotlarmilos
Copy link
Member

/azp run runtime-ioslike,runtime-ioslikesimulator

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@kotlarmilos
Copy link
Member

/azp run runtime-ioslikesimulator

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@build-analysis build-analysis bot mentioned this pull request Apr 17, 2023
Open
@mdh1418
Copy link
Member Author

mdh1418 commented Apr 17, 2023

/azp run runtime-ioslikesimulator

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@kotlarmilos
Copy link
Member

/azp run runtime-ioslike,runtime-ioslikesimulator

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@kotlarmilos
Copy link
Member

Failures shouldn't be related. The TCP failure is known issue dotnet/arcade#11683.

@kotlarmilos kotlarmilos mentioned this pull request Apr 18, 2023
Closed
@kotlarmilos
Copy link
Member

The failures are known #84979 and #84995.

@kotlarmilos kotlarmilos merged commit 790b14b into dotnet:main Apr 18, 2023
@kotlarmilos kotlarmilos mentioned this pull request Apr 28, 2023
Closed
@ghost ghost locked as resolved and limited conversation to collaborators May 18, 2023
@mdh1418 mdh1418 deleted the ios_investigate_eventpipe_failures branch June 10, 2024 20:34
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@fanyang-mono fanyang-mono fanyang-mono left review comments

@naricc naricc naricc left review comments

@kotlarmilos kotlarmilos kotlarmilos left review comments

@premun premun premun left review comments

@jkoritzinsky jkoritzinsky jkoritzinsky left review comments

@steveisok steveisok steveisok approved these changes

@akoeplinger akoeplinger Awaiting requested review from akoeplinger

@lateralusX lateralusX Awaiting requested review from lateralusX

@marek-safar marek-safar Awaiting requested review from marek-safar

@vargaz vargaz Awaiting requested review from vargaz

@lambdageek lambdageek Awaiting requested review from lambdageek

@thaystg thaystg Awaiting requested review from thaystg

@radical radical Awaiting requested review from radical

@lewing lewing Awaiting requested review from lewing

@pavelsavara pavelsavara Awaiting requested review from pavelsavara

@BrzVlad BrzVlad Awaiting requested review from BrzVlad

@SamMonoRT SamMonoRT Awaiting requested review from SamMonoRT

@MichalStrehovsky MichalStrehovsky Awaiting requested review from MichalStrehovsky

Assignees

@kotlarmilos kotlarmilos

@mdh1418 mdh1418

@naricc naricc

Labels
area-Infrastructure-mono runtime-mono specific to the Mono runtime
Projects
None yet
Milestone
No milestone
7 participants
@mdh1418 @naricc @kotlarmilos @steveisok @jkoritzinsky @premun @fanyang-mono