Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[wasm] Perf pipeline completely broken #88534

Closed
radical opened this issue Jul 7, 2023 · 4 comments
Closed

[wasm] Perf pipeline completely broken #88534

radical opened this issue Jul 7, 2023 · 4 comments
Assignees
@radical
Labels
arch-wasm WebAssembly architecture area-Infrastructure-libraries perf-pipeline Issues with dotnet-runtime-perf, or runtime-wasm-perf pipelines
Milestone
8.0.0

Comments

@radical
Copy link
Member

radical commented Jul 7, 2023 •
edited by build-analysis bot
Loading

https://dev.azure.com/dnceng/internal/_build/results?buildId=2216141&view=results

$ pushd "/home/helixbot/work/B79109DE/w/AF9E097B/e/performance/artifacts/bin/for-running/MicroBenchmarks"
$ dotnet exec MicroBenchmarks.dll --anyCategories Libraries Runtime "" --category-exclusion-filter NoInterpreter NoWASM NoMono --wasmEngine /home/helixbot/.jsvu/v8 "--wasmArgs=--expose_wasm --module " --cli /home/helixbot/work/B79109DE/p/dotnet/dotnet --wasmDataDir /home/helixbot/work/B79109DE/p/wasm-data --logBuildOutput --generateBinLog "" --partition-count 30 --partition-index 16 --artifacts /home/helixbot/work/B79109DE/w/AF9E097B/e/artifacts/BenchmarkDotNet.Artifacts --packages /home/helixbot/work/B79109DE/w/AF9E097B/e/performance/artifacts/packages --runtimes wasmnet80 --buildTimeout 1200
Unhandled exception. System.TypeLoadException: Could not load type 'System.Runtime.CompilerServices.NullableContextAttribute' from assembly 'System.Runtime, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
   at System.ModuleHandle.ResolveType(QCallModule module, Int32 typeToken, IntPtr* typeInstArgs, Int32 typeInstCount, IntPtr* methodInstArgs, Int32 methodInstCount, ObjectHandleOnStack type)
   at System.ModuleHandle.ResolveTypeHandle(Int32 typeToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
   at System.Reflection.RuntimeModule.ResolveType(Int32 metadataToken, Type[] genericTypeArguments, Type[] genericMethodArguments)
   at System.Reflection.CustomAttribute.FilterCustomAttributeRecord(MetadataToken caCtorToken, MetadataImport& scope, RuntimeModule decoratedModule, MetadataToken decoratedToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder`1& derivedAttributes, RuntimeType& attributeType, IRuntimeMethodInfo& ctorWithParameters, Boolean& isVarArg)
   at System.Reflection.CustomAttribute.AddCustomAttributes(ListBuilder`1& attributes, RuntimeModule decoratedModule, Int32 decoratedMetadataToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder`1 derivedAttributes)
   at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeMethodInfo method, RuntimeType caType, Boolean inherit)
   at BenchmarkDotNet.Extensions.ReflectionExtensions.<>c.<GetBenchmarks>b__14_0(MethodInfo method)
   at System.Linq.Enumerable.WhereArrayIterator`1.ToArray()
   at BenchmarkDotNet.Extensions.ReflectionExtensions.GetBenchmarks(TypeInfo typeInfo)
   at BenchmarkDotNet.Extensions.ReflectionExtensions.ContainsRunnableBenchmarks(Type type)
   at BenchmarkDotNet.Extensions.ReflectionExtensions.<>c.<GetRunnableBenchmarks>b__12_0(Type type)
   at System.Linq.Enumerable.WhereArrayIterator`1.ToArray()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.OrderedEnumerable`1.ToArray()
   at BenchmarkDotNet.Extensions.ReflectionExtensions.GetRunnableBenchmarks(Assembly assembly)
   at BenchmarkDotNet.Running.TypeFilter.GetTypesWithRunnableBenchmarks(IEnumerable`1 types, IEnumerable`1 assemblies, ILogger logger)
   at BenchmarkDotNet.Running.BenchmarkSwitcher.RunWithDirtyAssemblyResolveHelper(String[] args, IConfig config, Boolean askUserForInput)
   at BenchmarkDotNet.Running.BenchmarkSwitcher.Run(String[] args, IConfig config)
   at MicroBenchmarks.Program.Main(String[] args) in /home/helixbot/work/B79109DE/w/AF9E097B/e/performance/src/benchmarks/micro/Program.cs:line 43

Likely related to #87857 .

Known Issue Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Report

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 0 0
@radical radical added arch-wasm WebAssembly architecture blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' perf-pipeline Issues with dotnet-runtime-perf, or runtime-wasm-perf pipelines labels Jul 7, 2023
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Jul 7, 2023
@radical radical self-assigned this Jul 7, 2023
@ghost
Copy link

ghost commented Jul 7, 2023

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

https://dev.azure.com/dnceng/internal/_build/results?buildId=2216141&view=results

$ pushd "/home/helixbot/work/B79109DE/w/AF9E097B/e/performance/artifacts/bin/for-running/MicroBenchmarks"
$ dotnet exec MicroBenchmarks.dll --anyCategories Libraries Runtime "" --category-exclusion-filter NoInterpreter NoWASM NoMono --wasmEngine /home/helixbot/.jsvu/v8 "--wasmArgs=--expose_wasm --module " --cli /home/helixbot/work/B79109DE/p/dotnet/dotnet --wasmDataDir /home/helixbot/work/B79109DE/p/wasm-data --logBuildOutput --generateBinLog "" --partition-count 30 --partition-index 16 --artifacts /home/helixbot/work/B79109DE/w/AF9E097B/e/artifacts/BenchmarkDotNet.Artifacts --packages /home/helixbot/work/B79109DE/w/AF9E097B/e/performance/artifacts/packages --runtimes wasmnet80 --buildTimeout 1200
Unhandled exception. System.TypeLoadException: Could not load type 'System.Runtime.CompilerServices.NullableContextAttribute' from assembly 'System.Runtime, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
   at System.ModuleHandle.ResolveType(QCallModule module, Int32 typeToken, IntPtr* typeInstArgs, Int32 typeInstCount, IntPtr* methodInstArgs, Int32 methodInstCount, ObjectHandleOnStack type)
   at System.ModuleHandle.ResolveTypeHandle(Int32 typeToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
   at System.Reflection.RuntimeModule.ResolveType(Int32 metadataToken, Type[] genericTypeArguments, Type[] genericMethodArguments)
   at System.Reflection.CustomAttribute.FilterCustomAttributeRecord(MetadataToken caCtorToken, MetadataImport& scope, RuntimeModule decoratedModule, MetadataToken decoratedToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder`1& derivedAttributes, RuntimeType& attributeType, IRuntimeMethodInfo& ctorWithParameters, Boolean& isVarArg)
   at System.Reflection.CustomAttribute.AddCustomAttributes(ListBuilder`1& attributes, RuntimeModule decoratedModule, Int32 decoratedMetadataToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder`1 derivedAttributes)
   at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeMethodInfo method, RuntimeType caType, Boolean inherit)
   at BenchmarkDotNet.Extensions.ReflectionExtensions.<>c.<GetBenchmarks>b__14_0(MethodInfo method)
   at System.Linq.Enumerable.WhereArrayIterator`1.ToArray()
   at BenchmarkDotNet.Extensions.ReflectionExtensions.GetBenchmarks(TypeInfo typeInfo)
   at BenchmarkDotNet.Extensions.ReflectionExtensions.ContainsRunnableBenchmarks(Type type)
   at BenchmarkDotNet.Extensions.ReflectionExtensions.<>c.<GetRunnableBenchmarks>b__12_0(Type type)
   at System.Linq.Enumerable.WhereArrayIterator`1.ToArray()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.OrderedEnumerable`1.ToArray()
   at BenchmarkDotNet.Extensions.ReflectionExtensions.GetRunnableBenchmarks(Assembly assembly)
   at BenchmarkDotNet.Running.TypeFilter.GetTypesWithRunnableBenchmarks(IEnumerable`1 types, IEnumerable`1 assemblies, ILogger logger)
   at BenchmarkDotNet.Running.BenchmarkSwitcher.RunWithDirtyAssemblyResolveHelper(String[] args, IConfig config, Boolean askUserForInput)
   at BenchmarkDotNet.Running.BenchmarkSwitcher.Run(String[] args, IConfig config)
   at MicroBenchmarks.Program.Main(String[] args) in /home/helixbot/work/B79109DE/w/AF9E097B/e/performance/src/benchmarks/micro/Program.cs:line 43

Likely related to #87857 .

Author: radical
Assignees: -
Labels:

arch-wasm, blocking-clean-ci, perf-pipeline
Milestone: -

@radical radical added this to the 8.0.0 milestone Jul 7, 2023
@ghost ghost removed the untriaged New issue has not been triaged by the area owner label Jul 7, 2023
@radical
Copy link
Member Author

radical commented Jul 7, 2023

This will be fixed once we have an installer with dotnet/sdk#33841 . But I will investigate to how it is failing for the wasm case, because we use a sdk with updated assemblies for running the benchmarks.

@radical radical added the blocked Issue/PR is blocked on something - see comments label Jul 7, 2023
@ghost
Copy link

ghost commented Jul 10, 2023

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

Issue Details

https://dev.azure.com/dnceng/internal/_build/results?buildId=2216141&view=results

$ pushd "/home/helixbot/work/B79109DE/w/AF9E097B/e/performance/artifacts/bin/for-running/MicroBenchmarks"
$ dotnet exec MicroBenchmarks.dll --anyCategories Libraries Runtime "" --category-exclusion-filter NoInterpreter NoWASM NoMono --wasmEngine /home/helixbot/.jsvu/v8 "--wasmArgs=--expose_wasm --module " --cli /home/helixbot/work/B79109DE/p/dotnet/dotnet --wasmDataDir /home/helixbot/work/B79109DE/p/wasm-data --logBuildOutput --generateBinLog "" --partition-count 30 --partition-index 16 --artifacts /home/helixbot/work/B79109DE/w/AF9E097B/e/artifacts/BenchmarkDotNet.Artifacts --packages /home/helixbot/work/B79109DE/w/AF9E097B/e/performance/artifacts/packages --runtimes wasmnet80 --buildTimeout 1200
Unhandled exception. System.TypeLoadException: Could not load type 'System.Runtime.CompilerServices.NullableContextAttribute' from assembly 'System.Runtime, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
   at System.ModuleHandle.ResolveType(QCallModule module, Int32 typeToken, IntPtr* typeInstArgs, Int32 typeInstCount, IntPtr* methodInstArgs, Int32 methodInstCount, ObjectHandleOnStack type)
   at System.ModuleHandle.ResolveTypeHandle(Int32 typeToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
   at System.Reflection.RuntimeModule.ResolveType(Int32 metadataToken, Type[] genericTypeArguments, Type[] genericMethodArguments)
   at System.Reflection.CustomAttribute.FilterCustomAttributeRecord(MetadataToken caCtorToken, MetadataImport& scope, RuntimeModule decoratedModule, MetadataToken decoratedToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder`1& derivedAttributes, RuntimeType& attributeType, IRuntimeMethodInfo& ctorWithParameters, Boolean& isVarArg)
   at System.Reflection.CustomAttribute.AddCustomAttributes(ListBuilder`1& attributes, RuntimeModule decoratedModule, Int32 decoratedMetadataToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder`1 derivedAttributes)
   at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeMethodInfo method, RuntimeType caType, Boolean inherit)
   at BenchmarkDotNet.Extensions.ReflectionExtensions.<>c.<GetBenchmarks>b__14_0(MethodInfo method)
   at System.Linq.Enumerable.WhereArrayIterator`1.ToArray()
   at BenchmarkDotNet.Extensions.ReflectionExtensions.GetBenchmarks(TypeInfo typeInfo)
   at BenchmarkDotNet.Extensions.ReflectionExtensions.ContainsRunnableBenchmarks(Type type)
   at BenchmarkDotNet.Extensions.ReflectionExtensions.<>c.<GetRunnableBenchmarks>b__12_0(Type type)
   at System.Linq.Enumerable.WhereArrayIterator`1.ToArray()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.OrderedEnumerable`1.ToArray()
   at BenchmarkDotNet.Extensions.ReflectionExtensions.GetRunnableBenchmarks(Assembly assembly)
   at BenchmarkDotNet.Running.TypeFilter.GetTypesWithRunnableBenchmarks(IEnumerable`1 types, IEnumerable`1 assemblies, ILogger logger)
   at BenchmarkDotNet.Running.BenchmarkSwitcher.RunWithDirtyAssemblyResolveHelper(String[] args, IConfig config, Boolean askUserForInput)
   at BenchmarkDotNet.Running.BenchmarkSwitcher.Run(String[] args, IConfig config)
   at MicroBenchmarks.Program.Main(String[] args) in /home/helixbot/work/B79109DE/w/AF9E097B/e/performance/src/benchmarks/micro/Program.cs:line 43

Likely related to #87857 .

Known Issue Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Report

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 0 0
Author: radical
Assignees: radical
Labels:

arch-wasm, area-Infrastructure-libraries, blocked, blocking-clean-ci, perf-pipeline
Milestone: 8.0.0

@radical radical removed blocked Issue/PR is blocked on something - see comments blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' labels Jul 10, 2023
@radical
Copy link
Member Author

radical commented Jul 10, 2023

This is no longer failing.

@radical radical closed this as completed Jul 10, 2023
@ghost ghost locked as resolved and limited conversation to collaborators Aug 13, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@radical radical

Labels
arch-wasm WebAssembly architecture area-Infrastructure-libraries perf-pipeline Issues with dotnet-runtime-perf, or runtime-wasm-perf pipelines
Projects
None yet
Milestone
8.0.0
Development

No branches or pull requests
2 participants
@radical @jeffschwMSFT