Precompilation infinite loop on Windows

[BenChung]

Our package SCPToolbox.jl seems to cause precompilation to get stuck on Windows using 1.10-beta1 installed from the official binary installer (it works fine on a Debian system or when using 1.9 on Windows). When starting from an empty .julia folder, inputting

pkg> dev [email protected]:BenChung/SCPToolbox.jl.git
> using SCPToolbox

to the REPL causes the precompile process to get stuck with

Precompiling SCPToolbox
  Progress [>                                        ]  0/1
  ◓ SCPToolbox Waiting for background task / IO / timer. Interrupt to inspect

interrupting it causes

  1 dependency had warnings during precompilation:
┌ SCPToolbox [bd2bc758-a5dc-11eb-050f-6b2434548817]
│  [pid 25364] waiting for IO to finish:
│   TYPE[FD/PID]       @UV_HANDLE_T->DATA
│   timer              @0000021147a274f0->0000021152c123b0
│
│  [pid 25364] waiting for IO to finish:
│   TYPE[FD/PID]       @UV_HANDLE_T->DATA
│   timer              @0000021147a274f0->0000021152c123b0
│
│  [pid 25364] waiting for IO to finish:
│   TYPE[FD/PID]       @UV_HANDLE_T->DATA
│   timer              @0000021147a274f0->0000021152c123b0
│
│  [pid 25364] waiting for IO to finish:
│   TYPE[FD/PID]       @UV_HANDLE_T->DATA
│   timer              @0000021147a274f0->0000021152c123b0
│
│  [pid 25364] waiting for IO to finish:
│   TYPE[FD/PID]       @UV_HANDLE_T->DATA
│   timer              @0000021147a274f0->0000021152c123b0
│
│  [pid 25364] waiting for IO to finish:
│   TYPE[FD/PID]       @UV_HANDLE_T->DATA
│   timer              @0000021147a274f0->0000021152c123b0
│
│  [pid 25364] waiting for IO to finish:
│   TYPE[FD/PID]       @UV_HANDLE_T->DATA
│   timer              @0000021147a274f0->0000021152c123b0
│
│  [pid 25364] waiting for IO to finish:
│   TYPE[FD/PID]       @UV_HANDLE_T->DATA
│   timer              @0000021147a274f0->0000021152c123b0
│
│  [pid 25364] waiting for IO to finish:
│   TYPE[FD/PID]       @UV_HANDLE_T->DATA
│   timer              @0000021147a274f0->0000021152c123b0
└
[ Info: Waiting for another process (pid: 13848) to finish precompiling SCPToolbox [bd2bc758-a5dc-11eb-050f-6b2434548817]

The named process (13848) exists as a Julia process, but seems to be dormant. It's not immediately clear to me what's happening to cause this, particularly since it works on Linux.

[visr]

I've come across the same error several times, not always infinite loop though, often just a warning during precompilation. I tried to reduce it a bit, and see this when I add LinearMaps ChainRulesCore to an empty environment. This will trigger precompiling LinearMapsChainRulesCoreExt and LinearMapsSparseArraysExt, and then gives a warning on LinearMapsChainRulesCoreExt. If I delete .julia\compiled\v1.10\LinearMapsChainRulesCoreExt\ and run precompile it works fine, but if I delete both that and .julia\compiled\v1.10\LinearMapsSparseArraysExt\ and run precompile the issue occurs.

Looking at the LinearMaps and ChainRulesCore Project.toml files, this seems related to packages extensions. SparseArrays is both a strong and weak dependency of LinearMaps, and is a strong dependency of ChainRulesCore, which is a weakdep of LinearMaps.

(linmaps) pkg> add LinearMaps ChainRulesCore
Precompiling project...
  Progress [====================>                    ]  1/2
  ✓ LinearMaps → LinearMapsSparseArraysExt
  ◒ LinearMaps → LinearMapsChainRulesCoreExt Waiting for background task / IO / timer. Interrupt to inspect
 Interrupted: Exiting precompilation...

  1 dependency had warnings during precompilation:
┌ LinearMapsChainRulesCoreExt [80cc3b07-7966-5fa1-b050-64b4b3e87554]
│  [pid 14172] waiting for IO to finish:
│   TYPE[FD/PID]       @UV_HANDLE_T->DATA
│   timer              @00000126918ff4d0->0000012694495de0
└

(linmaps) pkg> st
Status `D:\temp\linmaps\Project.toml`
  [d360d2e6] ChainRulesCore v1.16.0
  [7a12625a] LinearMaps v3.11.0

julia> versioninfo()
Julia Version 1.10.0-beta1
Commit 6616549950 (2023-07-25 17:43 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: 24 × 12th Gen Intel(R) Core(TM) i7-12800HX
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, alderlake)
  Threads: 23 on 24 virtual cores
Environment:
  JULIA_EDITOR = code
  JULIA_NUM_THREADS = 16
  JULIA_PKG_PRESERVE_TIERED_INSTALLED = true
  JULIA_PKG_USE_CLI_GIT = true

[visr]

I can also reproduce on julia master (2c45e3b), and the issue goes away when turning off parallel precompilation.
It looks like #50757 is a duplicate of this issue (edit: maybe not), and possibly #50746 is related as well.

julia> Pkg.precompile(;strict=true, timing=true)
Precompiling project...
    817.3 ms  ✓ LinearMaps → LinearMapsSparseArraysExt
LinearMapsChainRulesCoreExt Waiting for background task / IO / timer.
[pid 24160] waiting for IO to finish:
 TYPE[FD/PID]       @UV_HANDLE_T->DATA
 timer              @000001ab6a744a50->000001ab68887190

  10976.2 ms  ✓ LinearMaps → LinearMapsChainRulesCoreExt
  2 dependencies successfully precompiled in 11 seconds. 10 already precompiled.
  1 dependency had warnings during precompilation:
┌ LinearMapsChainRulesCoreExt [80cc3b07-7966-5fa1-b050-64b4b3e87554]
│  [pid 24160] waiting for IO to finish:
│   TYPE[FD/PID]       @UV_HANDLE_T->DATA
│   timer              @000001ab6a744a50->000001ab68887190
└

# now remove both compiled extensions and try again with only one task
julia> ENV["JULIA_NUM_PRECOMPILE_TASKS"]=1

julia> Pkg.precompile(;strict=true, timing=true)
Precompiling project...
    749.8 ms  ✓ LinearMaps → LinearMapsSparseArraysExt
    580.5 ms  ✓ LinearMaps → LinearMapsChainRulesCoreExt
  2 dependencies successfully precompiled in 1 seconds. 10 already precompiled.

[Liozou]

Hi there! This bug appears to be due to a bug in the PyPlot dependency of SCPToolbox (see JuliaPy/PyPlot.jl#572) which was fixed in the meantime. It seems to be working locally, can you check on your version after updating your packages?