Only work-steal in the main loop

[Zoxc]

This PR removes work stealing from other threads, except when in the main loop. This avoids situations where we steal work we can't complete due to waiting on queries. This should fix bugs like rust-lang/rust#111521 and rust-lang/rust#115223, but it does have a negative performance impact.

Tests for 6 threads:

Benchmark	Before	After
	Time	Time	%
🟣 clap:check	0.6502s	0.7641s	💔 17.52%
🟣 hyper:check	0.1396s	0.1508s	💔 8.00%
🟣 regex:check	0.4033s	0.4413s	💔 9.43%
🟣 syn:check	0.6749s	0.7749s	💔 14.81%
🟣 syntex_syntax:check	2.1708s	2.4696s	💔 13.76%
Total	4.0389s	4.6007s	💔 13.91%
Summary	1.0000s	1.1270s	💔 12.70%

Benchmark	Before	After
	Time	Time	%
🟠 clap:debug	2.3400s	2.4645s	💔 5.32%
🟠 hyper:debug	0.3807s	0.3919s	💔 2.96%
🟠 regex:debug	1.7557s	1.7719s	0.92%
🟠 syn:debug	1.7401s	1.8426s	💔 5.89%
🟠 syntex_syntax:debug	6.6743s	6.9144s	💔 3.60%
Total	12.8908s	13.3854s	💔 3.84%
Summary	1.0000s	1.0374s	💔 3.74%

[SparrowLii]

The regression looks regrettable, and it will probably prevent us from landing parallel rustc.

I would like us to decide whether to work-stealing by querying dependencies instead of prohibiting it in inner loops.

I hope we can left this change and try other implementations first.

[Zoxc]

By increasing the thread count we can increase the work stealing done and thus reduce the regression:

Benchmark	Before PR (6 threads ,-j6)	After PR (12 threads ,-j6)
	Time	Time	%
🟣 clap:check	0.6556s	0.6647s	💔 1.38%
🟣 hyper:check	0.1485s	0.1519s	💔 2.32%
🟣 regex:check	0.4050s	0.4126s	💔 1.89%
🟣 syn:check	0.6795s	0.6895s	💔 1.47%
🟣 syntex_syntax:check	2.1858s	2.2038s	0.82%
Total	4.0743s	4.1224s	💔 1.18%
Summary	1.0000s	1.0158s	💔 1.58%

[Zoxc]

@cuviper Would you be up for reviewing this?

[zetanumbers]

Hello! Do you have any tips on currently triggering those bugs?

[cuviper]

I haven't had time to review this in depth -- but seeing how many deep areas it needs to change, along with existing patches diverging from rayon, I'm starting to think that we would be better off with a bespoke rustc_threadpool that doesn't try to be a modified rayon. Most of rayon's API is not used at all in rustc #[cfg(parallel_compiler)], and I think a lot of rustc-rayon's customization would be easier under a more limited API.

[zetanumbers]

I haven't had time to review this in depth -- but seeing how many deep areas it needs to change, along with existing patches diverging from rayon, I'm starting to think that we would be better off with a bespoke rustc_threadpool that doesn't try to be a modified rayon. Most of rayon's API is not used at all in rustc #[cfg(parallel_compiler)], and I think a lot of rustc-rayon's customization would be easier under a more limited API.

I am currently working on this and other parallelization issues.

[zetanumbers]

By increasing the thread count we can increase the work stealing done and thus reduce the regression

@Zoxc wouldn't that mess up with the jobserver? It raises the limit of number cores used in parallel.

[Zoxc]

I'm starting to think that we would be better off with a bespoke rustc_threadpool that doesn't try to be a modified rayon.

Yeah, this does seem to imply a move away from tracking Rayon. This approach doesn't match well with Rayon iterators and we may want to introduce fibers too. I'd like to see some more extensive testing to verify this solves the deadlock issues before moving further in this direction though.

@Zoxc wouldn't that mess up with the jobserver? It raises the limit of number cores used in parallel.

Not sure what you mean here? The jobserver would still limit active threads across cores as before.