Cargo check runs forever and memory usage grows unboundedly

[oliverb123]

Apologies, I really couldn't find a way to minimise this example, but this commit causes cargo check to run forever and consume an ever-growing amount of memory. I tried to narrow down what was causing the problem, but just pulling the various trait definitions out into another crate didn't reproduce it. I think the issue is related to (almost certainly) poorly-defined associated type bounds in the various traits under src/proxy/routes, as it seemed to start happening after I started adding those, but I'm really not very confident in that assessment.

I expected to see this happen: cargo check would run in a bounded amount of space and time

Instead, this happened: cargo check seems to run forever and consume all of my memory

I can confirm the issue occurs on both nightly and stable, with versions:

rustc --version --verbose:

rustc 1.73.0 (cc66ad468 2023-10-03)
binary: rustc
commit-hash: cc66ad468955717ab92600c770da8c1601a4ff33
commit-date: 2023-10-03
host: x86_64-unknown-linux-gnu
release: 1.73.0
LLVM version: 17.0.2

rustc --version --verbose:

rustc 1.75.0-nightly (09df6108c 2023-10-17)
binary: rustc
commit-hash: 09df6108c84fdec400043d99d9ee232336fd5a9f
commit-date: 2023-10-17
host: x86_64-unknown-linux-gnu
release: 1.75.0-nightly
LLVM version: 17.0.2

I can't include a backtrace because, naturally, the compiler doesn't exit, so setting RUST_BACKTRACE=1 has no effect

This is a screenshot of rustc consuming 23GB of ram while compiling the linked commit. I don't know why I thought this would be useful, but I guess I did, so here it is.

[oliverb123]

Good news! this commit has no dependencies and still reproduces the issue. cloc reports this as ~360 lines of code now, which is kind of small (not counting the example files, which I could have deleted, they're not relevant to the failure afaict), although I understand it's still not ideal, apologies again.

[basil-cow]

trait Filter {
    type ToMatch;
}

impl<T> Filter for T
where
    T: Fn(Self::ToMatch),
{
}

trait Rule {}

impl<T> Rule for T
where
    T: Filter,
{
}

struct JustFilter<F: Filter> {
    filter: F,
}

impl<T: Filter> Rule for JustFilter<T> {}

minimal repro, if you delete everything with Rule it correctly determines that the Filter impl overflows trying to check if T already implements Filter. interestingly enough, you seemingly cant change Fn into another trait that depends on Self::ToMatch, if i try to change the constraint into T: Trait<Self::ToMatch> rustc finds the overflow

[saethlin]

Some quick poking with a debugger suggests that the compiler is endlessly emitting diagnostics.

If this is a regression it isn't recent. This hangs on builds back to 1.60.

[estebank]

Running the repro with RUSTC_LOG=info I can see that calls to rustc_trait_selection::traits::project::project and rustc_trait_selection::traits::project::normalize_with_depth_to never go beyond depth=129.

I did a quick and dirty test to check for normalize_with_depth.*\( and recursion_depth and add + 1 to every call that didn't have it. Doing so did cause the compilation to stop, but it was too indiscriminate so it would also break existing tests.

[erickt]

I think we're running into this on Fuchsia, which may take a long time for us to make a reproduction, but I can try to explain what we're seeing. We're on rustc 1.75.0-nightly (9d1e4b787 2023-10-11). Normally cargo build --release can complete in a few seconds, but if we insert a simple bug, such as adding a HashMap to a struct but forgetting to add use std::collections::HashMap), the compiler will emit the "did you mean to import ...?", but then it'll fall into an apparent infinite loop around rustc_trait_selection::traits::project::normalize_with_depth_to. I've seen a build go past 30 minutes before I've killed it.

This sounds pretty analogous to a exponential expansion bug that was fixed in #114948. Perhaps diagnostics also needs to add something similar do avoid the same thing?

[lqd]

Not that it matters much but this looks quite old: it used to trigger a solving overflow but seems to have started infinite looping in nightly-2021-09-21. I seem to recall #89125 from this nightly being involved in some other compilation time issues.

[Zannick]

Does turning overflow-checks off solve this issue?

[profile.dev]
overflow-checks: false

I ask because I want to see if it's the same as my issue: I've been having a similar problem with cargo build only in dev profile spiking very sharply to 24 GiB during the build, and after trying some of the differences in profiles, I can now safely build and check in dev without a memory explosion, just by turning off overflow-checks. (More details)

[lqd]

@Zannick You're probably seeing a different issue. Here, the infinite loop is related to error diagnostics and trait solving.

[kpreid]

@rustbot label -E-needs-mcve

Triage: #116914 (comment) is a MCVE for this

[estebank]

I wonder if an early error checking for these unconditionally recursive definitions might be worth it while we track down the right place to fix this correctly.