Ambiguous reaction precedence rules cause deadlock

[petervdonovan]

See #1707 for a draft of a test that reproduces this. Let's not lose track of @lhstrh's comment that

I just want us to resist the temptation to treat this as just another bug that needs a quick fix.

Basically there are a number of people on the team who are in favor of rethinking the architecture a little bit rather than making minimal patches. This bug is of a sort that justifies careful thought because it is easy to create a partial solution that gets this one test to pass but that might not solve all of the ambiguities.

For the record, I should admit that the fed-gen refactoring, which I was in favor of and which I believed to be basically correct, probably did introduce this problem. Before, when we assigned levels globally, a problem of this kind could not occur. However, I and others are still unsure that global level assignments are the best solution.

[edwardalee]

On this:

I just want us to resist the temptation to treat this as just another bug that needs a quick fix.

Any bug fix should not be just about getting a test to run, of course. There is real risk, however, in treating an identified bug as a reason to reject and redo and entire design. The redesign may not be as easy as it looks, so this strategy could result in a bug persisting for years. Moreover, fixing the bug in the current implementation could lend insight into how to do a redesign.

[petervdonovan]

Thanks for this comment Edward. I still remember your earlier comment that we must make progress, and that a different design with different bugs that makes sense to different people does not constitute progress. I also agree that we shouldn't do a redesign just because we don't understand a current design. But it is also true (I hope) that we have reached more complete collective understanding of the challenges faced by the current design than what we had a year ago.