Make CheckUnused not slow. #20321
Merged
Make CheckUnused not slow. #20321
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sjrd
force-pushed
the
make-check-unused-not-slow
branch
5 times, most recently
from
933d8fa to
9ad7463
Compare
It is used for every single tree in `CheckUnused`, so this is worth it.
It is not efficient when the results are always used exactly once.
`Tree`s have structural equality. Even if `==` should be able to exit quickly either because of `eq` or an early difference, sets systematically call `hashCode`, which is going to recurse into the entire structure.
It is pointless to sort a list before converting it into a Set.
sjrd
force-pushed
the
make-check-unused-not-slow
branch
from
9ad7463 to
6587ab4
Compare
|
Sure, I will review this today. |
KacperFKorban
approved these changes
May 6, 2024
| val selector = selData.selector | ||
|
|
||
| if !selector.isWildcard then | ||
| if !altName.forall(explicitName => selector.rename == explicitName.toTermName) then |
There was a problem hiding this comment.
Can this be changed to altName.exists(explicitName => selector.rename != explicitName.toTermName)? (It would read like the comment then)
| // If the symbol is accessible in this scope without an import, do not register it for unused import analysis | ||
| val notForImport1 = | ||
| notForImport | ||
| || (!name.exists(_.toTermName != sym.name.toTermName) && sym.isAccessibleAsIdent) |
There was a problem hiding this comment.
Similarly to isInImport: can this first part be name.forall(_.toTermName == sym.name.toTermName)? (Less negation should make it simpler)
| if !isConstructorOfSynth(sym) && !doNotRegister(sym) then | ||
| if sym.isConstructor && sym.exists then | ||
| registerUsed(sym.owner, None) // constructor are "implicitly" imported with the class | ||
| def registerUsed(sym: Symbol, name: Option[Name], notForImport: Boolean = false, isDerived: Boolean = false)(using Context): Unit = |
There was a problem hiding this comment.
I find the logic harder to follow because of the not in notForImport, can we flip the value of this flag to make it mean "possibly form an import"?
Instead of dealing with entire `tpd.Import`s at the end of the scope, we eagerly flatten them into individual `ImportSelector`s. We store them along with some data, including a mutable flag for whether a selector has been used. This allows to dramatically simplify `isInImport`, as well as more aggressively cache the resolution of selectors. We also get rid of the `IdentityHashMap`. The algorithm is still O(n*m) where n is the number of imports in a scope, and m the number of references found in that scope. It is not entirely clear to me whether the previous logic was already O(n*m) or worse (it may have included an additional p factor for the number of possible selections from a given qualifier). Regardless, it is already quite a bit faster than before, thanks to smaller constant factors.
That test does not rely on any information dependent on the import selectors. It only relies on information at the use site. Eagerly checking it means we do not put as many symbols into the `usedInScope` set, which is good because it is one of the complexity factors of the unused-imports analysis.
sjrd
force-pushed
the
make-check-unused-not-slow
branch
from
6587ab4 to
8553bfc
Compare
noti0na1
approved these changes
May 7, 2024
|
@sjrd thank you very much for your work on this. You're going to make a lot of people happy with this! Out of curiosity, do you happen to know what release will these changes be included in? |
|
3.5.0-RC1 |
WojciechMazur
added a commit
that referenced
this pull request
Jul 6, 2024
Backports #20321 to the LTS branch. PR submitted by the release tooling. [skip ci]
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
It doesn't mean that it's fast yet, but it is already a significant step in that direction. In particular, this goes in the direction of addressing #19671.
The most important commit is "Simplify the logic for checking unused imports.", whose commit message follows:
Instead of dealing with entire
tpd.Imports at the end of the scope, we eagerly flatten them into individualImportSelectors. We store them along with some data, including a mutable flag for whether a selector has been used.This allows to dramatically simplify
isInImport, as well as more aggressively cache the resolution of selectors. We also get rid of theIdentityHashMap.The algorithm is still
O(n*m)where n is the number of imports in a scope, and m the number of references found in that scope. It is not entirely clear to me whether the previous logic was alreadyO(n*m)or worse (it may have included an additionalpfactor for the number of possible selections from a given qualifier).Regardless, it is already quite a bit faster than before, thanks to smaller constant factors.