-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Slow compilation times when inferring type HKT with intersection types #20120
Comments
During the meeting it was mentioned that #19931 might have fixed this issue, however it seems to have no effect on this particular problem - it has no effect when it comes to compilation times |
Intersection types can give a worst case exponential blowup of compilation times. So this is a nice goal, but not an achievable one 😉 |
I think I reached the end of my abilities here, so I will share my findings so hopefully the next person has an easier time.
Things I did to make my life easier (maybe they will be useful):
|
In scala#20120, we reach constraints with equal bounds that are intersection types, they are formed from multiple successive calls to `addOneBound`. We miss the `replace` optimization in this case because the bounds only become equal progressively, and we are only checking for equality with the constraint being added. The second tryReplace after updateEntry and isSub does not address this specific issue but scala#19955.
as an optimization In #20120, we reach constraints with equal bounds that are intersection types, they are formed from multiple successive calls to `addOneBound`. We miss the `replace` optimization in this case because the bounds only become equal progressively, and we are only checking for equality with the constraint being added. Additionally, we recheck for equal bounds after `constraint.updateEntry` as checking `isSub` can have narrowed the bounds further. #19955 is an example where this second optimization applies. Fix #20120 Close #20208 the original implementation
Thanks to changes in #20399 the compilation times has dropped significantly. In case of original project the compilation times presents as following:
Under |
@WojciechMazur I tried to compile the following code with Scala 3.5.0-RC1-bin-20240516-c608177-NIGHTLY and I got a significant raise of compilation time.
object Main {
trait A {}
trait B {}
trait C {}
trait D {}
trait E {}
trait F {}
trait G {}
trait H {}
trait I {}
trait J {}
trait K {}
trait L {}
trait M {}
trait N {}
trait O {}
trait P {}
trait Q {}
trait R {}
trait S {}
trait T {}
trait U {}
trait V {}
trait W {}
trait X {}
trait Y {}
trait Z {}
type AlphabeticServices = A & B & C & D & E & F & G & H & I & J & K & L & M & N & O & P & Q & R & S & T & U & V & W & X & Y & Z
type EnvOutA = B & C & D & E & F & G & H & I & J & K & L & M & N & O & P & Q & R & S & T & U & V & W & X & Y & Z
type EnvOutB = A & C & D & E & F & G & H & I & J & K & L & M & N & O & P & Q & R & S & T & U & V & W & X & Y & Z
type EnvOutC = A & B & D & E & F & G & H & I & J & K & L & M & N & O & P & Q & R & S & T & U & V & W & X & Y & Z
type EnvOutD = A & B & C & E & F & G & H & I & J & K & L & M & N & O & P & Q & R & S & T & U & V & W & X & Y & Z
type EnvOutE = A & B & C & D & F & G & H & I & J & K & L & M & N & O & P & Q & R & S & T & U & V & W & X & Y & Z
type EnvOutF = A & B & C & D & E & G & H & I & J & K & L & M & N & O & P & Q & R & S & T & U & V & W & X & Y & Z
type EnvOutG = A & B & C & D & E & F & H & I & J & K & L & M & N & O & P & Q & R & S & T & U & V & W & X & Y & Z
type EnvOutH = A & B & C & D & E & F & G & I & J & K & L & M & N & O & P & Q & R & S & T & U & V & W & X & Y & Z
type EnvOutI = A & B & C & D & E & F & G & H & J & K & L & M & N & O & P & Q & R & S & T & U & V & W & X & Y & Z
type EnvOutJ = A & B & C & D & E & F & G & H & I & K & L & M & N & O & P & Q & R & S & T & U & V & W & X & Y & Z
type EnvOutK = A & B & C & D & E & F & G & H & I & J & L & M & N & O & P & Q & R & S & T & U & V & W & X & Y & Z
type EnvOutL = A & B & C & D & E & F & G & H & I & J & K & M & N & O & P & Q & R & S & T & U & V & W & X & Y & Z
type EnvOutM = A & B & C & D & E & F & G & H & I & J & K & L & N & O & P & Q & R & S & T & U & V & W & X & Y & Z
type EnvOutN = A & B & C & D & E & F & G & H & I & J & K & L & M & O & P & Q & R & S & T & U & V & W & X & Y & Z
type EnvOutO = A & B & C & D & E & F & G & H & I & J & K & L & M & N & P & Q & R & S & T & U & V & W & X & Y & Z
type EnvOutP = A & B & C & D & E & F & G & H & I & J & K & L & M & N & O & Q & R & S & T & U & V & W & X & Y & Z
type EnvOutQ = A & B & C & D & E & F & G & H & I & J & K & L & M & N & O & P & R & S & T & U & V & W & X & Y & Z
type EnvOutR = A & B & C & D & E & F & G & H & I & J & K & L & M & N & O & P & Q & S & T & U & V & W & X & Y & Z
type EnvOutS = A & B & C & D & E & F & G & H & I & J & K & L & M & N & O & P & Q & R & T & U & V & W & X & Y & Z
type EnvOutT = A & B & C & D & E & F & G & H & I & J & K & L & M & N & O & P & Q & R & S & U & V & W & X & Y & Z
type EnvOutU = A & B & C & D & E & F & G & H & I & J & K & L & M & N & O & P & Q & R & S & T & V & W & X & Y & Z
type EnvOutV = A & B & C & D & E & F & G & H & I & J & K & L & M & N & O & P & Q & R & S & T & U & W & X & Y & Z
type EnvOutW = A & B & C & D & E & F & G & H & I & J & K & L & M & N & O & P & Q & R & S & T & U & V & X & Y & Z
type EnvOutX = A & B & C & D & E & F & G & H & I & J & K & L & M & N & O & P & Q & R & S & T & U & V & W & Y & Z
type EnvOutY = A & B & C & D & E & F & G & H & I & J & K & L & M & N & O & P & Q & R & S & T & U & V & W & X & Z
type EnvOutZ = A & B & C & D & E & F & G & H & I & J & K & L & M & N & O & P & Q & R & S & T & U & V & W & X & Y
trait Reader[-E, A] {
def map[B](f: A => B): Reader[E, B] = ???
def flatMap[E2 <: E, B](f: A => Reader[E2, B]): Reader[E2, B] = ???
}
def e1: Reader[EnvOutA, Unit] = ???
def e2: Reader[EnvOutB, Unit] = ???
def e3: Reader[EnvOutC, Unit] = ???
def e4: Reader[EnvOutD, Unit] = ???
def e5: Reader[EnvOutE, Unit] = ???
def e6: Reader[EnvOutF, Unit] = ???
def e7: Reader[EnvOutG, Unit] = ???
def e8: Reader[EnvOutH, Unit] = ???
def e9: Reader[EnvOutI, Unit] = ???
def e10: Reader[EnvOutJ, Unit] = ???
def e11: Reader[EnvOutK, Unit] = ???
def e12: Reader[EnvOutL, Unit] = ???
def e13: Reader[EnvOutM, Unit] = ???
def e14: Reader[EnvOutN, Unit] = ???
def e15: Reader[EnvOutO, Unit] = ???
def e16: Reader[EnvOutP, Unit] = ???
def e17: Reader[EnvOutQ, Unit] = ???
def e18: Reader[EnvOutR, Unit] = ???
def e19: Reader[EnvOutS, Unit] = ???
def e20: Reader[EnvOutT, Unit] = ???
def e21: Reader[EnvOutU, Unit] = ???
def e22: Reader[EnvOutV, Unit] = ???
def e23: Reader[EnvOutW, Unit] = ???
def e24: Reader[EnvOutX, Unit] = ???
def e25: Reader[EnvOutY, Unit] = ???
def e26: Reader[EnvOutZ, Unit] = ???
def program: Reader[AlphabeticServices, Unit] = for {
//1
_ <- e1
_ <- e2
_ <- e3
_ <- e4
_ <- e5
_ <- e6
_ <- e7
_ <- e8
_ <- e8
_ <- e9
_ <- e10
_ <- e11
_ <- e12
_ <- e13
_ <- e14
_ <- e15
_ <- e16
_ <- e17
_ <- e18
_ <- e19
_ <- e20
_ <- e21
_ <- e22
_ <- e23
_ <- e24
_ <- e25
_ <- e26
//2
_ <- e1
_ <- e2
_ <- e3
_ <- e4
_ <- e5
_ <- e6
_ <- e7
_ <- e8
_ <- e8
_ <- e9
_ <- e10
_ <- e11
_ <- e12
_ <- e13
_ <- e14
_ <- e15
_ <- e16
_ <- e17
_ <- e18
_ <- e19
_ <- e20
_ <- e21
_ <- e22
_ <- e23
_ <- e24
_ <- e25
_ <- e26
//3
_ <- e1
_ <- e2
_ <- e3
_ <- e4
_ <- e5
_ <- e6
_ <- e7
_ <- e8
_ <- e8
_ <- e9
_ <- e10
_ <- e11
_ <- e12
_ <- e13
_ <- e14
_ <- e15
_ <- e16
_ <- e17
_ <- e18
_ <- e19
_ <- e20
_ <- e21
_ <- e22
_ <- e23
_ <- e24
_ <- e25
_ <- e26
//4
_ <- e1
_ <- e2
_ <- e3
_ <- e4
_ <- e5
_ <- e6
_ <- e7
_ <- e8
_ <- e8
_ <- e9
_ <- e10
_ <- e11
_ <- e12
_ <- e13
_ <- e14
_ <- e15
_ <- e16
_ <- e17
_ <- e18
_ <- e19
_ <- e20
_ <- e21
_ <- e22
_ <- e23
_ <- e24
_ <- e25
_ <- e26
} yield ()
} |
as an optimization In #20120, we reach constraints with equal bounds that are intersection types, they are formed from multiple successive calls to `addOneBound`. We miss the `replace` optimization in this case because the bounds only become equal progressively, and we are only checking for equality with the constraint being added. Additionally, we recheck for equal bounds after `constraint.updateEntry` as checking `isSub` can have narrowed the bounds further. #19955 is an example where this second optimization applies. Fix #20120 Close #20208 the original implementation [Cherry-picked c608177]
Based on usage of Tapir in commercial project. Whenever one of the input/output types in the
Endpoint
type is contains an intersection type the compilation type are ~4x longer when compared with the same type using tuple instead of intersection type.In the original project compilation time of single endpoint val could take up to 1s - mostly due to additional implicit search for Codec instances and more complex method signatures. It is a blocker for finishing migration from Scala 2.13 to Scala 3, due to 4x longer compilation times (~90s Scala 2.13, 5 min Scala 3)
Compiler version
Tested only on fork #19897 containing prototype support for -Yprofile-trace. The issue can be observed in original codebase in all Scala 3.3 and 3.4 versions.
Minimized code
To be able to reproduce the compilation times observed in the original project the code cannot be fully minimized. Tapir uses implicits to infer result types for concatenation of input/output parameters, requiring us to include this mechanism in simplified form. Whole code takes ~200 lines of code.
Reordering
in
/out
method calls order can impact compilation times.repro.zip
Output
Compilation times based on -Yprofile-trace in Scala 3:
T0
- no intersection type or tuple type, doing classloading: 130msT1
/T1Bis
- using intersection types - 257ms / 148 msT2
/T2Bis
- using tuple types - 34 ms / 18 msCompilation times based on -Yprofile-trace in Scala 2.13:
T0
- no intersection type or tuple type, doing classloading: 134msT1
/T1Bis
- using intersection types - 48ms / 40 msT2
/T2Bis
- using tuple types - 45 ms / 39 msScala 3.5-nightly -Yprofile-trace (modified to show Apply trees)
The blank spots after
apply x
is probably a time spent in type comparer. Async-profiler flamegraph available belowScala 2.13 -Yprofile-trace
Archive below contains original outputs of -Yprofile-trace and flamegraph generated by async-profiler
profiler-outputs.zip
Type presented in presenation compiler
Expectation
Intersection types should affect compilation times with no more then 20% overhead.
The text was updated successfully, but these errors were encountered: