Fix performance regression for imgmath embedding
#10888
Conversation
Is your project open source? 26 minutes is incredibly long, I'd like to see if there are any other optimisations that can be made here. A |
|
it's not that long considering there are almost 6k math formulas to generate An idea might be generate images in batches to avoid running latex once per image |
|
Also I tried compilation with mathjax, which drops compilation to 14min but validity of formulas is not checked, which is a no-go |
We need the generated image files to be in one common folder to avoid having to re-generate several times the same image (the sha1 naming of the file acts as a cache to just read the depth) The location should be possibly common to all workers, that's why I choosed to just keep the original destination folder, ie the file is just moved from tempdir to the build folder before. This significantly drops the build times from 84min to 26min, but obviously leaves the unused image files in the output folder. Another solution would be to use another global temporary folder shared across all the workers, or to add a delete pass after all workers have joined, but there does not seem to be an api to do that.
jschueller
force-pushed
the
math_embed_v3
branch
from
467c0a2 to
c5fc7cb
Compare
|
Update: I added a callback to cleanup the unused latex images in the /math output folder afterwards |
jschueller
force-pushed
the
math_embed_v3
branch
from
c5fc7cb to
0e298c8
Compare
jschueller
changed the title
jschueller
changed the title
jschueller
force-pushed
the
math_embed_v3
branch
from
0e298c8 to
5ef54b6
Compare
jschueller
force-pushed
the
math_embed_v3
branch
from
5ef54b6 to
ddb826f
Compare
|
@AA-Turner please let me know if I should do anything else, I'd like to submit that for 5.3.0 |
# Conflicts: # sphinx/ext/imgmath.py
|
cool, thanks for taking over |
|
@jschueller if you could sanity check that your builds work with my changes it'd be appreciated! (Broadly I'm aiming to simplify |
|
yes, I just tested it and its ok |
AA-Turner
changed the title
imgmath embedding
We need the generated image files to be in one common folder to avoid having to re-generate several times the same image (the sha1 naming of the file acts as a cache to just read the depth)
The location should be possibly common to all workers, that's why I choosed to just keep the original destination folder, ie the file is just moved from tempdir to the build folder before.
This significantly drops my build time from 84min to 26min, but obviously leaves the unused image files in the output folder.
Another solution would be to use another global temporary folder shared across all the workers, or to add a delete pass after all workers have joined, but there does not seem to be an api to do that.
Bugfix: fix imgmath performance with embed=True
Follows #10816 #10878