How to speed up the build process?

[Thoufak]

As my project's code grew (both my own code and the included libraries) the build process has steadily slowed down. What are some measures I can take to speed it up? It currently takes 2-3 mins.

I'm using a one-file bundle. Pyinstaller 4.9.
If I need to provide more info or perform some tests, please let me know.

[rokm]

How large is the application bundle (if you build in onedir mode, or if you build in onefile and check the size of temporary _MEIXXXXXX directory at runtime)? Because if there are a lot of data and shared libs to compress, the onefile build process is expected to take a while just due to having to compress everything...

Therefore, the measures you can take are primarily aimed at reducing the application size:

1. always build in a clean environment with minimal required packages installed
2. do not use anaconda python in your build environment
3. explicitly exclude unused optional dependencies to prevent them from being collected (to prevent collection of their data, shared lib dependencies, and dependencies on other python packages)
4. go over collected resources to see if they are any that you are 100% sure you don't need (e.g., plugins, translation files, etc.). Then remove these by filtering Analysis.binaries and Analysis.datas in the spec file (i.e., after Analysis is instantiated and before its members are passed to PYZ, EXE, etc.).

[Thoufak]

Why does №1 speed up the build process though? I'd think that unused packages (that won't be packed into exe) don't slow down the build.

[rokm]

Why does №1 speed up the build process though? I'd think that unused packages (that won't be packed into exe) don't slow down the build.

If the package is really unused (not imported anywhere and thus does not end up in module graph), then it does not affect the build.

But often there's conditional imports (imports within try/catch blocks or even imports within if/else blocks), and those do end up in module graph, and are therefore analyzed and collected. And with the python ecosystem being what it is, you often end up with long import chains from some obscure sub-module of a package that you are using, which ends up pulling in large part of your environment.

[rokm]

Also, prior to #7066, hooks were ran for packages that appeared in module graph but would be later excluded, which also slowed down the build. See #7066 (comment).

[SaadZahem]

Why shouldn't I use anaconda though. I ran into a similar issue where pyinstaller bundled all globally installed packages despite using anaconda environment.

[rokm]

Why shouldn't I use anaconda though.

In the context of initial question, because at the time of writing (= PyInstaller 4.9), PyInstaller did not preserve symbolic links, resulting in drastically bloated builds on macOS and Linux when Anaconda-packaged numpy was used. That particular issue should be fixed with v6.