How to profile a PyInstaller application

[olumide-x]

How would a PyInstaller (one-dir) application be profiled? Its not truly a native application so standard profilers won't work, and as there are no sources there's no obvious way (to me at least) to load the cProfile module. e.g. python -m cProfile dir/app.py.

[bwoodsend]

You could use it's Python API although why can't you profile your original Python code instead?

[olumide-x]

I am familiar with the API technique(s) but as you noted it requires the sources.

The problem is that we notice a slowdown post packaging and obfuscation (I suspect the overhead of deobfuscation on the fly) but I need to prove it.

Is there a way to build a PyInstaller executable that to auto-loads the cProfiler?

[rokm]

Is there a way to build a PyInstaller executable that to auto-loads the cProfiler?

There is not (if we're talking out-of-the-box). Everything considered, why would there be?

The problem is that we notice a slowdown post packaging and obfuscation (I suspect the overhead of deobfuscation on the fly) but I need to prove it.

Surely the first test then should be to disable obfuscation, and determine whether the culprit is packaging itself or (de)obfuscation? (For example, to ensure that you are not running into something like #8211)

[olumide-x]

I'll consider that approach but I think it should be possible to profile an application at any stage of development or deployment.

Anyway, thanks for PyInstaller and your excellent support.

[rokm]

Does cProfile really require the sources, though? Looking at its main() implementation, it looks like you should be able to run it against an arbitrary module.

So if your entry-point is called program.py, create another entry-point, program_profile.py:

# program_profile.py
from cProfile import runctx
import runpy

code = "run_module(modname, run_name='__main__')"
globs = {
    'run_module': runpy.run_module,
    'modname': "program"
}

try:
    runctx(
        code,
        globs,
        None,
        None, # outfile
        2, # sort
    )
except BrokenPipeError as exc:
    # Prevent "Exception ignored" during interpreter shutdown.
    sys.stdout = None
    sys.exit(exc.errno)

And then build with pyinstaller --clean --noconfirm program_profile.py --hiddenimport program

(That's assuming multiprocessing is not in use).

[fenchu]

I've added timing to my programs and apart from the extraction into the tempdir and the cleanup when it stops it is very much alike. Which is should it is the same interpreter just in another directory