PyInstaller
-
|
TL;DR, how to setup pyinstaller for the app to be able to run scons as a subprocess? We are trying to convert an open sources project (Pypi 'apio' package) to pyinstaller based installation would appreciate help with the issue are facing. The app is a python command line which invokes an an scons subprocess and works well when installed via pip. To demonstrate the pyinstaller issue we face we created a small test case (see the .zip below) that has similar issues. It is made of main.py which determines the path to the sibling sconstruct.py file and invokes an 'scons' command on that file. This demo works as expected when installed as a pip package ('pip install -e .' in its root directory) but fails when run with pyinstaller installer, not being able to find the 'scons' binary (which is reality is a python script installed by My question is, how do I set pyinstaller to be able to run scons as a subprocess? Standard run (successful. With myapp installed as a pip package with Running under pyinstaller (with pip scons package uninstalled). |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 4 replies
-
|
Ideally, you want no subprocess at all – it's Python code so you should be allowed to just import and run it – although not all packages make that easy (and this one's pretty bad). from Scons.Script.Main import main
sys.argv[1:] = ["-f", arguments, here]
try:
main()
except SystemExit as ex:
if ex.code:
raiseOr if you must use subprocesses, use # At the top of your code
if sys.argv[1:3] == ["-m", "Scons"]:
sys.argv[1:] = sys.argv[3:]
from Scons.Script.Main import main
main()
sys.exit()
# When you want to invoke it
subprocess.run([sys.executable, "-m", "Scons", "-f", ...]) |
Beta Was this translation helpful? Give feedback.
-
|
@bwoodsend, the scons team discourage using it as a library and since it's not a conventional piece of python code, we prefer not to take a risk and and share a process with it. https://stackoverflow.com/questions/79303821/how-to-run-scons-in-the-same-python-process-that-invoked-it I will try your second suggestion, branching to it at the beginning of our main, this seems to be safer. I will report here. |
Beta Was this translation helpful? Give feedback.
-
|
@bwoodsend, I tried your second suggestion. It indeed invoked scons but I am getting a platform error that I don't get when running the test code as a standard python packge. I am running on Mac OSX apple silicon. Updated project zip |
Beta Was this translation helpful? Give feedback.
-
|
If you look at the code where the error originated, you will see that SCons is progammatically loading platform modules. So you will need to add missing modules to # -*- mode: python ; coding: utf-8 -*-
from PyInstaller.utils.hooks import collect_submodules
added_files = [
( '../myapp/sconstruct', '.' ),
]
hiddenimports = collect_submodules('SCons.Platform') # watch out for capitalization, it must match the package name
# alternatively, if you are building only for macOS, you might get away with just:
#hiddenimports = ['SCons.Platform.darwin']
a = Analysis(
['../myapp/main.py'],
pathex=[],
binaries=[],
datas=added_files,
hiddenimports=hiddenimports,
... |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @rokm, that works. I was able to use collect_modules to include all the missing modules. The next problem we face, is calling python code from the sconstruct file. This works when we install the package using pip but fails with pyinstaller. When the parent process calls the child (scons) process it passes to it the path of the sconstruct file so if pyinstaller needs it at a differnet locaiton, we can move it. Also, can add to it .py extension if it helps. The sconstruct file: Error log Updated test app |
Beta Was this translation helpful? Give feedback.
-
|
My first guess would be that since your entry-point script ( |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @rokm, I added @bwoodsend, @rokm, we greatly appreciate your help! |
Beta Was this translation helpful? Give feedback.
subprocess.run(["scons", ...])is invalid even without PyInstaller since there's no guarantee thatsconsis inPATH.Ideally, you want no subprocess at all – it's Python code so you should be allowed to just import and run it – although not all packages make that easy (and this one's pretty bad).
Or if you must use subprocesses, use
sys.executable -m Sconsto ensure Scons if findable without making assumptions about PATH. But in PyInstallersys.executableis your application rather thanpythonso you'll need to mimic python's-moption or your ap…