Random crashes only with Pyinstaller

[ghost]

Hi!

I have an app where i am using a lot of threading, Project works fine not compiled with Pyinstaller but when using Pyinstaller one of the processes randomly dies without having any errors or expectation logged at random times.

Do anyone have any tips on debugging this and figuring out why this is happening? Ive looked at the memory and its not high at the point where it "would" crash. The crashes are also very random, i cannot always reproduce it.

any help would be appreciated!

[xoviat]

You can try to use program tracing: https://github.com/pyinstaller/pyinstaller/wiki/Program-Tracing (modify it to log output to a file). Keep in mind this will generate a LOT of data (it could be up to 500 MB), and you will need to reduce it to the crash. But it does give you a starting point.

[ghost]

You can try to use program tracing: https://github.com/pyinstaller/pyinstaller/wiki/Program-Tracing (modify it to log output to a file). Keep in mind this will generate a LOT of data (it could be up to 500 MB), and you will need to reduce it to the crash. But it does give you a starting point.

Sounds like a dreadful job! Ill take a look and see if i can use it in my situation! So frustrating that it only happens in Pyinstaller!

[ghost]

Any other recommendations? Is there any types of crash logs or something, this error is not generated by my code its only happening when compiled with pyinstaller

[rokm]

Any other recommendations?

I'm afraid that without a way to reproduce this problem (e.g., by a minimal example that triggers it, even if occassionally) you're pretty much on your own.

[ghost]

Any other recommendations?

I'm afraid that without a way to reproduce this problem (e.g., by a minimal example that triggers it, even if occassionally) you're pretty much on your own.

I can replicate it, but its random. I cannot point to why and where its happening!

I looked at the event logs on the machine and see that theres some "stack overflow" exceptions but nothing more than that

[rokm]

What packages are you using in your application?

[ghost]

What packages are you using in your application?

Pretty long list, but it seems like the overflow is happening with threading and PyZMQ! I have a bunch threads that is waiting on responses from Queues.

[ghost]

Just got the crash while running it with https://docs.python.org/3/library/faulthandler.html

I got a traceback on the recent calls + "Windows Fatal exception: stack overflow"

[rokm]

Anything interesting in the traceback?

Also, I take it you are taking precautions against potential thread-safety issues in your use of PyZMQ?

[ghost]

Anything interesting in the traceback?

Also, I take it you are taking precautions against potential thread-safety issues in your use of PyZMQ?

The ZMQ thread is separate from everything and only grabs information from the queues sends it off and then receives and puts back the information it received into the queues.

[ghost]

Anything interesting in the traceback?

Also, I take it you are taking precautions against potential thread-safety issues in your use of PyZMQ?

Only interesting thing in the traceback is "Stack overflow" the most recent functions are all on the functions where its doing a .get() on the queues

[rokm]

Hmm, well, based on the symptoms, I'm still leaning towards this being some sort of concurrency issue...

While it could also be a PyInstaller-related bug, or perhaps a subtle behavior difference that exacerbates the issue, I don't think we'll be able to figure it out without a reproducer.

[ghost]

I see, thats very unfortunate. Ill stick to it and see if i can figure it out!

Thanks