Fixed docker support for the Python target

[cmnrd]

This PR implements several fixes:

• proper quoting of cmake arguments so that they work in the build image
• enabling Python docker tests
• ensuring that the docker generation runs after the Python code generation
• moving failing federated docker tests to a failing directory

Details

I noticed that the docker generation for Python is broken, producing errors like the following:

/lingua-franca/PingPongContainerized/src-gen/lib/pythontarget.c: In function ‘py_source_directory’:
#10 3.738 <command-line>: error: expected expression before ‘/’ token
#10 3.738 /lingua-franca/PingPongContainerized/src-gen/lib/pythontarget.c:171:36: note: in expansion of macro ‘LF_SOURCE_DIRECTORY’
#10 3.738   171 |   return PyUnicode_DecodeFSDefault(LF_SOURCE_DIRECTORY);
#10 3.738       |                                    ^~~~~~~~~~~~~~~~~~~
#10 3.738 /lingua-franca/PingPongContainerized/src-gen/lib/pythontarget.c: In function ‘py_package_directory’:
#10 3.738 <command-line>: error: expected expression before ‘/’ token
#10 3.738 /lingua-franca/PingPongContainerized/src-gen/lib/pythontarget.c:181:36: note: in expansion of macro ‘LF_PACKAGE_DIRECTORY’
#10 3.738   181 |   return PyUnicode_DecodeFSDefault(LF_PACKAGE_DIRECTORY);
#10 3.738       |                                    ^~~~~~~~~~~~~~~~~~~~
#10 3.743 make[2]: *** [CMakeFiles/LinguaFrancaPingPongContainerized.dir/build.make:146: CMakeFiles/LinguaFrancaPingPongContainerized.dir/lib/pythontarget.c.o] Error 1
#10 3.743 make[1]: *** [CMakeFiles/Makefile2:106: CMakeFiles/LinguaFrancaPingPongContainerized.dir/all] Error 2

This is related to the way we quote arguments that we pass to cmake. For some reason, the quoting behaves different when building for Python than when building for C. I assume this is because a different shell is used in the different base images. The implemented fix makes the generated quoting more robust.

While debugging this, I also noticed that the Python docker tests were disabled. So we did not notice the problems before... It looks like this was done because of another issue: #1564. However, the issue is not very descriptive, and I don't understand what it is about. @lhstrh Perhaps you can comment on whether we can consider the issue fixed?

After enabling the tests again, I noticed a fundamental flaw in the Python docker generation. It was completely relying on the C generator, which gets invoked by the Python generator before the Python code is generated. Thus, the docker image was incomplete, as it was created before any Python files were generated.

Finally, I had to mark two of the federated tests as failing. I don't know why they are failing and will open another issue.

[elgeeko1]

python:3.10-slim derives from Debian bookworm and the C image derives from Alpine. Debian will use bash as its shell, alpine will use POSIX Dash. However I'm surprised this would make a difference on something like quoting where the two shells should be interchangeable.

I think it would be a good idea to have the Python docker images use the same parent image as our C image, so that they have the same shell and package manager. python:3.10-alpine is an official image for Python 3.10 that is based on Alpine, so this would accomplish this.

[cmnrd]

python:3.10-slim derives from Debian bookworm and the C image derives from Alpine. Debian will use bash as its shell, alpine will use POSIX Dash. However I'm surprised this would make a difference on something like quoting where the two shells should be interchangeable.

I was surprised by this too. So basically the C and the Python dockerfile have similar content. In parrticular, the cmake arguments that define path C macros look like this in both docker files: -DLF_SOURCE_DIRECTORY="/some/path". However, when building the image, docker seems to convert this and add quoting. So when building for C, it actually passes '-DLF_SOURCE_DIRECTORY=/some/path' to cmake. In Python, it passes -DLF_SOURCE_DIRECTORY="/some/path". For some reason, the latter defines LF_SOURCE_DIRECTORY to /some/path, which lacks the quotes and is thus not a string in C.

I don't know what is causing the difference between C and Python.

I think it would be a good idea to have the Python docker images use the same parent image as our C image, so that they have the same shell and package manager. python:3.10-alpine is an official image for Python 3.10 that is based on Alpine, so this would accomplish this.

I agree, but I think we should treat using an alpine base image as a separate issue.

[lhstrh]

No idea why the federated tests are failing but please create an issue for them if you haven't already.

[lhstrh]

Actually, let's repurpose #1564 to track the failing tests, because it's neither clear that this PR fixes #1564 nor whether the "new" failures are related to the ones observed when I created #1564.