Removal of "No CMake" build option #1299

petervdonovan · 2022-07-20T02:13:16Z

This branch deletes over 500 LOC and breaks the "no CMake" build option that directly invokes the C compiler. The corresponding PRs in reactor-c and reactor-c-py should be merged first.

It's a bit of a work-in-progress, but I wish to rush this into master to mitigate conflicts. I promise to help resolve any conflicts that result in the fed-gen branch.

The #line directives make debugging impossible and obfuscate the generated code. I should re-enable them before merging to main.

Just to see what would happen.

We have informally discussed setting the margins in this way. I decided not to wait any longer for the opportune moment to make this change.

Do not use a C++ compiler for the runtime. Only use a C++ compiler for the generated code.

This is a work in progress. In particular, setup.py is broken.

This removes interaction with Python package management systems such as Pip from the LF build process. We do not need to install the C extension module in the user's environment because it is sufficient to have a naked .so or .dll sitting in the `src-gen` directory.

This includes an attempt to pass federated and docker tests.

This also keeps the file names that existed in master in org.lflang.generator.c and org.lflang.generator.python.

@arengarajan99

This follows up on a conversation with @arengarajan99 and Marten about the Arduino tests. The rationale is that soon, our Arduino support will be entirely different anyway, and that we can tolerate a temporary regression in the interim as a shortcut to save person-hours. Obviously I take responsibility if we have to scramble to get support back in time for a release.

This is a hack! The fact that it is needed is a reflection of the fact that the changes introduced by this PR leave the build system sensitive to idiosyncrasies in the way Python is installed and discovered on the system.

cmnrd · 2022-10-17T08:57:01Z

In this case it should be helpful to get the verbose output of cmake and compare the executed commands in CI to a local run.

petervdonovan · 2022-10-17T21:08:30Z

I did as Christian suggested and can confirm that CMake is discovers a different version of Python than what is discovered by which python. To be specific, invocations of gcc include the option -isystem /usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.10/include/python3.10 whereas when I SSH into the runner I see the following:

Mac-1666038353816:lingua-franca runner$ which python3
/usr/local/bin/python3
Mac-1666038353816:lingua-franca runner$ ls -al /usr/local/bin/python3
lrwxr-xr-x  1 runner  admin  40 Oct 11 00:31 /usr/local/bin/python3 -> ../Cellar/[email protected]/3.10.7/bin/python

Basically, one Python is in the Cellar, and the other is not.

There are ways to fix this, e.g., by explicitly specifying the path to Python when we invoke CMake.; furthermore, I know where in the code base this change would have to be made. However, there are a few reasons to stop here:

Every build environment is different, and an end user can always find ways to break their build by setting up their environment in special ways. Arguably, whether this is a bug in the build system or in the environment is a matter of perspective.
It is possible to work around this issue by using a virtual environment (which most Python programmers should do anyway).
This bug has never been observed except in CI, and it could be very rare.
It is not clear to me that other ways of integrating Python with CMake are recommended over the one I used; therefore, it is not obvious to me that fixing this bug would reduce the total number of bugs that we have.

lhstrh

This looks great! Congrats on finally getting this PR to pass the tests!! 🎆 I left some comments/requests. I wonder how much trouble these changes will give us in fed-gen... Perhaps you could assist with the merge? I haven't reviewed the changes in reactor-c yet. I will do that next.

org.lflang/src/org/lflang/ASTUtils.java

org.lflang/src/org/lflang/TargetProperty.java

org.lflang/src/org/lflang/federated/launcher/FedCLauncher.java

org.lflang/src/org/lflang/generator/c/CCompiler.java

org.lflang/src/org/lflang/generator/c/CGenerator.java

org.lflang/src/org/lflang/util/FileUtil.java

lhstrh · 2022-10-18T05:54:53Z

Looks like there is no accompanying PR for reactor-c-py?

Co-authored-by: Marten Lohstroh <[email protected]>

petervdonovan added 7 commits July 21, 2022 23:51

Update .gitignore.

f8da40e

Disable #line directives.

5a72323

The #line directives make debugging impossible and obfuscate the generated code. I should re-enable them before merging to main.

Delete some copyCoreFiles-related code.

58d2402

Just to see what would happen.

Delete code related to "no CMake" option.

e905019

Delete more "no CMake" code; address warnings.

24812d2

Update IntelliJ settings.

6c863d0

We have informally discussed setting the margins in this way. I decided not to wait any longer for the opportune moment to make this change.

Delete more "no CMake"-related code.

629e6f7

petervdonovan force-pushed the c-refactoring2 branch from 94a232c to fa5a530 Compare July 22, 2022 20:12

At least one test compiles.

cfe94ff

petervdonovan force-pushed the c-refactoring2 branch from fa5a530 to cfe94ff Compare July 22, 2022 20:31

petervdonovan added 2 commits July 22, 2022 19:05

Try again to compile a basic test.

d2eb15e

Update submodule.

1952c8f

petervdonovan force-pushed the c-refactoring2 branch from 350d199 to 1952c8f Compare July 23, 2022 02:39

petervdonovan added 17 commits July 23, 2022 23:28

Create reactor_common.h.

3960711

[c] Roll back a change.

d11b1b7

[c] Unit tests, ActionIsPresent compile again.

3030079

[c] Get CCpp to build again.

84fc226

Do not use a C++ compiler for the runtime. Only use a C++ compiler for the generated code.

Update C submodule.

d0eed31

Merge branch 'master' into c-refactoring2

21da5f9

Remove references to deprecated print functions.

7d03743

Update submodule ref.

a55f623

[build] Attempt to get Python target to compile.

43f5e85

This is a work in progress. In particular, setup.py is broken.

Failed attempt to build using scikit-build.

007dd70

[build] Address C, Python CI failures.

e671cd2

This includes an attempt to pass federated and docker tests.

[build] Minor fixes.

b60ae04

[build] Update submodule.

07560c8

[build] Address failures in CI.

1bb7ff5

[build] Get Docker working again for Python.

ea70b4e

[build] Organize C header files.

c6da31e

petervdonovan added 3 commits October 14, 2022 16:43

Update submodules.

51e1311

Merge branch 'master' into c-refactoring2-2

7c4d9b6

This also keeps the file names that existed in master in org.lflang.generator.c and org.lflang.generator.python.

Apply workaround for Python test failures.

34f9e76

petervdonovan force-pushed the c-refactoring2 branch from b7c45ab to 34f9e76 Compare October 15, 2022 00:42

petervdonovan added 3 commits October 14, 2022 17:55

Fix another char*/const char* issue.

1bab631

Address test failures.

fc1427f

Update ci.yml.

9ad319b

petervdonovan force-pushed the c-refactoring2 branch from 4a6f3a9 to 9ad319b Compare October 15, 2022 05:12

petervdonovan added 4 commits October 14, 2022 22:18

Do not run setup-python step in CI.

7a3e845

More cleanups.

7555435

Bring back setup-Python for Windows only.

3a34790

This is a hack! The fact that it is needed is a reflection of the fact that the changes introduced by this PR leave the build system sensitive to idiosyncrasies in the way Python is installed and discovered on the system.

petervdonovan marked this pull request as ready for review October 17, 2022 21:09

petervdonovan mentioned this pull request Oct 17, 2022

Draft: Arduino Support for Arduino IDE v2.0 #1409

Closed

petervdonovan requested a review from lhstrh October 17, 2022 23:49

lhstrh approved these changes Oct 18, 2022

View reviewed changes

This was referenced Oct 18, 2022

Removal of "No CMake" build option lf-lang/lf-python-support#20

Merged

Federated Launcher scripts are not designed to work except on Linux #1418

Open

petervdonovan and others added 2 commits October 18, 2022 13:20

Apply suggestions from code review

43ab1a8

Co-authored-by: Marten Lohstroh <[email protected]>

Address comments from code review.

1998c7e

petervdonovan changed the title ~~Remove "No CMake" build option.~~ Removal of "No CMake" build option. Oct 18, 2022

petervdonovan changed the title ~~Removal of "No CMake" build option.~~ Removal of "No CMake" build option Oct 18, 2022

Update submodules to main.

6f78ca9

petervdonovan merged commit 18892d7 into master Oct 18, 2022

petervdonovan deleted the c-refactoring2 branch October 18, 2022 20:57

petervdonovan mentioned this pull request Nov 5, 2022

Weird build errors occur when src-gen is not deleted prior to recompiling #1460

Closed

lhstrh added the refactoring Code quality enhancement label Jan 26, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Removal of "No CMake" build option #1299

Removal of "No CMake" build option #1299

petervdonovan commented Jul 20, 2022 •

edited

Loading

cmnrd commented Oct 17, 2022

petervdonovan commented Oct 17, 2022

lhstrh left a comment

lhstrh commented Oct 18, 2022

Removal of "No CMake" build option #1299

Removal of "No CMake" build option #1299

Conversation

petervdonovan commented Jul 20, 2022 • edited Loading

cmnrd commented Oct 17, 2022

petervdonovan commented Oct 17, 2022

lhstrh left a comment

Choose a reason for hiding this comment

lhstrh commented Oct 18, 2022

petervdonovan commented Jul 20, 2022 •

edited

Loading