Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unit testing #3

Merged
merged 3 commits into from
Jul 10, 2023
Merged

unit testing #3

merged 3 commits into from
Jul 10, 2023

Conversation

SimeonEhrig
Copy link
Member

@SimeonEhrig SimeonEhrig commented Jul 5, 2023

The PR add the tool SetupEnv and enable unit testing for QED.jl.

About SetupEnv

SetupEnv creates an environment for the unit tests. It reads the Project.toml of the project and installs all QED dependencies as development version from the current dev branch by default. All other dependencies are normally installed.
SetupEnv also allows to install QED dependencies from other repositories and branches.

  • On a local system, you can set the environment variable CI_UNIT_PKG_URL_<QED_package>=https://github.com/User/<QED_package>#<branch>. For example, if I want to use my fork of QEDbase and the feature branch fooFunction I have to set the environment variable CI_UNIT_PKG_URL_QEDbase=https://github.com/SimeonEhrig/QEDbase.jl.git#fooFunction.
  • In the CI, the dependency can set via commit message:

Example Git commit message:

Add function bar

- function bar uses the function foo from the QEDbase packge 

CI_UNIT_PKG_URL_QEDbase: https://github.com/SimeonEhrig/QEDbase.jl.git#fooFunction

Pipeline behavior

The pipeline has two stages for unit test. The first stage does the actual unit tests. If the stages passes, you code is fine. The second stage checks, if SetupEnv use custom URLs for QED packages. If this happens, the test will fail. The reason is, that no code should be merged, which rely on non merged functions. The solution is merging the code in the depending package first and than trigger the CI with CI_UNIT_PKG_URL_<package> in the message again.

Local unit tests

Local unit tests are enable in the .gitlab-ci.yml. Actual QED.jl will not have functionality, but with the unit tests, we can test the functionality of the SetupEnv.

unit testing in sub packages

Sub packages needs to clone this repository to get the SetupEnv tool, see QEDjl-project/QEDfields.jl#3

Documentation

Documentation is coming, when the integration tests are implement. I decided to split up the unit and integration tests in two separate PR for better review. Compare to it, it is better to write the documentation of the unit and integration test together. So, the unit test documentation is coming together with the integration test PR. A initial version can be found here: https://github.com/SimeonEhrig/GitlabCIMain/blob/master/docs/src/index.md

@SimeonEhrig SimeonEhrig force-pushed the initialGitlabCI branch 2 times, most recently from 0a0874e to 9d527cb Compare July 5, 2023 11:30
- Unit tests testing all function if the package itself
- The SetupEnv script add the develop version of all QED packages to julia environment.
- Run unit-test in the CI
@SimeonEhrig SimeonEhrig changed the title initial .gitlab-ci.yml unit testing Jul 6, 2023
@SimeonEhrig
Copy link
Member Author

SimeonEhrig commented Jul 6, 2023

Testing the PR

To test the PR, we need an unconventional setup. I added unit tests in the QEDfields project. Afterwards I created a feature branch, which implements an new function bar with a dependency to the function QEDbase::foo(). foo is not merged in dev branch and exists only on a feature branch on my fork. Like expected, the unit test fails, because the CI test against the dev branch, see this commit. In a second commit, I added CI_UNIT_PKG_URL_QEDbase: https://github.com/SimeonEhrig/QEDbase.jl.git#fooFunction at the end of the commit message. Now the CI is testing the unit tests with my feature branch and the CI passes.

In a third commit, I added the unit test dependency verification. Therefore the unit tests passes, but the CI failes.

@SimeonEhrig SimeonEhrig marked this pull request as ready for review July 6, 2023 08:08
@SimeonEhrig
Copy link
Member Author

@szabo137 Time for review 😎

@SimeonEhrig SimeonEhrig requested a review from szabo137 July 6, 2023 09:16
Copy link
Member

@szabo137 szabo137 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice PR, thanks! The only thing, which could be considered changing is the proper error handling to replace exit(0) and exit(1) entirely. However, maybe this would be a bit of overengineering 🤷‍♂️.

ci/SetupDevEnv/README.md Outdated Show resolved Hide resolved
ci/SetupDevEnv/README.md Outdated Show resolved Hide resolved
ci/SetupDevEnv/src/SetupDevEnv.jl Outdated Show resolved Hide resolved
ci/SetupDevEnv/src/SetupDevEnv.jl Outdated Show resolved Hide resolved
ci/SetupDevEnv/src/SetupDevEnv.jl Outdated Show resolved Hide resolved
ci/verify_env.jl Outdated Show resolved Hide resolved
ci/verify_env.jl Outdated Show resolved Hide resolved
ci/verify_env.jl Outdated Show resolved Hide resolved
ci/verify_env.jl Outdated Show resolved Hide resolved
ci/verify_env.jl Outdated Show resolved Hide resolved
- use logger instead println
- use custom error in verify_env.jl
Copy link
Member

@szabo137 szabo137 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything is fine :-)

@szabo137 szabo137 merged commit a06efb8 into QEDjl-project:dev Jul 10, 2023
szabo137 pushed a commit that referenced this pull request Jul 11, 2023
After PR #3 I recognized, that lines starting with `CI_UNIT_PKG_URL_`
exist in the commit message of the merge request. This triggers
accidentally custom URLs for packages in the `SetupDevEnv.jl` script.
Because it does not make any sense to use custom URLs on the `main` and
`dev` branch, this PR disable the feature for this branches.

Add also gitlab CI attribute `interruptible: true` that CI can be
aborted if a new commit was pushed to the same branch.
@SimeonEhrig SimeonEhrig mentioned this pull request Jul 13, 2023
szabo137 pushed a commit that referenced this pull request Aug 21, 2023
The PR adds the tool `integTestGen`.

The `integTestGen.jl` creates jobs for the CI, which checks if the
changes in PR does a API or behavior break which causes that other QED
packages which use the package of PR does not work anymore.

A detailed documentation is part of the PR. The doc also provides the
documentation for the unit tests, introduced in PR #3.
@AntonReinhard AntonReinhard added this to the Release-0.1.0 milestone Aug 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants