Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible issue with project imports in SQL incremental mode #674

Closed
DylanBaker opened this issue Jan 6, 2023 · 0 comments · Fixed by #675
Closed

Possible issue with project imports in SQL incremental mode #674

DylanBaker opened this issue Jan 6, 2023 · 0 comments · Fixed by #675
Assignees
Labels
Bug Something isn't working

Comments

@DylanBaker
Copy link
Collaborator

I think it's possible that SQL incremental mode doesn't work properly when run on a project with project imports.

If we imagine a Project A, which imports Project B, I believe the following is happening:

  • We first get the SQL on the commit/branch that is being tested. For this example, let's assume it's a commit. We therefore:
    • In Project A, we checkout a temp branch and hard reset it to the commit being tested.
    • For Project B, we find the production commit, checkout a temp branch and hard reset it to the production commit. At this stage, we add Project B to the skip_imports on the branch manager being used.
    • We gather the SQL for all explores in Project A.
    • We reset Project A to whatever branch it was on before and delete the temp branch.
    • We reset Project B to whatever branch it was on before and delete the temp branch.
  • We then need to get the SQL for the "production" state. We therefore:
    • In Project A, we find the production commit, checkout a temp branch and hard reset it to the production commit.
    • In Project B, we should also find the production commit, checkout a temp branch and hard reset it to the production commit, but I don't think we are. This is because Project B is in the skip_imports list and we therefore assume we don't need to do anything for it. This means Project B is left on whatever branch it started the run on, which could be old and is likely not in the production state.
    • The steps from here are the same as above, except we don't reset Project B, because it wasn't altered in the first place.

As a result of all this, the comparison of the SQL could often find differences, even when none are there, because the state of the imported project is left on whatever branch it started the run on.

I think the fix might be quite simple, which is to reset skip_imports to an empty list in the __aexit__ function. I think the skip_imports only really need to exist for one "go" of the branch manager.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants