Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Design to keep PullRequest PipelineResource in sync with reality #1286

Closed
bobcatfish opened this issue Sep 6, 2019 · 4 comments · Fixed by #1408
Closed

Design to keep PullRequest PipelineResource in sync with reality #1286

bobcatfish opened this issue Sep 6, 2019 · 4 comments · Fixed by #1408
Assignees

Comments

@bobcatfish
Copy link
Collaborator

Expected Behavior

If a Task uses a PullRequest Pipeline resource, as the Task progresses, if comments are added (or other changes represented in the filesystem) to the Pull Request, it should be possible:

  • For the executing Task to see these changes <-- this is kind of a stretch goal really
  • For the executing Task to use the PullRequest resource as an output and make additive changes to the PullRequest

Actual Behavior

When a Task is executing, the representation on the file system of the pull request is not updated. Even worse, the side effect is that when the PullRequestResource is used as an output, the data is treated as the source of truth and so any changes made between when the Task started and when it completed can be lost (e.g. comments made in the meantime will be deleted!!!)

Steps to Reproduce the Problem

  1. Create a Task that uses a PullRequest Resource as an output, e.g.:
    a. Task could sleep for a minute or something (to give you time for step 3)
    b. Task could also add a comment to the output (i.e. write to /workspace//comments/)
  2. Execute the Task, providing an existing PR as input
  3. Add a comment to the PR manually while the Task executes!
  4. After the task finishes, observe that the PR has the comment added by the Task, but the one you added manually has vanished :(

Additional Info

We should at least make it so changes are additive, but also we could be ambitious and do something really cool e.g. create our own fs? (@dlorenc had some cool ideas here)

@bobcatfish bobcatfish added this to the Pipelines 0.8 🐱 milestone Sep 6, 2019
@bobcatfish
Copy link
Collaborator Author

One way to work around this (reduce the risk) would be to use this with a Notification type (#49 )

@bobcatfish
Copy link
Collaborator Author

bobcatfish commented Sep 10, 2019

For our beta release, maybe more realistic to view PR outputs as additive only.

(would imply not being able to remove?)

@dibyom
Copy link
Member

dibyom commented Sep 10, 2019

might be useful: https://developer.github.com/v4/mutation/

@wlynch
Copy link
Member

wlynch commented Sep 24, 2019

I don't think notification types would solve this problem. The race condition exists even in that case, since a new comment could come in from when the PR is initialized on disk.

I think what we want here is a comment manifest. This would let us detect whether or not a comment was explicitly deleted, or just one we don't know about yet.

e.g. When we download the pull request resource, also create a file called /workspace/<resource>/comments/MANIFEST containing the comments pulled in the download:

1.json
2.json

When we upload the PR, if we see that 2.json is missing from the local filesystem, we know this was a deletion from the Task because it was originally listed in the manifest. If we see that 3.json is missing from the local filesystem, we shouldn't make any modifications to it because the task was not aware of it when the PR resource was initialized since it was not in the manifest.

wlynch added a commit to wlynch/pipeline that referenced this issue Oct 9, 2019
Introduces `.MANIFEST` files to comment and label directories to keep
track of sub-resources when the PullRequest resource is initialized.
This allows us to keep track of what resources the user explicitly
deleted, and not accidentally overwrite new comments/lables that were
introduced during execution.

Minor changes included:
* Changed fake GitHub server to make copies of PullRequests to avoid
  pointer collision across tests.
* Changed uploadLabels to Add/Delete labels explicitly instead of
  replacing all.

Change manifest path to .MANIFEST to avoid collisions.

Fixes tektoncd#1286
wlynch added a commit to wlynch/pipeline that referenced this issue Oct 9, 2019
Introduces `.MANIFEST` files to comment and label directories to keep
track of sub-resources when the PullRequest resource is initialized.
This allows us to keep track of what resources the user explicitly
deleted, and not accidentally overwrite new comments/lables that were
introduced during execution.

Minor changes included:
* Changed fake GitHub server to make copies of PullRequests to avoid
  pointer collision across tests.
* Changed uploadLabels to Add/Delete labels explicitly instead of
  replacing all.

Change manifest path to .MANIFEST to avoid collisions.

Fixes tektoncd#1286
wlynch added a commit to wlynch/pipeline that referenced this issue Oct 9, 2019
Introduces `.MANIFEST` files to comment and label directories to keep
track of sub-resources when the PullRequest resource is initialized.
This allows us to keep track of what resources the user explicitly
deleted, and not accidentally overwrite new comments/lables that were
introduced during execution.

Minor changes included:
* Changed fake GitHub server to make copies of PullRequests to avoid
  pointer collision across tests.
* Changed uploadLabels to Add/Delete labels explicitly instead of
  replacing all.

Change manifest path to .MANIFEST to avoid collisions.

Fixes tektoncd#1286
wlynch added a commit to wlynch/pipeline that referenced this issue Oct 9, 2019
Introduces `.MANIFEST` files to comment and label directories to keep
track of sub-resources when the PullRequest resource is initialized.
This allows us to keep track of what resources the user explicitly
deleted, and not accidentally overwrite new comments/lables that were
introduced during execution.

Minor changes included:
* Changed fake GitHub server to make copies of PullRequests to avoid
  pointer collision across tests.
* Changed uploadLabels to Add/Delete labels explicitly instead of
  replacing all.

Change manifest path to .MANIFEST to avoid collisions.

Fixes tektoncd#1286
wlynch added a commit to wlynch/pipeline that referenced this issue Oct 14, 2019
Introduces `.MANIFEST` files to comment and label directories to keep
track of sub-resources when the PullRequest resource is initialized.
This allows us to keep track of what resources the user explicitly
deleted, and not accidentally overwrite new comments/lables that were
introduced during execution.

Minor changes included:
* Changed fake GitHub server to make copies of PullRequests to avoid
  pointer collision across tests.
* Changed uploadLabels to Add/Delete labels explicitly instead of
  replacing all.

Change manifest path to .MANIFEST to avoid collisions.

Fixes tektoncd#1286
tekton-robot pushed a commit that referenced this issue Oct 15, 2019
Introduces `.MANIFEST` files to comment and label directories to keep
track of sub-resources when the PullRequest resource is initialized.
This allows us to keep track of what resources the user explicitly
deleted, and not accidentally overwrite new comments/lables that were
introduced during execution.

Minor changes included:
* Changed fake GitHub server to make copies of PullRequests to avoid
  pointer collision across tests.
* Changed uploadLabels to Add/Delete labels explicitly instead of
  replacing all.

Change manifest path to .MANIFEST to avoid collisions.

Fixes #1286
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants