feat(schema): do schema versioning and make backup non-blocking for indexing #7852
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
non-blocking for indexing
github-actions
bot
added
the
area/schema
NamanJain8
force-pushed
the
naman/index-nonblocking
branch
from
8edc558 to
e683f87
Compare
manishrjain
approved these changes
May 25, 2021
There was a problem hiding this comment.
Reviewed 13 of 13 files at r1.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @NamanJain8 and @vvbalaji-dgraph)
worker/draft.go, line 230 at r1 (raw file):
case opIndexing: for otherId, otherCloser := range n.ops { if otherId == opBackup {
Ideally, we should look at the backup timestamp. If indexing has a higher ts, then it's OK. If not, then not.
NamanJain8
changed the title
NamanJain8
commented
May 26, 2021
There was a problem hiding this comment.
Dismissed @jarifibrahim and @manishrjain from 2 discussions.
Reviewable status: 10 of 13 files reviewed, all discussions resolved (waiting on @manishrjain and @vvbalaji-dgraph)
worker/draft.go, line 230 at r1 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
Ideally, we should look at the backup timestamp. If indexing has a higher ts, then it's OK. If not, then not.
Done.
worker/draft.go, line 626 at r1 (raw file):
Previously, jarifibrahim (Ibrahim Jarif) wrote…
The startTs is actually the
MaxAssigned. See line 605. You might end up writing data at the wrong timestamp.
We should use the same timestamp as the one used for index rebuild (MaxAssigned). Max assigned is used here to index all the data that has been committed up until this point.
NamanJain8
added a commit
that referenced
this pull request
May 26, 2021
…ndexing (#7852) Issue: We used to write schema/types at timestamp=1. That single bit violates the Snapshot Isolation. Hence, we couldn't run schema updates and other operations like backup concurrently. This is an issue in a multi-tenant cluster because if the backup is running we couldn't update the schema. Fix: This PR writes the schema/types at the startTs of the transaction and enables the schema updates to be made even if the backup is running. (cherry picked from commit f376a40)
NamanJain8
added a commit
that referenced
this pull request
May 26, 2021
…g for i… (#7856) * feat(schema): do schema versioning and make backup non-blocking for indexing (#7852) Issue: We used to write schema/types at timestamp=1. That single bit violates the Snapshot Isolation. Hence, we couldn't run schema updates and other operations like backup concurrently. This is an issue in a multi-tenant cluster because if the backup is running we couldn't update the schema. Fix: This PR writes the schema/types at the startTs of the transaction and enables the schema updates to be made even if the backup is running. (cherry picked from commit f376a40) * fix the predicate move
NamanJain8
added a commit
that referenced
this pull request
Jun 1, 2021
… for i… (#7856) * feat(schema): do schema versioning and make backup non-blocking for indexing (#7852) Issue: We used to write schema/types at timestamp=1. That single bit violates the Snapshot Isolation. Hence, we couldn't run schema updates and other operations like backup concurrently. This is an issue in a multi-tenant cluster because if the backup is running we couldn't update the schema. Fix: This PR writes the schema/types at the startTs of the transaction and enables the schema updates to be made even if the backup is running. (cherry picked from commit f376a40) * fix the predicate move (cherry picked from commit eb0a04b)
NamanJain8
added a commit
that referenced
this pull request
Jun 1, 2021
… for i… (#7856) * feat(schema): do schema versioning and make backup non-blocking for indexing (#7852) Issue: We used to write schema/types at timestamp=1. That single bit violates the Snapshot Isolation. Hence, we couldn't run schema updates and other operations like backup concurrently. This is an issue in a multi-tenant cluster because if the backup is running we couldn't update the schema. Fix: This PR writes the schema/types at the startTs of the transaction and enables the schema updates to be made even if the backup is running. (cherry picked from commit f376a40) * fix the predicate move (cherry picked from commit eb0a04b)
NamanJain8
added a commit
that referenced
this pull request
Jun 1, 2021
#7856) (#7873) * feat(schema): do schema versioning and make backup non-blocking for indexing (#7852) Issue: We used to write schema/types at timestamp=1. That single bit violates the Snapshot Isolation. Hence, we couldn't run schema updates and other operations like backup concurrently. This is an issue in a multi-tenant cluster because if the backup is running we couldn't update the schema. Fix: This PR writes the schema/types at the startTs of the transaction and enables the schema updates to be made even if the backup is running. (cherry picked from commit f376a40) * fix the predicate move (cherry picked from commit eb0a04b)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue:
We used to write schema/types at
timestamp=1. That single bit violates the Snapshot Isolation. Hence, we couldn't run schema updates and other operations like backup concurrently. This is an issue in a multi-tenant cluster because if the backup is running we couldn't update the schema.Fix:
This PR writes the schema/types at the
startTsof the transaction and enables the schema updates to be made even if the backup is running.This change is