Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deduplicate wkis in mcq.StatementRefs #107

Merged
merged 2 commits into from
Dec 21, 2016
Merged

Deduplicate wkis in mcq.StatementRefs #107

merged 2 commits into from
Dec 21, 2016

Conversation

vyzo
Copy link
Contributor

@vyzo vyzo commented Dec 20, 2016

This fixes a subtle issue with duplicate wkis in compound statements:
It is possible for a compound statement to include simple statements which refer to the same wki, which would be included multiple times in the list returned by mcq.StatementRef.
As a result, the same statement would be mapped to the same wki multiple times in the Refs table.
This would create garbage in the db and have the unfortunate side effect of having the same statement appear multiple times in a query result set even when querying for a single wki.

The PR fixes this by constructing a set for compound statements, which deduplicates the wkis.
Measurements indicate no performance penalty for compound statement processing.

PS: Also added dedup for simple statements, to protect from faulty publishers who don't dedup their references.
Performance degradation for simple statements is also neglible.

Copy link
Contributor

@parkan parkan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logic is clear, no performance impact, looking good

@vyzo vyzo merged commit 4717e24 into master Dec 21, 2016
@vyzo vyzo deleted the vyzo-wki-dedup branch December 21, 2016 10:00
@vyzo vyzo changed the title Deduplicate wkis in mcq.StatementRefs for compound statements Deduplicate wkis in mcq.StatementRefs Dec 21, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants