Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(resolution): Prevent duplicate patterns in PackageResolver patternsByPackage array #5681

Merged

Conversation

rally25rs
Copy link
Contributor

@rally25rs rally25rs commented Apr 16, 2018

Summary

Fixes #5676. Previously duplicate patterns would be inserted into this array, causing unnecessary looping and manifest reading of the same patterns. It also caused a bug when duplicate patterns were marked as
peerDeps. We now prevent these duplicate entries.

Test plan

Added test to package-resolver tests.

…nsByPackage array

Previously duplicate patterns would be inserted into this array, causing unneccecary looping and
manifest reading of the same patterns. It also caused a bug when duplicate patterns were marked as
peerDeps (yarnpkg#5676). We now prevent these duplicate entries.

yarnpkg#5676
@rally25rs rally25rs requested a review from BYK April 16, 2018 22:39
@buildsize
Copy link

buildsize bot commented Apr 16, 2018

File name Previous Size New Size Change
yarn-[version].noarch.rpm 908.03 KB 907.71 KB -322 bytes (0%)
yarn-[version].js 3.94 MB 3.94 MB -1 KB (0%)
yarn-legacy-[version].js 4.1 MB 4.1 MB -1.06 KB (0%)
yarn-v[version].tar.gz 912.99 KB 913.03 KB 45 bytes (0%)
yarn_[version]all.deb 674.7 KB 674.52 KB -182 bytes (0%)

Copy link
Member

@BYK BYK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know why we were adding things multiple times, but as I mentioned earlier, I think the solution lies in using sets.

I'll look into that later on.


const config = await Config.create(
const config = await Config.create(
Object.assign(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: prefer object spread notation:

{
  cwd: loc,
  offline: false,
  cacheFolder,
  ...configOptions,
}

@@ -352,7 +353,9 @@ export default class PackageResolver {
this.patterns[pattern] = info;

const byName = (this.patternsByPackage[info.name] = this.patternsByPackage[info.name] || []);
byName.push(pattern);
if (byName.indexOf(pattern) === -1) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like it would be a lot more efficient to use a Set here since the ordering doesn't seem to be important?

I guess that'd be a refactor we need to do in a follow up?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that specific function yes, Set would be better, but the rest of the functions use Array.map and Array.filter on that object, which are not available on Set so we would have to convert Set to Array all over the place: [...patterns].map( which is probably worse overall for memory and performance usage. (expanding the set to an array frequently, vs infrequent .indexOf calls on small arrays)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that specific function yes, Set would be better, but the rest of the functions use Array.map and Array.filter on that object, which are not available on Set so we would have to convert Set to Array all over the place: [...patterns].map( which is probably worse overall for memory and performance usage. (expanding the set to an array frequently, vs infrequent .indexOf calls on small arrays)

Shameless plug: https://github.com/BYK/superset

@rally25rs
Copy link
Contributor Author

rally25rs commented Apr 17, 2018

Somehow all the CI environments now fail for a snapshot that would not have changed based on my last commit. Cool... 😠

This is due to a new version of lockfile that was published, and the yarn test is pulling the real npm published version, not mocked data. Fails on master too.

@BYK
Copy link
Member

BYK commented Apr 17, 2018

@rally25rs submitted #5687 to fix the failures on master.

@arcanis arcanis merged commit f69ecf1 into yarnpkg:master Apr 18, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Upgrade command fails with Yarn 1.5.1/1.6.0
3 participants