Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

assert: improve deepEqual Set and Map worst case #14258

Merged
merged 2 commits into from
Jul 19, 2017

Conversation

BridgeAR
Copy link
Member

@BridgeAR BridgeAR commented Jul 15, 2017

This change improves the algorithm for the worst case from O(n^2)
to O(n log n) by using a lazily initiated set for object keys and a special handling for loose equal primitives.

In addition a few comments got fixed and a few statements simplified.

There is still 100% coverage and I heavily tested weird cases.

Benchmarks:

 assert/deepequal-set.js method="deepEqual_looseMatches" len=500 n=500             6810.50 %        *** 4.878337e-16
 assert/deepequal-set.js method="deepEqual_mixed" len=500 n=500                    1004.55 %        *** 6.872133e-24
 assert/deepequal-set.js method="deepEqual_objectOnly" len=500 n=500               1077.10 %        *** 7.799749e-21
 assert/deepequal-set.js method="deepEqual_primitiveOnly" len=500 n=500              62.80 %        *** 8.981486e-11
 assert/deepequal-set.js method="deepStrictEqual_mixed" len=500 n=500               994.50 %        *** 1.788847e-15
 assert/deepequal-set.js method="deepStrictEqual_objectOnly" len=500 n=500         1089.84 %        *** 7.120327e-17
 assert/deepequal-set.js method="deepStrictEqual_primitiveOnly" len=500 n=500         3.58 %            2.607197e-01
 assert/deepequal-set.js method="notDeepEqual_looseMatches" len=500 n=500          6795.11 %        *** 4.752399e-13
 assert/deepequal-set.js method="notDeepEqual_mixed" len=500 n=500                  777.34 %        *** 2.503238e-27
 assert/deepequal-set.js method="notDeepEqual_objectOnly" len=500 n=500             409.91 %        *** 3.435125e-17
 assert/deepequal-set.js method="notDeepEqual_primitiveOnly" len=500 n=500          183.85 %        *** 8.764397e-29
 assert/deepequal-set.js method="notDeepStrictEqual_mixed" len=500 n=500              4.43 %          * 2.496594e-02
 assert/deepequal-set.js method="notDeepStrictEqual_objectOnly" len=500 n=500       437.70 %        *** 1.406055e-27
 assert/deepequal-set.js method="notDeepStrictEqual_primitiveOnly" len=500 n=500      3.13 %            3.144997e-01


 assert/deepequal-map.js method="deepEqual_looseMatches" len=500 n=500             1814.34 %        *** 1.700744e-20
 assert/deepequal-map.js method="deepEqual_mixed" len=500 n=500                     952.35 %        *** 1.807387e-17
 assert/deepequal-map.js method="deepEqual_objectOnly" len=500 n=500               1042.38 %        *** 1.482305e-16
 assert/deepequal-map.js method="deepEqual_primitiveOnly" len=500 n=500             133.51 %        *** 1.363842e-16
 assert/deepequal-map.js method="deepStrictEqual_mixed" len=500 n=500               996.73 %        *** 3.463207e-19
 assert/deepequal-map.js method="deepStrictEqual_objectOnly" len=500 n=500         1072.64 %        *** 2.516677e-17
 assert/deepequal-map.js method="deepStrictEqual_primitiveOnly" len=500 n=500        81.57 %        *** 2.526982e-18
 assert/deepequal-map.js method="notDeepEqual_looseMatches" len=500 n=500          1787.48 %        *** 4.458942e-20
 assert/deepequal-map.js method="notDeepEqual_mixed" len=500 n=500                  598.92 %        *** 1.020276e-14
 assert/deepequal-map.js method="notDeepEqual_objectOnly" len=500 n=500             407.62 %        *** 7.984672e-23
 assert/deepequal-map.js method="notDeepEqual_primitiveOnly" len=500 n=500          179.22 %        *** 8.525884e-17
 assert/deepequal-map.js method="notDeepStrictEqual_mixed" len=500 n=500             -1.87 %            7.260855e-01
 assert/deepequal-map.js method="notDeepStrictEqual_objectOnly" len=500 n=500       429.53 %        *** 3.240289e-20
 assert/deepequal-map.js method="notDeepStrictEqual_primitiveOnly" len=500 n=500     63.90 %        *** 1.195866e-14

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines
Affected core subsystem(s)

assert

@nodejs-github-bot nodejs-github-bot added the assert Issues and PRs related to the assert subsystem. label Jul 15, 2017
@refack refack self-assigned this Jul 16, 2017
@mscdex mscdex added the performance Issues and PRs related to the performance of Node.js. label Jul 16, 2017
@BridgeAR
Copy link
Member Author

BridgeAR commented Jul 18, 2017

I added benchmarks and reworked the code again and fixed a regression for deepEqual with Map by doing so. deepEqual with Map got a bit ugly but I think there are not many other possibilities. I also simplified two more statements.

I still have to check the coverage but PTAL already.

Update: I moved the benchmarks to the top.

@refack
Copy link
Contributor

refack commented Jul 18, 2017

@BridgeAR things are hectic with trying to release 6.11.2 & 8.2.0, so it might take a few more days...

@BridgeAR BridgeAR force-pushed the improve-assert-o-notation branch from b6ccc6f to fdbd79a Compare July 19, 2017 14:31
@BridgeAR BridgeAR changed the title assert: improve deepEqual Set and Map worst case WIP - assert: improve deepEqual Set and Map worst case Jul 19, 2017
@BridgeAR BridgeAR force-pushed the improve-assert-o-notation branch from fdbd79a to 37e0950 Compare July 19, 2017 20:44
@BridgeAR BridgeAR changed the title WIP - assert: improve deepEqual Set and Map worst case assert: improve deepEqual Set and Map worst case Jul 19, 2017
@BridgeAR
Copy link
Member Author

I finished the code and the coverage is kept at 100% and I heavily tested weird edge cases.

I decided not to add a memory set for loose equal keys even though as it would complicate things again and the numbers should be good enough even without that optimization.
The loose comparison was a bit more challenging then I first anticipated and the solution is definitely more complex but it is also fast.

I am running the benchmarks for maps right now and I am adding them to the main entry as soon as they are done. The numbers for sets are already there.

@refack
Copy link
Contributor

refack commented Jul 19, 2017

BridgeAR added 2 commits July 19, 2017 19:25
This change improves the algorithm for the worst case from O(n^2)
to O(n log n) by using a lazily initiated set with object or
not strict equal primitives keys.

In addition a few comments got fixed and a statement simplified.

PR-URL: nodejs#14258
Reviewed-By: Refael Ackermann <[email protected]>
@refack refack force-pushed the improve-assert-o-notation branch from 37e0950 to 462b58e Compare July 19, 2017 23:27
@refack refack merged commit 462b58e into nodejs:master Jul 19, 2017
@refack
Copy link
Contributor

refack commented Jul 19, 2017

Landed in 462b58e 5203bb0

addaleax pushed a commit that referenced this pull request Jul 22, 2017
@refack refack mentioned this pull request Jul 22, 2017
4 tasks
Fishrock123 pushed a commit that referenced this pull request Jul 24, 2017
@addaleax addaleax mentioned this pull request Jul 24, 2017
@addaleax
Copy link
Member

The first commit (5203bb0) doesn’t land cleanly on 8.x; if you can, please follow the guide and raise a backport PR, if you don’t think it’s worth it let me know and we’ll add the dont-land-on label.

@BridgeAR
Copy link
Member Author

BridgeAR commented Jul 27, 2017

@addaleax there was one commit that should land first. In that case it should land cleanly (hopefully).
be20e9e

EDIT I first listed two commits but it is actually only one as the other one was a semver major.

@addaleax
Copy link
Member

@BridgeAR Correct me if I’m wrong, but the first commit seems to depend on a semver-major one, semantically, so it can’t be backported. If you can backport the second one and this one cherry-picks cleanly after that, great. :)

@BridgeAR
Copy link
Member Author

@addaleax yes, I was to quick with sending the message and I already removed that from the entry again^^.

@addaleax
Copy link
Member

Lands cleanly now :)

addaleax pushed a commit that referenced this pull request Jul 28, 2017
This change improves the algorithm for the worst case from O(n^2)
to O(n log n) by using a lazily initiated set with object or
not strict equal primitives keys.

In addition a few comments got fixed and a statement simplified.

PR-URL: #14258
Reviewed-By: Refael Ackermann <[email protected]>
@MylesBorins
Copy link
Contributor

Should this be landed on v6.x?

@BridgeAR
Copy link
Member Author

@MylesBorins Set and Map support was added in 8.0 as a semver-major. Due to that backporting is not possible.

@BridgeAR BridgeAR mentioned this pull request Sep 20, 2017
4 tasks
@refack refack removed their assignment Oct 20, 2018
@BridgeAR BridgeAR deleted the improve-assert-o-notation branch April 1, 2019 23:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
assert Issues and PRs related to the assert subsystem. performance Issues and PRs related to the performance of Node.js.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants