zlib: do not emit event on *Sync() methods #5707
Conversation
jasnell
added
the
semver-major
|
Marking as semver-major due to the nature of the change... tho, I certainly hope no one had come to rely on this behavior ;-) |
|
LGTM |
|
A citgm run, perhaps? cc @thealphanerd |
|
citgm: https://ci.nodejs.org/job/thealphanerd-smoker/116/ There has been some weirdness lately including
|
|
@thealphanerd Is the failure in that CITGM run a false positive? This is a work-in-progress so another run may be necessary later. |
|
Actually, it looks like I did in fact get all the instances I needed to get, so it may not need another CITGM run. I do need to add a test though, for the every-day CI stuff. |
|
So some of those failures on ppc are new. Glad everything else was green!!! I'm running it again on master to see if that give us different results. Will add finding to this comment. |
|
It seems like the only likely way someone might be tripped up by this change is if they are calling |
|
Test added. Woot. CI: https://ci.nodejs.org/job/node-test-pull-request/1945/ |
|
Not seeing any of the ppc citgm problems on master specifically
Running citgm one more time to be sure. If vinyl-fs is having a regression then gulp would definitely be affected too |
|
eslint is only failure this time. That's a known flaky? |
|
Only CI failure is known-flaky Someone, please feel free to figure out how to fix that one. :-/ |
|
Removing |
| require('../common'); | ||
| const zlib = require('zlib'); | ||
|
|
||
| const zipper = new zlib.Gzip(); |
There was a problem hiding this comment.
shouldn't this be testing zlib.gzipSync()?
There was a problem hiding this comment.
If it used zlib.gzipSync() then the object that emits the close event would not be accessible by the test.
So instead, this is emulating zlibBufferSync() and calling _processChunk() without a callback so that it can have access to the event if it fires.
|
Clean citgm run this time |
|
Since @jasnell labeled this |
| @@ -453,7 +453,9 @@ Zlib.prototype.flush = function(kind, callback) { | |||
| } | |||
| }; | |||
|
|
|||
| Zlib.prototype.close = function(callback) { | |||
| Zlib.prototype.close = function(callback, options) { | |||
| options = options || {}; | |||
There was a problem hiding this comment.
Object.assign() is safer than this pattern in the event that a non-object truthy value is passed. Not sure if we want to cover up those types of programming errors, but thought I'd at least mention it.
There was a problem hiding this comment.
Might as well do it right the first time. I'll make that change.
| zipper.on('close', () => { throw new Error('unexpected `close` event'); }); | ||
|
|
||
| const buffer = new Buffer('hello world'); | ||
| zipper._processChunk(buffer, zlib.Z_FINISH); |
There was a problem hiding this comment.
For sake of paranoia, is there an event that would notify that the chunk has been processed and called before 'close' would be called? I'd like to assert that that event is called in process exit.
There was a problem hiding this comment.
@trevnorris I don't believe so, but would this more complete test address the concern?
'use strict';
require('../common');
const zlib = require('zlib');
const assert = require('assert');
const shouldNotBeCalled = () => { throw new Error('unexpected event'); };
const message = 'Come on, Fhqwhgads.';
const zipper = new zlib.Gzip();
zipper.on('close', shouldNotBeCalled);
const buffer = new Buffer(message);
const zipped = zipper._processChunk(buffer, zlib.Z_FINISH);
const unzipper = new zlib.Gunzip();
unzipper.on('close', shouldNotBeCalled);
const unzipped = unzipper._processChunk(zipped, zlib.Z_FINISH);
assert.notEqual(zipped.toString(), message);
assert.strictEqual(unzipped.toString(), message);
There was a problem hiding this comment.
You know, I think I like this more complete test regardless, so I'll push that up too.
|
fwiw, if anyone disagrees with the semver-major, feel free to downgrade it! |
WIP right now, still need to do it for more methods than just gunzipSync(). Refs: nodejs#1668
|
The more I think about this, the less happy I am with the |
|
I'm not normally in zlib, but change looks clean. LGTM |
I'm all for "when in doubt, make it major", so I have no problem with it. This is doubly true considering that a semver major release is coming up in about a month, so if this lands relatively soon, it will be released in the not-too-distant future. |
|
LGTM. Speed and memory usage remain stable after this change with my test here: #1665 (comment). |
Asynchronous functions in `zlib` should not emit the close event. This fixes an issue where asynchronous calls in a for loop could exhaust memory because the pending event prevents the objects from being garbage collected. Fixes: nodejs#1668 PR-URL: nodejs#5707 Reviewed-By: jasnell - James M Snell <[email protected]> Reviewed-By: trevnorris - Trevor Norris <[email protected]> Reviewed-By: Сковорода Никита Андреевич <[email protected]>
|
Landed in 8b43d3f |
Pull Request check-list
Please make sure to review and check all of these items:
make -j8 test(UNIX) orvcbuild test nosign(Windows) pass withthis change (including linting)?
test (or a benchmark) included?
existing APIs, or introduces new ones)?
Affected core subsystem(s)
zlib
Description of change
WIP right now, still need to do it for more methods than just
gunzipSync().
Needs a test, too.
Refs: #1668