Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

repl: fix crash with large buffer tab completion #13817

Closed
wants to merge 7 commits into from
Closed

repl: fix crash with large buffer tab completion #13817

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
a59ca52
repl: fix crash with large buffer tab completion
XadillaX Jun 20, 2017
c08093e
update after reviewing
XadillaX Jun 20, 2017
402070d
add some test cases
XadillaX Jun 20, 2017
59649f3
use this._writeToOutput to instead of process.stdout.write
XadillaX Jun 20, 2017
876ab6e
update after reviewing again
XadillaX Jun 21, 2017
11db662
update after @TimothyGu's reviewing
XadillaX Jun 21, 2017
573e252
update after reviewing again
XadillaX Jun 21, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 31 additions & 5 deletions lib/repl.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ const Module = require('module');
const domain = require('domain');
const debug = util.debuglog('repl');
const errors = require('internal/errors');
const Buffer = require('buffer').Buffer;

const parentModule = module;
const replMap = new WeakMap();
Expand Down Expand Up @@ -689,8 +690,31 @@ function intFilter(item) {
return /^[A-Za-z_$]/.test(item);
}

const defaultProperties = {
ARRAY: Object.getOwnPropertyNames([]).filter(intFilter),
BUFFER: Object.getOwnPropertyNames(Buffer.alloc(1)).filter(intFilter)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not Buffer.alloc(0)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I don't know that we can alloc a zero-size buffer.

};

function mayBeLargeObject(obj) {
return (Array.isArray(obj) || Buffer.isBuffer(obj));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't address other TypedArray types.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, currently, Buffer.prototype passes the Buffer.isBuffer test. On the other hand, getting Buffer.prototype.length will throw the following error:

> Buffer.prototype.length
TypeError: Method get TypedArray.prototype.length called on incompatible receiver [object Object]
    at Uint8Array.get length [as length] (<anonymous>)
    at repl:1:17
    at ContextifyScript.Script.runInThisContext (vm.js:44:33)
    at REPLServer.defaultEval (repl.js:239:29)
    at bound (domain.js:301:14)
    at REPLServer.runBound [as eval] (domain.js:314:12)
    at REPLServer.onLine (repl.js:433:10)
    at emitOne (events.js:120:20)
    at REPLServer.emit (events.js:210:7)
    at REPLServer.Interface._onLine (readline.js:278:10)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, currently, Buffer.prototype passes the Buffer.isBuffer test.

How about obj instanceof Buffer?

}

function filteredOwnPropertyNames(obj) {
if (!obj) return [];
if (mayBeLargeObject(obj) && obj.length > 1e6) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you use a constant for 1e6 like const ARRAY_LENGTH_THRESHOLD?

this._writeToOutput('\r\n');
process.emitWarning(
'Instance is too large so the completion may missing some custom ' +
'properties.',
'REPLWarning',
undefined,
undefined,
true);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally I'm okay with not showing the warning. Autocomplete is not a feature that necessarily has to work 100% of the time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But if there's no warning, I think after this PR landed, developers use Node.js may open several issue about this bug "why autocompletion is wrong", though it's not exactly a bug.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@XadillaX fwiw, the autocompletion in REPL is far from perfect and I, as a user, would not and do not expect it to always show all the completions (though it would be an awesome thing). Even IDEs don't do that reliably for JavaScript.

Copy link
Contributor Author

@XadillaX XadillaX Jun 21, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aqrln The point is:

> var ele = Buffer.alloc(1);
> ele.biu = 1;
> ...
> ele.<tab>
ele.__defineGetter__      ele.__defineSetter__      ele.__lookupGetter__      ele.__lookupSetter__
ele.__proto__             ele.constructor           ele.hasOwnProperty        ele.isPrototypeOf
ele.propertyIsEnumerable  ele.toLocaleString        ele.toString              ele.valueOf

...

ele.biu

> var ele = Buffer.alloc(1e6 + 1); ele.biu = 1;
> ele.<tab>
(node:3635) REPLWarning: Instance is too large so the completion may missing some custom properties.

ele.__defineGetter__      ele.__defineSetter__      ele.__lookupGetter__      ele.__lookupSetter__
ele.__proto__             ele.constructor           ele.hasOwnProperty        ele.isPrototypeOf
ele.propertyIsEnumerable  ele.toLocaleString        ele.toString              ele.valueOf

...

Without warning, the developers may be confused that why there's no biu, that completion is what truly they want to find.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay. In that case, the message could be clearer:

The current object has too many own properties. Completion output may be truncated.

or

The current array, Buffer, or TypedArray has too many entries. Certain properties may be missing from completion output.


return Array.isArray(obj) ?
defaultProperties.ARRAY :
defaultProperties.BUFFER;
}
return Object.getOwnPropertyNames(obj).filter(intFilter);
}

Expand Down Expand Up @@ -843,9 +867,11 @@ function complete(line, callback) {
if (this.useGlobal || vm.isContext(this.context)) {
var contextProto = this.context;
while (contextProto = Object.getPrototypeOf(contextProto)) {
completionGroups.push(filteredOwnPropertyNames(contextProto));
completionGroups.push(
filteredOwnPropertyNames.call(this, contextProto));
}
completionGroups.push(filteredOwnPropertyNames(this.context));
completionGroups.push(
filteredOwnPropertyNames.call(this, this.context));
addStandardGlobals(completionGroups, filter);
completionGroupsLoaded();
} else {
Expand All @@ -865,13 +891,13 @@ function complete(line, callback) {
}
} else {
const evalExpr = `try { ${expr} } catch (e) {}`;
this.eval(evalExpr, this.context, 'repl', function doEval(e, obj) {
this.eval(evalExpr, this.context, 'repl', (e, obj) => {
// if (e) console.log(e);

if (obj != null) {
if (typeof obj === 'object' || typeof obj === 'function') {
try {
memberGroups.push(filteredOwnPropertyNames(obj));
memberGroups.push(filteredOwnPropertyNames.call(this, obj));
} catch (ex) {
// Probably a Proxy object without `getOwnPropertyNames` trap.
// We simply ignore it here, as we don't want to break the
Expand All @@ -889,7 +915,7 @@ function complete(line, callback) {
p = obj.constructor ? obj.constructor.prototype : null;
}
while (p !== null) {
memberGroups.push(filteredOwnPropertyNames(p));
memberGroups.push(filteredOwnPropertyNames.call(this, p));
p = Object.getPrototypeOf(p);
// Circular refs possible? Let's guard against that.
sentinel--;
Expand Down
30 changes: 30 additions & 0 deletions test/parallel/test-repl-tab-complete.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -305,6 +305,36 @@ testMe.complete('.b', common.mustCall((error, data) => {
assert.deepStrictEqual(data, [['break'], 'b']);
}));

// tab completion for large buffer
const warningRegEx =
/\(node:\d+\) REPLWarning: Instance is too large so the completion may missing some custom properties\./;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is longer than 80 characters, but the linter passes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall I split it into two lines?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say yes.

[ Array, Buffer ].forEach((type) => {
putIn.run(['.clear']);

if (type === Array) {
putIn.run(['var ele = []; for (let i = 0; i < 1e7; i++) ele.push(i);']);
} else {
putIn.run(['var ele = Buffer.alloc(1e8)']);
}

common.hijackStderr(common.mustCall((err) => {
process.nextTick(() => {
assert.ok(warningRegEx.test(err));
});
}));
testMe.complete('ele.', common.mustCall((err, data) => {
common.restoreStderr();
assert.ifError(err);

const ele = (type === Array) ? [] : Buffer.alloc(1);

data[0].forEach((key) => {
if (!key) return;
assert.notStrictEqual(ele[key.substr(4)], undefined);
});
}));
});

const testNonGlobal = repl.start({
input: putIn,
output: putIn,
Expand Down