Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: handle errors from file#createReadStream #615

Merged
merged 5 commits into from
Feb 28, 2019
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions src/file.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1212,6 +1212,8 @@ class File extends ServiceObject<File> {
return;
}

rawResponseStream.on('error', onComplete);

const headers = rawResponseStream.toJSON().headers;
const isCompressed = headers['content-encoding'] === 'gzip';
const shouldRunValidation = !rangeRequest && (crc32c || md5);
Expand All @@ -1235,9 +1237,9 @@ class File extends ServiceObject<File> {
rawResponseStream.pipe(pumpify.obj(throughStreams));
}

rawResponseStream.on('end', onComplete).pipe(throughStream, {
end: false
});
rawResponseStream.on('error', onComplete)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is having 2 error listeners with the same callback intentional?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code above this actually re-assigns what rawResponseStream is. When we did that, it was to avoid duplicating the event handler assignments.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh ok, sorry missed that part.

.on('end', onComplete)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there any scenarios where onComplete could fire multiple times? I sort of wonder if we should be using something like once to ensure that doesn't happen somehow.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I 'spose anything is possible! I added new code for that, please take a look!

.pipe(throughStream, {end: false});
};

// This is hooked to the `complete` event from the request stream. This is
Expand Down
83 changes: 83 additions & 0 deletions test/file.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,15 @@ const fakePromisify = {
const fsCached = extend(true, {}, fs);
const fakeFs = extend(true, {}, fsCached);

const zlibCached = extend(true, {}, zlib);
let createGunzipOverride: Function|null;
const fakeZlib = extend(true, {}, zlib, {
createGunzip() {
return (createGunzipOverride || zlibCached.createGunzip)
.apply(null, arguments);
},
});

let hashStreamValidationOverride: Function|null;
const hashStreamValidation = require('hash-stream-validation');
function fakeHashStreamValidation() {
Expand Down Expand Up @@ -148,6 +157,7 @@ describe('File', () => {
'hash-stream-validation': fakeHashStreamValidation,
os: fakeOs,
'xdg-basedir': fakeXdgBasedir,
zlib: fakeZlib,
}).File;
});

Expand Down Expand Up @@ -178,6 +188,7 @@ describe('File', () => {
directoryFile = new File(BUCKET, 'directory/file.jpg');
directoryFile.request = util.noop;

createGunzipOverride = null;
handleRespOverride = null;
hashStreamValidationOverride = null;
makeWritableStreamOverride = null;
Expand Down Expand Up @@ -957,6 +968,39 @@ describe('File', () => {
})
.resume();
});

it('should emit errors from the request stream', done => {
const error = new Error('Error.');
const rawResponseStream = through();
// tslint:disable-next-line:no-any
(rawResponseStream as any).toJSON = () => {
return {headers: {}};
};
const requestStream = through();

handleRespOverride =
(err: Error, res: {}, body: {}, callback: Function) => {
callback(null, null, rawResponseStream);
setImmediate(() => {
rawResponseStream.emit('error', error);
});
};

file.requestStream = () => {
setImmediate(() => {
requestStream.emit('response', rawResponseStream);
});
return requestStream;
};

file.createReadStream()
.on('error',
(err: Error) => {
assert.strictEqual(err, error);
done();
})
.resume();
});
});
});

Expand Down Expand Up @@ -995,6 +1039,24 @@ describe('File', () => {
})
.resume();
});

it('should emit errors from the gunzip stream', done => {
const error = new Error('Error.');
const createGunzipStream = through();
createGunzipOverride = () => {
setImmediate(() => {
createGunzipStream.emit('error', error);
});
return createGunzipStream;
};
file.createReadStream()
.on('error',
(err: Error) => {
assert.strictEqual(err, error);
done();
})
.resume();
});
});

describe('validation', () => {
Expand Down Expand Up @@ -1022,6 +1084,27 @@ describe('File', () => {
};
});

it('should emit errors from the validation stream', done => {
const error = new Error('Error.');

hashStreamValidationOverride = () => {
setImmediate(() => {
fakeValidationStream.emit('error', error);
});
return fakeValidationStream;
};

file.requestStream = getFakeSuccessfulRequest(data);

file.createReadStream()
.on('error',
(err: Error) => {
assert.strictEqual(err, error);
done();
})
.resume();
});

it('should pass the userProject to getMetadata', done => {
const fakeOptions = {
userProject: 'grapce-spaceship-123',
Expand Down