-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DynamoDB and Throttling #402
Comments
Great questions, Carl.
Note that setting a maxRetries value of 0 means the SDK will not retry throttling errors, which is probably not what you want. If the SDK is taking longer, it's usually because you are being throttled or there is some other retryable error being thrown. If you want to debug how the SDK is retrying, you can add a handler to inspect these retries: AWS.events.on('retry', function(resp) {
if (resp.error && resp.error.retryable) {
var date = new Date();
console.log(date, '| Retrying request for the ' + resp.retryCount + 'th time.');
console.log(date, '| Retry triggered by', err.code, err.message);
}
}); That event fires whenever the SDK decides to retry. If AWS.events.on('retry', function(resp) {
// retry all requests with a 5sec delay (if they are retryable)
if (resp.error) resp.error.retryDelay = 5000;
// or alternatively, disable retries completely.
// This is equivalent to setting maxRetries to 0.
if (resp.error) resp.error.retryable = false;
}); Hope that helps! |
Hi, Yes that helps a lot. It works pretty much as I thought it did :) I agree that in general you want the sdk to execute the retries but in our specific case we're not being throttled on the table but rather on a partition but that's another story. Just so that I don't misunderstand, when you mention overriding the properties in AWS.events.on('retry', ...) I assume that doing so is still in the global scope and not possible to do for a specific operation, such as a putItem request? Cheers |
You can add event hooks for individual requests, I was just trying to provide some simple debugging code. To attach the event to an individual request: var req = dynamodb.putItem(params);
req.on('retry', function() { ... });
req.send(function(err, data) {
console.log(err, data);
}); |
Sorry, I completely misread that. Br,
|
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread. |
Hi,
This isn't so much an issue as a question regarding the implementation.
I wonder if and how exponential back offs are implemented in the sdk.
If I create a new dynamo object i see that maxRetries is undefined but I'm not sure exactly what that implies.
var AWS = require('aws'-sdk');
var dynamo = new AWS:DynamoDB();
console.log(dynamo);
When we get throttled on occasion I see that it takes a lot longer for our callback to be called, sometime up to 25 seconds. I haven't had the possibility to debug this so I'm not sure exactly what is happening which is why I am curious as to if and how the maxRetries is used, especially if it is not explicitly passed when creating the dynamo object.
I'm guessing that this might have something to do with this.
From: https://github.com/aws/aws-sdk-js/blob/master/lib/services/dynamodb.js
For arguments sake I will assume that the default retires are in fact 10 and that this is the logic that is applied for the exponential back off and have a follow up question on this:
Is there any way to control the number of retires for a specific call. I.e. I have my dynamo object with the default settings and I call putItem once and for that specific call I'd like to have a different maxRetries (in my case 0) but still use the same object. I suspect this is not feasible?
Looking forward to your response and some additional insight on this fine module :)
Br,
Carl
The text was updated successfully, but these errors were encountered: