Timeouts

[GeoffreyPlitt]

Sometimes my jobs get stuck, just because of some error that fails to call the callback correctly. But then the queue stops processing and nothing happens--forever.

I was surprised to see there isn't a default timeout, or a way to set timeouts, on jobs. Is this on purpose?

[behrad]

Sometimes my jobs get stuck, just because of some error that fails to call the callback correctly. But then the queue stops processing and nothing happens--forever.

with any stuck active job increase you loose one job of you workers concurrency limit, so when stuck active jobs reaches that limit, you worker feels full and no job will be processed, until you restart node.js process!
So the best practice is to guard your process against errors using node.js domains OR promises so that you can catch uncaughtExcaptions and call done

I was surprised to see there isn't a default timeout, or a way to set timeouts, on jobs. Is this on purpose?

Redis doesn't support EXPIRE on ZADD or LPUSH operations, so we should have implemented it ourselves in kue with a helper timeout set (=are they really STUCK!?), but I think it isn't worth it since it is a work around for the actual problem, this limits your application logic, in many cases you can't say my job will SURE go no longer than t seconds, and BIG ts won't really help in high traffic deployments. So you should finally do proper exception handling.
Another workaround is to watch for active job list, and delete jobs older than some age (=are they really STUCK!?) this is what was doing in production.

I had written a patch using domains inside workers, so when something goes wrong, Kue can automatically call done(err) but at that time I had two unclear points about the whole error handling thing:

1. How should the err passed to you as the worker code? (with 'error' emitted?)
2. What happens if caller already uses domains/promises for error handling? worker domains then omits error from being catched there. It seems very hard to chain domain errors, should Kue throw someError ?

I've postponed this patch till 0.9, However I think it's time to make it experimentally available in 0.9

[behrad]

please read #403 AND #391 AND #418

[behrad]

Resolved? shall it be closed?

[GeoffreyPlitt]

Sure.

On Fri, Nov 7, 2014 at 1:54 AM, Behrad [email protected] wrote:

Resolved? shall it be closed?

—
Reply to this email directly or view it on GitHub
#456 (comment).

http://www.geoffplitt.com
http://facebook.com/geoffrey.plitt
https://twitter.com/GeoffreyPlitt
773.339.0915