feature request: thread start/exit callbacks

[beevik]

Some programs require (1) the ability to initialize per-thread resources whenever a new thread starts executing, and (2) the ability to clean up resources just before the thread exits. For example, performance monitoring tools relying on code instrumentation often initialize per-thread counters on thread start-up and flush them to an output device just before the thread exits.

In libuv there appear to be 3 ways threads can be created:

1. The user directly calls uv_thread_create.
2. The thread pool creates a number of worker threads in response to various libuv work scheduling requests (e.g., fs jobs, DNS address lookups).
3. On OSX, uv__stream_try_select can create a new thread.

In only the first case can the user control the initialization and clean-up of per-thread resources. In the remaining two cases, it's impossible because there is no direct call into the user's code when the thread starts or exits.

To remedy this, I'd like to propose two new libuv APIs:

• One to set an optional callback to be called from within a new thread right after it begins executing.
• One to set an optional callback to be called from within a thread right before it exits.

I've created a branch with the proposed change. See https://github.com/beevik/libuv/releases/tag/thread-cb for the first draft. If you think this is a change worth considering for inclusion in libuv, let me know and I'll make a pull request out of it.

It's possible I missed a way to do this within the current libuv API, so please let me know if that's the case.

[saghul]

What programs do require doing that exactly? Also what is it that needs to be initialized / cleaned up?

Also note that on Windows we use some of Windows's internal thread pools, which we have no control of.

[txdv]

You can get an idea when stuff is being executed, measure how long it takes.

[beevik]

This came up in our project (the one that required the ability to customize the memory allocator). We have a memory and resource tracing system that tracks the resources allocated and freed by each thread. At thread startup, we initialize a thread-local pointer to a trace context. As memory is allocated and freed within a thread, we update the thread's trace context. When the thread exits, we flush data from the trace context to external storage (network or disk) and destroy the context.

The reason this works for us is because our threading model assumes we fully manage all worker and helper threads. Unfortunately, because libuv spawns threads without making our code aware of it, any resource allocation performed within these threads becomes invisible to our tracing system.

The fact that libuv uses Windows thread pools does seem to expose a fatal flaw in my proposal. I'm going to need to think further about this and reconsider whether we can use libuv.

[beevik]

After looking at the code a bit further, it appears libuv uses the Windows threadpool for pipe I/O, "slow" polling, and tty reading. On the other hand, it uses its own worker threadpool for file operations and DNS. Is there any reason why pipes, tty and polling use the Windows threadpool instead of the libuv one?

There appear to be several different threading models being used by different libuv subsystems, although I'm not familiar enough with the code to understand why.

[bnoordhuis]

The reason this works for us is because our threading model assumes we fully manage all worker and helper threads. Unfortunately, because libuv spawns threads without making our code aware of it, any resource allocation performed within these threads becomes invisible to our tracing system.

I want to move to a system where the embedder can plug in a thread pool that's under full control of the embedder (io.js needs it do proper workload management between libuv and V8.) I think that would solve your issue?

[beevik]

I want to move to a system where the embedder can plug in a thread pool that's under
full control of the embedder (io.js needs it do proper workload management between
libuv and V8.) I think that would solve your issue?

I think it would, assuming all the places in libuv that create helper threads also switch to using this thread pool. (If I recall correctly, there is at least one place in the code where a one-off helper thread is created but is not part of any thread pool.)

[bnoordhuis]

You mean the OS X select() thread? I don't think that would be part of it because what it does is not unit-of-work based. It's not really amenable to handing off to a thread pool because it would just hog a thread and never let go.

I don't think you'd have to care about what it does though because it's completely internal to libuv.

[beevik]

I don't think you'd have to care about what it does though because it's completely internal to libuv.

That's probably true for most, but our use case entails overriding the memory allocator. So if the long-lived thread function calls uv__malloc and we are trying to associate those allocations with their threads, it might still be a problem for us.

I realize our use case is somewhat esoteric, and it may not be appropriate to change libuv in ways that will suit only us.

[saghul]

Mi first impression about this is that it sounds too much like a corner case to make it into the library.

You would be able to roll your own threadpool which does whatever when threads are spawned by libuv (or not), see libuv/leps#4, but the select thread is internal to libuv so I don't see a really good reason to make internal threads visible like that. There is also the fact that Windows uses those Windows-y builtin thread pools.

Would it be possible for you to check if the TLS block was created in your uv__malloc replacement function? Or even go The Hard Way (R) and use LD_PRELOAD perhaps?

[beevik]

I also previously considered the lazy uv__malloc initialization solution, but that would be only half of the solution. We'd still need a way to know when the thread exited to clean up any allocated resources. To get around this, perhaps we could add some sort of thread-safe garbage collection mechanism, but then things are starting to get quite complicated.

The LD_PRELOAD idea is interesting, but not cross-platform, which our project is.