[Discussion] API to associate binding state (pointers) with the JerryScript context

[martijnthe]

A while ago, JERRY_CONTEXT define and jerry_context_t had been added.
Before that change, all the fields in jerry_context_t were statics that were sprinkled throughout the JerryScript code base. This wasn't great for projects where multiple instances of JerryScript are needed, because statics are by definition "singleton".

Luckily, the addition of the JERRY_CONTEXT define made it easier to create multiple instances of JerryScript. Typically, to support multiple instances, one would provide a custom definition of JERRY_CONTEXT, for example to look up the current jerry_context_t in thread local storage.

However, often, binding code also needs to keep state regarding the native resources that it uses. In a multi-context scenario, each instance of JerryScript needs to have its own "instance" of this binding state as well. Therefore, using static variables cannot be used. Again, because static equals "singleton".

Using __thread or a platform-specific thread local storage (TLS) API could be used of course, but this would make the binding code less portable / re-usable.

Therefore, I think it make sense to try to come up with an API to associate binding state with a JerryScript context in an easy way that promotes re-use and portability of the binding code.

Some design goals that I have in mind for this an API to associate binding state with a JerryScript context:

• enable multiple bindings to each associate their own state with a JerryScript context.
• making binding code "portable" / re-usable: avoid __thread or a platform specific TLS API.
• should not require having to change JerryScript code (incl. jcontext.[h|c]).
• "scales down": no bloat when using a single-context (same speed/RAM/code size as today).
• easy to use from a developer point of view (probably not as easy as static, but hopefully close?)

I imagining something like a void *user_binding_state[] array in JERRY_CONTEXT, plus a couple of jerry_... functions to register/allocate/get/free the state for a particular binding. But I'm not quite sure how such a solution would "scale down".

Thoughts?

[martijnthe]

cc @jiangzidong @zherczeg ideas?

[zherczeg]

What about adding two functions: one for setting a global user pointer and another to getting this pointer?

[martijnthe]

What about adding two functions: one for setting a global user pointer and another to getting this pointer?

That would mean you'd have to put all the pieces of state from various bindings into one big struct: not great for keeping the various bindings modular.
Or have some custom mechanism on top of the single user pointer to "mux" the single global one into a separate pointer per binding module: also not great, because everyone would implement their own mechanism for this over and over and bindings would not be easily reusable across projects (unless we magically all implement the exact same thing).

I added a "design goal" to the list: enable multiple bindings to each associate their own state with a JerryScript context.

[jiangzidong]

About "scale down":
IMO, we don't need the perfect "scale down"
It's ok to add a pointer member in the jerry_context_t, and the pointer points to a struct/array which contains binding states. Then the memory for each jerry context increases sizeof(void *), which should be acceptable.

[jiangzidong]

@martijnthe

because everyone would implement their own mechanism for this over and over and bindings would not be easily reusable across projects

It's hard to implement a general binding state struct/array so that we can reuse other binding codes directly, because binding scenarios are various.
@zherczeg 's method, binding-writer should define what's the pointer points to. A struct or array, totally up to the binding-writer.
@martijnthe 's array method, binding-write should also define the length of the array, and meanings of index.

[zherczeg]

As for me I would offer such things as an option. From JerryScript core side it would only be a pointer, but there would be utilities which offer more, and people could select what fits their use case. I described this concept in #1716.

[gabrielschulhof]

You have to also consider that there may be multiple mutually exclusive uses for adding a pointer to the context. For example, V8's isolate has SetData() and the corresponding GetData(), but when we were implementing N-API we wanted to tack binding-specific data onto the Isolate using those APIs but we couldn't, because there are other native modules out there that also do that, making use of one of the four slots.

So, if you're a binding and you retrieve the pointer from the context, is this the pointer you set, or is this the pointer some other binding set, because its init ran before your init?

Fortunately, we're dealing with a JavaScript engine here, right? So, we could create an object that is not exposed to JavaScript and attach it to the context, and then various bindings could claim various property keys on the object and attach their data as the value.

So, like

#define MY_BINDING_DATA_KEY "my_binding_data_key"
...
my_binding_data *my_data = get_my_binding_data();
jerry_value_t context_object = jerry_get_context_object(context);
jerry_value_t my_property_value = jerry_create_object();
jerry_set_object_native_handle(my_property_value, (uintptr_t)my_data, free_my_binding_data);
jerry_value_t prop_name = jerry_create_string_from_utf8(MY_BINDING_DATA_KEY);
jerry_set_property(context_object, prop_name, my_property_value);
...

Basically, I wish to caution that a single pointer is insufficient because there may be multiple independent reasons for attaching data to a context. Thus, we should be exploiting the fact that JavaScript objects are string-keyed hashes.

The example above could be made more efficient with a custom API, of course. After all, we do not need entire JS objects hanging from the one context object, but only native pointers. The important thing IMO is that the pointers are stored in a string-keyed hash, though.

[gabrielschulhof]

For reference: nodejs/abi-stable-node#206 (comment)

[gabrielschulhof]

Basically, we ended up going with a per-module context, allocated when the module is first initialized, and then stored in the context for all native callbacks. This has the following downside: nodejs/node#12246 (review)

[gabrielschulhof]

Another nice benefit of implementing the JS external-data-plus-deleter approach is that, when a context is discarded, the deleters associated with the data will also be called, and so the bindings will be informed that a certain context has been discarded. This is also missing in V8.

[zherczeg]

Hm, what about reversing the question? Instead allowing setting pointers in JerryScript context, we could retrieve the starting address of the context, which should be unique. This could be a key to access various local properties.

[gabrielschulhof]

So have a context-keyed hash stored statically and globally in the binding, right? That's also possible, of course, as long as there is one well-known, well-documented key that can be used. Currently it sounds like the jerry_context_t* is that key.

[gabrielschulhof]

Then the binding won't know that a context has been discarded though and so it won't know that it can delete a key from the hash. There should be a callback to inform about that.

[gabrielschulhof]

Also it might be real nice if bindings didn't have to implement their own hash table, because five different bindings will implement it five different ways.

[gabrielschulhof]

It just ends up being boilerplate.

[zherczeg]

Yes. I do think the global context must be unique for each JerryScript instances, regardless where it is stored. Otherwise they would mess up the global state of others.

The other thing which could be acceptable is having a chain list of pages, where the head is stored in JerryScript, and each page must start with a next and key pointer pair. The key pointer should point to a static const data, where this data could start with a pointer to a free function. This free function could be called when a JerryScript instance is destroyed. The downside is that each data binding must do a linear search to find their corresponding page, which could be costly if too many of them.

The problem is no matter what we offer, there will be downsides, and there will be people who will don't like those downsides. That is why I prefer a system, where people can choose their appropriate solution, rather then forcing one onto them.

[gabrielschulhof]

If that's the case, you're basically asking various independent projects to agree on a way to attach arbitrary data to a context, and implement a library that does that. You will then make room in the context by giving them a void* so they can actually use that library.

[zherczeg]

It will likely working this way. Probably there will be 2-3 major libraries which projects can use. They will offer different advantages and their perf/memory cost will be different. Probably there will be a very high level library, which offers generic interfaces at high perf/memory costs, and another utility based library which just offers useful utility functions. The primary aim for JerryScript is that it must work well on a system with 64K RAM and 100MHz 32 bit CPU. That does not offer much space to generic hash tables, especially if the memory is highly fragmented.

[zherczeg]

However, when you have 512K or more memory and the CPU is 600MHz, the story is different.

[zherczeg]

Probably shared interfaces could help this. Think about malloc. We have several malloc implementations, TCmalloc, JEmalloc, libc malloc, etc, and you can select the best according to your needs. E.g. TCmalloc is faster but use more memory than others, etc. However, they still offer the same malloc/free/realloc interface. Probably what we should do is standardizing certain interfaces, but their implementation could be replaceable. The recommended interfaces could go into a common interface library.

[gabrielschulhof]

What about if instead of a void* a context as made available to the public would be a jerry_value_t that was created with global_context.public_context = jerry_value_acquire(jerry_create_object())? That would be a very easy common interface, and the entire JerryScript API that deals with associating external data can be made available.

Is that really so heavy?

In fact, internally, jerry_init() could perform

jerry_set_object_native_handle(global_context.public_context,
  (uintptr_t)&global_context, NULL);

and you could document that if a caller ever did that with the context object the behaviour would be undefined (because they would overwrite internal JerryScript state).

[gabrielschulhof]

Wait a sec, I just realized you need a context to do jerry_get_object_native_handle() later on. Ha! Ha!

Anyway, instead of a void*, you could still provide a jerry_acquire_value(jerry_create_object())-based jerry_value_t instead.

[gabrielschulhof]

There won't be that many contexts lying around, right? Just, like, one per thread, so it's not like you're creating thousands of new objects here.

[martijnthe]

Re. @gabrielschulhof 's idea of using a JS object that is non-reachable from the JS code itself: that would add quite a bit of overhead in terms of runtime speed to access the state.

How about something like:

// jerryscript.h

// Module read-only data.
typedef struct
{
    size_t state_size_bytes;
    void (*init_cb)(void *module_state);
    void (*deinit_cb)(void *module_state);
} jerry_module_t;

// Module index. Assigned at run-time.
// This is not part of jerry_module_t, so that jerry_module_t can be read-only.
typedef int jerry_module_idx_t;

// Registers a module and returns the assigned module index.
// Should only be called once upon booting the system.
jerry_module_idx_t jerry_module_register(const jerry_module_t *module);

// Getter that a module can call to get its state (after being instantiated).
void *jerry_module_get_state(int jerry_module_idx_t);

// Macro Sugar:
#define JERRY_NATIVE_MODULE_DECLARE(name, state_size_bytes, init_cb, deinit_cb) \
static jerry_module_idx_t _jerry_module_id__##name; \
static const _jerry_module_##name = { \
	state_size_bytes, \
	init_cb, \
	deinit_cb, \
}; \
__attribute__((constructor)) \
static void _jerry_module_auto_register__##name (void) \
{ g_jerry_module_id__##name = jerry_module_register(&_jerry_module_##name ); }

#define JERRY_NATIVE_MODULE_GET_STATE(name) jerry_module_get_state(_jerry_module_id__##name)

// my_binding_a.c

typedef struct {
    	int my_binding_x;
    	char my_binding_yz;
} my_binding_state_t;

static void my_init(void *module_state) {
	my_binding_state_t *my_state = module_state;
	// init all the things:
	my_state->my_binding_x = 1;
	my_state->my_binding_yz = 'I';
	// ... any other things it needs to do, open files, start jet engines, etc.
}

static void my_deinit(void *module_state) {
	// clean up here: close files, stop the engines, ...

	// Do NOT free(module_state), will already be taken care of after this function returns.
}

static jerry_value_t my_native_method(...) {
         my_binding_state_t *state = JERRY_NATIVE_MODULE_GET_STATE(my_binding);
        state->my_binding_x = jerry_get_number_value(arg_p[0]);
}

JERRY_NATIVE_MODULE_DECLARE(my_binding, sizeof(my_binding_state_t), my_init, my_deinit);

// jcontext.h
typedef struct
{
    // ... all the existing stuff here ...
    void **module_state_map;
} jerry_context_t;

// config.h
#define CONFIG_MAX_MODULES (8)

// jerry-module.c
static jerry_module_idx_t g_jerry_module_next_index;
static jerry_module_t g_jerry_module_map[CONFIG_MAX_MODULES];

jerry_module_idx_t jerry_module_register(const jerry_module_t *module) {
	g_jerry_module_map[g_jerry_module_next_index] = module;
	return g_jerry_module_next_index++;
}

// Called right after jerry_init()
void jerry_module_init(jerry_module_idx_t *idxs, size_t num_idxs) {
	const int num_modules = g_jerry_module_next_index;
	JERRY_CONTEXT(module_state_map) = jerry_port_alloc(num_modules * sizeof(void *));
	JERRY_ASSERT(JERRY_CONTEXT(module_state_map));
	for (int i = 0; i < i < num_idxs; ++i) {
                const jerry_module_idx_t idx = g_jerry_module_map[idxs[i]];
                JERRY_ASSERT(module_idx < g_jerry_module_next_index);
		void *module_state = jerry_port_alloc(g_jerry_module_map[idx].state_size_bytes);
		JERRY_CONTEXT(module_state_map)[idx] = module_state;
		g_jerry_module_map[idx].init_cb(module_state);
	}
}

// Called just before jerry_deinit()
void jerry_module_deinit(jerry_module_idx_t *idxs, size_t num_idxs) {
	// Basically similar to jerry_module_init()
	// but then to call deinit_cb() and after that free all the things.
}

[zherczeg]

I don't mind creating a local object. It could be created at the first time when its value is requested. I have a question though: the object properties are accessed by strings, what will guarantee that different modules use different strings? The address of a static variable is more unique.

[zherczeg]

// config.h
#define CONFIG_MAX_MODULES (8)

The whole point so far was that we don't know the number of modules at compile time. What happens if a module is loaded/unloaded? Unused modules should not be loaded, that consumes a lot of memory.

[martijnthe]

What happens if a module is loaded/unloaded? Unused modules should not be loaded, that consumes a lot of memory.

OK, well you could have something like:

void jerry_module_init(jerry_module_idx_t *idxs, size_t num_idxs) {
  // .. only create state for indices that were passed in.
}

The whole point so far was that we don't know the number of modules at compile time.

I think as a project developer you do know the maximum possible number at compile time. I'm not considering loading dynamic libs or anything like that, I think that's probably out of scope ;)

However, a module developer does not know what other modules will be included (both compile+run time) by the project developer.

[gabrielschulhof]

Name clashes with strings are easy to resolve. See DNS or npm. It's your project name. Or you can use uuidgen :)

[zherczeg]

I'm not considering loading dynamic libs or anything like that, I think that's probably out of scope ;)

Why? In iot.js only those modules are loaded which are actually used to save memory.

[gabrielschulhof]

Also, the project developer chooses which modules to bundle so I don't think they're likely to bundle two modules woth an identical name.

[martijnthe]

require_impl() needs to at least take a jerry_context_t* the_context and set a value on it

@gabrielschulhof re. "set a value" (the binding's state): happens in jerry_module_init(), see my earlier sketch (#1717 (comment)).

I guess I didn't show using the state. I'll edit the earlier sketch to show this as well.

[gabrielschulhof]

Oh, right ... I see.

[martijnthe]

The address of the global static module definition structure can serve as the unique key that identifies the module.

Yes, but... After thinking about it a bit longer: I think it's worthwhile assigning an index for each module (at boot, "registration time"). That would make the look-up of the binding's state fast and simple vs. having to look up by address (need a data structure more complicated than a simple array to make that possible).

I think it makes sense to optimize for fast look-up because that is what is likely to happen a lot.

If we don't like...

// config.h
#define CONFIG_MAX_MODULES (8)

... we could also make that array dynamic and realloc when needed, but I suspect most project developers will know at compile time what the max number of modules is.

[gabrielschulhof]

Yeah, in Zephyr.js, before going with the linked list I also thought about making an array of these module definitions. In JerryScript a run-time fixed-size, compile-time configurable array totally makes sense.

[gabrielschulhof]

Then, I guess each context will have an array where the indices are the same as what the modules receive at startup and the values are the module state.

[zherczeg]

I don't really see a theoretical difference between a compile time defined array and a single pointer. Both has the same limitations.

Based on this conversion, my proposal is the following:

JerryScript side (pseudo code only, the names should be better of course)

/* Returns a unique pointer */
void * jerry_get_instance (void);
/* Gets/sets a global pointer in the JerryScript context. */
void *jerry_get_context_pointer (void);
void jerry_set_context_pointer (void *context_p);
/* Registers a function which is called when the context is about to be destroyed. */
void jerry_set_on_destroy (jerry_on_destroy_callback_t on_destroy_callback);

In the JerryScript utility library, there would be an interface:

/* When this interface is used, it controls the context pointer of JerryScript. */

/* Registers a module. Context destroy callback is called when the JerryScript context is destroyed.
   Initial value for the own pointer of the module is also set. Returns 0 on error. */
size_t jerry_module_register (jerry_on_destroy_callback_t on_destroy_callback, void *module_p);
/* Unregisters a module. */
void jerry_module_unregister (size_t);
/* Get/set the module pointer. */
void * jerry_module_get_pointer (size_t module_id, void *module_p);
void jerry_module_set_pointer (size_t module_id, void *module_p);

All modules must use this interface. There could be several implementations: fixed array, chain list, hashmap, and the developer chooses the appropriate one depending on the use case. However the modules must not depend on any implementation, only the interface.

[jiangzidong]

@zherczeg I agree with you except for 2 minor questions. Just to make sure.

1. jerry_get_instance means return the address of current jerry_context_t right?

2. About the two types of interface (jerry side and utility side), I think only utilities can use the jerry-side interface (like jerry_get_context_pointer ), and bindings/modules can only use the utility-side inferface (like jerry_module_register ), right?

[zherczeg]

I just realized that a module cannot store the module id, since it needs a storage. Hence we need a unique pointer (id). Also it would be a good idea to specify the size of the module specific area, since the manager could allocate a single area for a header and a module specific space (useful for a chain list or hashmap based implementation). The modified interface:

// id - it must be a unique pointer, an address of a const data, or a callback
//  function address, a constant string address, whatever
// on_destroy_callback is called when the JerryScript instance is destroyed
// area size - size of the module specific area. If a module needs a re-allocatable
// space, it should be sizeof (void*), and the module just store a pointer there
// returns the same value as jerry_module_get_pointer, or NULL if failed
void * jerry_module_register (void *id_p, jerry_on_destroy_callback_t on_destroy_callback, size_t area_size);
// unregisters a module
void jerry_module_unregister (void *id_p);
// gets the module specific space starting address
void * jerry_module_get_pointer (void *id_p);

@jiangzidong:

1. Not necessarily. Only one thing is sure: it is a unique pointer for each jerryscript instance. At first we will return with the address of jerry_context_t, but it could be changed in the future. Modifying this area is not allowed.

2. Yes, it is a dependency. If somebody uses a standard module manager utility (what a name:) ), it will exclusively control this pointer. However if somebody wants to have an own module manager system, or their system has only a single module, they can directly control this pointer. Standard modules should use standard module managers, that can be a dependency as well.

I think "standard module" could be defined somewhere, and each standard module must follow these guidelines (this is independent from JerryScript). Non-standard modules can do whatever they want, and the system developers must handle them.

[zherczeg]

Another idea: id could be a static string, and it should be the name of the module, so the name of the active modules could be printed for standard modules. This is just an idea, we could drop it if you don't like it.

Also jerry_get_instance retuns a unique id for the active JerryScript instances. If an instance is deleted, its instance id can be reused later by a new JerryScript instance.

[martijnthe]

@zherczeg Questions about your proposal:

• jerry_module_register(): would it be called once per boot or every time when initing JerryScript (after jerry_init())?

I like your idea of having a "standard" interface that supports multiple implementations that have different design goals.

Suggestion: I'd rather roll up jerry_on_destroy_callback_t on_destroy_callback and size_t area_size into a typedef'd struct. The pointer to that struct can can be used instead of the void *id. So:

// Module read-only data.
typedef struct
{
    size_t area_size_bytes;
    void (*on_destroy_cb)(void *module_state_p);
    char *name; // if we want to add this we can do so w/o breaking the `jerry_module_register` interface.
} jerry_module_t;

void * jerry_module_register(const jerry_module_t *module_p);

[zherczeg]

It is up to you. I think you have more experience with modules, and for me this is kind of outside the scope of jerryscript core, so I am open to all ideas. I also like your struct based register idea.

The only thing I don't want is adding too much module specific stuff to JerryScript core, which is a JS engine not a NodeJS like system.

I don't mind if we create a new top level directory in jerryscript, e.g. jerry-utilites, and add a module directory there (jerry-utilities/module), where the documentation and source code of standard modules could be placed. I also don't mind adding compilation support to these utilities to the core build system.

[gabrielschulhof]

After having spoken with @martijnthe we believe we have identified the minimum necessary support from core:

typedef void *(*jerry_user_context_init_cb)(void);
typedef void (*jerry_user_context_deinit_cb)(void *context);

typedef struct {
  // ... existing stuff
  void *user_context;
  jerry_user_context_deinit_cb deinit_cb;
} jerry_context_t;

// init_cb is called before jerry_init_with_user_context() returns
// (so no need to save in jerry_context_t)
void jerry_init_with_user_context (jerry_init_flag_t flags,
					  jerry_user_context_init_cb init_cb,
				  jerry_user_context_deinit_cb deinit_cb);

void *jerry_get_user_context(void);

This allows each context to be initialized with a custom "user context", which is allocated and stored during context initialization. It cannot be re-initialized during the lifetime of the context, and it will be deallocated using the provided deleter upon context disposal. The original jerry_init() then becomes

void jerry_init (jerry_init_flag_t flags) {
  jerry_init_with_user_context (flags, NULL, NULL);
}

I will make a separate PR to this effect, affecting only core. I will then render the module utility PR as a second modification that requires this change (and that will likely use #1725 for testing).

[martijnthe]

Note that by design, we did not provide a "setter", but instead an init_cb, to enforce that the context is only set once. (Having a "setter" might otherwise be used by multiple libraries and therefore cause problems.)

[gabrielschulhof]

Exactly, and we want the init_cb and the deinit_cb to appear next to each other to emphasize that there is one user context manager for each context, established when the context is instantiated.

[zherczeg]

I like this proposal! +1 from me

[poussa]

LGTM