Feature request: Support __attribute__((constructor)) for easy module loading

[gabrielschulhof]

Related to #1717 would it be possible to implement support for constructors (__attribute__((constructor))) in jerry's C library? I think that's where it needs to be implemented, in the jerry-asm.S file.

Having support for constructors makes it easy to register modules independently of the main program, since all you have to do is compile in a module and it will be available at runtime without any additional code.

[akosthekiss]

@gabrielschulhof What is your use case (hardware, OS, toolchain, program)? Jerry-libc is a very limited/restricted libc variant. Are you really using it instead of the compiler's default libc? (I'm not saying you shouldn't, but you have to know/accept that jerry-libc is really just the bare minimum, at least for now.)

[gabrielschulhof]

The use case is simply to make mixing and matching different modules easy. That is, adding a new module should be as simple as adding a few source files to your project, without having to have those source files exports some well known entry point, and without having to modify your main.c (or whatever) to call those entry points whenever the module mix changes for your project.

Consider the following example. You make one small change to your Makefile:

...
diff a/Makefile b/Makefile
+obj-$(MY_COOL_MODULE) = my_cool_module.c
...

and now, without any further code changes, your project supports

var my_cool_module = require( "my_cool_module" );

Without this feature, you would need

...
diff a/Makefile b/Makefile
+obj-$(MY_COOL_MODULE) = my_cool_module.c
...
diff a/main.c b/main.c
+#ifdef BUILD_WITH_MY_COOL_MODULE
+#include "my_cool_module.h"
+#endif /* def BUILD_WITH_MY_COOL_MODULE */
...
+#ifdef BUILD_WITH_MY_COOL_MODULE
+my_cool_module_init();
+#endif /* def BUILD_WITH_MY_COOL_MODULE */
...

which is a lot harder to maintain.

Also, Node.js uses constructors to start the process of loading modules, so this would increase interoperability with Node.js native addons - a project I'm currently working on 😀

I'm fairly certain that I'm using jerry-libc when the constructor doesn't work, because if I turn off -static and -nostdlib, the constructor works and it links to /lib64/libc.so.6.

I understand that jerry-libc is the bare minimum, which is why I would like to discuss the cost/benefit of adding this feature. The fact is that constructors are something that cannot be implemented by writing a few functions, because people (including me) would need to know assembly to implement that support, whereas features that can be expressed as a set of functions, even if they should go into a C library, need not go in if it is to be minimal, because they can be maintained outside - because they're just plain C code, and a lot more people know how to read/write/maintain that.

And yes, the above example is also C code, so yes, it could be maintained without constructors, but it's all boilerplate which could be avoided with constructors. Besides, constructors are supported by quite a few platforms - all the ones Node.js runs on, at any rate.

[zherczeg]

Question: could this be done in a way that the linker can cut the support when it is not used? At least the majority of it. If yes, I don't mind having this feature.

[akosthekiss]

@zherczeg I don't think so. But it could be a compile time option for jerry-libc whether to support constructors or not. (BTW, it ctors are supported, it's nice to support destructors, too, as they walk hand-in-hand, quite often.)

[akosthekiss]

@gabrielschulhof I get the usefulness of constructors. Where I'd like to pick your mind is why you'd like to use jerry-libc? Especially as you are mentioning Node.js native addons. Because of the minimalistic implementation of jerry-libc, the lack of constructors will be the least of your problems, I'd guess. You have been warned :)

[gabrielschulhof]

Actually, after module registration, things are pretty straight forward, because I'm implementing the new Node.js N-API which maps ECMAScript pretty closely. The only challenges are scopes (which I have implemented as linked lists), and externals (a primitive type not part of ES, but which both N-API-supporting engines (V8 and ChakraCore) expose in order to make passing pointers into JS easier than having to wrap objects).

[gabrielschulhof]

My ultimate goal is to use N-API to make it possible to build Node.js modules on JerryScript without any code changes. So, like, the same C-file will build against Node.js and against JerryScript. This could be good for both projects, because it would make Node.js addons more memory-consumption-conscious, and it would bring more addons to JerryScript. Of course it's all one big experiment 😃

[akosthekiss]

@gabrielschulhof There are two major issues I see:

• JerryScript is not the scaled-down version of Node.js, they are not at the same level of a software stack. Node.js is a whole ecosystem, something that is built on top of a JavaScript engine (V8 in that case). JerryScript is not that. It is a JS engine, say, the scaled-down version of V8 (actually, not, but the analogy works). I know that there is a related issue where discussion is already ongoing - even though a bit late, I'll join the club and speak my thoughts.
• I envision a lot of problems trying to build Node.js native modules against jerry-libc. Constructors are not the only feature that are there in standard libc's (and projects rely upon) but are not there in jerry-libc.

That being said, constructors (and destructors) can be supported by jerry-libc, most probably. I'll try to come up with an implementation in the coming days.

[LaszloLango]

@gabrielschulhof, have you tried to build with newlib or ulibc? IMHO those would be more suitable for your usecase.

[gabrielschulhof]

@akosthekiss I understand that JerryScript is a JS engine and not a full runtime. The fact though is that by its very nature of being designed for constrained environments it provides some features of a runtime - such as a C library. Also, don't forget that I'm not talking about Node.js C++ modules linked directly against V8. I'm talking about the new N-API that has just landed in Node.js master, which is a C interface that seeks to be independent of the underlying JS engine and tracking ES itself.

We have already ported many of the most popular Node.js modules which were using the V8 API to N-API so that we can validate its function, performance, and completeness, and now I'm trying to see if it's possible to provide JerryScript's API as an implementation of N-API.

@LaszloLango I can certainly build with a different C library. In fact, until a few days ago I didn't even know that the system C library will run constructors even if they are linked into an executable :) I thought constructors only work with dlopen() :) So, I created two different entry points for my module - one for Node.js via the library constructor, and one via an exported function. This kept the central code providing the module functionality engine-agnostic, but it made the module registration code engine-specific. I figured that it may be worth implementing constructors in JerryScript to iron out this difference, and then we can use a single header for the API definition in both Node.js and JerryScript.

[akosthekiss]

@gabrielschulhof have a look at #1725, see whether it works for you.

(BTW, one more potential issue came to my mind: I'm not sure how you'd like to register modules but note that a lot of jerry API functions cannot be called until jerry_init is called. Which means that jerry API is effectively blocked for constructor functions.)

[gabrielschulhof]

@akosthekiss Aaaah, I'm glad you brought that up :) The module loading is broken into two steps. Indices into the pointer array are assigned at boot time in response to the constructor call. The module name is also added to a linked list for subsequent lookup by require(). This is context-independent and requires no cooperation from jerryscript. It's purely runtime initialization.

Only when require() gets called from JS is the resulting linked list consulted and, when a module is found, its initialization routine (which returns a jerry_value_t) called. Since the process starts from JS, an initialized context must be present.

This is in contrast to Node.js, where bootup and initialization are one and the same. But then in Node.js every module gets dlopen()-ed in response to a require(), so obviously the environment is already there.

[akosthekiss]

Closing as #1725 has landed