Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

addons: allow multiple requires of the same addon #19731

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 30 additions & 16 deletions src/node.cc
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@
#include <sys/types.h>

#include <string>
#include <unordered_map>
#include <vector>

#if defined(NODE_HAVE_I18N_SUPPORT)
Expand Down Expand Up @@ -186,7 +187,7 @@ static int v8_thread_pool_size = v8_default_thread_pool_size;
static bool prof_process = false;
static bool v8_is_profiling = false;
static bool node_is_initialized = false;
static node_module* modpending;
static std::unordered_map<std::string, node_module*> mods;
static node_module* modlist_builtin;
static node_module* modlist_internal;
static node_module* modlist_linked;
Expand Down Expand Up @@ -2118,8 +2119,8 @@ extern "C" void node_module_register(void* m) {
mp->nm_flags = NM_F_LINKED;
mp->nm_link = modlist_linked;
modlist_linked = mp;
} else {
modpending = mp;
} else if (mods.count(mp->nm_modname) == 0) {
mods[mp->nm_modname] = mp;
Copy link
Contributor

@gabrielschulhof gabrielschulhof Apr 8, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need a stronger key because this also breaks those projects which use two different versions of the same module. So, like,

project
  |
  +-dep1
  |   |
  |   +-native-addon1(@2.2.0)
  |
  +-native-addon1(@1.2.0)

The distinction between the two modules with identical nm_modname is the path from which they are loaded. So we should save the absolute path of the module we are about to load to a global variable declared next to modpending right before calling dlib.Open(), and then use that path as the key here.

So, like

 static node_module* modpending;
+std::string modpending_filename;

and then

mods[modpending_filename.c_str()] = mp;

in node_module_register(), and

+modpending_filename = std::string(*filename);
 bool is_opened = dlib.Open();

in DLOpen().
In general, I have reservations about attempting to duplicate what the library loader is doing, because that code has very likely been refined to a very high degree over the course of the decades.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are absolutely right, the basename is not unambiguous enough.

I can change it as you suggest - in fact we don't even need to save the pending filename, we can just he the pending node_module * as we do today and use its file name as the key in the std::unordered_map.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@frutiger we must also ensure that two different file names which refer to the same file in reality are treated correctly. I'm not sure to what extent the JS side of require() takes care of resolving files in this way.

Copy link
Contributor

@gabrielschulhof gabrielschulhof Apr 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@frutiger it might be worth also taking a look at the JS side, and maybe adding some more resolution steps to the portion of the code path following the decision to load a native module.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@frutiger I guess we should at least follow symlinks.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not actually sure just following symlinks is enough, nor do I think we would be able to do a good job of maintaining compatibility of this logic with the loader logic in JS. Perhaps we could hash the contents of the file itself?

Copy link
Contributor

@gabrielschulhof gabrielschulhof Apr 17, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally, the JS loader logic would cause the resulting call to dlopen() to always return a new module, and run the constructors. However, this means that we will have reproduced dlopen()'s decision as to whether to load a library or increment the refcount of an existing one. I'm pretty sure we haven't done that.

The decision made by dlopen() as to whether to load a new library, thereby running its constructors, or incrementing the reference count of an existing library, thereby not running its constructors, is what we're trying to replicate here. The code for that decision is very old, very well-vetted, and may even be platform-dependent. This is why I said I was reluctant to attempt to reproduce that code.

In an ideal world we'd have dlreopen() or the RTLD_REOPEN flag which would do exactly what we need. But alas.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess @bnoordhuis' modification whereby it falls back to looking for a well-known symbol is a way to kick-start the addon if it fails to register. But, like you said, this won't work for existing modules. sigh

}
}

Expand Down Expand Up @@ -2227,16 +2228,10 @@ inline InitializerCallback GetInitializerCallback(DLib* dlib) {

// DLOpen is process.dlopen(module, filename, flags).
// Used to load 'module.node' dynamically shared objects.
//
// FIXME(bnoordhuis) Not multi-context ready. TBD how to resolve the conflict
// when two contexts try to load the same shared object. Maybe have a shadow
// cache that's a plain C list or hash table that's shared across contexts?
static void DLOpen(const FunctionCallbackInfo<Value>& args) {
Environment* env = Environment::GetCurrent(args);
auto context = env->context();

CHECK_EQ(modpending, nullptr);

if (args.Length() < 2) {
env->ThrowError("process.dlopen needs at least 2 arguments.");
return;
Expand All @@ -2260,12 +2255,6 @@ static void DLOpen(const FunctionCallbackInfo<Value>& args) {
DLib dlib(*filename, flags);
bool is_opened = dlib.Open();

// Objects containing v14 or later modules will have registered themselves
// on the pending list. Activate all of them now. At present, only one
// module per object is supported.
node_module* const mp = modpending;
modpending = nullptr;

if (!is_opened) {
Local<String> errmsg = OneByteString(env->isolate(), dlib.errmsg_.c_str());
dlib.Close();
Expand All @@ -2278,12 +2267,37 @@ static void DLOpen(const FunctionCallbackInfo<Value>& args) {
return;
}

size_t lastdot = dlib.filename_.find_last_of('.');
CHECK_NE(lastdot, std::string::npos);
CHECK_EQ(lastdot, dlib.filename_.find(".node", lastdot));

#ifdef _WIN32
size_t lastslash = dlib.filename_.find_last_of('\\');
#else
size_t lastslash = dlib.filename_.find_last_of('/');
#endif
size_t basenameloc = 0;
if (lastslash == std::string::npos) {
// No separators in the path
basenameloc = 0;
} else {
// At least one separator in the path
basenameloc = lastslash + 1;
}

std::string name = dlib.filename_.substr(basenameloc, lastdot - basenameloc);
node_module* mp = nullptr;
if (mods.find(name) != mods.end()) {
mp = mods[name];
}

if (mp == nullptr) {
if (auto callback = GetInitializerCallback(&dlib)) {
callback(exports, module, context);
} else {
dlib.Close();
env->ThrowError("Module did not self-register.");
env->ThrowError("Module did not self-register or module required and "
"registered with different names.");
}
return;
}
Expand Down
19 changes: 19 additions & 0 deletions test/addons-napi/9_multiply_required/binding.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#include <node_api.h>
#include "../common.h"
#include <string.h>

napi_value Method(napi_env env, napi_callback_info info) {
napi_value world;
const char* str = "world";
size_t str_len = strlen(str);
NAPI_CALL(env, napi_create_string_utf8(env, str, str_len, &world));
return world;
}

napi_value Init(napi_env env, napi_value exports) {
napi_property_descriptor desc = DECLARE_NAPI_PROPERTY("hello", Method);
NAPI_CALL(env, napi_define_properties(env, exports, 1, &desc));
return exports;
}

NAPI_MODULE(NODE_GYP_MODULE_NAME, Init)
8 changes: 8 additions & 0 deletions test/addons-napi/9_multiply_required/binding.gyp
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"targets": [
{
"target_name": "binding",
"sources": [ "binding.c" ]
}
]
}
8 changes: 8 additions & 0 deletions test/addons-napi/9_multiply_required/test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
'use strict';
const common = require('../../common');

const bindingPath = require.resolve(`./build/${common.buildType}/binding`);

require(bindingPath);
delete require.cache[bindingPath];
require(bindingPath);
5 changes: 0 additions & 5 deletions test/addons/dlopen-ping-pong/test.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,3 @@ process.dlopen(module, bindingPath,
os.constants.dlopen.RTLD_NOW | os.constants.dlopen.RTLD_GLOBAL);
module.exports.load(`${path.dirname(bindingPath)}/ping.so`);
assert.strictEqual(module.exports.ping(), 'pong');

// Check that after the addon is loaded with
// process.dlopen() a require() call fails.
const re = /^Error: Module did not self-register\.$/;
assert.throws(() => require(`./build/${common.buildType}/binding`), re);
15 changes: 15 additions & 0 deletions test/addons/multiply-required-addon/binding.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#include <node.h>
#include <v8.h>

void Method(const v8::FunctionCallbackInfo<v8::Value>& args) {
v8::Isolate* isolate = args.GetIsolate();
args.GetReturnValue().Set(v8::String::NewFromUtf8(isolate, "world"));
}

void define(v8::Local<v8::Object> exports,
v8::Local<v8::Value> module,
v8::Local<v8::Context> context) {
NODE_SET_METHOD(exports, "hello", Method);
}

NODE_MODULE(binding, &define);
9 changes: 9 additions & 0 deletions test/addons/multiply-required-addon/binding.gyp
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
'targets': [
{
'target_name': 'binding',
'defines': [ 'V8_DEPRECATION_WARNINGS=1' ],
'sources': [ 'binding.cc' ]
}
]
}
8 changes: 8 additions & 0 deletions test/addons/multiply-required-addon/test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
'use strict';
const common = require('../../common');

const bindingPath = require.resolve(`./build/${common.buildType}/binding`);

require(bindingPath);
delete require.cache[bindingPath];
require(bindingPath);
2 changes: 1 addition & 1 deletion test/addons/not-a-binding/test.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@
const common = require('../../common');
const assert = require('assert');

const re = /^Error: Module did not self-register\.$/;
const re = /^Error: Module did not self-register or module required and registered with different names\.$/;
assert.throws(() => require(`./build/${common.buildType}/binding`), re);