ES Module loading with abolute path fails on windows

[jonerer]

• Version: v13.8.0
• Platform: Windows 10 64-bit version 1809
• Subsystem: "esm" maybe?

What steps will reproduce the bug?

1. Create a new project in C:\prog\node\genast
2. Add "type": "module" in package.json
3. Make a "other.js" with some content, like

const thing = "hej"
export default thing

4. Make "index.js" with the following:

import other from "./other.js"
console.log(other)

> node index.js
hej

(as expected!)

6. Change index.js to:

import other from "C:\\prog\\node\\genast\\other.js"
console.log(other)

It errors out with:
internal/modules/run_main.js:54
internalBinding('errors').triggerUncaughtException(
^

Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only file and data URLs are supported by the default ESM loader
at Loader.defaultResolve [as _resolve] (internal/modules/esm/resolve.js:33:11)
at Loader.resolve (internal/modules/esm/loader.js:85:40)
at Loader.getModuleJob (internal/modules/esm/loader.js:188:40)
at ModuleWrap. (internal/modules/esm/module_job.js:42:40)
at link (internal/modules/esm/module_job.js:41:36) {
code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'
}

Using forward- och backward slashes doesn't make a difference.

How often does it reproduce? Is there a required condition?

100%

What is the expected behavior?

Absolute includes should work the same as relative

What do you see instead?

Additional information

The same issue is seen with both static ES imports and dynamic ES imports.

[devsnek]

You'd need to write file:///c:/x/y/z instead of c:/x/y/z

[kiranpwr1260]

As per node.js documentation:

The specifier of an import statement is the string after the from keyword, e.g. 'path' in import { sep } from 'path'. Specifiers are also used in export from statements, and as the argument to an import() expression.

There are four types of specifiers:

• Bare specifiers like 'some-package'. They refer to an entry point of a package by the package name.

• Deep import specifiers like 'some-package/lib/shuffle.mjs'. They refer to a path within a package prefixed by the package name.

• Relative specifiers like './startup.js' or '../config.mjs'. They refer to a path relative to the location of the importing file.

• Absolute specifiers like 'file:///opt/nodejs/config.js'. They refer directly and explicitly to a full path.

Bare specifiers, and the bare specifier portion of deep import specifiers, are strings; but everything else in a specifier is a URL.

Only file: and data: URLs are supported. A specifier like 'https://example.com/app.js' may be supported by browsers but it is not supported in Node.js.

Specifiers may not begin with / or //. These are reserved for potential future use. The root of the current volume may be referenced via file:///.

[jonerer]

@kiranpwr1260 thanks!

It works on mac os though. Maybe not the same way you posted, but it works according to expectation.

On macos, if you do this:

1. type: module in package.json
2. In index.js, put

import "/Users/<username>/prog/node/genast/other.mjs"

3. In other.mjs, put console.log('wep')
4. node index.js
   Outputs, as expected, wep

I expected the same to be true on windows, hence the issue report :)

Thanks @devsnek as well!
In the issue I posted over on "jest" about this, they said you could use require('url').pathToFileURL(specifier).href to turn a path into a "file url". But it seems a bit weird if this is required for windows but not macos (and presumably linux). A lot of the packages I depend on will probably not know about this

[SimenB]

Yeah, I think this is either a bug on macos & linux or windows.

Mac and Linux (alpine docker image, to be precise):

$ echo 'export default "hello!";' > file.mjs
$ node -e 'import(require.resolve("./file.mjs")).then(console.log, console.error)'
(node:28) ExperimentalWarning: The ESM module loader is experimental.
[Module] { default: 'hello!' }

Windows:

$ echo 'export default "hello!";' > file.mjs
$ node -e 'import(require.resolve("./file.mjs")).then(console.log, console.error)'
(node:3036) ExperimentalWarning: The ESM module loader is experimental.
Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only file and data URLs are supported by the default ESM loader
    at Loader.defaultResolve [as _resolve] (internal/modules/esm/resolve.js:33:11)
    at Loader.resolve (internal/modules/esm/loader.js:85:40)
    at Loader.getModuleJob (internal/modules/esm/loader.js:188:40)
    at Loader.import (internal/modules/esm/loader.js:163:28)
    at importModuleDynamically (internal/modules/cjs/loader.js:1094:27)
    at exports.importModuleDynamicallyCallback (internal/process/esm_loader.js:37:14)
    at Object.<anonymous> (C:\Users\IEUser\script.js:1:16)
    at Module._compile (internal/modules/cjs/loader.js:1151:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
    at Module.load (internal/modules/cjs/loader.js:1000:32) {
  code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'
}

With url.pathToFileURL it works, so we can use that in Jest. I'd still consider this a bug in node, though

[devsnek]

the issue is that C:/ is ambiguous with a URL. (like HTTP:/) and we use URLs in the loader.

[SimenB]

From a user perspective it's really weird that absolute paths work for some OSes, but not all. Would it make sense to throw on absolute paths without file:// on mac and linux as well?

[devsnek]

you should use https://nodejs.org/api/url.html#url_url_pathtofileurl_path

[SimenB]

Yeah, I've changed to use that in Jest now, but it's still odd to me that it behaves differently across OSes, even though pathToFileURL fixes it. You have to actually test on windows to notice absolute paths doesn't work (in our case, we only tested windows on Node 12, while our test only runs on Node 13), then find about that API (followed by either fixing your own code, or reporting it to some library).

Maybe the error message could include something about passing the file path provided through pathToFileURL? Should hopefully mean less googling for people hitting it

[ExE-Boss]

This is because C:/path/to/module.js gets parsed as:

URL {
	scheme: "c:",
	host: null,
	path: "/path/to/module.js",
}

And Node doesn’t know how to deal with a URL with a scheme of c:.

What you want is file:///C:/path/to/module.js:

URL {
	scheme: "file:",
	host: "",
	path: "/C:/path/to/module.js",
}

[SimenB]

Yeah, I understand why it happens, I just disagree with the behavior that absolute paths are treated differently between platforms, and would prefer it to just accept either relative paths or absolute paths that has the file protocol to avoid the foot gun of code working fine on one platform and not the other.

That ship might have sailed though 🙂

[devsnek]

@SimenB it doesn't accept paths, it accept urls. like i said, the correct thing to do if you have a path is to use pathToFileURL.

[SimenB]

Absolute paths (either unix or windows style) aren't valid URLs, though.

$ node -p 'new URL("/Users/user/file.js").href'
internal/url.js:243
  throw new ERR_INVALID_URL(input);
  ^

TypeError [ERR_INVALID_URL]: Invalid URL: /Users/user/file.js

(same behavior in Chrome)

I 100% agree with passing the path through url.pathToFileURL first (that's the fixed I've applied in Jest), but I think it should be required on all platforms, not just Windows.

[jonerer]

So to summarize and clarify, ESM importing right now works like this:

• relative paths:
  • works on mac/linux
  • works on windows
• absolute paths:
  • works on mac/linux
  • doesn't work on windows
• "local file urls"
  • works on mac/linux
  • works on windows

This issue report is about the inconsistency regarding "absolute paths".

For my money, I would love to have absolute paths work on windows as well. It only makes sense that a function to include other files can actually handle paths. At least when running in a node.js environment.

I understand the difficulties differentiating between a windows absolute path and an url, since the drive letter could be confused with a URL scheme. And single-letter schemes seem to be valid according to this RFC: https://tools.ietf.org/html/rfc3986#section-3.1 . But I think it's safe to assume that introducing single-letter URL schemes in a world where windows exists is not likely to happen

[devsnek]

@SimenB they aren't valid absolute urls, but they are valid as relative urls (https://url.spec.whatwg.org/#path-absolute-url-string), which is why they're accepted in import.

I'm not sure how we could detect the intent of someone when they typed it, since that's all that separates an absolute path from a relative path absolute url string.

[SimenB]

Aha, gotcha! Didn't think about the case when it's a path with a separately provided base. Thanks for being patient with me 👍

$ node -p 'new URL("/Users/user/file.js", "http://example.com/some/path").href'
http://example.com/Users/user/file.js

So what essentially happens is import(specifier) => import(new URL(specifier, import.meta.url)) to resolve the absolute path? In that case I don't really have any ideas on how to best detect it 😅 I assume being stricter (i.e. only treating paths starting with ./ as relative and not /) is not an option? Would probably be bad for browser interop.

I think this should be highlighted in the docs somewhere, although I'm not entirely sure where. pathToFileURL is only mentioned in one place in the current ESM docs, and that's in an example transform loader. Maybe in the section talking about relative specifiers to import()? The 2 relative examples use ./, might be an idea to mention / will be treated as relative to the root of the FS, which breaks on Windows? While I now (finally) understand why it behaves the way it does, it feels like a footgun if you're migrating from require to import.

[devsnek]

adding something to the docs seems reasonable to me.

[guybedford]

As discussed, this behaviour is by design. A docs PR to help clarify further would be very welcome.

[d3x0r]

It would be nice to be able to load an absolute path... in this case I don't have a way to set the current working directory, and file:///c:/... or file://c:/... do not work. It ends up prepending the current path (which again I don't have control over) and attempting to load like c:\tmp\file:\c:\ for node file:///c:/tmp/shell/node_modules/... I just happen to be in c:\tmp when testing this, but that is not likely to be the path.