[BUG] Npm install ETIMEDOUT

[MrNakai]

Is there an existing issue for this?

• I have searched the existing issues

This issue exists in the latest npm version

• I am using the latest npm

Current Behavior

I am having problems on any installation with npm. See an example log below :
__
0 verbose cli [
0 verbose cli '/home/pi/.config/nvm/versions/node/v17.1.0/bin/node',
0 verbose cli '/home/pi/.config/nvm/versions/node/v17.1.0/bin/npm',
0 verbose cli 'i',
0 verbose cli 'npm',
0 verbose cli '--verbose'
0 verbose cli ]
1 info using [email protected]
2 info using [email protected]
3 timing npm:load:whichnode Completed in 1ms
4 timing config:load:defaults Completed in 9ms
5 timing config:load:file:/home/pi/.config/nvm/versions/node/v17.1.0/lib/node_modules/npm/npmrc Completed in 4ms
6 timing config:load:builtin Completed in 4ms
7 timing config:load:cli Completed in 13ms
8 timing config:load:env Completed in 2ms
9 timing config:load:file:/home/pi/Melting-potes/.npmrc Completed in 1ms
10 timing config:load:project Completed in 8ms
11 timing config:load:file:/home/pi/.npmrc Completed in 7ms
12 timing config:load:user Completed in 8ms
13 timing config:load:file:/home/pi/.config/nvm/versions/node/v17.1.0/etc/npmrc Completed in 0ms
14 timing config:load:global Completed in 2ms
15 timing config:load:validate Completed in 7ms
16 timing config:load:credentials Completed in 6ms
17 timing config:load:setEnvs Completed in 6ms
18 timing config:load Completed in 66ms
19 timing npm:load:configload Completed in 66ms
20 timing npm:load:setTitle Completed in 1ms
21 timing npm:load:setupLog Completed in 2ms
22 timing config:load:flatten Completed in 16ms
23 timing npm:load:cleanupLog Completed in 10ms
24 timing npm:load:configScope Completed in 0ms
25 timing npm:load:projectScope Completed in 2ms
26 timing npm:load Completed in 133ms
27 timing arborist:ctor Completed in 4ms
28 timing arborist:ctor Completed in 1ms
29 timing arborist:ctor Completed in 0ms
30 timing arborist:ctor Completed in 1ms
31 timing idealTree:init Completed in 128ms
32 timing idealTree:userRequests Completed in 18ms
33 silly idealTree buildDeps
34 silly fetch manifest npm@*
35 silly placeDep ROOT npm@ OK for: want: *
36 timing idealTree:#root Completed in 464709ms
37 timing idealTree:node_modules/npm Completed in 0ms
38 timing idealTree:buildDeps Completed in 464721ms
39 timing idealTree:fixDepFlags Completed in 3ms
40 timing idealTree Completed in 464884ms
41 timing command:install Completed in 464967ms
42 verbose type system
43 verbose stack FetchError: request to http://registry.npmjs.org/npm failed, reason: connect ETIMEDOUT 2606:4700::6810:1423:80
43 verbose stack at ClientRequest. (/home/pi/.config/nvm/versions/node/v17.1.0/lib/node_modules/npm/node_modules/minipass-fetch/lib/index.js:110:14)
43 verbose stack at ClientRequest.emit (node:events:390:28)
43 verbose stack at Socket.socketErrorListener (node:_http_client:447:9)
43 verbose stack at Socket.emit (node:events:402:35)
43 verbose stack at emitErrorNT (node:internal/streams/destroy:164:8)
43 verbose stack at emitErrorCloseNT (node:internal/streams/destroy:129:3)
43 verbose stack at processTicksAndRejections (node:internal/process/task_queues:83:21)
44 verbose cwd /home/pi/Melting-potes
45 verbose Linux 5.10.63-v7+
46 verbose argv "/home/pi/.config/nvm/versions/node/v17.1.0/bin/node" "/home/pi/.config/nvm/versions/node/v17.1.0/bin/npm" "i" "npm" "--verbose"
47 verbose node v17.1.0
48 verbose npm v8.1.2
49 error code ETIMEDOUT
50 error syscall connect
51 error errno ETIMEDOUT
52 error network request to http://registry.npmjs.org/npm failed, reason: connect ETIMEDOUT 2606:4700::6810:1423:80
53 error network This is a problem related to network connectivity.
53 error network In most cases you are behind a proxy or have bad network settings.
53 error network
53 error network If you are behind a proxy, please make sure that the
53 error network 'proxy' config is set properly. See: 'npm help config'
54 verbose exit 1
__
no proxy problem (see config below)
__
I tried to set registry in http or https: same result ( respond to ping by the way).
I also tried to reinstall everything, clean the cache several times, delete the package-lock file, deactivate the firewall of my box.

node version v17.1.0
npm version 8.1.2

I have exhausted all the leads I could find online, can you help me with this problem? thanks in advance

Expected Behavior

normal install from npm

Steps To Reproduce

I'm not really sure how, but each time i reinstall node and npm I meet the same outcome

Environment

• npm: 8.1.2
• Node: 17.1.0
• OS :Raspbian 11 bullseye
• platform: Raspberry Pi 3
• npm config:
  ; "default" config from default values

_auth = (protected)
access = null
all = false
allow-same-version = false
also = null
audit = true
audit-level = null
auth-type = "legacy"
before = null
bin-links = true
browser = null
ca = null
cache = "/home/pi/.npm"
cache-max = null
cache-min = 0
cafile = null
call = ""
cert = null
ci-name = null
cidr = null
color = true
commit-hooks = true
depth = null
description = true
dev = false
diff = []
diff-dst-prefix = "b/"
diff-ignore-all-space = false
diff-name-only = false
diff-no-prefix = false
diff-src-prefix = "a/"
diff-text = false
diff-unified = 3
dry-run = false
editor = "vi"
engine-strict = false
fetch-retries = 2
fetch-retry-factor = 10
fetch-retry-maxtimeout = 60000
fetch-retry-mintimeout = 10000
fetch-timeout = 300000
force = false
foreground-scripts = false
format-package-lock = true
fund = true
git = "git"
git-tag-version = true
global = false
global-style = false
globalconfig = "/home/pi/.config/nvm/versions/node/v17.1.0/etc/npmrc"
heading = "npm"
https-proxy = null
if-present = false
ignore-scripts = false
include = []
include-staged = false
include-workspace-root = false
init-author-email = ""
init-author-name = ""
init-author-url = ""
init-license = "ISC"
init-module = "/home/pi/.npm-init.js"
init-version = "1.0.0"
init.author.email = ""
init.author.name = ""
init.author.url = ""
init.license = "ISC"
init.module = "/home/pi/.npm-init.js"
init.version = "1.0.0"
json = false
key = null
legacy-bundling = false
legacy-peer-deps = false
link = false
local-address = null
location = "user"
lockfile-version = null
; loglevel = "notice" ; overridden by cli
logs-max = 10
; long = false ; overridden by cli
maxsockets = 15
message = "%s"
metrics-registry = "http://registry.npmjs.org/"
node-options = null
node-version = "v17.1.0"
noproxy = [""]
npm-version = "8.1.2"
offline = false
omit = []
only = null
optional = null
otp = null
pack-destination = "."
package = []
package-lock = true
package-lock-only = false
parseable = false
prefer-offline = false
prefer-online = false
prefix = "/home/pi/.config/nvm/versions/node/v17.1.0"
preid = ""
production = null
progress = true
proxy = null
read-only = false
rebuild-bundle = true
; registry = "https://registry.npmjs.org/" ; overridden by user
save = true
save-bundle = false
save-dev = false
save-exact = false
save-optional = false
save-peer = false
save-prefix = "^"
save-prod = false
scope = ""
script-shell = null
searchexclude = ""
searchlimit = 20
searchopts = ""
searchstaleness = 900
shell = "/bin/bash"
shrinkwrap = true
sign-git-commit = false
sign-git-tag = false
sso-poll-frequency = 500
sso-type = "oauth"
strict-peer-deps = false
; strict-ssl = true ; overridden by user
tag = "latest"
tag-version-prefix = "v"
timing = false
tmp = "/tmp"
umask = 0
unicode = true
update-notifier = true
usage = false
user-agent = "npm/{npm-version} node/{node-version} {platform} {arch} workspaces/{workspaces} {ci}"
userconfig = "/home/pi/.npmrc"
version = false
versions = false
viewer = "man"
which = null
workspace = []
workspaces = null
yes = null

; "user" config from /home/pi/.npmrc

registry = "http://registry.npmjs.org/"
strict-ssl = true

; "cli" config from command line options

loglevel = "silent"
long = true

[rlucha]

We have observed the same issue recently.

[tpeter29]

I've noticed this too. It seems to hang and then disconnect when trying to generate a new angular application. If I change the npm version to 6.14.14, it works fine. Once I move to version 7 or 8, it stops working. I am behind a corporate proxy and use the appropriate settings in the .npmrc file.

[douph1]

With this old version (npm v6) I have no problem.

added 3962 packages from 2188 contributors and audited 3981 packages in 114.588s
found 583 vulnerabilities (24 low, 176 moderate, 295 high, 88 critical)
run npm audit fix to fix them, or npm audit for details

2838 https connection acquired and released but no more than 50 connection established at a time !
duration 114seconds

npm doc v6
https://docs.npmjs.com/cli/v6/using-npm/config?v=true#proxy
A proxy to use for outgoing https requests. If the HTTPS_PROXY or https_proxy or HTTP_PROXY or http_proxy environment variables are set, proxy settings will be honored by the underlying request library.
https://docs.npmjs.com/cli/v6/using-npm/config?v=true#https-proxy
A proxy to use for outgoing https requests. If the HTTPS_PROXY or https_proxy or HTTP_PROXY or http_proxy environment variables are set, proxy settings will be honored by the underlying request library.

npm was using request library

The next 15.x.y version doesn't work with http proxy :
node:15.0.0(npm 7.0.2) / 15.0.1(npm 7.0.3) / 15.1.0(npm 7.0.8)
npm doc v7
https://docs.npmjs.com/cli/v7/using-npm/config?v=true#proxy
A proxy to use for outgoing http requests. If the HTTP_PROXY or http_proxy environment variables are set, proxy settings will be honored by the underlying request library.
https://docs.npmjs.com/cli/v7/using-npm/config?v=true#https-proxy
A proxy to use for outgoing https requests. If the HTTPS_PROXY or https_proxy or HTTP_PROXY or http_proxy environment variables are set, proxy settings will be honored by the underlying make-fetch-happen library.

npm verb stack TypeError [ERR_INVALID_PROTOCOL]: Protocol "https:" not supported. Expected "http:"
npm verb stack at new NodeError (node:internal/errors:258:15)
npm verb stack at new ClientRequest (node:_http_client:155:11)
npm verb stack at request (node:https:313:10)
npm verb stack at /usr/local/lib/node_modules/npm/node_modules/minipass-fetch/lib/index.js:80:17
npm verb stack at new Promise ()
npm verb stack at fetch (/usr/local/lib/node_modules/npm/node_modules/minipass-fetch/lib/index.js:39:10)
npm verb stack at /usr/local/lib/node_modules/npm/node_modules/make-fetch-happen/index.js:297:14
npm verb stack at /usr/local/lib/node_modules/npm/node_modules/promise-retry/index.js:29:24

The next version since 15.2.0 are all creating more than 3000 https connection simultaneous throug the proxy !
node 15.2.0(npm 7.0.8), 15.14.0 (npm 7.7.6)

When sometime I didn't get the timeout, the time to get the package is up from 114s in npm v6 to ~ 6min in npm v7+
as too much parallelism is probably not well handle by the proxy or the network.

I have also tested with latest NPM
[email protected] with npm 9.1.3 ( update with npm -g install npm@9 )
It also consume 3600 parallel connection to the proxy.

All test done under docker with squid or mitmproxy
Both proxy show far too many parallel CONNECT request through the proxy ( no reuse ) and no wait the first finish to ask another connection.

To allow squid to handle more than ~800 parallel connection simultaneously by default and I have adapted the number file limit.
In both case, because 3600 parallel connection simultaneously to the proxy is to much or delayed, we got
ECONNRESET like #4652 or ETIMEDOUT

Thanks CLOUDFLARE to handle easily so many new https connection.

According to the official online doc and the test above, the problem is clearly since NPM v7 and probably the new make-fetch-happen library instead of request library

Older library managed to not consume too many parallel connection to the https proxy ( max 50 seen ).

--maxsockets x has no effect.

As a possible workaround for people who manage Squid on Ubuntu Focal if you think than handle more than 1000 parallel connection is needed ( and your infra can handle it )

https://bugs.launchpad.net/ubuntu/+source/squid/+bug/1943536

systemctl show squid.service | grep NOFILE
LimitNOFILE=524288
LimitNOFILESoft=1024

squidclient mgr:info | grep descriptors
Maximum number of file descriptors: 1024
Available number of file descriptors: 1018
Reserved number of file descriptors: 100

echo "max_filedescriptors 65535" >> /etc/squid/squid.conf

service squid restart

squidclient mgr:info | grep descriptors
Maximum number of file descriptors: 65535
Available number of file descriptors: 65529
Reserved number of file descriptors: 100

[douph1]

PS: I forgot to mention that the 3000+ https connexion to the proxy are made in 2 or 3 seconds
(reproduced on CPU with 8 core in Jenkins and locally on laptop with 8 core too )

[Thai22]

but is it necessary to do downgrade npm version in order to work with react next js

[douph1]

Maybe soon fixed with @npmcli/[email protected] when it will be included on new Node/Npm release :

• [BUG] maxSockets option not respected for proxied connections make-fetch-happen#59
• https://github.com/npm/agent/releases/tag/v2.1.0
• dbb18f4

Thanks @lukekarrys