Still experiencing memory leak with version [email protected] #576

weemsdj · 2019-09-10T14:26:30Z

We have also seen the memory leak in our production environment with [email protected] and recently updated to 2.6.1. It seems memory leak is still an issue and we had to revert back to using 2.5.3 to stabilize our code.

Not certain how soon this can be addressed, but we are staying with version 2.5.3

RabeaWahab · 2019-09-25T11:55:02Z

We are also experiencing it with 2.6.1, rolling back and will keep watching this issue.

peterikladious · 2019-09-27T16:15:50Z

Also seeing it -- In my case a 16x increase in memory. 2.5.3 works for us.

balajic · 2019-09-30T18:51:54Z

I am also seeing the same issue.. Kindly fix it.

balajic · 2019-09-30T20:19:16Z

BTW, Thanks to @peterikladious suggestion.
2.5.3 version saved me. No leaks..
I am staying with that..

balajic · 2019-10-15T17:21:48Z

@bimalkjha Any update on this issue? I am struck with 2.5.3

bimalkjha · 2019-10-23T13:39:25Z

@balajic Please install latest code from github using command npm install git+https://[email protected]/ibmdb/node-ibm_db.git and verify it. I have delivered some code for src/odbc.cpp file which is expected to fix blob data corruption and memory leak issue. I suspect memory leak was happening due to use of Nan::Utf8String, that I removed now. Thanks.

balajic · 2019-10-31T14:29:29Z

@bimalkjha
I am trying to install it on a new folder /tmp/ibmtest folder on my RHEL server and it fails.
Before that I did execute npm init on this folder.

First I tried version 2.5.3 and it worked fine.

admin@rhel:/tmp/ibmtest$ npm install [email protected]

> [email protected] install /tmp/ibmtest/node_modules/ibm_db
> node installer/driverInstall.js

IBM_DB_HOME environment variable have already been set to -> /home/db2inst1/sqllib

Downloading of clidriver skipped - build is in progress...

make: Entering directory `/tmp/ibmtest/node_modules/ibm_db/build'
  CXX(target) Release/obj.target/odbc_bindings/src/odbc.o
  CXX(target) Release/obj.target/odbc_bindings/src/odbc_connection.o
  CXX(target) Release/obj.target/odbc_bindings/src/odbc_statement.o
  CXX(target) Release/obj.target/odbc_bindings/src/odbc_result.o
  SOLINK_MODULE(target) Release/obj.target/odbc_bindings.node
  COPY Release/odbc_bindings.node
make: Leaving directory `/tmp/ibmtest/node_modules/ibm_db/build'

npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN [email protected] No description
npm WARN [email protected] No repository field.

+ [email protected]
added 107 packages from 97 contributors and audited 182 packages in 12.626s
found 1 high severity vulnerability
  run `npm audit fix` to fix them, or `npm audit` for details

Then I tried the Current Production Version 2.6.1 - worked fine

admin@rhel:/tmp/ibmtest$ rm -rf node_modules/
admin@rhel:/tmp/ibmtest$ npm install ibm_db

> [email protected] install /tmp/ibmtest/node_modules/ibm_db
> node installer/driverInstall.js

platform =  linux
IBM_DB_HOME environment variable have already been set to -> /home/db2inst1/sqllib

Downloading of clidriver skipped - build is in progress...

make: Entering directory `/tmp/ibmtest/node_modules/ibm_db/build'
  CXX(target) Release/obj.target/odbc_bindings/src/odbc.o
  CXX(target) Release/obj.target/odbc_bindings/src/odbc_connection.o
  CXX(target) Release/obj.target/odbc_bindings/src/odbc_statement.o
  CXX(target) Release/obj.target/odbc_bindings/src/odbc_result.o
  SOLINK_MODULE(target) Release/obj.target/odbc_bindings.node
  COPY Release/odbc_bindings.node
make: Leaving directory `/tmp/ibmtest/node_modules/ibm_db/build'

npm WARN [email protected] No description
npm WARN [email protected] No repository field.

+ [email protected]
added 107 packages from 97 contributors and audited 182 packages in 9.211s
found 0 vulnerabilities

Finally the latest version from git and failed. No firewall issues with accessing the site.

admin@rhel:/tmp/ibmtest$ rm -rf node_modules/
admin@rhel:/tmp/ibmtest$ npm install git+https://[email protected]/ibmdb/node-ibm_db.git
npm ERR! path git
npm ERR! code ENOENT
npm ERR! errno ENOENT
npm ERR! syscall spawn git
npm ERR! enoent Error while executing:
npm ERR! enoent undefined ls-remote -h -t https://[email protected]/ibmdb/node-ibm_db.git
npm ERR! enoent
npm ERR! enoent
npm ERR! enoent spawn git ENOENT
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/aradmin/.npm/_logs/2019-10-31T14_25_48_314Z-debug.log


admin@rhel:/tmp/ibmtest$ curl https://[email protected]/ibmdb/node-ibm_db.git
<html>
<head><title>301 Moved Permanently</title></head>
<body>
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx</center>
</body>
</html>
admin@rhel:/tmp/ibmtest$

Some more logs

admin@rhel:/tmp/ibmtest$ head -n 50 /home/aradmin/.npm/_logs/2019-10-31T14_25_48_314Z-debug.log
0 info it worked if it ends with ok
1 verbose cli [ '/usr/bin/node',
1 verbose cli   '/usr/bin/npm',
1 verbose cli   'install',
1 verbose cli   'git+https://[email protected]/ibmdb/node-ibm_db.git' ]
2 info using [email protected]
3 info using [email protected]
4 verbose npm-session 93a1bfb0e0973655
5 silly install loadCurrentTree
6 silly install readLocalPackageData
7 silly fetchPackageMetaData error for git+https://[email protected]/ibmdb/node-ibm_db.git Error while executing:
7 silly fetchPackageMetaData undefined ls-remote -h -t https://[email protected]/ibmdb/node-ibm_db.git
7 silly fetchPackageMetaData
7 silly fetchPackageMetaData
7 silly fetchPackageMetaData spawn git ENOENT
8 timing stage:rollbackFailedOptional Completed in 8ms
9 timing stage:runTopLevelLifecycles Completed in 54ms
10 verbose stack Error: spawn git ENOENT
10 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:240:19)
10 verbose stack     at onErrorNT (internal/child_process.js:415:16)
10 verbose stack     at process._tickCallback (internal/process/next_tick.js:63:19)
11 verbose cwd /tmp/ibmtest
12 verbose Linux 3.10.0-1062.1.2.el7.x86_64
13 verbose argv "/usr/bin/node" "/usr/bin/npm" "install" "git+https://[email protected]/ibmdb/node-ibm_db.git"
14 verbose node v10.16.3
15 verbose npm  v6.9.0
16 error path git
17 error code ENOENT
18 error errno ENOENT
19 error syscall spawn git
20 error enoent Error while executing:
20 error enoent undefined ls-remote -h -t https://[email protected]/ibmdb/node-ibm_db.git
20 error enoent
20 error enoent
20 error enoent spawn git ENOENT
21 error enoent This is related to npm not being able to find a file.
22 verbose exit [ 1, true ]

Tail of the log

admin@rhel:/tmp/ibmtest$ tail -n 50 /home/aradmin/.npm/_logs/2019-10-31T14_25_48_314Z-debug.log
0 info it worked if it ends with ok
1 verbose cli [ '/usr/bin/node',
1 verbose cli   '/usr/bin/npm',
1 verbose cli   'install',
1 verbose cli   'git+https://[email protected]/ibmdb/node-ibm_db.git' ]
2 info using [email protected]
3 info using [email protected]
4 verbose npm-session 93a1bfb0e0973655
5 silly install loadCurrentTree
6 silly install readLocalPackageData
7 silly fetchPackageMetaData error for git+https://[email protected]/ibmdb/node-ibm_db.git Error while executing:
7 silly fetchPackageMetaData undefined ls-remote -h -t https://[email protected]/ibmdb/node-ibm_db.git
7 silly fetchPackageMetaData
7 silly fetchPackageMetaData
7 silly fetchPackageMetaData spawn git ENOENT
8 timing stage:rollbackFailedOptional Completed in 8ms
9 timing stage:runTopLevelLifecycles Completed in 54ms
10 verbose stack Error: spawn git ENOENT
10 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:240:19)
10 verbose stack     at onErrorNT (internal/child_process.js:415:16)
10 verbose stack     at process._tickCallback (internal/process/next_tick.js:63:19)
11 verbose cwd /tmp/ibmtest
12 verbose Linux 3.10.0-1062.1.2.el7.x86_64
13 verbose argv "/usr/bin/node" "/usr/bin/npm" "install" "git+https://[email protected]/ibmdb/node-ibm_db.git"
14 verbose node v10.16.3
15 verbose npm  v6.9.0
16 error path git
17 error code ENOENT
18 error errno ENOENT
19 error syscall spawn git
20 error enoent Error while executing:
20 error enoent undefined ls-remote -h -t https://[email protected]/ibmdb/node-ibm_db.git
20 error enoent
20 error enoent
20 error enoent spawn git ENOENT
21 error enoent This is related to npm not being able to find a file.
22 verbose exit [ 1, true ]

balajic · 2019-10-31T14:32:33Z

@bimaljha Hold on. This server doesn't have git. Let me try after installing it

balajic · 2019-10-31T14:51:28Z

@bimalkjha Installation went through fine after installing git.. However it didn't address the Memory Leak.
See below.

admin@rhel:/tmp/ibmtest$ npm install git+https://[email protected]/ibmdb/node-ibm_db.git

> [email protected] install /tmp/ibmtest/node_modules/ibm_db
> node installer/driverInstall.js

platform =  linux
IBM_DB_HOME environment variable have already been set to -> /home/db2inst1/sqllib

Downloading of clidriver skipped - build is in progress...

make: Entering directory `/tmp/ibmtest/node_modules/ibm_db/build'
  CXX(target) Release/obj.target/odbc_bindings/src/odbc.o
  CXX(target) Release/obj.target/odbc_bindings/src/odbc_connection.o
  CXX(target) Release/obj.target/odbc_bindings/src/odbc_statement.o
  CXX(target) Release/obj.target/odbc_bindings/src/odbc_result.o
  SOLINK_MODULE(target) Release/obj.target/odbc_bindings.node
  COPY Release/odbc_bindings.node
make: Leaving directory `/tmp/ibmtest/node_modules/ibm_db/build'

npm WARN [email protected] No repository field.

+ [email protected]
updated 1 package and audited 239 packages in 24.454s
found 0 vulnerabilities

Before Start

While Running

BTW, I am (ab)using ibm_db to replace DataStage and implemented true massive child process to extract the data in parallel. I am just pushing the limit on the server, firing 50 parralel queries at the same time and hence you see the Memory/CPU shoots up in a sec on this 16 Core 62 GB Memory server.

However when I use [email protected] memory consumption hardly goes up, may be 100 - 200 MB maximum 1GB.

wnwaynelee · 2019-11-10T14:37:46Z

@bimalkjha After running a bunch of 2.5.3 vs master tests, I also confirmed that master's memory usage is not as good as 2.5.3's.

In terms of memoryUsage.heapTotal, memoryUsage.heapUsed, and memoryUsage.external, master is very close to 2.5.3. That's why the current tests for memory leak do not flag any issue. In terms of memoryUsage.rss, both 2.5.3 and master grow. However, 2.5.3's growth rate is so slow, when compared to master's, it looks like 2.5.3 levels off and master keeps growing.

Attached is a zip file with

test-blob-insert-memoryUsage.js - a test I created base on test-blob-insert.js to show the issue
memoryUsage.2.5.3vMaster.html - chart of my test results from running test-blob-insert-memoryUsage.js comparing 2.5.3 and master

memoryUsage.zip

dexdexdex · 2019-11-18T22:38:34Z

This is something our project is experiencing as well. Memory usage has been fine on 2.5.0, we recently upgraded to 2.6.1 and noticed bad behavior.

alanramsay · 2019-11-29T13:16:25Z

We have experienced this also with 2.6.1. Going back to 2.5.1 fixed our memory leak.

hiro5id · 2019-12-08T08:18:18Z

Also having problems with memory on 2.6.1 with Ubuntu Linux. Unfortunately I don't have the option to go back to 2.5.1 because we are running Node 12 and 2.5.1 fails to install on Node 12

* doc: update for issue #593 (Bimal Jha) * fix: update windows binary using latest code (Bimal Jha) * fix: for memory leak issue #576 (Bimal Jha) * fix: update windows binary for vscode 1.40.x (Bimal Jha) * update windows binaries (Bimal Jha) * fix: add executeNonQuerySync in odbc.js issue #595 (Bimal Jha) * fix: ignore sqlcode 100 by executeNonQuery, issue #591 (Bimal Jha) * fix: use v8::Isolate for nodev >= 11 (Bimal Jha) * fix: update unzipper version, issue #588 (Bimal Jha) * Support for node v12 on z/OS, and fix some test cases for z/OS (#586) (alexcfyung) * force push connection to the queue when poolSize breaches maxPool boundry (#581) (ashutoshrnjn) * doc: update doc for executeNonQuerySync API, issue #583 (Bimal Jha) * fix: blob data corruption issue #582 (Bimal Jha) * support for install --debug option on windows (Priyanka Manoharan) * doc: Docker Linux Container instructions (Bimal Jha)

bimalkjha · 2019-12-09T14:45:20Z

@weemsdj @RabeaWahab @peterikladious @balajic @wnwaynelee @dexdexdex @alanramsay @hiro5id We have delivered fix of memory leak issue and restored the behavior at 2.5.3 level.
Please install new version of [email protected] and verify. Thanks.

wahlbergfredrik · 2019-12-17T13:57:47Z

The [email protected] update seems to resolve the memory leak issues for us. But it also seems to make the overall performance of our service worse by 50-100% (the service takes ~50-100% longer time to complete) in comparison to the version running on [email protected].

The only difference for our Node.js application between these tests is the update to [email protected] (from [email protected]) and update from Node 10 to Node12.

Is there any reason for this drop in performance?

bimalkjha · 2019-12-18T07:28:06Z

@wahlbergfredrik Thanks for confirming that memory leak issue is fixed.

For performance issues, we have another issue #269 in open state. We can use that open issue for performance thing.

Also, I have doubt about the performance degradation claim. If you see the last two comments of issue 269, node.js v12.x and [email protected] has far better performance compared to [email protected]. That issue has a test program too and I too have added perTest.js to check the performance. If you test schenario is different, please share a similar test program like perfTest.js so that I can check the performance issue. You can update on issue 269 or open a new issue.

Thanks.

RabeaWahab · 2019-12-24T20:17:25Z

@bimaljha memory leak is not reproducible anymore, thank you.

bimalkjha added a commit that referenced this issue Dec 9, 2019

fix: for memory leak issue #576

585b3b0

bimalkjha closed this as completed Dec 9, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Still experiencing memory leak with version [email protected] #576

Still experiencing memory leak with version [email protected] #576

weemsdj commented Sep 10, 2019

RabeaWahab commented Sep 25, 2019

peterikladious commented Sep 27, 2019

balajic commented Sep 30, 2019

balajic commented Sep 30, 2019

balajic commented Oct 15, 2019

bimalkjha commented Oct 23, 2019 •

edited

Loading

balajic commented Oct 31, 2019

balajic commented Oct 31, 2019

balajic commented Oct 31, 2019

wnwaynelee commented Nov 10, 2019

dexdexdex commented Nov 18, 2019

alanramsay commented Nov 29, 2019

hiro5id commented Dec 8, 2019 •

edited

Loading

bimalkjha commented Dec 9, 2019

wahlbergfredrik commented Dec 17, 2019

bimalkjha commented Dec 18, 2019

RabeaWahab commented Dec 24, 2019

Still experiencing memory leak with version [email protected] #576

Still experiencing memory leak with version [email protected] #576

Comments

weemsdj commented Sep 10, 2019

RabeaWahab commented Sep 25, 2019

peterikladious commented Sep 27, 2019

balajic commented Sep 30, 2019

balajic commented Sep 30, 2019

balajic commented Oct 15, 2019

bimalkjha commented Oct 23, 2019 • edited Loading

balajic commented Oct 31, 2019

balajic commented Oct 31, 2019

balajic commented Oct 31, 2019

wnwaynelee commented Nov 10, 2019

dexdexdex commented Nov 18, 2019

alanramsay commented Nov 29, 2019

hiro5id commented Dec 8, 2019 • edited Loading

bimalkjha commented Dec 9, 2019

wahlbergfredrik commented Dec 17, 2019

bimalkjha commented Dec 18, 2019

RabeaWahab commented Dec 24, 2019

bimalkjha commented Oct 23, 2019 •

edited

Loading

hiro5id commented Dec 8, 2019 •

edited

Loading