clang-12 + Release mode causes zlib failures (apparent data corruption)

[djmarcin]

Version

16.3.0, 16.4.0, 16.4.2

Platform

Linux 5.8.0-53-generic #60~20.04.1-Ubuntu SMP Thu May 6 09:52:46 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Subsystem

zlib

What steps will reproduce the bug?

Install clang 12.

$ sudo apt install clang-12
$ clang-12 --version
Ubuntu clang version 12.0.1-++20210605063018+0826268d59c6-1~exp1~20210605043737.99
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

Download and build Node with clang-12

$ curl -sL https://github.com/nodejs/node/archive/refs/tags/v16.4.2.tar.gz | tar xz
$ cd node-16.4.2 && export CC="clang-12" CXX="clang++-12" && ./configure && make -j$(nproc) BUILDTYPE=Release V=0 -C out

Run the following script with the produced binary:

$ curl -o /tmp/prettier.tgz https://registry.npmjs.org/prettier/-/prettier-2.2.1.tgz
$ cat - << EOF | out/Release/node
const fs = require('fs');
const { createGunzip } = require('zlib');
const { pipeline } = require('stream');
let inFile = fs.createReadStream('/tmp/prettier.tgz');
let outFile = fs.createWriteStream('/tmp/prettier.tar');
pipeline(inFile, createGunzip(), outFile, (err) => {
  if (err) {
    console.error('An error occurred:', err);
    process.exitCode = 1;
  }
});
EOF

Observe output:

An error occurred: Error: incorrect data check
    at Zlib.zlibOnError [as onerror] (node:zlib:191:17) {
  errno: -3,
  code: 'Z_DATA_ERROR'
}

How often does it reproduce? Is there a required condition?

Always when compilied in Release mode with clang-12

What is the expected behavior?

prettier.tar should be produced without errors

What do you see instead?

An error occurred: Error: incorrect data check
    at Zlib.zlibOnError [as onerror] (node:zlib:191:17) {
  errno: -3,
  code: 'Z_DATA_ERROR'
}

Additional information

This error does not appear when:

• Compiling with BUILDTYPE=Debug and using out/Debug/node
• Compiling with clang-11 (tested with Ubuntu clang version 11.0.0-2~ubuntu20.04.1)
• Compiling with GCC (tested with gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0)

This error is also not present when compiling upstream zlib (https://zlib.net/) with clang-12.

This issue breaks npm install for prettier and several other packages as well because the files cannot be decompressed.

Built on an AMD Threadripper CPU.

[jesec]

Refs: vercel/pkg-fetch#208

I can reproduce this issue with Apple Clang compiled macos-x64 executable.

clang -v
Apple clang version 13.0.0 (clang-1300.0.29.3)
Target: arm64-apple-darwin21.0.0
Thread model: posix
InstalledDir: /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

./configure && make

At least Node 14 and 16 release lines are impacted.

Interestingly, the issue does not appear with arm64 executable compiled with the same toolchain.

This could become a release-critical bug, once the new Clang/Xcode versions roll in. I suggest that we re-triage this issue as zlib, confirmed-bug.

I am looking at the issue right now, and to raise attention here are some pings to recent committers to the zlib parts: @RaisinTen, @addaleax, @mscdex, @aduh95, @ronag .

[RaisinTen]

Perhaps we need to apply the fix referred to in #31800 (comment) to solve this issue. But we need #33044 to land first to be able to update our copy of zlib.

[meta-github]

Newer version of clang (tested on macOS with Xcode 13.2.1) seem to fail to generate correct code when compiling zlib/contrib/optimizations/inffast_chunk.c with -O2 or higher optimization. -O1 seems to work fine.

[lpinca]

#45387 landed. I cannot reproduce on macOS with clang 14. Can this be closed?

[meta-github]

Did the fix find its way into all currently supported versions of node.js?

[lpinca]

Did the fix find its way into all currently supported versions of node.js?

#45387 is not yet included in any version.

[meta-github]

I suggest to close the bug after the upstream fix has been merged into all currently supported versions of node.js.

[lpinca]

Why? we usually close the issue when the fix lands on main. If the issue is fixed on main there is no reason to keep it open.

[meta-github]

I can confirm that the bug no longer exists on main. So, you might close this if you want. I still need to keep my workaround in my build script for releases until all releases are fixed.

[lpinca]

Thank you for the confirmation.

[fabOnReact]

I experience this issue with clang-14, nvm, node 16.15.1, npm 8.11.0, macOS 13.3.1 (22E261). Using nvm install v16.15.1 --shared-zlib fixed the issue. npm install would fail with Error code Z_DATA_ERROR Zlib: incorrect data check -3. I followed this instructions to fix it

• https://stackoverflow.com/a/71528293/7295772
• https://github.com/nvm-sh/nvm#macos-troubleshooting