[v6.x] test: investigate test-fs-read-buffer-tostring-fail

[refack]

• Version: v6.x-staging
• Platform: macOS
• Subsystem: test

not ok 386 parallel/test-fs-read-buffer-tostring-fail
  ---
  duration_ms: 61.861
  severity: fail
  stack: |-
    timeout
  ...

https://ci.nodejs.org/job/node-test-commit-osx/11278/nodes=osx1010/

[refack]

Test file - https://github.com/nodejs/node/blob/v6.x-staging/test/parallel/test-fs-read-buffer-tostring-fail.js
/cc @nodejs/platform-macos @nodejs/streams @nodejs/buffer

[mcollina]

@refack I don't think it's possible to not have this test as flaky. We are generating a very big file on memory, then we try to save that extremely big file on disk, and then load it. Everything can happen in all this I/O and GC() activity.

However, one thing we can do: convert https://github.com/nodejs/node/blob/v6.x-staging/test/parallel/test-fs-read-buffer-tostring-fail.js#L29-L33 into the pattern as the end of the https://nodejs.org/api/stream.html#stream_writable_write_chunk_encoding_callback (the one using the 'drain' event). That would reduce memory allocation on the Node.js process during the setup phase.

I'm happy to review a PR that addresses this issue, and I think it would be good for someone that wants to improve their streams knowledge.

[refack]

Also in the worse case, could create a PR adding

test-fs-read-buffer-tostring-fail      : PASS,FLAKY

to https://github.com/nodejs/node/blob/v6.x-staging/test/parallel/parallel.status#L7

[Jeyanthinath]

Can I take and contribute to this ?

[bnoordhuis]

@Jeyanthinath Go for it.

[Jeyanthinath]

PR #14472 ( deleted )

[Jeyanthinath]

New PR #14495

[Trott]

I don't think it's possible to not have this test as flaky.

If the test is necessarily flaky on 6.x, then I'm not sure it makes sense to mark it as flaky on 6.x. By marking it as flaky, the results will always be ignored. For a test where there's a chance of fixing the issue, that's fine.

For a test where there's no possible fix and/or no intention of fixing, it may be better to remove the test rather than have a test whose results are ignored.

Or if the test is minimally flaky, then leave the status alone. You'll get red once in a month or whatever, but if it starts going red 100% of the time, you'll know something is (probably) broken.

Not a blocking objection on the "mark as flaky" approach. Just suggestions. Feel free to proceed with the current plan if you disagree.

[Trott]

Oh, not sure it's reasonable if the test is resource-intensive, but another option would be to program in a retry into the test before deciding the test has failed. Not great, but probably better than not having a test or having a test whose failures are discounted.

[refack]

If the test is necessarily flaky on 6.x, then I'm not sure it makes sense to mark it as flaky on 6.x. By marking it as flaky, the results will always be ignored. For a test where there's a chance of fixing the issue, that's fine.

IMHO yellow ⚠️ is better than ❌. It communicates that something it sort of wrong, and if it becomes consistent, it will get investigated. Red just makes people dig, find it's unrelated, and lose faith in the CI.

I do agree that it the test could be rewritten to stabilize it, that is the better option in the longer term.

[mcollina]

Considering the fact that the test is gone from master as it is targeting a deprecated API, I'm definitely 👍 in flagging it as flaky.

[apapirovski]

As far as I can tell this has been resolved by marking it as flaky. Closing.