Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test failures due to #8550 #8676

Open
1 task done
cpcallen opened this issue Nov 28, 2024 · 2 comments · May be fixed by #8689
Open
1 task done

Test failures due to #8550 #8676

cpcallen opened this issue Nov 28, 2024 · 2 comments · May be fixed by #8689
Assignees
Labels
issue: bug Describes why the code or behaviour is wrong

Comments

@cpcallen
Copy link
Contributor

cpcallen commented Nov 28, 2024

Check for duplicates

  • I have searched for similar issues before opening a new one.

Description

Following the merge of #8550, there are now several mocha tests with fail with a considerable degree of reliability (about 100% reproducible on my local machine, and reliably enough in CI to cause test failures to prevent PRs from being merged:

I am planning to temporarily disable these, but we should fix them and then reenable them.

Reproduction steps

  1. Run npm ci
  2. Run npm test
  3. Observe errors.

Stack trace

[16:02:36] Starting 'mocha'...
=======================================
== mocha
Starting webdriverio...
Loading URL: file:///Users/cpcallen/src/blockly/tests/mocha/index.html
============Blockly Mocha Test Failures================
"WorkspaceSvg cleanUp two blocks first at (10, 15) second at (0, 0) do not switch places" failed: block1 is below block2: expected Coordinate{ x: +0, y: 52 } to deeply equal Coordinate{ x: +0, y: 50 }
"WorkspaceSvg cleanUp two overlapping blocks are moved to origin and below" failed: block2 is below block1: expected Coordinate{ x: +0, y: 52 } to deeply equal Coordinate{ x: +0, y: 50 }
"WorkspaceSvg cleanUp two overlapping blocks are moved to origin and below including children" failed: block2 is below block1: expected Coordinate{ x: +0, y: 52 } to deeply equal Coordinate{ x: +0, y: 50 }
"WorkspaceSvg cleanUp two large overlapping blocks are moved to origin and below" failed: block2 is below block1: expected Coordinate{ x: +0, y: 149 } to deeply equal Coordinate{ x: +0, y: 144 }
============Blockly Mocha Test Summary=================
4 tests failed
============Blockly Mocha Test Summary=================
Mocha tests failed
FAILED: mocha
[16:02:54] Finished 'mocha' after 18 s

Screenshots

No response

Browsers

No response

@cpcallen cpcallen added issue: bug Describes why the code or behaviour is wrong issue: triage Issues awaiting triage by a Blockly team member labels Nov 28, 2024
cpcallen added a commit to cpcallen/blockly that referenced this issue Nov 28, 2024
@cpcallen cpcallen removed the issue: triage Issues awaiting triage by a Blockly team member label Nov 29, 2024
@BenHenning
Copy link
Contributor

Did some analysis on this. Across >4500 runs on my local machine of WorkspaceSvg, I saw zero test failures. I suspect this is an environment-specific problem which might explain why it's a 100% repro for you @cpcallen, 0% for me, and flaky on CI.

@BenHenning
Copy link
Contributor

Either way, despite the oddity of why the test behavior even differs across different systems, I think we can possibly improve the robustness of the tests by replacing some of the deep equals checks with relative checks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
issue: bug Describes why the code or behaviour is wrong
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants