Incorrect NVDA Bot responses: "Navigate to the bottom of a modal dialog" (Modal Dialog Example, Test 14.1, V24.05.09)

[IsaDC]

Description of Behavior

I still getting the results from the previous NVDA version.

Test Setup

• Test File: test-14-navToEndModalDialog-nvda.collected.html
• AT: NVDA (Version 2024.1)
• Browser: Chrome (Version 124)

[IsaDC]

Now I get this when trying to re-run the test plan:

• AT: NVDA 2023.3.3
• Browser: Chrome 99
  There are no results to evaluate.
  I have already tried signing out and back in and performing a hard refresh, but the issue persists.

Cc @jugglinmike

[IsaDC]

Then, when I finally get some results, they don't match with the manual results. E.G.

<style> </style>

Test	Bot Results	Manual Results
Test 6:Navigate to the bottom of a modal dialog	A blank line	button    Verify Address  button    Add  button    Cancel
Test 8:Bump into the top edge of a modal dialog	A blank line	heading    level 2  Add Delivery Address

 

[gnarf]

I set this up for modal tests with NVDA/chrome on the staging site (which is using the same automation backend) and it responded using 2024.1, I'm not sure where the old version results were sneaking in, but all new NVDA automation runs should be 2024.1 since May 9

The "blank line" results are confusing, they don't seem related to the other keypress errors, I'll have to do some local testing to see if I can get a screen recording of it running this test.

[gnarf]

The empty results were tracked down to a problem with these tests rowNumber property being a decimal (14.1 as well as 14).

PR w3c/aria-at-app#1105 solves this issue and the fix was deployed to staging. The results of that run are still streaming in, but show the same as your manual testing for number 6 https://aria-at-staging.w3.org/run/575?user=9999

[ccanash]

Hi @IsaDC , this was released . I will close it but please let me know if anything is not working as expected