CreateAxeHTMLReport : seems to double the counts (violations, incomplete, etc)

[juprog]

Hi,
I was trying to compare the html report created, with what the chrome extension shows. And I noticed that it seems that the "counts" are double... It seems that it adds up both the "html" and the "selectors" under each violation to produce a count on the report.

is this correct?

I also tried to use the .WithOutputFile(), and in the json produced, the number of "nodes" is correct (that would be the actual count I am looking for...)

any ideas? or has anyone noticed this?

[TroyWalshProf]

Just checked against google.com and I get the same number of violations with the Axe Chrome extension and the HTML report.
*If you have an example I can take a look.

[juprog]

I will try to provide the example as soon as I can : in the one sample I produced, I was targeting a specific element by ID, so that I would get the smallest number of issues... that I passed to the analyze(), in my company's application.
We have a whole lot of frames and nested elements. I will try to see if I can make sure to remove company stuff in order to provide the example in question, as soon as I can. -- all other samples I produced from single "pages" in our app have 200,300 issues (due to all the nested frames) and that was too hard for my brain to parse and compare.

[TroyWalshProf]

Suggest looking at a simpler page like "https://www.w3schools.com/tags/tryit.asp?filename=tryhtml_iframe" to reproduce something like this.
Note* If you have have 2 iframes with the same content we would expect you to have what look like duplicates.

[juprog]

I am not sure if it is going to help, or make it clearer - but here is how I use it where I get the expected count that matches Chrome AxeDevTools count for the same element - but where the CreateAxeHtmlReport seems to differ on the count

  AxeResult results_MyCRMModal4 = new AxeBuilder(driver)
                  .Include("#bodyIframe", "#modalFrame", "#subbody", ".tableWrapper")
                  .WithRules("tabindex")
                  .WithOutputFile(@"c:\temp\axehtmlreports\democounts\tabindex4.json")
                  .Analyze();

                driver.CreateAxeHtmlReport(results_MyCRMModal4, @"c:\temp\axehtmlreports\democounts\tabindex4.html");

Explanation : I am targeting a single element with class "tableWrapper", but I need to give it the hierarchy so it can find it : frame#bodyIframe > div#modalframe > frame#subbody > div.tableWrapper

in visual studio, when inspecting the returned response : results_MyCRMModal4 returns 1 violation node, and inside that it returns 8 nodes (all the results that violate tabindex rule) - so this is what I expect.

in the html report : Violations count = 32 , but inside the section it shows 1 rule violation and the 8 nodes under it...

Chrome Axe DevTools : 8 results for tabindex

so I know the count is correct in the code when it returns the response from .analyze() ... but if I pass that response object into .CreateAxeHtmlReport, then it outputs the 32 count for violations...

I do not know where to put screenshots, but I have some based on the explanation above...

[juprog]

so here are my observations so far :
The report output in json is what I am looking for.
If I pass the axeresults to the CreateAxeHTMLReport method, then it doubles or quadruples the section counts (for violations, incomplete)

So right now, it looks like at least the results returned in JSON in the output file are correct, so I can try to parse those myself in code and make my own type of reporting (it would have been too easy to use reporting out of the box even though you went to the trouble of making that available :) .

if I search for an element with 2 levels (in the include instruction) - it returns counts x 2 (using the CreateAxeHTMLReport and passing the axeresults as parameter)
if I search for an element with 4 levels (in the include instruction) - it returns counts x4 (using the CreateAxeHTMLReport and passing the axeresults as parameter)

could be some coincidence , could be our application is just really weirdly composed (iframes and all)

//this returns counts x2 in the html report

AxeResult results_MyCRMModal2 = new AxeBuilder(driver)
                  .Include("#bodyIframe", "#modalFrame")
                  .WithRules("tabindex")
                  .WithOutputFile(@"c:\temp\axehtmlreports\democounts\tabindex2.json")
                  .Analyze();
driver.CreateAxeHtmlReport(results_MyCRMModal2, @"c:\temp\axehtmlreports\democounts\tabindex2.html");

//this returns counts x4 in the html report

AxeResult results_MyCRMModal4 = new AxeBuilder(driver)
                  .Include("#bodyIframe", "#modalFrame", "#subbody", ".tableWrapper")
                  .WithRules("tabindex")
                  .WithOutputFile(@"c:\temp\axehtmlreports\democounts\tabindex4.json")
                  .Analyze();

                driver.CreateAxeHtmlReport(results_MyCRMModal4, @"c:\temp\axehtmlreports\democounts\tabindex4.html");

[TroyWalshProf]

Cool, that gives me something to work with.

[TroyWalshProf]

@juprog - Working on a fix, should have it out the first half of this week.
#172

[juprog]

very cool ! I will try to check back in whenever it is ready and hopefully can test it against my use cases

[TroyWalshProf]

This should be addressed in 3.0.0

[juprog]

I did a crazy cowgirl move and even though I have a demo planned to my team today, I updated my solution with latest

So far it seems to have fixed the total counts per category in the html reports : Violations, Incomplete, Passes when there are frames being analyzed.

I only looked at a few examples so far, and I did not check anything else so I will check more but right now
page with no frames - counts are correct (like they were before)
page where I use 2 levels/identifiers : .Include("#bodyIframe", "#modalCategories") - counts are correct (where they were double)
page where I use 3 levels/identifiers: .Include("#bodyIframe", "#modalFrame","#infoContactWrapper") - counts are correct (where they were tripled)

thank you :)