Test is marked Pass, when Canopy assertion "==" fails and returns error

[adalenthya]

I am new to Canopy and F#, still trying to get my head around these.
In the following code, I just want a simple assertion "==" to make my test fail and mark it as Fail in the Reporter file.
For some reason, the test is Pass, although the error was thrown.

"ILS Make a search by selecting each option from the AutoComplete list" &&& fun _ ->
            "#Location" << "California"
            sleep 5
            highlight "#psrk-autocomplete"
            sleep 1
            let autoCompleteListOfOptions = (elements "#psrk-autocomplete a")
            for i = 0 to autoCompleteListOfOptions.Length - 1 do
                let optionToBeSelected = (nth i "#psrk-autocomplete a")
                let optionToBeSelectedText = read optionToBeSelected
                sleep 1
                click (optionToBeSelected)
                sleep 1
                "#Location" == "blabla" // I want to make sure, that when this assertion fails, the test is marked as FAIL
                sleep 3
                clear "#Location"
                "#Location" << "California"
                sleep 3

I tried other assertions too : "===", "!=" and same thing happens.
What am i missing? Please help me.

[lefthandedgoat]

The test did not pass, the two above it did. It is still white which makes me think that the runner has not marked it as failed yet, or that there is a bug in the reporter.

Can you set it up to run just these three tests and see how it behaves?

Do the values in the console look different than the html report?

[adalenthya]

I ran just these 3 tests and i get in the console "2 pass, 1 failed"

yes, the values are marked correctly in the Console, but in the Reported that 1 test that failed, is not marked correctly.

=== Updated screen-shot.

[lefthandedgoat]

Ok that failure is different than the failure in your first screenshot. Did this cause the html report to show a failure correctly?

Either way I will look at it this afternoon and try to reproduce or determine what could be the problem. Fortunately fixing bugs in the reporter does not require you to make any changes on your end.

[adalenthya]

My bad, I added the wrong screen-shot, so I Re-run the exact code from above and attached the correct screen-shot.
Thank you so much for looking into this.

[lefthandedgoat]

ok thanks!

[lefthandedgoat]

I can't reproduce it. I even renamed my tests to the be the same (in case it was a character encoding problem) and I used the test code your provided pointed at the public website.

Can you reproduce it, and open up dev tools in the report browser and see if there are any javascript errors?

[adalenthya]

I managed to reproduce it again, here is my report file:
ILS_Report-11-17-16-9-11-38.zip
There aren't any JS errors in the console, for the report file.

There must be some inconsistency, but not sure what.

[lefthandedgoat]

Using your zip, I run this in the console

updateTestInContext('CommercialCafe ILS [Home page]: Input Search Auto Complete Functionality', 'ILS Make a search by selecting each option from the AutoComplete list','Fail', '');

Which is (minus some encoding) what canopy would run, and it marks it as red.

The only thing that I can think of is that the it is doing 'swalloedjS' will which runn the js, but if it fails it will keep going, and it is failing once, but not consistently. Maybe I can make it retry.

Let me try to repro and keep looking.

[lefthandedgoat]

I was able to repro so I am looking now.

[lefthandedgoat]

You can see this message in your console screenshot:

[9004:7028:1117/164130:ERROR:http_connection.cc(37)] Too large read data is pending: capacity=1048576, max_buffer_size=1048576, read=1048576
[19.505][SEVERE]: Unable to receive message from renderer

When an error occurs canopy takes a screenshot of the image and then base64 encodes it and puts it in the report. When I take a screenshot of your site, its around 900k, and when encoded its 1.2 megs. It looks like chrome/chromedriver only allows a message of ~1 meg, which is too small for a screenshot of your size.

For example on my test page which is really basic and just black and white text, its only 60k. I think its because your page has a city scape image which does not compress well because each pixel is relatively unique.

You can resize the browser so that the image is smaller

start chrome
resize (600,400)

Not really great per se but it works.

You can also disable image injection, but images in your report is one of the most valuable features in my opinion.

configuration.failureScreenshotsEnabled <- false

It may be possible to send a flag to chrome to accept more data, but I don't have time to look into that right now, sorry.

I hope this helps and gets your pointed in a good direction.

[lefthandedgoat]

I thought of one other solution. I may be able to convert the image from .png to .jpeg (which offers more compression) and that would solve it.

I will look into that today.

[adalenthya]

The screen-shots are very important and relevant for my reports.
Resizing the browser, seems to cover some parts of the page that are relevant for the tests, so i'd have to scroll a lot trough the page (up/down, left/right).
So i am still trying to find a way to make chrome accept more data.

Please let me know, if the .png to .jpeg conversion is working fine.

[lefthandedgoat]

This is now fixed in canopy 1.0.4 which you can download here: https://www.nuget.org/packages/canopy/1.0.4

If you have any more problems please let me know. Thank you for reporting this bug!

[adalenthya]

It's working great! Thank you, too, for looking into it and finding a very good fix! :)

[lefthandedgoat]

I am going to close this issue, if you have any more bugs/questions please feel free to open new issues!