Protractor tests: random failures

[pascalgrimaud]

Our protractor tests still failed randomly.

See:

• https://travis-ci.com/jhipster/jhipster-vuejs/jobs/175764686
• logs:

  Account
    ✓ should fail to login with bad password
    1) should login with admin account
    2) should be able to sign up
    3) should load user management

As it is always at the same tests, I suspect there is something to do here.

[pascalgrimaud]

Some improvements have been done here #341
But there are still some work to do before we can close this

[pascalgrimaud]

Regularly, there is a random failure at should delete previously created fake user

  Account
    ✓ should fail to login with bad password
    ✓ should login with admin account (3703ms)
    ✓ should be able to sign up (1858ms)
    ✓ should load user management (2016ms)
    ✓ should activate the new registered user
    ✓ should not be able to sign up if login already taken
    ✓ should not be able to sign up if email already taken
    ✓ should be able to log in with new registered account
    ✓ should login with admin account (1506ms)
    ✓ should fail to update password when using incorrect current password
    ✓ should login with admin account
    ✓ should be able to update password
    ✓ should be able to log in with new password (1988ms)
    ✓ should login with user_test account
    ✓ should be able to change user_test settings
    ✓ should login with admin account
    ✓ should not be able to change admin settings if email already exists
    ✓ should preview details from previously created fake user
    ✓ should edit details from previously created fake user
    1) should delete previously created fake user

[pascalgrimaud]

Another random failure, at should login with admin account

  Account
    ✓ should fail to login with bad password
    1) should login with admin account
    2) should be able to sign up
    ✓ should load user management (1969ms)
    3) should activate the new registered user
    4) should not be able to sign up if login already taken
    5) should not be able to sign up if email already taken
    6) should be able to log in with new registered account
    7) should login with admin account
    8) should fail to update password when using incorrect current password
    9) should login with admin account
    10) should be able to update password
    11) should be able to log in with new password
    12) should login with user_test account
    13) should be able to change user_test settings
    14) should login with admin account
    15) should not be able to change admin settings if email already exists
    16) should preview details from previously created fake user
    17) should edit details from previously created fake user
    18) should delete previously created fake user

[pascalgrimaud]

I'm adding a bounty for this issue, as it's very annoying.
We can migrate completely to Azure because of this.

[pascalgrimaud]

I'm still trying to fix this, and I found the problem could come from these codes:

• https://github.com/jhipster/jhipster-vuejs/blob/master/generators/entity-client/templates/vue/src/test/javascript/e2e/entities/entity-page-object.ts.ejs#L71-L74
  https://github.com/jhipster/jhipster-vuejs/blob/master/generators/entity-client/templates/vue/src/test/javascript/e2e/entities/entity-page-object.ts.ejs#L88-L90

Sometimes, the count is not correct...

[pascalgrimaud]

@vishal423 : thanks for your hard work but we still have some issue, so I'm reopening this ticket

[vishal423]

@pascalgrimaud, I do see an open discussion on migration to cypress #401 and being favored by many folks. If we choose to move ahead with protractor, then, I can spend some time this week to analyze remaining intermittent issues. Let me know how you want to proceed ahead on this.

[pascalgrimaud]

Yes, I'm waiting @sahbi-ktifa for that, we'll discuss about it at JHipster Conf, I think

[pascalgrimaud]

I'm increasing the bounty as it's really annoying since months, and you work so hard to fix this @vishal423
Thanks so much.

Can you wait few days before claiming the bounty plz ?
So we'll be sure Protractor has been improved.

[vishal423]

@pascalgrimaud, IMO, we should keep this issue open for a week or two and list down the intermittent issues observed during that time frame. This will help to identify areas that can be improved.

At moment, I see below issues:

• Remove entity dialog is visible after successful delete operation (fails in after all hook). It also causes subsequent tests to fail. - should be addressed with e2e: Replace structural checks, reduce timeouts, fix intermittent failures #417

   "after all" hook: ret for "should delete last xxx":
 WebDriverError: element click intercepted: Element <li data-v-39b7de9c="" id="account-menu" class="pointer nav-item b-nav-dropdown dropdown" href="javascript:void(0);" active-class="active">...</li> is not clickable at point (1197, 31). Other element would receive the click: <div id="removeEntity" role="dialog" class="modal fade d-block" aria-hidden="true" style="">...</div>

• In some cases, tests fail with an element present in DOM, but, not visible on UI. - should be addressed with e2e: Replace structural checks, reduce timeouts, fix intermittent failures #417
• Count of elements after successful delete is incorrect. - should be addressed with e2e: Replace structural checks, reduce timeouts, fix intermittent failures #417

[vishal423]

I think we can close this issue as e2e tests seems stable at moment.

[pascalgrimaud]

Yes, don't forget to claim the bounty @vishal423
Thanks so much

[vishal423]

Bounty claimed https://opencollective.com/generator-jhipster/expenses/9112

[pascalgrimaud]

Just approuved
Thanks so much for fixing this @vishal423 !

[vishal423]

thanks @pascalgrimaud I really enjoyed exploring protractor 😄

[jdubois]

Bounty paid, thanks & congrats!!