WaitTimeoutError during test

[ryuran]

Helo!

After an error on Travis (OnetapInc/chromy#62), I try to reduce async limit to pass test.

asyncCaptureLimit: 5,
asyncCompareLimit: 50

But nothing change 😢 : https://travis-ci.org/20minutes/colette/builds/271137302

Then I try with

asyncCaptureLimit: 1,
asyncCompareLimit: 1

https://travis-ci.org/20minutes/colette/builds/271199288
I don’t know why it’s green even if i have no result of the test…

[garris]

Did you try to reduce to 1?

[ryuran]

@garris yes, my second try is at 1:1
It’s better but travis return positive whereas backstop was not able to finish test and I have some errors in logs.

ChromyJS returned an unexpected error while attempting to capture a selector. { WaitTimeoutError: evaluate() timeout
    at EvaluateTimeoutError.ExtendableBuiltin (/home/travis/build/20minutes/colette/node_modules/chromy/dist/error.js:44:28)
    at EvaluateTimeoutError.TimeoutError (/home/travis/build/20minutes/colette/node_modules/chromy/dist/error.js:73:128)
    at new EvaluateTimeoutError (/home/travis/build/20minutes/colette/node_modules/chromy/dist/error.js:118:145)
    at Chromy._evaluateWithReplaces$ (/home/travis/build/20minutes/colette/node_modules/chromy/dist/document.js:484:21)
    at tryCatch (/home/travis/build/20minutes/colette/node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:65:40)
    at Generator.invoke [as _invoke] (/home/travis/build/20minutes/colette/node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:299:22)
    at Generator.prototype.(anonymous function) [as throw] (/home/travis/build/20minutes/colette/node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:117:21)
    at tryCatch (/home/travis/build/20minutes/colette/node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:65:40)
    at invoke (/home/travis/build/20minutes/colette/node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:155:20)
    at /home/travis/build/20minutes/colette/node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:167:13
    at <anonymous> name: 'WaitTimeoutError' }

[garris]

I have just verified that asyncCaptureLimit and asyncCompareLimit are working as designed. At this point I think you need to look at why your particular test is failing.

[ryuran]

I add log for test scenario label.
It seems fail on ./docs/item-theme-skins.html or the next one ./docs/item-theme-styles.html.
I check this files with html validator, they are valid.
So I’m blocked. All is fine in local environment.

[kiran-redhat]

@ryuran 1. are you about to find that in chrome console ?

2. Do you see sale error with asyncCaptureLimit: 1,?

If 1 &2 are yes, clone latest master and test. asyncCaptureLimit and asyncCompareLimit we're working fine for me

[ryuran]

@kiran-redhat

1. I don’t know how to return more info from browser in travis-CI logs
2. This travis build is running with asyncCaptureLimit: 1 https://travis-ci.org/20minutes/colette/builds/271948792 I have the error

[kiran-redhat]

for number 1: open your app in Chrome browser press f12 then click on console then copy and paste your selector and hit enter. if you find element wait for your selector to be visible (probably chromy.visible() )

…

On Sep 5, 2017 9:24 PM, "Yvain Liechti" ***@***.***> wrote: @kiran-redhat <https://github.com/kiran-redhat> 1. I don’t know how to return more info from browser from travis-CI 2. This travis build is running with asyncCaptureLimit: 1 https://travis-ci.org/20minutes/colette/builds/271948792 <https://travis-ci.org/20minutes/colette/builds/271948792> I have the error — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#537 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AQUrIz_uD5ROOyWVDt21w2YFEBIEIiIwks5sfRNCgaJpZM4PK4Y4> .

[ryuran]

No error in my local chrome.
Backstop.js run fine in local environment with all test to green.

I update backstopjs:

└─┬ backstopjs@3.0.24
  └── chromy@0.5.2

And I try to change delay to 200 (it was 100), but it stops always on the same test.
Without error this time but with travis timeout:
https://travis-ci.org/20minutes/colette/builds/271982035

[ryuran]

It seems fail most often on test with <img>

[reynaldoapb]

Try to kill chrome with shell command taskkill /F /IM chrome.exe and run everything again.

[axe312ger]

Turning it down to

asyncCaptureLimit: 1,
asyncCompareLimit: 1

worked for me on travis :)

[mhavelant]

Hi!

I got the same error. The async limits at 1 did not help, however setting "selectors" to "viewport" from "document" did. So it might be something with chromy with 'document' as the selector.

Edit: Setting asyncCaptureLimit to 1 AND not using "document" together worked. Setting the capture limit to something higher broke it.

[garris]

What version of chrome and backstop are you using?

[mhavelant]

The newest ones:

• Google Chrome 62.0.3202.52 beta
• BackstopJS v3.0.29

Note: I'm running under a dockerized Ubuntu 16.04