Use akka.test.timefactor

[seglo]

Purpose

Increase total time (max time dilation) for expect calls in akka streams testkit. This will help reduce the number of timeout test failures in slow test environments like Travis and Jenkins.

This is useful for tests that assert consistency where lots of client restarts occur that can cause congestion on the test broker.

i.e). Transactional tests that use RestartSource to assert consistency for streams with transient failures

#826

References

https://doc.akka.io/docs/akka/current/testing.html#accounting-for-slow-test-systems

Changes

• Increase akka.test.timefactor for all tests

Background Context

[seglo]

Repeated test run showing the variability of test run times. This uses the test config from this PR.

sbt:akka-stream-kafka> tests/testOnly *.TransactionsSpec -- -z "must complete with messages filtered out and transient failure causing an abort with restartable source" -DtimesToRepeat=100
[info] TransactionsSpec:
        ℹ︎ Checking the system...
        ✔ Docker version should be at least 1.6.0
        ✔ Docker environment should have more than 2GB free disk space
[info] A consume-transform-produce cycle
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (14 seconds, 165 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 954 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 654 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 701 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 560 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 531 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 582 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 453 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 421 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 576 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 455 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 443 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 620 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 427 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 419 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 399 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 395 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 425 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 409 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 530 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 515 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 405 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 419 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 439 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 515 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 329 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 426 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 477 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 410 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 365 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (20 seconds, 379 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 489 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 377 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 328 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 385 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 389 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 400 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 402 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 464 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 481 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 286 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 359 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 594 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 415 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 404 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 363 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 366 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 476 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 352 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 389 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 398 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 275 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 666 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 323 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 401 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 469 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 393 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 354 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 354 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 311 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 475 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 351 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 501 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 478 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 395 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 365 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 338 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 359 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 304 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 288 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 402 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 450 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 464 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 341 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 446 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 360 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 379 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 408 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 348 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 321 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 435 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 379 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 391 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 365 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 421 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 394 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 474 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 323 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 414 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 352 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 388 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 387 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 408 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 356 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 334 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 333 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 366 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (20 seconds, 210 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 327 milliseconds)
[info] - must complete with messages filtered out and transient failure causing an abort with restartable source (3 seconds, 369 milliseconds)
[info] ScalaTest
[info] Run completed in 7 minutes, 10 seconds.
[info] Total number of tests run: 100
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 100, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[info] Passed: Total 100, Failed 0, Errors 0, Passed 100
[success] Total time: 434 s, completed 29-Nov-2019 12:46:04 PM

[ennru]

LGTM.