Investigating flaky tests

[Avasam]

No description provided.

[Kalmat]

Hi!

I've noticed that when you run the tests, they randomly fail, but typically not when I run them... weird.

I see you have some "suspects" for this behavior, like the max time or number of retries in some functions when you set wait parameter to True. Currently the max amount of time to wait is 1.375 seconds, which I fully agree it might not be enough fo CI tests, but I wonder the impact in "real-life" when raising that max value. The problem here is that the developers can not set that value according to their needs, so perhaps changing the wait parameter from bool to float, the developers can define the value which best fits their needs (thus, being 0 = no wait, any other value = max time in seconds to wait). The "new" problem with this solution is retro-compatibility...

Anyway, if you need help or just discuss some concepts, just let me know!

By the way, impressive work you've done so far... Thank you!