Has Element stability deteriorated in recent VSCode?

[ahmedneilhussain]

Hi,

We use vscode-extension-tester but with our own wrapper so that we can use Bazel to manage the version of VSCode that we test against. We recently bumped the VSCode version quite a bit (from 1.91 to 1.105 or more) and have found the tests to have suddenly become a lot flakier. Lots of stale reference/element not intractable errors where previously things were stable.

I noticed the recent major commit b5c5ef5#diff-b9d9204b97112d663da9a2b5c45dab5445d06a2c430c8ecb988ec96069a8914a and wondered whether this was a long-planned improvement or was motivated by specific issues with recent versions of VSCode. @djelinek would you be able to clarify?

If it was motivated by a significant change in VSCode's behaviour (I also wondered about whether they'd bumped the version of Chromium that gets bundled), is there an issue somewhere that outlines the problem and how best to handle it?

Many thanks

[djelinek]

Hello,
All of these changes were made as an initial step to improve the stability of the tests. Although it was planned, the recent VS Code changes meant that it was expedited, as ExTester started to become very unreliable for UI testing after the release of VS Code 1.100+. VS Code started using more and more parts of the workbench asynchronously, and the DOM started refreshing quite often, which always leads to StaleElementErrors. This is why more dynamic functionality was added to ExTester.

By the way, bear in mind that ExTester officially supports only the three latest versions. You can use the 'vscode-min' placeholder, which currently points to VS Code 1.107.x.

[ahmedneilhussain]

Thanks - confirms that VSCode has definitely become harder to work with in terms of DOM stability in recent versions.

I guess the safest thing to do is to avoid holding onto and reusing web element handles/page object model items (other than for top elements) for more than brief periods, and instead requery them?