Scraper is retrieving wrong thumbnail resolutions on some occasions

[benoit74]

See https://farm.openzim.org/pipeline/a134da85-d1e8-46b8-a626-9ab20d5c66af/debug

[benoit74]

The problem partially comes from https://phabricator.wikimedia.org/T436302

If I focus on thumbnails of https://commons.wikimedia.org/wiki/File:1871_Descent_F937.1_fig03.jpg, then most articles (e.g. Eye) is using correct URL https://upload.wikimedia.org/wikipedia/commons/thumb/c/ca/1871_Descent_F937.1_fig03.jpg/120px-1871_Descent_F937.1_fig03.jpg

For some reason, there is one article (Deiters'_cells) which uses a wrong URL (https://upload.wikimedia.org/wikipedia/commons/thumb/c/ca/1871_Descent_F937.1_fig03.jpg/100px-1871_Descent_F937.1_fig03.jpg).

The fact that URL is wrong is not mwoffliner problem.

However, mwoffliner should have chose the 120px URL instead of the 100px one since we always choose the higher res URL we encounter.

This is a bug in mwoffliner URL selection logic, which can be traced back to #2631 where we introduced a confusion between width of the image in the URL which is expected to be used for download and highest width at which the image has been used so far in HTML.

We need to keep both values in memory and make a distinction between both values in general, and making this distinction would even help to solve this particular situation.

It looks like it would even probably solve (most) mdwiki.org image issues (maybe a bit by chance) because in general we do have articles with proper thumbnail URL (using proper resolution) in the selection, and this URL always has a higher res than the bad URL since this is de-facto what Wikimedia had to implement when moving to a limited set of thumbnail sizes as documented in https://www.mediawiki.org/wiki/Common_thumbnail_sizes (it always uses a higher res than what is used in the page to only downscale, never upscale).

[benoit74]

Not a blocker for 2.0.0 release