perf: reduce memory footprint and add backpressure limit

[alambare]

Files were fully loaded into memory.

The back-pressure limit prevents the explosion of the buffer size when waiting on long pending requests.

[github-actions]

Test Results

    4 files  ±0      4 suites  ±0   3m 42s ⏱️ -5s
  843 tests ±0    843 ✅ ±0  0 💤 ±0  0 ❌ ±0 
3 422 runs  ±0  3 420 ✅ ±0  2 💤 ±0  0 ❌ ±0 

Results for commit f833e4e. ± Comparison against base commit 8c36c36.

♻️ This comment has been updated with latest results.

[eodag-bot]

Code Coverage (Ubuntu)

Details

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                      8       0  100.00%
cli.py                                         322      65  79.81%   63-64, 88, 689-753, 855-906, 910
config.py                                      452      30  93.36%   81-83, 92, 100, 104-106, 177, 188, 205-206, 729-731, 855-858, 902-903, 912-913, 947, 1027, 1086-1091, 1093
crunch.py                                        5       5  0.00%    20-24
api/__init__.py                                  0       0  100.00%
api/core.py                                    788      76  90.36%   337, 604, 635, 679-682, 720, 764, 798, 843-848, 874, 960, 1288-1300, 1340, 1342, 1372, 1376-1387, 1400-1406, 1491-1494, 1531-1551, 1620-1625, 1637-1640, 1658-1670, 1965-1971, 2222, 2226-2230, 2239-2241, 2285-2286, 2315-2316
api/search_result.py                           126      11  91.27%   87, 96, 103, 117, 282, 287-288, 336, 359-360, 366
api/product/__init__.py                         18       1  94.44%   62
api/product/_assets.py                          52       5  90.38%   97, 183, 191, 194-198
api/product/_product.py                        213      19  91.08%   72-74, 238-239, 345, 372, 473, 491, 515-518, 527-530, 585-588
api/product/metadata_mapping.py                726      63  91.32%   123-125, 256-257, 314-326, 328, 339, 364, 416-417, 454, 475-478, 501, 512-514, 615-616, 640-641, 651-654, 669-670, 827, 878, 1031, 1040-1044, 1061-1066, 1193, 1207-1227, 1247, 1252, 1405, 1419, 1444, 1490, 1551, 1574-1575, 1595-1599, 1615, 1623
api/product/drivers/__init__.py                 20       2  90.00%   33-36
api/product/drivers/base.py                     29       1  96.55%   98
api/product/drivers/generic.py                   7       0  100.00%
api/product/drivers/sentinel1.py                15       0  100.00%
api/product/drivers/sentinel2.py                15       0  100.00%
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 21       2  90.48%   48, 55
plugins/manager.py                             173      19  89.02%   105-110, 145-150, 154, 192, 214, 218, 242, 281-282, 382-385, 397-398
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           97      10  89.69%   165-167, 214-215, 241-243, 296-297
plugins/apis/usgs.py                           180      26  85.56%   156, 258, 292, 327-329, 334, 360-361, 366, 396-403, 414-419, 441-447, 478
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py              20       0  100.00%
plugins/authentication/base.py                  17       2  88.24%   45, 58
plugins/authentication/generic.py               16       3  81.25%   50, 55, 65
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              46       7  84.78%   153-156, 177-182
plugins/authentication/oauth.py                 15       7  53.33%   48-50, 54-57
plugins/authentication/openid_connect.py       232      28  87.93%   91-92, 104-122, 169, 175-203, 211, 350-353, 379, 420
plugins/authentication/qsauth.py                34       1  97.06%   91
plugins/authentication/sas_auth.py              53       3  94.34%   67, 88, 134
plugins/authentication/token.py                128       9  92.97%   180, 217, 289-290, 340-344
plugins/authentication/token_exchange.py        36      14  61.11%   75, 93-121
plugins/crunch/__init__.py                       0       0  100.00%
plugins/crunch/base.py                          10       1  90.00%   43
plugins/crunch/filter_date.py                   59      14  76.27%   52-57, 69, 78, 87, 90, 100-102, 109-111, 118
plugins/crunch/filter_latest_intersect.py       47       8  82.98%   52-53, 69, 78-81, 83, 90-93
plugins/crunch/filter_latest_tpl_name.py        31       1  96.77%   83
plugins/crunch/filter_overlap.py                66      18  72.73%   28-30, 66-69, 76-79, 85, 93, 104-120
plugins/crunch/filter_property.py               30       5  83.33%   55-60, 63-64
plugins/download/__init__.py                     0       0  100.00%
plugins/download/aws.py                        493     126  74.44%   271, 284, 353-356, 370-374, 466-468, 472, 505-506, 512-516, 547, 594, 598, 605, 635-643, 647, 682-690, 701-703, 764-852, 863-868, 921, 936-938, 941, 951-960, 968-982, 992-1015, 1022-1035, 1073, 1099, 1144-1146, 1366
plugins/download/base.py                       260      35  86.54%   129, 158, 305-306, 364-365, 407, 411-422, 436, 504-508, 538-539, 564-572, 574-578, 628, 649, 671, 679
plugins/download/creodias_s3.py                 26      10  61.54%   58-73
plugins/download/http.py                       545      99  81.83%   236, 278-281, 342-345, 348, 355-360, 391-393, 410, 425, 479, 514, 528, 542, 550-554, 570-575, 586, 604, 640-643, 664, 674, 681, 701, 779, 797-826, 833-848, 868, 898-907, 943, 968-969, 988-993, 1002, 1017-1019, 1023, 1026, 1041-1042, 1120-1121, 1125, 1177, 1223-1224, 1230, 1240, 1268, 1303, 1323, 1360-1362
plugins/download/s3rest.py                     122      26  78.69%   123, 157, 164, 199, 226-233, 236-238, 242, 253-259, 267-268, 271-275, 290-293, 302, 323-326
plugins/search/__init__.py                      22       0  100.00%
plugins/search/base.py                         170      14  91.76%   103, 107, 131-137, 274, 295, 434, 456-459, 468
plugins/search/build_search_result.py          479      56  88.31%   246-247, 283, 286, 311, 314, 345-347, 559, 561, 565, 568, 695, 697, 758, 765, 789, 826, 877, 927-942, 990, 1015, 1018, 1022, 1143-1144, 1153-1162, 1225, 1246, 1265-1274, 1391-1392, 1437, 1446-1448, 1508, 1554
plugins/search/cop_marine.py                   254      47  81.50%   56, 64-66, 76-77, 82, 87-88, 104, 106, 109, 175-176, 232, 238, 242, 246, 259, 270-271, 279, 312, 316, 337, 341, 345, 349, 353-357, 363-366, 369-383, 400, 449-453, 458, 470
plugins/search/creodias_s3.py                   25       1  96.00%   51
plugins/search/csw.py                          105      81  22.86%   98-99, 103-104, 112-160, 166-179, 187-219, 237-278
plugins/search/data_request_search.py          208      72  65.38%   195-198, 214, 225, 229-230, 241, 246, 251, 258, 271-274, 328-329, 333, 343-349, 354, 372-376, 392-395, 403-414, 431, 433, 440-443, 445-446, 464-468, 501, 511, 522, 535, 541-556, 561
plugins/search/qssearch.py                     724      95  86.88%   416-417, 461, 519, 533, 537-543, 558-559, 571-575, 632-648, 708-720, 765-768, 839-840, 891, 910, 917, 929, 986, 1007, 1010-1011, 1020-1021, 1030-1031, 1040-1041, 1068, 1154-1159, 1163-1172, 1206, 1229, 1302-1303, 1309, 1399, 1564, 1567, 1573-1574, 1595, 1623-1635, 1642, 1674-1676, 1686-1692, 1722, 1745, 1750-1751, 1766, 1782, 1869, 1876, 1887, 1910-1915, 1924, 1933
plugins/search/stac_list_assets.py              25      10  60.00%   44-51, 75-85
plugins/search/static_stac_search.py            79      13  83.54%   100-126, 170, 183
rest/__init__.py                                 4       2  50.00%   21-22
rest/cache.py                                   33       7  78.79%   35-37, 53-55, 59, 68
rest/config.py                                  25       0  100.00%
rest/constants.py                                6       0  100.00%
rest/core.py                                   259      60  76.83%   257, 296, 311-347, 443, 480-519, 596, 626-629, 702, 709-756
rest/errors.py                                  72       5  93.06%   117, 127, 138, 163-164
rest/server.py                                 194      24  87.63%   86, 109-111, 285-290, 318, 518-520, 537-542, 571, 573, 577-578, 582-583
rest/stac.py                                   331      71  78.55%   312, 334, 386-389, 416-443, 474-476, 499, 531-532, 614-654, 676-692, 784-788, 846-847, 857-862, 872-876, 925, 1015-1017
rest/types/__init__.py                           0       0  100.00%
rest/types/collections_search.py                13      13  0.00%    18-44
rest/types/eodag_search.py                     180       6  96.67%   225-229, 282, 285, 353, 375
rest/types/queryables.py                        64      12  81.25%   61, 71-79, 105-110, 174
rest/types/stac_search.py                      125       7  94.40%   128, 174, 189-191, 199, 203
rest/utils/__init__.py                          94      12  87.23%   101-102, 121-123, 175, 185-199
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           23       3  86.96%   48, 60, 62
types/__init__.py                              161      40  75.16%   67, 80-84, 95-107, 135-137, 144-149, 252, 262-278, 283, 285, 307, 312, 320, 330
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                            111       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 103
utils/__init__.py                              587      44  92.50%   71, 197-198, 207-233, 236, 251, 331-335, 410-414, 433-435, 449, 541, 556, 596-597, 626, 1001-1004, 1012-1013, 1051-1052, 1099-1100, 1240, 1493, 1508, 1517, 1529
utils/cache.py                                  22       0  100.00%
utils/env.py                                     3       0  100.00%
utils/exceptions.py                             47       0  100.00%
utils/free_text_search.py                       65       2  96.92%   83, 91
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                28       1  96.43%   41
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/repr.py                                   38       8  78.95%   51, 53, 57, 98, 122-129
utils/requests.py                               55      29  47.27%   51-52, 64, 85-96, 107-124, 128
utils/rest.py                                   36       1  97.22%   55
utils/s3.py                                    263      11  95.82%   207-208, 249, 265, 287, 509, 558-559, 622, 697, 723
utils/stac_reader.py                           113      44  61.06%   63-85, 95-97, 101, 138, 154-159, 206-216, 226-256
TOTAL                                        10867    1628  85.02%

Diff against develop

Filename       Stmts    Miss  Cover
-----------  -------  ------  -------
utils/s3.py      +14      -2  +1.04%
TOTAL            +14      -2  +0.04%

Results for commit: 76bf758

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

[eodag-bot]

Code Coverage (Windows)

Details

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                      8       0  100.00%
cli.py                                         322      65  79.81%   63-64, 88, 689-753, 855-906, 910
config.py                                      452      31  93.14%   81-83, 92, 100, 104-106, 177, 188, 205-206, 729-731, 855-858, 902-903, 912-913, 947, 1027, 1056, 1086-1091, 1093
crunch.py                                        5       5  0.00%    20-24
api/__init__.py                                  0       0  100.00%
api/core.py                                    788      76  90.36%   337, 604, 635, 679-682, 720, 764, 798, 843-848, 874, 960, 1288-1300, 1340, 1342, 1372, 1376-1387, 1400-1406, 1491-1494, 1531-1551, 1620-1625, 1637-1640, 1658-1670, 1965-1971, 2222, 2226-2230, 2239-2241, 2285-2286, 2315-2316
api/search_result.py                           126      11  91.27%   87, 96, 103, 117, 282, 287-288, 336, 359-360, 366
api/product/__init__.py                         18       1  94.44%   62
api/product/_assets.py                          52       5  90.38%   97, 183, 191, 194-198
api/product/_product.py                        213      19  91.08%   72-74, 238-239, 345, 372, 473, 491, 515-518, 527-530, 585-588
api/product/metadata_mapping.py                726      63  91.32%   123-125, 256-257, 314-326, 328, 339, 364, 416-417, 454, 475-478, 501, 512-514, 615-616, 640-641, 651-654, 669-670, 827, 878, 1031, 1040-1044, 1061-1066, 1193, 1207-1227, 1247, 1252, 1405, 1419, 1444, 1490, 1551, 1574-1575, 1595-1599, 1615, 1623
api/product/drivers/__init__.py                 20       2  90.00%   33-36
api/product/drivers/base.py                     29       1  96.55%   98
api/product/drivers/generic.py                   7       0  100.00%
api/product/drivers/sentinel1.py                15       0  100.00%
api/product/drivers/sentinel2.py                15       0  100.00%
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 21       3  85.71%   48, 55, 68
plugins/manager.py                             173      19  89.02%   105-110, 145-150, 154, 192, 214, 218, 242, 281-282, 382-385, 397-398
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           97      10  89.69%   165-167, 214-215, 241-243, 296-297
plugins/apis/usgs.py                           180      26  85.56%   156, 258, 292, 327-329, 334, 360-361, 366, 396-403, 414-419, 441-447, 478
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py              20       0  100.00%
plugins/authentication/base.py                  17       2  88.24%   45, 58
plugins/authentication/generic.py               16       3  81.25%   50, 55, 65
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              46       7  84.78%   153-156, 177-182
plugins/authentication/oauth.py                 15       7  53.33%   48-50, 54-57
plugins/authentication/openid_connect.py       232      28  87.93%   91-92, 104-122, 169, 175-203, 211, 350-353, 379, 420
plugins/authentication/qsauth.py                34       1  97.06%   91
plugins/authentication/sas_auth.py              53       3  94.34%   67, 88, 134
plugins/authentication/token.py                128       9  92.97%   180, 217, 289-290, 340-344
plugins/authentication/token_exchange.py        36      14  61.11%   75, 93-121
plugins/crunch/__init__.py                       0       0  100.00%
plugins/crunch/base.py                          10       1  90.00%   43
plugins/crunch/filter_date.py                   59      14  76.27%   52-57, 69, 78, 87, 90, 100-102, 109-111, 118
plugins/crunch/filter_latest_intersect.py       47      33  29.79%   49-54, 67-112
plugins/crunch/filter_latest_tpl_name.py        31       1  96.77%   83
plugins/crunch/filter_overlap.py                66      18  72.73%   28-30, 66-69, 76-79, 85, 93, 104-120
plugins/crunch/filter_property.py               30       5  83.33%   55-60, 63-64
plugins/download/__init__.py                     0       0  100.00%
plugins/download/aws.py                        493     126  74.44%   271, 284, 353-356, 370-374, 466-468, 472, 505-506, 512-516, 547, 594, 598, 605, 635-643, 647, 682-690, 701-703, 764-852, 863-868, 921, 936-938, 941, 951-960, 968-982, 992-1015, 1022-1035, 1073, 1099, 1144-1146, 1366
plugins/download/base.py                       260      37  85.77%   129, 158, 225-227, 305-306, 364-365, 407, 411-422, 436, 504-508, 538-539, 564-572, 574-578, 628, 649, 671, 679
plugins/download/creodias_s3.py                 26      10  61.54%   58-73
plugins/download/http.py                       545      99  81.83%   236, 278-281, 342-345, 348, 355-360, 391-393, 410, 425, 479, 514, 528, 542, 550-554, 570-575, 586, 604, 640-643, 664, 674, 681, 701, 779, 797-826, 833-848, 868, 898-907, 943, 968-969, 988-993, 1002, 1017-1019, 1023, 1026, 1041-1042, 1120-1121, 1125, 1177, 1223-1224, 1230, 1240, 1268, 1303, 1323, 1360-1362
plugins/download/s3rest.py                     122      26  78.69%   123, 157, 164, 199, 226-233, 236-238, 242, 253-259, 267-268, 271-275, 290-293, 302, 323-326
plugins/search/__init__.py                      22       0  100.00%
plugins/search/base.py                         170      14  91.76%   103, 107, 131-137, 274, 295, 434, 456-459, 468
plugins/search/build_search_result.py          479      63  86.85%   246-247, 283, 286, 311, 314, 345-347, 557-568, 695, 697, 758, 765, 789, 826, 877, 893, 927-942, 990, 1015, 1018, 1022, 1143-1144, 1153-1162, 1225, 1240, 1246, 1265-1274, 1391-1392, 1437, 1446-1448, 1508, 1554
plugins/search/cop_marine.py                   254      47  81.50%   56, 64-66, 76-77, 82, 87-88, 104, 106, 109, 175-176, 232, 238, 242, 246, 259, 270-271, 279, 312, 316, 337, 341, 345, 349, 353-357, 363-366, 369-383, 400, 449-453, 458, 470
plugins/search/creodias_s3.py                   25       1  96.00%   51
plugins/search/csw.py                          105      81  22.86%   98-99, 103-104, 112-160, 166-179, 187-219, 237-278
plugins/search/data_request_search.py          208      72  65.38%   195-198, 214, 225, 229-230, 241, 246, 251, 258, 271-274, 328-329, 333, 343-349, 354, 372-376, 392-395, 403-414, 431, 433, 440-443, 445-446, 464-468, 501, 511, 522, 535, 541-556, 561
plugins/search/qssearch.py                     724     107  85.22%   416-417, 461, 519, 533, 537-543, 558-559, 571-575, 632-648, 708-720, 765-768, 839-840, 891, 910, 917, 929, 986, 1007, 1010-1011, 1020-1021, 1030-1031, 1040-1041, 1068, 1154-1159, 1163-1172, 1206, 1229, 1302-1303, 1309, 1399, 1496-1500, 1564, 1567, 1573-1574, 1595, 1623-1635, 1642, 1674-1676, 1686-1692, 1722, 1745, 1750-1751, 1766, 1782, 1857, 1869, 1876, 1887, 1910-1915, 1924, 1932-1942, 1960, 1976-1980
plugins/search/stac_list_assets.py              25      10  60.00%   44-51, 75-85
plugins/search/static_stac_search.py            79      13  83.54%   100-126, 170, 183
rest/__init__.py                                 4       2  50.00%   21-22
rest/cache.py                                   33      22  33.33%   35-37, 44-70
rest/config.py                                  25       1  96.00%   35
rest/constants.py                                6       0  100.00%
rest/core.py                                   259     151  41.70%   156, 158, 160, 163-164, 178-188, 197-198, 204, 207, 248-298, 311-347, 379-417, 432-448, 464-473, 480-519, 536, 578-663, 702, 709-756
rest/errors.py                                  72      51  29.17%   63, 68-111, 116-119, 126-129, 137-167, 175-180, 195-201
rest/server.py                                 194     194  0.00%    18-596
rest/stac.py                                   331      76  77.04%   246, 312, 334, 386-389, 416-443, 474-476, 499, 531-532, 614-654, 676-692, 719, 784-788, 846-847, 853, 857-862, 872-876, 925, 963, 996, 1015-1017
rest/types/__init__.py                           0       0  100.00%
rest/types/collections_search.py                13      13  0.00%    18-44
rest/types/eodag_search.py                     180      20  88.89%   225-229, 262-264, 282, 285, 291, 295, 353, 371-386
rest/types/queryables.py                        64      20  68.75%   54-55, 60-63, 68-79, 105-110, 119-120, 174
rest/types/stac_search.py                      125      11  91.20%   126-128, 174, 189-191, 199, 203, 251, 254
rest/utils/__init__.py                          94      30  68.09%   72-78, 98, 101-102, 121-123, 136, 143, 168-176, 183-204
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           23       5  78.26%   43-44, 48, 60, 62
types/__init__.py                              161      41  74.53%   67, 80-84, 95-107, 135-137, 144-149, 214, 252, 262-278, 283, 285, 307, 312, 320, 330
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                            111       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 103
utils/__init__.py                              587      44  92.50%   71, 197-198, 207-233, 236, 251, 331-335, 410-414, 433-435, 449, 541, 556, 596-597, 626, 1001-1004, 1012-1013, 1051-1052, 1099-1100, 1240, 1493, 1508, 1517, 1529
utils/cache.py                                  22       0  100.00%
utils/env.py                                     3       0  100.00%
utils/exceptions.py                             47       0  100.00%
utils/free_text_search.py                       65       2  96.92%   83, 91
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                28       1  96.43%   41
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/repr.py                                   38       8  78.95%   51, 53, 57, 98, 122-129
utils/requests.py                               55      29  47.27%   51-52, 64, 85-96, 107-124, 128
utils/rest.py                                   36       1  97.22%   55
utils/s3.py                                    263      11  95.82%   207-208, 249, 265, 287, 509, 558-559, 622, 697, 723
utils/stac_reader.py                           113      44  61.06%   63-85, 95-97, 101, 138, 154-159, 206-216, 226-256
TOTAL                                        10867    2051  81.13%

Diff against develop

Filename       Stmts    Miss  Cover
-----------  -------  ------  -------
utils/s3.py      +14      -1  +0.64%
TOTAL            +14      -1  +0.04%

Results for commit: 76bf758

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

[github-actions]

Benchmark comparison

Compared baseline: origin/develop
Compared candidate: c16f973

------------------------------- benchmark: 7 tests, 2 sources -------------------------------
Name (time in ms)                                 baseline Median  candidate Median   ΔMedian
---------------------------------------------------------------------------------------------
test_benchmark_stac_reader_fetch_recursive                 4.0474            4.0463     -0.0%
test_benchmark_cli_version                                21.2564           21.4724     +1.0%
test_benchmark_cli_without_args_subprocess               130.9582          131.3889     +0.3%
test_benchmark_core_search_with_provider                 166.1277          164.0441     -1.3%
test_benchmark_core_search_with_count                    218.1798          212.5057     -2.6%
test_benchmark_cli_list_collections                      282.4256          284.9882     +0.9%
test_benchmark_eodag_instantiation_subprocess          1,020.6749        1,008.6160     -1.2%
---------------------------------------------------------------------------------------------

Legend:
  Cyan: reference source for comparison. Green: improvement, Red: regression.
  Δ: percentage change from reference source.

[github-actions]

Code Coverage (Ubuntu)

Details

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                     19       2  89.47%   60, 69
cli.py                                         259      11  95.75%   99-110, 376, 658
config.py                                      336      27  91.96%   71-73, 76, 79, 82, 86, 90, 94-96, 590-592, 746-748, 772-776, 795, 803, 833-838, 840
crunch.py                                        2       0  100.00%
api/__init__.py                                  0       0  100.00%
api/collection.py                              151       8  94.70%   177, 213, 216, 323, 362, 365, 383, 386
api/core.py                                    776      58  92.53%   262, 548, 596, 639, 679, 699, 740-745, 770, 854-873, 887, 893, 1046, 1051, 1150, 1189-1190, 1286-1287, 1313, 1344-1345, 1371, 1384, 1445-1446, 1477-1478, 1552-1557, 1569-1572, 1684, 1909, 2066, 2070, 2179, 2267-2268
api/provider.py                                389      42  89.20%   128-132, 199, 203-204, 341-362, 430, 546, 550-551, 556-559, 570, 646-656, 846-847, 895, 902, 916, 939-942, 976-983, 987-988
api/search_result.py                           138      12  91.30%   111, 123, 133, 154, 208, 266, 278, 413, 468-471
api/product/__init__.py                         26       4  84.62%   51-54
api/product/_assets.py                          58       5  91.38%   101, 195-196, 208-212
api/product/_product.py                        315      24  92.38%   217, 373, 399-402, 439, 458-459, 568, 597, 604, 644, 756, 799-802, 811-814, 868, 934, 946, 1013
api/product/metadata_mapping.py                887      57  93.57%   136-138, 259-264, 288, 347-348, 428, 448-449, 454, 517-518, 555, 576-579, 602, 614-615, 656, 679, 704-707, 709-714, 793, 799, 815, 823, 1054, 1207, 1229, 1238-1242, 1259-1264, 1397, 1420, 1429, 1451, 1456, 1523, 1620, 1641, 1667, 1681, 1779, 1874, 1970, 2036, 2080
api/product/drivers/__init__.py                 11       0  100.00%
api/product/drivers/base.py                     32       0  100.00%
api/product/drivers/generic.py                  11       0  100.00%
api/product/drivers/sentinel1.py                33       0  100.00%
api/product/drivers/sentinel2.py                33       0  100.00%
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 38       3  92.11%   50, 84-85
plugins/manager.py                             192      17  91.15%   103-108, 138, 220, 242, 260-261, 273, 312-313, 413-416, 428-429
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             8       0  100.00%
plugins/apis/ecmwf.py                           95      10  89.47%   163-165, 213-214, 240-242, 273-274
plugins/apis/usgs.py                           175      25  85.71%   158, 246, 279, 322-324, 329, 357-358, 363, 393-400, 411-416, 438-444
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py             124      35  71.77%   52-54, 69-70, 142-149, 177-203, 226, 258-262, 279, 303, 319-320
plugins/authentication/base.py                  26       4  84.62%   58, 71, 94, 108
plugins/authentication/eoiam.py                 98       1  98.98%   170
plugins/authentication/generic.py               16       3  81.25%   50, 55, 65
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              46       4  91.30%   154, 177-182
plugins/authentication/openid_connect.py       235      19  91.91%   98-99, 107-125, 172, 180, 194, 214, 353-356, 382, 423
plugins/authentication/qsauth.py                34       1  97.06%   91
plugins/authentication/sas_auth.py              57       3  94.74%   68, 89, 135
plugins/authentication/token.py                141       9  93.62%   192, 229, 309-310, 358-362
plugins/authentication/token_exchange.py        36      14  61.11%   75, 93-121
plugins/crunch/__init__.py                       6       0  100.00%
plugins/crunch/base.py                          12       0  100.00%
plugins/crunch/filter_date.py                   47       0  100.00%
plugins/crunch/filter_latest_intersect.py       52       6  88.46%   87-88, 95-96, 98-102
plugins/crunch/filter_latest_tpl_name.py        35       0  100.00%
plugins/crunch/filter_overlap.py                66      10  84.85%   118-121, 137-163
plugins/crunch/filter_property.py               30       2  93.33%   81-84
plugins/download/__init__.py                     4       0  100.00%
plugins/download/aws.py                        403      77  80.89%   271, 305, 354-357, 387-388, 396-400, 480-483, 523-525, 529, 560-561, 567-571, 602, 667-675, 739-833, 845-850, 888, 914, 959-961, 1013
plugins/download/base.py                       297      24  91.92%   148, 178, 394-395, 439, 473, 550-554, 584, 619-620, 645-654, 713, 734, 756, 764
plugins/download/http.py                       578      74  87.20%   242, 284-287, 349-352, 355, 362-367, 398-400, 417, 432, 493, 528, 542, 556, 566-570, 586-591, 602, 621, 657-660, 681, 691, 698, 754, 862, 894, 926-935, 971, 1016-1021, 1030, 1045-1047, 1051, 1054, 1069-1070, 1080, 1155, 1207, 1249-1250, 1262, 1272, 1328-1329, 1359, 1397, 1477-1478
plugins/search/__init__.py                      25       0  100.00%
plugins/search/base.py                         207      19  90.82%   118, 122, 146-152, 161, 227-230, 290, 322, 343, 469, 554-557, 566
plugins/search/build_search_result.py          527      78  85.20%   138-139, 175, 179, 199, 258, 427-441, 520-521, 570, 582-584, 608, 610, 675, 683-687, 707, 717, 743, 792, 817, 879-884, 945, 973, 976, 1031-1052, 1093, 1120-1121, 1130-1139, 1229, 1244, 1247, 1252, 1271-1280, 1401-1402, 1446, 1455-1457, 1514, 1562-1572
plugins/search/cop_ghsl.py                     410      84  79.51%   73-74, 109-110, 174, 176, 218, 249-252, 298, 314, 360, 364-369, 384, 406, 427, 461, 477, 512, 527, 541-542, 556-558, 565, 569-571, 574-577, 595-655, 666-681
plugins/search/cop_marine.py                   297      65  78.11%   56, 64-66, 82, 104, 106, 109, 175-176, 194, 218, 224, 228, 250-251, 259, 294-297, 303, 313, 326, 334, 340-344, 350-353, 359-412, 428-431, 481, 493, 537-538
plugins/search/creodias_s3.py                   29       1  96.55%   59
plugins/search/csw.py                          112      87  22.32%   99-100, 104-105, 113-170, 176-189, 197-229, 246-287
plugins/search/eumetsat_ds.py                   25       0  100.00%
plugins/search/geodes.py                        66       0  100.00%
plugins/search/oar.py                           82      13  84.15%   50, 57-60, 172-180, 186-188, 195
plugins/search/qssearch.py                     889     108  87.85%   381, 425-426, 543-544, 567-568, 580-584, 816-822, 880, 946, 980, 987, 1058, 1079, 1082-1083, 1101, 1110-1111, 1129, 1200, 1209, 1214-1231, 1240, 1255, 1264, 1277, 1300, 1339, 1412, 1435, 1450, 1523-1524, 1526-1528, 1530-1531, 1618, 1625, 1733-1737, 1803, 1806, 1810-1811, 1830-1833, 1845, 1867-1884, 1892-1898, 1933-1935, 1956-1962, 1969, 2007, 2023, 2046, 2051-2052, 2067, 2073, 2087, 2193, 2197, 2208, 2232, 2245, 2253-2263, 2289
plugins/search/stac_list_assets.py              25      10  60.00%   44-51, 75-85
plugins/search/static_stac_search.py            84      17  79.76%   99-127, 166-169, 182
resources/__init__.py                            0       0  100.00%
resources/shp/__init__.py                        0       0  100.00%
types/__init__.py                              180      43  76.11%   58, 71-75, 86-98, 126-128, 135-140, 155, 165, 247, 280, 290-306, 311, 313, 335, 340, 348, 358
types/bbox.py                                   39      19  51.28%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                          10       0  100.00%
types/queryables.py                            113       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 103
types/stac_extensions.py                       162       4  97.53%   455-461, 640
types/stac_metadata.py                         159      22  86.16%   98, 120, 153, 186-188, 202-216, 225-232, 259-262
utils/__init__.py                              637      58  90.89%   57, 186-241, 254-256, 274, 307, 387-391, 466-470, 550, 590-591, 620, 1002-1005, 1056, 1075-1076, 1108, 1126-1127, 1291, 1293, 1301, 1305, 1311, 1313, 1315, 1321, 1323, 1423, 1555, 1620, 1795
utils/cache.py                                  22       0  100.00%
utils/dates.py                                 191       1  99.48%   182
utils/deserialize.py                            74      10  86.49%   53, 55, 103, 108-109, 148, 162, 184-185, 191
utils/env.py                                     3       0  100.00%
utils/exceptions.py                             58       0  100.00%
utils/free_text_search.py                       65       2  96.92%   83, 91
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                31       1  96.77%   41
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/repr.py                                   38       0  100.00%
utils/requests.py                               55      29  47.27%   51-52, 64, 85-96, 107-124, 128
utils/s3.py                                    269      11  95.91%   206-207, 248, 264, 286, 508, 555-556, 624, 699, 725
utils/stac_reader.py                           113      44  61.06%   63-85, 95-97, 101, 138, 154-159, 206-216, 226-256
utils/streamresponse.py                         91       4  95.60%   106, 166, 175, 187
utils/yaml.py                                   42       7  83.33%   49, 65, 70-74
TOTAL                                        11624    1389  88.05%

Diff against develop

Filename       Stmts    Miss  Cover
-----------  -------  ------  -------
utils/s3.py      +15      -1  +0.63%
TOTAL            +15      -1  +0.02%

Results for commit: f833e4e

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

[github-actions]

Code Coverage (Windows)

Details

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                     19       2  89.47%   60, 69
cli.py                                         259      11  95.75%   99-110, 376, 658
config.py                                      336      27  91.96%   71-73, 76, 79, 82, 86, 90, 94-96, 590-592, 746-748, 772-776, 795, 803, 833-838, 840
crunch.py                                        2       0  100.00%
api/__init__.py                                  0       0  100.00%
api/collection.py                              151       8  94.70%   177, 213, 216, 323, 362, 365, 383, 386
api/core.py                                    776      58  92.53%   262, 548, 596, 639, 679, 699, 740-745, 770, 854-873, 887, 893, 1046, 1051, 1150, 1189-1190, 1286-1287, 1313, 1344-1345, 1371, 1384, 1445-1446, 1477-1478, 1552-1557, 1569-1572, 1684, 1909, 2066, 2070, 2179, 2267-2268
api/provider.py                                389      42  89.20%   128-132, 199, 203-204, 341-362, 430, 546, 550-551, 556-559, 570, 646-656, 846-847, 895, 902, 916, 939-942, 976-983, 987-988
api/search_result.py                           138      12  91.30%   111, 123, 133, 154, 208, 266, 278, 413, 468-471
api/product/__init__.py                         26       4  84.62%   51-54
api/product/_assets.py                          58       5  91.38%   101, 195-196, 208-212
api/product/_product.py                        315      24  92.38%   217, 373, 399-402, 439, 458-459, 568, 597, 604, 644, 756, 799-802, 811-814, 868, 934, 946, 1013
api/product/metadata_mapping.py                887      57  93.57%   136-138, 259-264, 288, 347-348, 428, 448-449, 454, 517-518, 555, 576-579, 602, 614-615, 656, 679, 704-707, 709-714, 793, 799, 815, 823, 1054, 1207, 1229, 1238-1242, 1259-1264, 1397, 1420, 1429, 1451, 1456, 1523, 1620, 1641, 1667, 1681, 1779, 1874, 1970, 2036, 2080
api/product/drivers/__init__.py                 11       0  100.00%
api/product/drivers/base.py                     32       0  100.00%
api/product/drivers/generic.py                  11       0  100.00%
api/product/drivers/sentinel1.py                33       0  100.00%
api/product/drivers/sentinel2.py                33       0  100.00%
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 38       3  92.11%   50, 84-85
plugins/manager.py                             192      17  91.15%   103-108, 138, 220, 242, 260-261, 273, 312-313, 413-416, 428-429
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             8       0  100.00%
plugins/apis/ecmwf.py                           95      10  89.47%   163-165, 213-214, 240-242, 273-274
plugins/apis/usgs.py                           175      25  85.71%   158, 246, 279, 322-324, 329, 357-358, 363, 393-400, 411-416, 438-444
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py             124      35  71.77%   52-54, 69-70, 142-149, 177-203, 226, 258-262, 279, 303, 319-320
plugins/authentication/base.py                  26       4  84.62%   58, 71, 94, 108
plugins/authentication/eoiam.py                 98       1  98.98%   170
plugins/authentication/generic.py               16       3  81.25%   50, 55, 65
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              46       4  91.30%   154, 177-182
plugins/authentication/openid_connect.py       235      19  91.91%   98-99, 107-125, 172, 180, 194, 214, 353-356, 382, 423
plugins/authentication/qsauth.py                34       1  97.06%   91
plugins/authentication/sas_auth.py              57       3  94.74%   68, 89, 135
plugins/authentication/token.py                141       9  93.62%   192, 229, 309-310, 358-362
plugins/authentication/token_exchange.py        36      14  61.11%   75, 93-121
plugins/crunch/__init__.py                       6       0  100.00%
plugins/crunch/base.py                          12       0  100.00%
plugins/crunch/filter_date.py                   47       0  100.00%
plugins/crunch/filter_latest_intersect.py       52       6  88.46%   87-88, 95-96, 98-102
plugins/crunch/filter_latest_tpl_name.py        35       0  100.00%
plugins/crunch/filter_overlap.py                66      10  84.85%   118-121, 137-163
plugins/crunch/filter_property.py               30       2  93.33%   81-84
plugins/download/__init__.py                     4       0  100.00%
plugins/download/aws.py                        403      77  80.89%   271, 305, 354-357, 387-388, 396-400, 480-483, 523-525, 529, 560-561, 567-571, 602, 667-675, 739-833, 845-850, 888, 914, 959-961, 1013
plugins/download/base.py                       297      26  91.25%   148, 178, 253-255, 394-395, 439, 473, 550-554, 584, 619-620, 645-654, 713, 734, 756, 764
plugins/download/http.py                       578      74  87.20%   242, 284-287, 349-352, 355, 362-367, 398-400, 417, 432, 493, 528, 542, 556, 566-570, 586-591, 602, 621, 657-660, 681, 691, 698, 754, 862, 894, 926-935, 971, 1016-1021, 1030, 1045-1047, 1051, 1054, 1069-1070, 1080, 1155, 1207, 1249-1250, 1262, 1272, 1328-1329, 1359, 1397, 1477-1478
plugins/search/__init__.py                      25       0  100.00%
plugins/search/base.py                         207      19  90.82%   118, 122, 146-152, 161, 227-230, 290, 322, 343, 469, 554-557, 566
plugins/search/build_search_result.py          527      78  85.20%   138-139, 175, 179, 199, 258, 427-441, 520-521, 570, 582-584, 608, 610, 675, 683-687, 707, 717, 743, 792, 817, 879-884, 945, 973, 976, 1031-1052, 1093, 1120-1121, 1130-1139, 1229, 1244, 1247, 1252, 1271-1280, 1401-1402, 1446, 1455-1457, 1514, 1562-1572
plugins/search/cop_ghsl.py                     410      84  79.51%   73-74, 109-110, 174, 176, 218, 249-252, 298, 314, 360, 364-369, 384, 406, 427, 461, 477, 512, 527, 541-542, 556-558, 565, 569-571, 574-577, 595-655, 666-681
plugins/search/cop_marine.py                   297      65  78.11%   56, 64-66, 82, 104, 106, 109, 175-176, 194, 218, 224, 228, 250-251, 259, 294-297, 303, 313, 326, 334, 340-344, 350-353, 359-412, 428-431, 481, 493, 537-538
plugins/search/creodias_s3.py                   29       1  96.55%   59
plugins/search/csw.py                          112      87  22.32%   99-100, 104-105, 113-170, 176-189, 197-229, 246-287
plugins/search/eumetsat_ds.py                   25       0  100.00%
plugins/search/geodes.py                        66       0  100.00%
plugins/search/oar.py                           82      13  84.15%   50, 57-60, 172-180, 186-188, 195
plugins/search/qssearch.py                     889     108  87.85%   381, 425-426, 543-544, 567-568, 580-584, 816-822, 880, 946, 980, 987, 1058, 1079, 1082-1083, 1101, 1110-1111, 1129, 1200, 1209, 1214-1231, 1240, 1255, 1264, 1277, 1300, 1339, 1412, 1435, 1450, 1523-1524, 1526-1528, 1530-1531, 1618, 1625, 1733-1737, 1803, 1806, 1810-1811, 1830-1833, 1845, 1867-1884, 1892-1898, 1933-1935, 1956-1962, 1969, 2007, 2023, 2046, 2051-2052, 2067, 2073, 2087, 2193, 2197, 2208, 2232, 2245, 2253-2263, 2289
plugins/search/stac_list_assets.py              25      10  60.00%   44-51, 75-85
plugins/search/static_stac_search.py            84      17  79.76%   99-127, 166-169, 182
resources/__init__.py                            0       0  100.00%
resources/shp/__init__.py                        0       0  100.00%
types/__init__.py                              180      43  76.11%   58, 71-75, 86-98, 126-128, 135-140, 155, 165, 247, 280, 290-306, 311, 313, 335, 340, 348, 358
types/bbox.py                                   39      19  51.28%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                          10       0  100.00%
types/queryables.py                            113       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 103
types/stac_extensions.py                       162       4  97.53%   455-461, 640
types/stac_metadata.py                         159      22  86.16%   98, 120, 153, 186-188, 202-216, 225-232, 259-262
utils/__init__.py                              637      58  90.89%   57, 186-241, 254-256, 274, 307, 387-391, 466-470, 550, 590-591, 620, 1002-1005, 1056, 1075-1076, 1108, 1126-1127, 1291, 1293, 1301, 1305, 1311, 1313, 1315, 1321, 1323, 1423, 1555, 1620, 1795
utils/cache.py                                  22       0  100.00%
utils/dates.py                                 191       1  99.48%   182
utils/deserialize.py                            74      10  86.49%   53, 55, 103, 108-109, 148, 162, 184-185, 191
utils/env.py                                     3       0  100.00%
utils/exceptions.py                             58       0  100.00%
utils/free_text_search.py                       65       2  96.92%   83, 91
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                31       1  96.77%   41
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/repr.py                                   38       0  100.00%
utils/requests.py                               55      29  47.27%   51-52, 64, 85-96, 107-124, 128
utils/s3.py                                    269      11  95.91%   206-207, 248, 264, 286, 508, 555-556, 624, 699, 725
utils/stac_reader.py                           113      44  61.06%   63-85, 95-97, 101, 138, 154-159, 206-216, 226-256
utils/streamresponse.py                         91       4  95.60%   106, 166, 175, 187
utils/yaml.py                                   42       7  83.33%   49, 65, 70-74
TOTAL                                        11624    1391  88.03%

Diff against develop

Filename       Stmts    Miss  Cover
-----------  -------  ------  -------
utils/s3.py      +15      -1  +0.63%
TOTAL            +15      -1  +0.02%

Results for commit: f833e4e

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

[jlahovnik]

PR fixing pre-commit should be merged first