Skip to content

Actions: huggingface/lighteval

Actions

Slow end to end tests

Actions

Loading...
Loading

Show workflow options

Create status badge

Loading
slow_tests.yaml
878 workflow runs
878 workflow runs

Filter by Event

⌥ + click/return to exclude

Filter by Status

⌥ + click/return to exclude

Filter by Branch

⌥ + click/return to exclude

Filter by Actor

⌥ + click/return to exclude
don't score an empty F1 gold reference as zero Slow end to end tests #1603: Pull request #1354 opened by gyanu2507
Action required gyanu2507:fix/f1-empty-gold
gyanu2507:fix/f1-empty-gold
Action required
docs: fix broken code examples in evaluating-a-custom-model.mdx Slow end to end tests #1602: Pull request #1352 opened by mohansree14
Action required mohansree14:fix/custom-model-docs-syntax
mohansree14:fix/custom-model-docs-syntax
Action required
Fix duplicate custom task module imports Slow end to end tests #1601: Pull request #1351 opened by yangweigbh
Action required yangweigbh:fix/reuse-custom-task-module
yangweigbh:fix/reuse-custom-task-module
Action required
fix: keep every task's responses when reloading them from details Slow end to end tests #1600: Pull request #1350 opened by tonycoder-hub
Action required tonycoder-hub:cursor/fix-details-reload-multi-task-responses-3037
tonycoder-hub:cursor/fix-details-reload-multi-task-responses-3037
Action required
Fix relational comparisons with different condition counts Slow end to end tests #1599: Pull request #1349 opened by Excelius-Wang
Action required Excelius-Wang:fix/relational-condition-count
Excelius-Wang:fix/relational-condition-count
Action required
Parse hellaswag_arabic endings with ast.literal_eval Slow end to end tests #1598: Pull request #1348 opened by Nakul-Sinha
Action required Nakul-Sinha:fix/arabic-hellaswag-literal-eval
Nakul-Sinha:fix/arabic-hellaswag-literal-eval
Action required
Add add_generation_prompt=True when tokenizing judge prompts Slow end to end tests #1597: Pull request #1347 opened by Nakul-Sinha
Action required Nakul-Sinha:fix/judge-add-generation-prompt
Nakul-Sinha:fix/judge-add-generation-prompt
Action required
fix(metrics): avg@n scores raw text instead of post-processed text, and skips preprocessing Slow end to end tests #1586: Pull request #1335 opened by arthi-arumugam-git
Action required arthi-arumugam-git:fix-avg-at-n-post-processed-text
arthi-arumugam-git:fix-avg-at-n-post-processed-text
Action required
Fix few-shot sampler isolation Slow end to end tests #1585: Pull request #1334 opened by atirna
20d 1h 46m 7s atirna:fix/fewshot-sampler-isolation
atirna:fix/fewshot-sampler-isolation
20d 1h 46m 7s
Fix BBEH generative evaluation Slow end to end tests #1584: Pull request #1333 opened by lakshayxi
Action required lakshayxi:fix-bbeh-evaluation
lakshayxi:fix-bbeh-evaluation
Action required
Fix DetailsLogger compatibility with xxhash 4.0 Slow end to end tests #1583: Pull request #1332 opened by Honorllq
Action required Honorllq:agent/fix-xxhash-compat
Honorllq:agent/fix-xxhash-compat
Action required
Fix inspect-ai GenerateConfig field names in lighteval eval Slow end to end tests #1582: Pull request #1331 opened by harshasiddartha
Action required harshasiddartha:fix/inspect-ai-generateconfig-field-names
harshasiddartha:fix/inspect-ai-generateconfig-field-names
Action required
fix(inspect): align GenerateConfig field names with current inspect-ai (fixes #1327) Slow end to end tests #1581: Pull request #1328 synchronize by sk8ordie84
Action required sk8ordie84:fix/inspect-ai-generateconfig-fields
sk8ordie84:fix/inspect-ai-generateconfig-fields
Action required
fix(inspect): align GenerateConfig field names with current inspect-ai (fixes #1327) Slow end to end tests #1580: Pull request #1328 opened by sk8ordie84
Action required sk8ordie84:fix/inspect-ai-generateconfig-fields
sk8ordie84:fix/inspect-ai-generateconfig-fields
Action required
Store provider credentials as SecretStr in model configs (#1326) Slow end to end tests #1579: Commit 932e1f2 pushed by NathanHB
3m 3s main
main
3m 3s
Store provider credentials as SecretStr in model configs Slow end to end tests #1578: Pull request #1326 synchronize by NathanHB
3m 12s fix/litellm-api-key-leak
fix/litellm-api-key-leak
3m 12s
Store provider credentials as SecretStr in model configs Slow end to end tests #1577: Pull request #1326 synchronize by NathanHB
26s fix/litellm-api-key-leak
fix/litellm-api-key-leak
26s
Store provider credentials as SecretStr in model configs Slow end to end tests #1576: Pull request #1326 synchronize by NathanHB
23s fix/litellm-api-key-leak
fix/litellm-api-key-leak
23s
Store provider credentials as SecretStr in model configs Slow end to end tests #1575: Pull request #1326 opened by NathanHB
25s fix/litellm-api-key-leak
fix/litellm-api-key-leak
25s
Fix unsafe eval in Arabic HellaSwag prompt parsing Slow end to end tests #1574: Pull request #1325 opened by chosch801
30d 0h 5m 15s chosch801:fix/1293-safe-arabic-endings
chosch801:fix/1293-safe-arabic-endings
30d 0h 5m 15s
Fix stale openai reference in endpoint command help text Slow end to end tests #1573: Pull request #1324 opened by RudrenduPaul
30d 0h 5m 0s RudrenduPaul:fix-openai-endpoint-help-960
RudrenduPaul:fix-openai-endpoint-help-960
30d 0h 5m 0s
Expose inspect-ai sample_shuffle in the eval command Slow end to end tests #1572: Pull request #1323 opened by mshimkus-positron
30d 0h 4m 21s mshimkus-positron:expose-sample-shuffle
mshimkus-positron:expose-sample-shuffle
30d 0h 4m 21s
[codex] Preserve provider usage metadata Slow end to end tests #1571: Pull request #1322 opened by Tanisha-Katara
30d 0h 0m 26s Tanisha-Katara:codex/preserve-provider-usage-metadata
Tanisha-Katara:codex/preserve-provider-usage-metadata
30d 0h 0m 26s
fix: restore hydra-core compatibility Slow end to end tests #1570: Pull request #1321 synchronize by Kloudedge-apex
30d 0h 1m 30s Kloudedge-apex:agent/fix-hydra-antlr-compatibility
Kloudedge-apex:agent/fix-hydra-antlr-compatibility
30d 0h 1m 30s
fix: restore hydra-core compatibility Slow end to end tests #1569: Pull request #1321 opened by Kloudedge-apex
30d 0h 1m 56s Kloudedge-apex:agent/fix-hydra-antlr-compatibility
Kloudedge-apex:agent/fix-hydra-antlr-compatibility
30d 0h 1m 56s