Skip to content

Logging: reduce duplication of jobs-submit cmd in scheduler log#7127

Draft
MetRonnie wants to merge 2 commits intocylc:masterfrom
MetRonnie:log-warnings
Draft

Logging: reduce duplication of jobs-submit cmd in scheduler log#7127
MetRonnie wants to merge 2 commits intocylc:masterfrom
MetRonnie:log-warnings

Conversation

@MetRonnie
Copy link
Copy Markdown
Member

@MetRonnie MetRonnie commented Dec 8, 2025

Partially addresses #3647

When investigating an issue, the repeated logging of a batch jobs-submit command was rather irritating and hampered looking through the scheduler log.

Repro

Have a workflow with lots of tasks that will be batch-submitted but will fail to submit.

# flow.cylc
[task parameters]
    x = 1..10
[scheduling]
    [[graph]]
        R1 = foo<x>
[runtime]
    [[foo<x>]]
        platform = borked
# global.cylc
[platforms]
    [[borked]]
        hosts = localhost
        install target = localhost
        job runner = sge

Logging diff:

 ERROR - [jobs-submit cmd] cylc jobs-submit --path=/bin --path=/usr/bin --path=/usr/local/bin --path=/sbin --path=/usr/sbin --path=/usr/local/sbin -- '$HOME/cylc-run/diamond/log/job' 1/foo_x01/01 1/foo_x02/01 1/foo_x03/01 1/foo_x04/01 1/foo_x05/01 1/foo_x06/01 1/foo_x07/01 1/foo_x08/01 1/foo_x09/01 1/foo_x10/01
     [jobs-submit ret_code] 1
-    [jobs-submit out] 2025-12-08T18:23:34Z|1/foo_x01/01|1|
 ERROR - [1/foo_x01/01:preparing] submission failed
 INFO - [1/foo_x01/01:preparing] => submit-failed
 WARNING - [1/foo_x01/01:submit-failed] did not complete the required outputs:
     x  succeeded
-ERROR - [jobs-submit cmd] cylc jobs-submit --path=/bin --path=/usr/bin --path=/usr/local/bin --path=/sbin --path=/usr/sbin --path=/usr/local/sbin -- '$HOME/cylc-run/diamond/log/job' 1/foo_x01/01 1/foo_x02/01 1/foo_x03/01 1/foo_x04/01 1/foo_x05/01 1/foo_x06/01 1/foo_x07/01 1/foo_x08/01 1/foo_x09/01 1/foo_x10/01
-    [jobs-submit ret_code] 1
-    [jobs-submit out] 2025-12-08T18:23:34Z|1/foo_x02/01|1|
 ERROR - [1/foo_x02/01:preparing] submission failed
 INFO - [1/foo_x02/01:preparing] => submit-failed
 WARNING - [1/foo_x02/01:submit-failed] did not complete the required outputs:
     x  succeeded
-ERROR - [jobs-submit cmd] cylc jobs-submit --path=/bin --path=/usr/bin --path=/usr/local/bin --path=/sbin --path=/usr/sbin --path=/usr/local/sbin -- '$HOME/cylc-run/diamond/log/job' 1/foo_x01/01 1/foo_x02/01 1/foo_x03/01 1/foo_x04/01 1/foo_x05/01 1/foo_x06/01 1/foo_x07/01 1/foo_x08/01 1/foo_x09/01 1/foo_x10/01
-    [jobs-submit ret_code] 1
-    [jobs-submit out] 2025-12-08T18:23:34Z|1/foo_x03/01|1|
 ERROR - [1/foo_x03/01:preparing] submission failed
 INFO - [1/foo_x03/01:preparing] => submit-failed
 WARNING - [1/foo_x03/01:submit-failed] did not complete the required outputs:
     x  succeeded
-ERROR - [jobs-submit cmd] cylc jobs-submit --path=/bin --path=/usr/bin --path=/usr/local/bin --path=/sbin --path=/usr/sbin --path=/usr/local/sbin -- '$HOME/cylc-run/diamond/log/job' 1/foo_x01/01 1/foo_x02/01 1/foo_x03/01 1/foo_x04/01 1/foo_x05/01 1/foo_x06/01 1/foo_x07/01 1/foo_x08/01 1/foo_x09/01 1/foo_x10/01
-    [jobs-submit ret_code] 1
-    [jobs-submit out] 2025-12-08T18:23:34Z|1/foo_x04/01|1|
 ERROR - [1/foo_x04/01:preparing] submission failed
 INFO - [1/foo_x04/01:preparing] => submit-failed
 WARNING - [1/foo_x04/01:submit-failed] did not complete the required outputs:
     x  succeeded
-ERROR - [jobs-submit cmd] cylc jobs-submit --path=/bin --path=/usr/bin --path=/usr/local/bin --path=/sbin --path=/usr/sbin --path=/usr/local/sbin -- '$HOME/cylc-run/diamond/log/job' 1/foo_x01/01 1/foo_x02/01 1/foo_x03/01 1/foo_x04/01 1/foo_x05/01 1/foo_x06/01 1/foo_x07/01 1/foo_x08/01 1/foo_x09/01 1/foo_x10/01
-    [jobs-submit ret_code] 1
-    [jobs-submit out] 2025-12-08T18:23:34Z|1/foo_x05/01|1|
 ERROR - [1/foo_x05/01:preparing] submission failed
 INFO - [1/foo_x05/01:preparing] => submit-failed
 WARNING - [1/foo_x05/01:submit-failed] did not complete the required outputs:
     x  succeeded
 ...

Check List

  • I have read CONTRIBUTING.md and added my name as a Code Contributor.
  • Contains logically grouped changes (else tidy your branch by rebase).
  • Does not contain off-topic changes (use other PRs for other changes).
  • No dependency changes
  • Tests are included
  • No changelog needed as minor
  • No docs needed
  • If this is a bug fix, PR should be raised against the relevant ?.?.x branch.

@MetRonnie MetRonnie added small could be better Not exactly a bug, but not ideal. labels Dec 8, 2025
@MetRonnie MetRonnie self-assigned this Dec 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

could be better Not exactly a bug, but not ideal. small

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant