I have a question about Digdag workflows.
In a .dig file, I have the following configuration:
_export:
plugin:
repositories:
- https://jitpack.io/
dependencies:
- com.github.takemikami:digdag-plugin-shresult:0.0.3
+find_dirs:
sh_result>: |
find ./ -maxdepth 1 -type d -exec basename {} \; | grep -v "^.$" | grep -E '^\[' | sort -u | tr '\n' ',' | sed 's/,$//'
destination_variable: dirs
stdout_format: text
+call_dig:
for_each>:
dir: "${dirs.split(',')}"
_parallel: true
_do:
+loop_dig:
call>: ${dir}/unload.dig
In the ‘+find_dirs’ task, I’m attempting to extract directory names that start with ‘[’ and store them in the ‘dirs’ variable. However, during Digdag execution, there are instances where items other than directory names starting with ‘[’ are being stored in ‘dirs’.
Is this potentially a bug in Digdag? I would appreciate your confirmation.
Here are the ‘dirs’ values that I want to extract and the ‘dirs’ values that were mistakenly extracted in this operation:
Desired ‘dirs’:
dirs: [foo1]bar1,[foo2]bar2,[foo3]bar3,[foo4]bar4
‘Dirs’ extracted in the current operation (Example 1: Code from the executed workflow’s .dig file is being extracted):
dirs: >+
2023-09-14 21:45:48 +0000 [INFO] (1913@[0:test_wf]+test_coordinator+call_dig^sub^sub+find_dirs) io.digdag.core.agent.OperatorManager: sh_result>: find ./ -maxdepth 1 -type d -exec basename {} \; | grep -v "^.$" | grep -E '^\[' | sort -u | tr '\n' ',' | sed 's/,$//'
‘Dirs’ extracted in the current operation (Example 2: Logs from other projects executed at the same time are being extracted):
dirs: >-
2023-09-07 09:15:02 +0000 [INFO] (5117@[0:cost_alert]+cost_change+scripts+notice) io.digdag.core.agent.OperatorManager: sh>: cd cost_alert python3 [main.py](http://main.py/) -t increase [foo1]bar1,[foo2]bar2,[foo3]bar3,[foo4]bar4
I have a question about Digdag workflows.
In a .dig file, I have the following configuration:
In the ‘+find_dirs’ task, I’m attempting to extract directory names that start with ‘[’ and store them in the ‘dirs’ variable. However, during Digdag execution, there are instances where items other than directory names starting with ‘[’ are being stored in ‘dirs’.
Is this potentially a bug in Digdag? I would appreciate your confirmation.
Here are the ‘dirs’ values that I want to extract and the ‘dirs’ values that were mistakenly extracted in this operation:
Desired ‘dirs’:
dirs: [foo1]bar1,[foo2]bar2,[foo3]bar3,[foo4]bar4‘Dirs’ extracted in the current operation (Example 1: Code from the executed workflow’s .dig file is being extracted):
‘Dirs’ extracted in the current operation (Example 2: Logs from other projects executed at the same time are being extracted):