FC: add assignment preset for controller on :5020 + clearer diagnostics

README.md

@@ -227,6 +227,12 @@ If you started the task server with the lite preset, you can also run the lite e
 python -m src.assigner --config configs/assignments/lite.yaml
 ```
 
+If you started the AgentBench FC stack via docker compose (`extra/docker-compose.yml`), use the FC assignment preset (controller on port 5020):
+
+```bash
+python -m src.assigner --config configs/assignments/default_fc.yaml
+```
+
 ## Next Steps
 
 If you wish to launch more tasks or use other models, you can refer to the content

configs/assignments/default_fc.yaml

@@ -0,0 +1,25 @@
+# Default assignment preset for AgentBench FC (AgentRL controller on :5020).
+#
+# Bring up the stack:
+#   docker compose -f extra/docker-compose.yml up
+#
+# Run:
+#   python -m src.assigner --config configs/assignments/default_fc.yaml
+
+import: definition_fc.yaml
+
+concurrency:
+  task:
+    dbbench-std: 5
+    os-std: 5
+  agent:
+    gpt-3.5-turbo-0613: 5
+
+assignments:
+  - agent:
+      - gpt-3.5-turbo-0613
+    task:
+      - dbbench-std
+      - os-std
+
+output: "outputs/{TIMESTAMP}"

configs/assignments/definition_fc.yaml

@@ -0,0 +1,22 @@
+# Definition preset for AgentBench FC (AgentRL controller).
+#
+# The docker-compose stack in extra/docker-compose.yml uses an AgentRL controller
+# that listens on port 5020 (host network mode). Use this definition when you
+# start the benchmark with:
+#
+#   docker compose -f extra/docker-compose.yml up
+#
+# Then run assigner with:
+#   python -m src.assigner --config configs/assignments/default_fc.yaml
+
+definition:
+  task:
+    overwrite:
+      module: src.client.TaskClient
+      parameters:
+        controller_address: "http://localhost:5020/api"
+    import: ../tasks/task_assembly.yaml
+  agent:
+    import:
+      - ../agents/api_agents.yaml
+      - ../agents/fs_agent.yaml

docs/Entrance_en.md

@@ -56,6 +56,7 @@ real-time to the specified output directory.
 Parameters for assigner:
 
 - `[--config CONFIG]`: Specifies the configuration file to read. Default is `configs/assignments/default.yaml`.
+  - If you started the AgentBench FC stack via `docker compose -f extra/docker-compose.yml up`, use `configs/assignments/default_fc.yaml` (controller on `http://localhost:5020/api`).
 - `[--auto-retry]`: Auto retry failed samples.
 
 If the `output` field in the configuration contains `{TIMESTAMP}`, it will be replaced with the current time for

src/client/task.py

@@ -37,7 +37,14 @@ def get_concurrency(self) -> int:
                 self.controller_address + "/list_workers"
             )
         except Exception as e:
-            print(ColorMessage.yellow(f"Warning task {self.name} cannot connect to controller {e}"))
+            print(
+                ColorMessage.yellow(
+                    "Warning: task '{}' cannot connect to controller '{}': {}. "
+                    "If you started the AgentRL controller via `docker compose -f extra/docker-compose.yml up`, "
+                    "the controller is typically on http://localhost:5020/api; set `controller_address` in the assignment config accordingly."
+                    .format(self.name, self.controller_address, e)
+                )
+            )
             return 0
         if result.status_code != 200:
             raise AgentBenchException(result.text, result.status_code, self.name)