Skip to content

Commit 6cfa296

Browse files
XianBWXianBW
authored
fix: harden log server web boundaries (#1468)
* fix: harden log server web boundaries * style: sort log server imports * style: format DS trace link * fix: avoid exposing server exception details * docs: document secure log server configuration * fix: authenticate internal web storage updates
1 parent 2c878f9 commit 6cfa296

10 files changed

Lines changed: 500 additions & 64 deletions

File tree

‎README.md‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -399,6 +399,65 @@ rdagent server_ui --port 19899
399399
400400
After that, open `http://127.0.0.1:19899` in your browser.
401401
402+
##### Web UI security and remote access
403+
404+
The Flask backend listens on `127.0.0.1` by default. This keeps its process-control, upload, and trace APIs accessible only from the local machine. No authentication token is required while the server is bound to localhost.
405+
406+
To access the Web UI from another machine, explicitly bind it to a non-local address and configure an authentication token:
407+
408+
```sh
409+
export UI_SERVER_AUTH_TOKEN='<a-long-random-token>'
410+
rdagent server_ui --port 19899 --host 0.0.0.0
411+
```
412+
413+
Then open the following URL once to establish an authenticated browser session:
414+
415+
```text
416+
http://<server-host>:19899/?token=<a-long-random-token>
417+
```
418+
419+
The server removes the token from the address bar by redirecting to `/` and stores it in an HTTP-only, same-site cookie. API clients can instead send it in the request header:
420+
421+
```text
422+
Authorization: Bearer <a-long-random-token>
423+
```
424+
425+
The server refuses to bind to a non-local address unless `UI_SERVER_AUTH_TOKEN` is set. When exposing it outside a trusted development machine, put it behind an HTTPS reverse proxy and avoid recording token-bearing query strings in proxy logs. The `--host` option controls the address when the server is started through the CLI; `UI_SERVER_HOST` is the corresponding default for direct use of the backend entry point.
426+
427+
Cross-origin browser access is disabled by default. If the frontend and backend are served from different origins, configure an explicit JSON allowlist rather than enabling every origin:
428+
429+
```sh
430+
export UI_CORS_ALLOWED_ORIGINS='["https://ui.example.com"]'
431+
```
432+
433+
##### Web UI storage and compatibility settings
434+
435+
The Flask backend uses the following environment variables. Uploaded input files are deliberately kept outside the trace directory so that they cannot be discovered and deserialized as persisted traces.
436+
437+
| Environment variable | Default | Description |
438+
| --- | --- | --- |
439+
| `UI_STATIC_PATH` | `./git_ignore_folder/static` | Directory containing the built Web UI assets. |
440+
| `UI_TRACE_FOLDER` | `./git_ignore_folder/traces` | Directory containing generated trace data and process logs. |
441+
| `UI_UPLOAD_FOLDER` | `./git_ignore_folder/uploads` | Isolated directory for uploaded input files. Mount, back up, and clean it separately from the trace directory. |
442+
| `UI_SERVER_HOST` | `127.0.0.1` | Default host used by the backend entry point. Use `server_ui --host` when starting it through the CLI. |
443+
| `UI_SERVER_AUTH_TOKEN` | empty | Bearer/cookie authentication token. Required for any non-localhost binding. |
444+
| `UI_CORS_ALLOWED_ORIGINS` | `[]` | JSON list of allowed browser origins. CORS is disabled when the list is empty. |
445+
| `UI_MAX_UPLOAD_MB` | `20` | Maximum size in MiB of an entire HTTP request, including all uploaded files and form data. |
446+
| `UI_LOAD_LEGACY_PICKLE_TRACES` | `false` | Whether to deserialize persisted pickle traces when the server starts. Enable only for a fully trusted trace directory. |
447+
448+
Uploads whose filenames end in `.dill`, `.pickle`, `.pkl`, `.py`, `.pyc`, or `.pyo` are rejected. Existing workflows that use these formats as uploaded inputs must convert them to a non-executable data format or provide them through another trusted mechanism.
449+
450+
Legacy pickle trace loading is disabled by default because pickle deserialization can execute code. Consequently, after a server restart, an existing trace may still appear in the history list but its saved messages will not be loaded into the Web UI. If compatibility with trusted historical traces is required, opt in explicitly:
451+
452+
```sh
453+
export UI_LOAD_LEGACY_PICKLE_TRACES=true
454+
rdagent server_ui --port 19899
455+
```
456+
457+
Only enable this setting when every file under `UI_TRACE_FOLDER` is trusted and the directory is not writable by untrusted users or services.
458+
459+
Data-science trace share links no longer accept a URL-controlled `log_folder`. A link can preserve the selected trace, but the recipient must have the corresponding log folder configured or select it in the UI.
460+
402461
#### Common Notes
403462
404463
Port `19899` is used in the examples above. Before starting either UI, check whether this port is already occupied. If it is, please change it to another available port.

‎docs/ui.rst‎

Lines changed: 140 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ Introduction
88

99
RD-Agent will generate some logs during the R&D process. These logs are very useful for debugging and understanding the R&D process. However, just viewing the terminal log is not intuitive enough. RD-Agent provides a web app as UI to visualize the R&D process. You can easily view the R&D process and understand the R&D process better.
1010

11-
A Quick Demo
11+
Streamlit UI
1212
============
1313

1414
Start Web App
@@ -47,3 +47,142 @@ Use Web App
4747
- Next Loop: Show one success **R&D Loop**.
4848
- One Evolving: Show one **evolving** step of **development** part.
4949
- refresh logs: clear shown logs.
50+
51+
52+
Flask Web UI
53+
============
54+
55+
RD-Agent also provides a separate frontend in ``web/`` backed by the Flask log
56+
server started with ``rdagent server_ui``. This UI provides real-time trace,
57+
upload, process-control, and user-interaction APIs.
58+
59+
Build and start
60+
---------------
61+
62+
Install the frontend dependencies and build the static assets:
63+
64+
.. code-block:: bash
65+
66+
cd web
67+
npm install
68+
npm run build:flask
69+
cd ..
70+
71+
The generated assets are served from ``./git_ignore_folder/static`` by default.
72+
Set ``UI_STATIC_PATH`` before starting the server to use another directory.
73+
74+
Start the server locally:
75+
76+
.. code-block:: bash
77+
78+
rdagent server_ui --port 19899
79+
80+
Then open ``http://127.0.0.1:19899``. The server listens on localhost by
81+
default, so its process-control, upload, and trace APIs are not exposed to
82+
other machines.
83+
84+
Remote access and authentication
85+
--------------------------------
86+
87+
To access the Flask Web UI remotely, explicitly select a non-local address and
88+
configure a long, random authentication token:
89+
90+
.. code-block:: bash
91+
92+
export UI_SERVER_AUTH_TOKEN='<a-long-random-token>'
93+
rdagent server_ui --port 19899 --host 0.0.0.0
94+
95+
The server refuses to bind to a non-local address unless
96+
``UI_SERVER_AUTH_TOKEN`` is set. Open the following URL once to establish an
97+
authenticated browser session:
98+
99+
.. code-block:: text
100+
101+
http://<server-host>:19899/?token=<a-long-random-token>
102+
103+
The server redirects to ``/`` after storing the token in an HTTP-only,
104+
same-site cookie. API clients can supply the same token without using a cookie:
105+
106+
.. code-block:: text
107+
108+
Authorization: Bearer <a-long-random-token>
109+
110+
Put remotely accessible deployments behind an HTTPS reverse proxy. Avoid
111+
recording the initial token-bearing URL in proxy logs or sharing it through an
112+
untrusted channel. When the server is started through the CLI, ``--host``
113+
controls the listening address; ``UI_SERVER_HOST`` is the corresponding default
114+
when invoking the backend entry point directly.
115+
116+
CORS is disabled by default. If a browser frontend is hosted on another origin,
117+
configure an explicit JSON allowlist:
118+
119+
.. code-block:: bash
120+
121+
export UI_CORS_ALLOWED_ORIGINS='["https://ui.example.com"]'
122+
123+
Configuration
124+
-------------
125+
126+
The Flask Web UI supports the following environment variables:
127+
128+
.. list-table::
129+
:header-rows: 1
130+
:widths: 30 25 70
131+
132+
* - Environment variable
133+
- Default
134+
- Description
135+
* - ``UI_STATIC_PATH``
136+
- ``./git_ignore_folder/static``
137+
- Directory containing the built Web UI assets.
138+
* - ``UI_TRACE_FOLDER``
139+
- ``./git_ignore_folder/traces``
140+
- Directory containing generated trace data and process logs.
141+
* - ``UI_UPLOAD_FOLDER``
142+
- ``./git_ignore_folder/uploads``
143+
- Isolated directory for uploaded inputs. Mount, back up, and clean it
144+
separately from the trace directory.
145+
* - ``UI_SERVER_HOST``
146+
- ``127.0.0.1``
147+
- Default host used by the backend entry point. Use ``server_ui --host``
148+
when starting the server through the CLI.
149+
* - ``UI_SERVER_AUTH_TOKEN``
150+
- empty
151+
- Bearer/cookie authentication token. Required for non-localhost bindings.
152+
* - ``UI_CORS_ALLOWED_ORIGINS``
153+
- ``[]``
154+
- JSON list of allowed browser origins. CORS is disabled when empty.
155+
* - ``UI_MAX_UPLOAD_MB``
156+
- ``20``
157+
- Maximum size in MiB of the complete HTTP request, including all files and
158+
form data.
159+
* - ``UI_LOAD_LEGACY_PICKLE_TRACES``
160+
- ``false``
161+
- Whether to deserialize persisted pickle traces at startup. Enable only
162+
for a fully trusted trace directory.
163+
164+
Upload and trace safety
165+
-----------------------
166+
167+
Uploaded input files are stored outside ``UI_TRACE_FOLDER`` so they cannot be
168+
discovered and deserialized as persisted traces. Uploads ending in ``.dill``,
169+
``.pickle``, ``.pkl``, ``.py``, ``.pyc``, or ``.pyo`` are rejected. Workflows
170+
using these formats as uploaded inputs must convert them to a non-executable
171+
data format or provide them through another trusted mechanism.
172+
173+
Legacy pickle trace loading is disabled by default because deserializing a
174+
pickle can execute code. After a restart, a historical trace may still appear
175+
in the history list while its saved messages remain unloaded. To browse trusted
176+
historical traces, opt in explicitly:
177+
178+
.. code-block:: bash
179+
180+
export UI_LOAD_LEGACY_PICKLE_TRACES=true
181+
rdagent server_ui --port 19899
182+
183+
Only enable this setting when every file under ``UI_TRACE_FOLDER`` is trusted
184+
and the directory is not writable by untrusted users or services.
185+
186+
Data-science trace share links no longer accept a URL-controlled ``log_folder``.
187+
A link can preserve the selected trace, but the recipient must configure or
188+
select the corresponding log folder in the UI.

‎rdagent/app/cli.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -62,13 +62,13 @@ def ui(port=19899, log_dir="", debug: bool = False, data_science: bool = False):
6262
subprocess.run(cmds)
6363

6464

65-
def server_ui(port=19899):
65+
def server_ui(port: int = 19899, host: str = "127.0.0.1") -> None:
6666
"""
6767
start the Flask log server in real time
6868
"""
6969
from rdagent.log.server.app import main as log_server_main
7070

71-
log_server_main(port=port)
71+
log_server_main(port=port, host=host)
7272

7373

7474
def ds_user_interact(port=19900):

0 commit comments

Comments
 (0)