Forward stdin through --connect to compilation server

[Copilot]

haxe --connect blocks on recv without forwarding stdin to the server, so programs executed via --interp/--run through the compilation server cannot receive piped input.

# Previously: stdin data is silently discarded
echo "hello" | haxe --connect 6000 --main Main --interp

Changes

• compilationContext.ml: Add stdin : in_channel option to communication record. Refactor server_accept from a tuple to a server_connection record with named fields (support_nonblock, read, write, close, get_stdin) for readability.

• serverCompilationContext.ml: Add current_stdin and current_stdin_pipe fields to ServerCompilationContext.t, replacing the former global refs.

• server.ml (client — Connect module):

  • Extract client-side connection logic into a Connect module with poll and do_connect
  • poll uses a single Unix.select loop to multiplex reading from both the server socket and local stdin — no separate thread needed
  • On stdin EOF, call shutdown SHUTDOWN_SEND to signal completion
  • Explicit exit 0/exit 1 to ensure clean process termination

• server.ml (server — SocketRequest module):

  • Extract null-terminated request reading and stdin pipe forwarding into a dedicated SocketRequest module
  • read returns {data; stdin} where stdin is a pipe fed by a forwarding thread via setup_client_stdin_forward
  • Scan for \000 anywhere in recv buffer instead of only checking the last byte — handles TCP segmentation where stdin data arrives in the same segment as the args terminator

• server.ml (server — init_wait_socket):

  • After reading the request via SocketRequest.read, store the returned stdin channel in a connection-local ref exposed via get_stdin on the server_connection record
  • Ignore SIGPIPE via raw signal 13 (Sys.sigpipe is -8 in OCaml 5, unusable)

• server.ml (server — parse_file): Takes sctx instead of cs to access sctx.current_stdin directly rather than a global ref.

• compiler.ml (create_compilation_context): Use comm.stdin when present instead of creating an immediately-EOF pipe

Testing

• tests/misc/connect_stdin/: Integration tests using sys.io.Process to orchestrate a --wait server and --connect clients:
  • Single line stdin forwarding through --connect
  • Multiline stdin forwarding through --connect
  • No-stdin request completes without hanging
• Registered in CI under the Macro target (Linux) in tests/runci/targets/Macro.hx

Known limitation

The server-side forwarding thread blocks on recv(sin) until the connection closes, which can delay cleanup for requests that don't consume stdin (e.g. --version). A follow-up could run process in a separate thread so the main thread owns the stdin forwarding lifecycle directly, or replace the pipe+thread with Unix.dup on the socket fd.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[Simn]

@copilot A few follow-up questions. I always struggle with Unix pipes and sockets so bear with me!

1. Could the server-side be simplified if we launched the process call inside wait_loop in a thread? This is something we've been discussing anyway, so perhaps now might be a good opportunity to do so. The idea would be to only run a single process at a time and have further requests queue up to execute afterwards, which should prevent the vast majority of shared data concerns. My thinking is that while the actual compilation is going on, the main thread could keep managing further incoming data independently.
2. Please look into refactoring so that we don't need the global current_stdin reference and the new current_stdin_pipe one. I'm not sure where these should go but we definitely don't want to have any globals.
3. Isn't there a way to use Unix.select on the client-side in do_connect to read from either the socket or stdin which lets us avoid the thread there?
4. The Find null terminator part seems like a necessary evil indeed. I don't want to abstract this into a complete pipeline implementation, but could we try to wrap this in a separate module to hide some of the ugliness there and have a cleaner init_wait_socket?

[Copilot]

@copilot A few follow-up questions. I always struggle with Unix pipes and sockets so bear with me!

1. Could the server-side be simplified if we launched the process call inside `w...

Here's what I did for points 2-4, and my analysis for point 1:

Points 2-4 are addressed in commit e4bc032:

• Point 2: current_stdin and current_stdin_pipe moved from global refs to ServerCompilationContext.t fields. parse_file takes sctx instead of cs. The stdin pipe from init_wait_socket is stored in a local ref returned via get_stdin on the connection record.
• Point 3: Client-side do_connect now uses a single Unix.select loop multiplexing the server socket and stdin — no thread needed.
• Point 4: Null-terminated request parsing extracted into SocketRequest module with read returning {data; overflow}.

Also refactored server_accept from a 5-tuple to a server_connection record.

Point 1 — threading process in wait_loop:

This would be a meaningful architectural improvement. With process in a separate thread, the main thread could own the full connection lifecycle: accept → read request → manage stdin forwarding → wait for compilation → close. The main benefit for stdin specifically: the main thread could sit in a select loop forwarding stdin data to the pipe while the compilation thread runs, and cleanly stop when compilation finishes. This would eliminate the server-side forwarding thread entirely.

The "one compilation at a time" constraint makes this tractable. However, there's substantial global mutable state to consider:

• type_module_hook, TypeloadParse.parse_hook (set once in enable_cache_mode, should be fine)
• Parser.reset_state(), Parallel.enable, DeprecationCheck.warned_positions (reset per-compilation in reset)
• MacroContext.macro_interp_cache (shared interpreter, accessed in cleanup)
• stats counters (reset and incremented during compilation)

With one-at-a-time and a mutex, most of these are safe since they're only accessed during compilation. The tricky part is the gap between compilation finishing and cleanup running — the main thread would need to ensure cleanup happens before the next compilation starts.

I think this is better suited for a follow-up PR since it's a larger refactor orthogonal to stdin forwarding. It would also resolve the known limitation where the server-side forwarding thread blocks on recv(sin) after compilation finishes.

[Simn]

@copilot I have pushed two refactoring commits. Please look at each of them and trace that I didn't change any behavior!

And then, we'll have to find a way to add at least some basic tests for this. I don't know the best approach for that, so please investigate!

[Copilot]

@copilot I have pushed two refactoring commits. Please look at each of them and trace that I didn't change any behavior!

And then, we'll have to find a way to add at least some basi...

I traced both refactoring commits:

"move" (467347f): ✅ Pure behavioral refactor. Moves the stdin pipe+thread setup from inline in init_wait_socket into SocketRequest.setup_client_stdin_forward, and changes the return type from {data; overflow} to {data; stdin}. The pipe forwarding logic is identical.

"split" (cba7977): ⚠️ Found one behavioral change — process_response() was moved from after the loop() call (where it flushes any remaining response data when the server closes the connection) to before it (where it's a no-op on an empty buffer). Fixed in c361514.

For testing, I added tests/misc/connect_stdin/ with three integration tests:

1. Single line stdin forwarding through --connect
2. Multiline stdin forwarding through --connect
3. No-stdin request completes without hanging

These run as --interp using sys.io.Process to orchestrate a --wait server and --connect clients. Registered in CI under the Macro target (Linux) in c361514.

[Simn]

No way this works...

Edit: It works!

[Simn]

There's more work to be done here for sure, but all the ugly parts are nicely isolated for now so I'll go ahead and merge this as-is because I have some big refactoring on top of this coming up.