Skip to content

mDNS peer discovery broken when running from git source (uv sync + uv run) #1788

@Sofille65

Description

@Sofille65

Description

When running exo from a fresh git clone + uv sync (as opposed to the pre-built EXO.app), nodes on the same local network do not discover each other via mDNS. Each node starts, elects itself as Master, and runs in isolation — no peers are ever found.

The same machines running EXO.app v1.0.68 (pre-built DMG) form a 4-node cluster within seconds.

Environment

  • Version: v1.0.68 (tag ba611f9c)
  • Platform: macOS 26.3.1 (4x Mac Studio, Apple M3 Ultra, ARM64)
  • Network: Same LAN (192.168.86.0/24), multicast working
  • Python: 3.13.12 via uv 0.10.7
  • exo_pyo3_bindings: 0.1.0, built from source with maturin

Steps to reproduce

  1. git clone https://github.com/exo-explore/exo.git
  2. cd exo && git checkout v1.0.68
  3. cd dashboard && npm install && npm run build && cd ..
  4. uv sync
  5. uv run exo
  6. Repeat on a second machine on the same LAN
  7. Nodes never discover each other — each remains an isolated Master indefinitely

Investigation details

  • Firewall: Disabled on all machines
  • Multicast: Working — ping 224.0.0.251 gets responses from all machines on the LAN
  • mDNS port: exo process correctly binds to UDP *:5353 (3 sockets confirmed via lsof)
  • libp2p swarm: TCP LISTEN port is open (random high port)
  • NETWORK_VERSION: Identical b"v0.0.1" on all nodes, no EXO_LIBP2P_NAMESPACE override
  • Preshared key (pnet): Same source code = same SHA3 hash
  • mDNS query interval: We patched MDNS_QUERY_INTERVAL from 1500s down to 5s in discovery.rs, force-rebuilt exo_pyo3_bindings, still no peers discovered after several minutes
  • Connection updates: Zero ConnectionUpdate events — the Rust networking layer never reports any peer
  • No errors: No error or warning in logs related to networking

Expected behavior

Nodes should discover each other via mDNS and form a cluster, as they do when running from the pre-built EXO.app.

Workaround

Use the pre-built EXO.app from GitHub releases instead of running from git source.

Hypothesis

The issue likely relates to how exo_pyo3_bindings is compiled via maturin during uv sync (dev/editable mode) vs how it is compiled and bundled in the PyInstaller-based EXO.app release. Possible causes:

  • Different Rust feature flags or build profile (debug vs release)
  • The maturin editable install may not correctly link or initialize the libp2p networking stack
  • The tokio runtime spawned inside the Python process (via pyo3) may handle mDNS multicast sockets differently in dev mode

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions