release(v1.10.5): fix WSS supervisor wedge, debounce key-exchange storm, skip direct dial in compat mode

Three daemon-side bugs surfaced while diagnosing why service-agents
appeared unresponsive from a v1.10.4 compat-mode daemon: the WSS
supervisor exited permanently on the first failed redial, peers fired
4-5 "encrypted tunnel established" events per handshake because direct
and relay copies of the same PILA were not coalesced, and every
compat-mode dial wasted 25-78s on direct retries that physically
cannot succeed.

Fixes:
- pkg/daemon/transport/wss: supervisor exits only on Close. A nil
  conn at loop top just skips drainReads and proceeds to backoff +
  redial. Regression test pins the failure mode.
- pkg/daemon/keyexchange: same-pubkey PILA arriving within
  DuplicateHandshakeDebounce (250ms) of a freshly-installed Crypto
  skips the log line, bus event, and PostInstallHook. The
  asymmetric-recovery reply-on-stale path is intentionally outside
  the gate so the rc3 list-agents recovery scenario still works.
- pkg/daemon: compat-mode dial pre-flips the peer to relay before
  the first SYN. directRetries=0 branch then skips the dead direct
  budget.

Verified against beacon.pilotprotocol.network with a fresh v1.10.5
daemon: TCP/443-only, list-agents query round-trips in ~3s end-to-end.

Author: teovl

CHANGELOG.md

@@ -7,6 +7,87 @@ project uses [Semantic Versioning](https://semver.org/).
 Detailed per-release notes are on the
 [GitHub Releases page](https://github.com/TeoSlayer/pilotprotocol/releases).
 
+## [1.10.5] - 2026-05-20
+
+### Fixed
+- **WSS reconnect supervisor no longer wedges on a failed first redial.**
+  v1.10.4 introduced the reconnect supervisor but contained an
+  early-exit bug: when the supervisor cleared `t.conn` after a read
+  failure and then the first redial attempt itself failed (network
+  hiccup, beacon 5xx, nginx restart still in progress), the next
+  loop iteration saw `conn == nil` at the top and `return`ed,
+  killing the supervisor goroutine for the lifetime of the daemon.
+  Operators saw a long stretch of `wss: reconnecting` Warn lines
+  followed by silence; every subsequent `Send` then returned
+  `ErrReconnecting` forever and only a daemon restart recovered.
+  Observed in production today on a v1.10.4 daemon that ran ~10 h
+  before the supervisor exited.
+
+  The supervisor now treats `t.closed.Load()` as the only exit
+  condition. A nil `conn` at iteration start just skips
+  `drainReads` and goes straight to backoff + redial — so a
+  transient outage drains the backoff budget instead of killing
+  the goroutine. Regression-pinned by
+  `TestReconnect_SurvivesFailedRedialAttempts` (3 forced 503
+  redial failures followed by recovery).
+
+- **Duplicate key_exchange frames coalesced; no more "encrypted
+  tunnel established" log storm.** Direct + relay copies of the
+  same PILA frame plus peer-side retransmits caused the daemon to
+  fire the full side-effect path (log, `tunnel.established` bus
+  event, PostInstallHook with salvage replay + flushPending) 4–5×
+  per peer within milliseconds. A new
+  `DuplicateHandshakeDebounce = 250 ms` window coalesces
+  same-X25519-pubkey frames arriving on top of a freshly-installed
+  Crypto.
+
+  Crucially, the SendKeyExchangeToNode-when-stale path is NOT
+  gated by the debounce: the asymmetric-recovery scenario (peer
+  dropped crypto for us, retransmits PILA, our InboundDecryptStale
+  is true) still elicits our PILA reply so the peer can re-derive
+  the shared secret. Regression-pinned by
+  `TestDuplicatePILACoalescedSuppressesLogAndHook`,
+  `TestDuplicatePILAOutsideDebounceFiresHookAgain`, and
+  `TestDuplicatePILAStillRepliesForAsymmetricRecovery`. The
+  existing `TestAsymmetricRecoveryRepliesOnDuplicatePILAWhenStale`
+  also still passes.
+
+- **Compat-mode dial no longer burns 25–78 s on direct-retry
+  attempts that can't succeed.** A compat-mode daemon has no
+  public UDP socket — its data plane lives entirely on the WSS
+  bridge to the beacon. Pre-v1.10.5, every outbound dial still
+  ran the full 3-attempt direct phase before falling back to
+  relay; each attempt consumed an ephemeral port and idled for
+  the SYN-RTO budget. Under fan-out (e.g. the trusted-agents
+  auto-handshake on first dial) this exhausted local ephemeral
+  ports on macOS.
+
+  `DialConnectionContext` now pre-flips the peer's routing state
+  to relay BEFORE sending the initial SYN when `-transport=compat`,
+  and the existing `relayActive → directRetries=0` branch picks
+  it up — first SYN goes out via the WSS bridge on the first try.
+
+### Tests
+- `pkg/daemon/transport/wss/zz_wss_test.go::TestReconnect_SurvivesFailedRedialAttempts`
+  — `fakeBeacon` accepts an initial dial, kills the conn on
+  the 2nd binary frame, then refuses the next 3 redial attempts
+  with 503. Pre-fix: supervisor exits, test wedges out.
+  Post-fix: supervisor exhausts the 503 streak with exponential
+  backoff and the 4th attempt succeeds; a probe frame round-trips
+  on the fresh conn.
+- `pkg/daemon/keyexchange/zz_duplicate_debounce_test.go` — three
+  scenarios covering coalescing inside the window, re-firing
+  after the window, and asymmetric-recovery preservation.
+
+### Verified
+- Live smoke test against `beacon.pilotprotocol.network` from a
+  fresh v1.10.5 daemon: `-transport compat` binds only TCP/443,
+  WSS auth completes, trust handshake with `list-agents` lands
+  within ~88 s of first dial, `send-message list-agents` returns
+  2521 bytes of structured JSON in ~3 s end-to-end. No
+  `wss: reconnecting` storm, no "tunnel established" log noise
+  burst.
+
 ## [1.10.4] - 2026-05-19
 
 ### Fixed

pkg/daemon/daemon.go

@@ -2924,6 +2924,17 @@ func (d *Daemon) dialConnectionLocked(ctx context.Context, dstAddr protocol.Addr
 		return nil, err
 	}
 
+	// Compat mode: the daemon has no public UDP socket, so any direct
+	// SYN we send out can't reach the peer. Pre-flip the routing
+	// state to relay BEFORE the first SYN so it goes out via the
+	// beacon WSS bridge on the first try, not after a 25-78s direct
+	// retry budget burns. Same reasoning as the directRetries=0
+	// branch in the retransmit loop below — we just need it earlier
+	// so the initial SYN is correctly routed.
+	if d.config.TransportMode == "compat" && !d.tunnels.IsRelayPeer(dstAddr.Node) {
+		d.tunnels.SetRelayPeer(dstAddr.Node, true)
+	}
+
 	// Auto-initiate handshake toward known trusted agents when we have no
 	// local trust entry yet. Scoped to the trusted-agents list so we don't
 	// spray handshakes at arbitrary peers. Fires non-blocking so the
@@ -2993,9 +3004,12 @@ func (d *Daemon) dialConnectionLocked(ctx context.Context, dstAddr protocol.Addr
 	retries := 0
 	directRetries := DialDirectRetries
 	maxRetries := DialMaxRetries
-	relayActive := d.tunnels.IsRelayPeer(dstAddr.Node) // may already be relay from prior attempt
+	// relayActive is set when this peer is pinned to relay via either a
+	// prior attempt's fallback OR the compat-mode pre-flip above. Either
+	// way the direct retry budget would burn for nothing — skip it.
+	relayActive := d.tunnels.IsRelayPeer(dstAddr.Node)
 	if relayActive {
-		directRetries = 0 // skip direct phase, go straight to relay
+		directRetries = 0
 	}
 	rto := DialInitialRTO
 	timer := time.NewTimer(rto)

pkg/daemon/keyexchange/handle.go

@@ -7,6 +7,7 @@ import (
 	"encoding/binary"
 	"log/slog"
 	"net"
+	"time"
 
 	"github.com/TeoSlayer/pilotprotocol/internal/crypto"
 )
@@ -89,6 +90,12 @@ func (m *Manager) HandleAuthFrame(data []byte, from *net.UDPAddr, fromRelay bool
 	oldPC := m.env.Get(peerNodeID)
 	hadCrypto := oldPC != nil
 	keyChanged := hadCrypto && oldPC.PeerX25519Key != pc.PeerX25519Key
+	// duplicate := same-pubkey frame arriving inside the debounce
+	// window of a freshly-installed Crypto. See
+	// DuplicateHandshakeDebounce — the goal is to coalesce the direct+
+	// relay arrival pair and peer-side retransmits without dropping
+	// real recovery work.
+	duplicate := hadCrypto && !keyChanged && time.Since(oldPC.CreatedAt) < DuplicateHandshakeDebounce
 	// Only replace the installed envelope.Crypto when there is no
 	// existing entry OR the peer's X25519 ephemeral key actually changed.
 	// Replacing on a duplicate key_exchange (same pubkey — common under
@@ -103,31 +110,45 @@ func (m *Manager) HandleAuthFrame(data []byte, from *net.UDPAddr, fromRelay bool
 	// Cache the verified peer pubkey.
 	m.SetPeerPubKey(peerNodeID, peerEd25519PubKey)
 
-	if keyChanged {
-		slog.Info("peer rekeyed (auth), re-establishing tunnel", "peer_node_id", peerNodeID)
+	// Side-effect gate: the log line, tunnel.established bus event, and
+	// PostInstallHook fire ONLY when this is not a coalesced duplicate.
+	// A real rekey (keyChanged) or a first-time install always falls
+	// through to the side effects. The recovery-reply path below (the
+	// SendKeyExchangeToNode call) is NOT gated on `duplicate` — the
+	// asymmetric-recovery case (B dropped crypto for A while A retains
+	// it) requires A to reply on B's retransmit even though A sees it
+	// as a duplicate. Pinned by
+	// TestAsymmetricRecoveryRepliesOnDuplicatePILAWhenStale.
+	if duplicate {
+		slog.Debug("auth key exchange: duplicate frame coalesced",
+			"peer_node_id", peerNodeID, "age", time.Since(oldPC.CreatedAt), "relay", fromRelay)
 	} else {
-		slog.Info("encrypted tunnel established", "auth", true,
-			"peer_node_id", peerNodeID, "endpoint", from, "relay", fromRelay)
-	}
-	m.publish("tunnel.established", map[string]any{
-		"peer_node_id":  peerNodeID,
-		"authenticated": true,
-		"relay":         fromRelay,
-		"rekeyed":       keyChanged,
-	})
-
-	if m.postInstall != nil {
-		m.postInstall(PostInstallEvent{
-			PeerNodeID:    peerNodeID,
-			From:          from,
-			FromRelay:     fromRelay,
-			Authenticated: true,
-			HadCrypto:     hadCrypto,
-			KeyChanged:    keyChanged,
-			OldCrypto:     oldPC,
-			NewCrypto:     pc,
-			PeerEd25519:   peerEd25519PubKey,
+		if keyChanged {
+			slog.Info("peer rekeyed (auth), re-establishing tunnel", "peer_node_id", peerNodeID)
+		} else {
+			slog.Info("encrypted tunnel established", "auth", true,
+				"peer_node_id", peerNodeID, "endpoint", from, "relay", fromRelay)
+		}
+		m.publish("tunnel.established", map[string]any{
+			"peer_node_id":  peerNodeID,
+			"authenticated": true,
+			"relay":         fromRelay,
+			"rekeyed":       keyChanged,
 		})
+
+		if m.postInstall != nil {
+			m.postInstall(PostInstallEvent{
+				PeerNodeID:    peerNodeID,
+				From:          from,
+				FromRelay:     fromRelay,
+				Authenticated: true,
+				HadCrypto:     hadCrypto,
+				KeyChanged:    keyChanged,
+				OldCrypto:     oldPC,
+				NewCrypto:     pc,
+				PeerEd25519:   peerEd25519PubKey,
+			})
+		}
 	}
 
 	if !hadCrypto || keyChanged {
@@ -218,36 +239,45 @@ func (m *Manager) HandleUnauthFrame(data []byte, from *net.UDPAddr, fromRelay bo
 	oldPC := m.env.Get(peerNodeID)
 	hadCrypto := oldPC != nil
 	keyChanged := hadCrypto && oldPC.PeerX25519Key != pc.PeerX25519Key
+	// duplicate := same-pubkey frame arriving inside DuplicateHandshakeDebounce
+	// of a freshly-installed Crypto. See HandleAuthFrame for the
+	// rationale; same coalescing window applies to PILK as well.
+	duplicate := hadCrypto && !keyChanged && time.Since(oldPC.CreatedAt) < DuplicateHandshakeDebounce
 	// Same rationale as HandleAuthFrame: don't replace on a duplicate
 	// key_exchange (same pubkey).
 	if !hadCrypto || keyChanged {
 		m.env.Install(peerNodeID, pc)
 	}
 
-	if keyChanged {
-		slog.Info("peer rekeyed, re-establishing tunnel", "peer_node_id", peerNodeID)
+	if duplicate {
+		slog.Debug("unauth key exchange: duplicate frame coalesced",
+			"peer_node_id", peerNodeID, "age", time.Since(oldPC.CreatedAt), "relay", fromRelay)
 	} else {
-		slog.Info("encrypted tunnel established",
-			"peer_node_id", peerNodeID, "endpoint", from, "relay", fromRelay)
-	}
-	m.publish("tunnel.established", map[string]any{
-		"peer_node_id":  peerNodeID,
-		"authenticated": false,
-		"relay":         fromRelay,
-		"rekeyed":       keyChanged,
-	})
-
-	if m.postInstall != nil {
-		m.postInstall(PostInstallEvent{
-			PeerNodeID:    peerNodeID,
-			From:          from,
-			FromRelay:     fromRelay,
-			Authenticated: false,
-			HadCrypto:     hadCrypto,
-			KeyChanged:    keyChanged,
-			OldCrypto:     oldPC,
-			NewCrypto:     pc,
+		if keyChanged {
+			slog.Info("peer rekeyed, re-establishing tunnel", "peer_node_id", peerNodeID)
+		} else {
+			slog.Info("encrypted tunnel established",
+				"peer_node_id", peerNodeID, "endpoint", from, "relay", fromRelay)
+		}
+		m.publish("tunnel.established", map[string]any{
+			"peer_node_id":  peerNodeID,
+			"authenticated": false,
+			"relay":         fromRelay,
+			"rekeyed":       keyChanged,
 		})
+
+		if m.postInstall != nil {
+			m.postInstall(PostInstallEvent{
+				PeerNodeID:    peerNodeID,
+				From:          from,
+				FromRelay:     fromRelay,
+				Authenticated: false,
+				HadCrypto:     hadCrypto,
+				KeyChanged:    keyChanged,
+				OldCrypto:     oldPC,
+				NewCrypto:     pc,
+			})
+		}
 	}
 
 	// Respond with our key if this is a new peer or the peer rekeyed.

pkg/daemon/keyexchange/keyexchange.go

@@ -93,6 +93,31 @@ const (
 	// Prevents the "bounce" where a transient outbound packet immediately
 	// restarts a cycle that just exhausted MaxRekeyAttempts.
 	rekeyGaveUpCooldown = 60 * time.Second
+
+	// DuplicateHandshakeDebounce is the window during which a repeated
+	// key_exchange frame from the same peer carrying the SAME X25519
+	// ephemeral key is treated as a duplicate of an already-handled
+	// handshake. Inside the window the frame is logged at Debug and the
+	// observable side effects are skipped: no "encrypted tunnel
+	// established" log line, no tunnel.established bus event, no
+	// PostInstallHook invocation, and no reply key_exchange.
+	//
+	// Why this matters: peers retransmit their PILA via the
+	// keyexchange.loop's RekeyRetransmitInterval (4 s) PLUS direct +
+	// relay copies of the same frame can both land within ms when the
+	// beacon relay is hot. Both legs of every duplicate used to fire
+	// the postInstall hook (peer-endpoint update, salvage replay,
+	// flushPending) and produce a noisy log burst that operators
+	// mistook for a real handshake storm. Real rekey (peer's ephemeral
+	// key actually changed) is NOT debounced — keyChanged forces the
+	// full path so salvage + flushPending run.
+	//
+	// 250 ms is short enough that an actual second handshake (caused
+	// by a real key rotation, not a duplicate frame) still progresses
+	// promptly, and long enough to coalesce the direct+relay arrival
+	// window plus most peer-side retransmits caused by an early ACK
+	// being dropped.
+	DuplicateHandshakeDebounce = 250 * time.Millisecond
 )
 
 // PendingRekeyState tracks a key-exchange we sent and are waiting on.