Expose Hyperliquid TWAP WS feeds as custom data

[graceyangfan]

Problem / goal

Hyperliquid WebSocket already streams userTwapHistory and userTwapSliceFills, but strategies cannot consume them through Nautilus custom data. Issue #4673 asks to expose these as opt-in adapter custom data (research / TWAP lifecycle), not as core market-data types and not via the default execution user bootstrap.

Closes #4673

Scope

In this PR

• Domain types: HyperliquidTwapHistory, HyperliquidTwapSliceFill (#[custom_data], registered for Arrow + JSON)
• Wire alignment from live mainnet payloads: optional twapId on history rows, optional description, Decimal sizes on TWAP state, seconds-vs-ms venue time handling
• Parse → handler emit (NautilusWsMessage::CustomData) → WS client subscribe/unsubscribe → data client branches
• Identity: DataType type name + metadata["user"] + identifier = user address
• Python surface: package exports, PyO3 methods, generated stubs
• Integration docs: capability rows, field tables, subscribe examples
• Tests: data-client routing + missing user, WS emit fixtures, live JSON parse fixtures, Arrow + Parquet catalog round-trips

Out of scope (explicit)

• REST twapPlace / TWAP order placement
• twapStates channel
• Folding TWAP into subscribe_all_user_channels (execution default)
• Core MD types / folding into generic fills or order reports
• Dual-actor engine refcount test (engine DataClientAdapter already refcounts custom DataType topics; these channels are not a competing shared-stream profile like depth-for-latency)

Design notes (reviewer map)

Sibling path: Hyperliquid PublicTrade / user-keyed custom data (same transport, handler emit, data-client branch style). User-keyed (not instrument-keyed): strategies pass metadata={"user": "0x..."}; the address need not be the adapter trading account.

flowchart LR
  S[Strategy subscribe_data] --> DC[HyperliquidDataClient]
  DC --> WS[WS subscribe_user_twap_*]
  WS --> V[Venue userTwapHistory / userTwapSliceFills]
  V --> H[FeedHandler parse + CustomData]
  H --> MB[msgbus / on_data]
  H --> CAT[Optional Parquet catalog via Arrow]

Loading

Non-obvious choices:

Topic	Choice	Why
Opt-in only	Not in subscribe_all_user_channels	Issue requirement; avoids auto-opening research feeds for every exec account
Status enum	Reuse HyperliquidTwapStatus (activated/terminated/finished/error + Unknown)	Matches venue docs / live; unknown strings stay lenient
History twap_id	Option<u64>	Live mainnet history rows include twapId; keep optional for older shapes
Unknown coin	instrument_id = None, do not drop row	HIP-3 / uncached coins still useful with raw coin
Precision	Decimal for sizes/prices/fees	Avoid f64 on money-adjacent fields; catalog/Arrow use serde Utf8 for Option/enum/Decimal
Snapshot	is_snapshot from venue envelope	Consumers can clear/rebuild local TWAP state on snapshot batches

Testing

Local (paths that fail without this change):

export PATH="$HOME/.local/bin:/opt/homebrew/opt/node@24/bin:/opt/homebrew/bin:$PATH"

cargo test -p nautilus-hyperliquid --features "python,arrow,high-precision" --lib data_types::tests
# 8 passed (includes TWAP Arrow schema + encode/decode)

cargo test -p nautilus-hyperliquid --features "arrow,high-precision" --test catalog
# 4 passed (PublicTrade + TWAP history/slice catalog round-trips)

cargo test -p nautilus-hyperliquid --features "python,arrow,high-precision" --test data_client -- twap
# 3 passed (subscribe/unsubscribe + missing metadata['user'])

cargo test -p nautilus-hyperliquid --features "python,arrow,high-precision" --test websocket -- twap
# 2 passed (handler emit of concrete custom data)

cargo test -p nautilus-hyperliquid --features "python,arrow,high-precision" --lib parse::tests::test_parse_ws_twap
# 2 passed (live mainnet fixtures under test_data/)

make py-stubs
bash scripts/ci/check-generated-drift.bash
# No generated file drift detected

prek run --files \
  crates/adapters/hyperliquid/src/common/enums.rs \
  crates/adapters/hyperliquid/src/data.rs \
  crates/adapters/hyperliquid/src/data_types.rs \
  crates/adapters/hyperliquid/src/python/mod.rs \
  crates/adapters/hyperliquid/src/python/websocket.rs \
  crates/adapters/hyperliquid/src/websocket/client.rs \
  crates/adapters/hyperliquid/src/websocket/handler.rs \
  crates/adapters/hyperliquid/src/websocket/messages.rs \
  crates/adapters/hyperliquid/src/websocket/parse.rs \
  crates/adapters/hyperliquid/tests/catalog.rs \
  crates/adapters/hyperliquid/tests/data_client.rs \
  crates/adapters/hyperliquid/tests/websocket.rs \
  crates/adapters/hyperliquid/test_data/ws_user_twap_history.json \
  crates/adapters/hyperliquid/test_data/ws_user_twap_slice_fills.json \
  docs/integrations/hyperliquid.md \
  python/nautilus_trader/adapters/hyperliquid/__init__.py \
  python/nautilus_trader/adapters/hyperliquid/__init__.pyi
# all hooks green (fmt/clippy/docs/ruff/…)

Note (macOS only): --test catalog with the python feature can trip linker_messages (__eh_frame too large). Catalog tests are validated with arrow,high-precision (no python). Linux CI is unaffected.

Follow-ups

• REST TWAP place / management APIs (separate issue/PR)
• twapStates if product needs live state snapshots beyond history
• Optional Python to_arrow_record_batch_bytes helper (only HyperliquidPublicTrade has this today among HL custom types)

Reviewer checklist (self)

• Issue-scoped; v2 only
• Sibling-first (PublicTrade / user channels)
• Subscribe and unsubscribe
• Not default-exec bootstrap
• Python export + generated stubs
• Docs table + example with HYPERLIQUID_CLIENT_ID
• At least one test fails without the feature
• Local prek + focused cargo tests green before push
• Draft for early design/API feedback; mark ready after CI

[cjdsellers]

Hi @graceyangfan,

Thank you for the PR. I found one issue in the custom-data routing: custom_user trims metadata["user"] before opening the venue subscription, while the actor's DataType topic retains the original metadata. The handler then builds the emitted DataType from the venue's echoed user. With surrounding whitespace, the venue subscription succeeds but events are published on a different topic, so on_data never receives them.

Could you either reject metadata that differs from its canonical form before subscribing, or retain the original subscription identity when emitting, and add a test covering the subscribe-to-emit topic identity?