interstream_datamosh

A real-codec datamosh component for TouchDesigner — video in, moshed video out, with every parameter exposed for live performance (MIDI/OSC/CHOP-mappable).

Unlike shader "fake" datamosh, this does a genuine keyframe-free H.264 P-frame encode/decode per frame (via PyAV) to produce authentic macroblock bloom and pixel-smear — plus a GPU motion-vector engine for a lighter, full-res alternative.

Inspired by Interstream, the real-time datamosh app by Signal Culture.

---

What it does

• Two mosh engines, switchable live (Engine ▸ Mosh Engine):
  • Codec — per-frame keyframe-free H.264 encode→decode. The authentic look; heavier (software encode on the main thread).
  • GPU-MV — harvests real macroblock motion vectors once per frame, then re-applies them on the GPU. ~2× cheaper, keeps real motion; a slightly different, organic character.
• Two-input pre-mosh Blend — composite/crossfade/lumakey two sources before moshing.
• Region Mask, video Feedback, LFO/drift Modulation, audio-reactivity, an auto-mosh "Fluid" engine, snapshot+morph Presets, and a Master stage with output-resolution control and Syphon/Spout output.

Requirements

• TouchDesigner 2025 or newer.
• PyAV (av) 18.x with libx264 — installed automatically on first load. PyAV is a compiled library and can't be embedded in a .tox, so the component bootstraps it: if av isn't importable, it uses TouchDesigner's own Python (guaranteed version/arch match) to pip install av==18.0.0 into a small per-user folder (one-time ~40 MB download), then wires it in. No terminal, no parameters, no restart. Watch Setup ▸ PyAV Status walk through installing → OK.
  • Requires internet on first load. Turn the behavior off with Setup ▸ Auto-Install PyAV if you manage Python yourself.
  • Manual alternatives (checked before auto-install, always win): point Setup ▸ PyAV Site at any Python 3.11 site-packages containing av (the bundled ./scripts/install_pyav.sh builds one and prints the path), set a PYAV_SITE env var, or pip install -r requirements.txt into TouchDesigner's own Python.
  • The component fails loud, not silent: if PyAV still can't be imported, the mosh passes the input through unmoshed and the Setup ▸ PyAV Status line + a red error on the mosh node tell you why (including the pip log path if the auto-install failed). The GPU-MV engine needs PyAV too (it harvests real codec motion vectors).
• macOS for Syphon out / Windows for Spout out (optional; the component works fine without it).

Install / Quick start

1. Drag interstream_datamosh.tox into your project (PyAV auto-installs on first load — see Requirements).
2. Wire a video TOP → in1. Output comes from out1.
3. On the Engine page, toggle Mosh on. That's it.

The component ships in a clean pass-through state — nothing moshes, warps, feeds back, or modulates until you switch it on, and every parameter's default is that clean state, so "Revert to Default" anywhere gets you back to safety. The resting knob values are the Classic preset, so the first switch you flip already sounds like the instrument.

Inputs: in1, in2 (video TOPs — in2 is the second Blend layer), mask_in (optional mask TOP for the Region Mask). Output: out1.

Parameters (7 pages)

Pages are grouped with header sections so everything fits the dialog at normal width.

Page	What it controls
Engine	Moshengine — Codec vs GPU-MV (hard switch; the unselected engine doesn't cook), then one section per engine. Codec Engine: Mosh, Bloom, Livebleed, Reset, Pixelpersist, Encoderes, Bitrate, Codec, Framerate, Bypass. GPU-MV Engine: Mvamt (displacement gain), Mvkeep (smear persistence), Mvbright, Mvmotiongain, Mvblocky/Mvblocksize (macroblock look), Mvquantfb (displacement stepping), Mvflipdir, Mvswirl, Mvchromasplit, Mvtrails, tint (Mvtintr/g/b + Mvtintamt), and the MV feedback set (Mvfbzoom/Mvfbrotate/Mvfbdecay/Mvfbhue).
Layer	Blend: Blendmode (12 modes incl. crossdissolve/lumakey), Blendmix crossfade (dry side = the unfaded top layer), Layertop, Topopacity (fades the top layer out of the blend AND out of the key; symmetric modes converge to the bottom layer at 0 — subtractive/multiplicative modes converge to black), real luma key (Lumakeythresh/Lumakeysoft — keys on top-layer luminance). Warp: per-input Warpinput, Zoom/Rotate/Pan (+ speeds), Normalize, per-input reset.
FX	Feedback: Feedback, Feedbackamount, Feedbackmix (Pre-Warp / Pre-Mosh / Masked — the moshed element feeds only itself, outside the mask stays dry / Post-Mask — the final composite loops back in), Feedbackreset (clears both loops); Masked falls back to Pre-Mosh when no usable matte; both loops are bounded (fixed 1280×720, convex cross ≤ 0.98, held in reset when unused). Mask: Maskenable, Maskinvert, Maskfeather, Masksource. Fluid Mosh: Fluidmosh, Fluidrate (5–1000 ms), Fluidspread, Fluidintensity, Fluidautoreset, Fluidtargets (cumulative surge ladder — Mosh Only ⊂ +Bloom ⊂ +Feedback ⊂ +GPU-MV Amt; engine-aware), Fluidramp, Fluidjitter, Fluidcodechop (random codec per burst, Codec engine). While Fluid runs it owns its surge targets — don't also route mod-matrix slots at them.
Modulation	Patch bay: 6 slots (Slot{1-6}src/tgt/depth/on → rows 1–6 of the mod matrix; rows 7+ stay free-form), 4 LFOs (shape/rate/phase), Driftrate, Extsmooth, 10 read-only source meters (Mtr*). Default routing (ships off): onset→Mosh 1.0, level→Bloom 3.0, low→Feedbackamount 0.3, lfo1→Mvswirl 0.5. Depth slider is ±1; type up to ±8 for wide-range targets. Turning a slot off releases its target back to base; modulated writes are always bounded to the target's designed range. ⚠ Audio sources need Audio ▸ Enable ON (meters read 0 and audio-routed slots sit still without it). ⚠ While a slot is ON the matrix is the single writer for its target — manual edits to that parameter are overridden each frame until the slot or Modenable goes off. ⚠ While Fluidmosh runs it owns Mosh/Bloom — don't also route slots at them.
Audio	Self-contained audio reactivity: Audiosource (device/file/chop), device picker (Audiodevice + Audiorefresh; empty = system default), file transport (Audiofile — empty = built-in demo, Audioplay, Audiofilevol, loops), Audiochop external input, Audioenable, Audiogain, Audiosmooth, Lowsplit/Midsplit crossovers in Hz, low/mid/high trims. Machine-specific pars (Audiodevice/Audiofile/Audiochop) are never captured into presets.
Presets	Presetenable, Presetname + Snapshot (capture), Presetdelete (removes the preset selected in Preset A), Presetdice (randomize safe look-params), Preseta/Presetb, Morph, Morphmode — snapshot & interpolate looks. 9 baked presets: Clean, Classic, Heavy, VHS Ghost, Block Storm, Liquid Metal, Rainbow Tunnel, Subtle Drift, Full Wreck.
Setup	Output: Masterres (output resolution, independent of encode res), Syphonout, Syphonname. About: version, mechanism, Pyavsite (manual override), Pyavautoinstall (zero-config bootstrap, on by default), Pyavstatus.

Every custom parameter is host-mappable — bind a MIDI/OSC CHOP to drive it live.

Performance notes

• The Codec engine encodes on the main thread, so cost scales with Encode Res (Engine ▸ Encoderes) and how many instances run at once. For multiple simultaneous instances, drop Encode Res (e.g. 360p/480p) and use Master Res to output higher.
• Feather Mosh on the Codec engine runs a second software encoder — at 720p that lands around 36 fps. Two clean outs: drop Encode Res to 480p, or switch to the GPU-MV engine (both hold ~60).
• The GPU-MV engine is cheaper per instance and keeps real motion, but the per-frame encode it still needs means 3+ full-res instances at 720p won't hold 60fps either — it's an improvement, not a silver bullet.
• Idle cost of the unselected engine is ~0 (it isn't pulled).

Credits & License

Created by Arson Rivvers.

Inspired by Interstream, the real-time datamosh app from Signal Culture — its real-codec approach (keyframe-free moshing, timed blooming, perpetual mosh) is what this component reimagines as a native TouchDesigner instrument. Signal Culture is a small nonprofit building tools, residencies, and resources for experimental media art — check out their Modular Apps and support their work.

Licensed under CC BY-NC 4.0 — free to use, remix, and share for non-commercial work with attribution. For commercial licensing, get in touch.

Built in TouchDesigner. Datamosh core in Python via PyAV / libx264.