Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
fa10f69
Initial implementation
wcandillon May 19, 2026
cf63d0a
:wrench:
wcandillon May 19, 2026
87fb8ac
:wrench:
wcandillon May 19, 2026
e9c0406
:wrenhc:
wcandillon May 19, 2026
69dcd30
:wrench:
wcandillon May 19, 2026
be9b29f
:wrench:
wcandillon May 19, 2026
afd5bdb
Merge branch 'main' into external-texture
wcandillon May 20, 2026
41b6b96
Merge branch 'main' into external-texture
wcandillon May 20, 2026
7c26343
:wrench:
wcandillon May 20, 2026
6f9b11a
:wrench:
wcandillon May 20, 2026
bb74056
:wrench:
wcandillon May 20, 2026
026c2f2
:wrench:
wcandillon May 20, 2026
054f90a
:wrench:
wcandillon May 20, 2026
258710e
Merge branch 'main' into external-texture
wcandillon May 21, 2026
b7cf5ce
:wrench:
wcandillon May 21, 2026
f4db134
:wrench:
wcandillon May 22, 2026
9a283c6
:wrench:
wcandillon May 22, 2026
c7a116f
:wrench:
wcandillon May 22, 2026
34c0ff3
:wrench:
wcandillon May 22, 2026
6e36091
:wrench:
wcandillon May 22, 2026
5de331e
:wrech:
wcandillon May 22, 2026
f9073a2
:wrench:
wcandillon May 22, 2026
2de307b
:wrench:
wcandillon May 22, 2026
3f57926
CameraHelmet: build the env cubemap with THREE.CubeCamera
claude May 22, 2026
3eba2c8
:wrench:
wcandillon May 22, 2026
bd849b6
CameraHelmet: reflect the front cam as a planar screen, not a panorama
claude May 27, 2026
55c8faf
:wrench:
wcandillon May 27, 2026
c0b6f78
:wrench:
wcandillon May 27, 2026
a1b60e1
:wrench:
wcandillon May 27, 2026
1be4efa
Merge branch 'claude/festive-planck-2EOky' into external-texture
wcandillon May 27, 2026
8107d60
Merge branch 'main' into external-texture
wcandillon May 31, 2026
7c66119
:wrench:
wcandillon May 31, 2026
b0ac50c
:wrench:
wcandillon Jun 1, 2026
55a137d
Merge branch 'external-texture' of https://github.com/wcandillon/reac…
wcandillon Jun 1, 2026
f02bfe2
Merge origin/main into external-texture
Copilot Jun 1, 2026
2075e97
:wrench:
wcandillon Jun 1, 2026
f89f708
:wrench:
wcandillon Jun 1, 2026
85ecdc2
:wrench:
wcandillon Jun 1, 2026
8adcf2d
:wrench:
wcandillon Jun 1, 2026
be66dda
:wrench:
wcandillon Jun 1, 2026
9be78c7
:wrench:
wcandillon Jun 1, 2026
77f0e9c
:wrench:
wcandillon Jun 1, 2026
322f925
:wrench:
wcandillon Jun 1, 2026
dd5d61f
:wrench:
wcandillon Jun 1, 2026
6f5a00e
:wrench:
wcandillon Jun 1, 2026
5b22987
:wrench:
wcandillon Jun 1, 2026
0600883
:wrench:
wcandillon Jun 1, 2026
8b0e6f1
Merge branch 'main' into external-texture-min
wcandillon Jun 1, 2026
542caba
:wrench:
wcandillon Jun 1, 2026
d67b3b2
:wrench:
wcandillon Jun 2, 2026
27b99d8
:wrench:
wcandillon Jun 2, 2026
2eeea58
:wrench:
wcandillon Jun 2, 2026
942b4d5
chore(🧵): worklet-reachable native-buffer factory via RNWebGPU captur…
wcandillon Jun 2, 2026
1692605
:wrench:
wcandillon Jun 2, 2026
c3d35d3
Merge branch 'external-texture-min' of https://github.com/wcandillon/…
wcandillon Jun 2, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 57 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,19 +226,21 @@ device.queue.copyExternalImageToTexture(

React Native WebGPU exposes Dawn's `SharedTextureMemory` so you can import a native pixel surface (an `IOSurface`-backed `CVPixelBuffer` on iOS, an `AHardwareBuffer` on Android) as a sampleable `GPUTexture` without copying pixels through the CPU. This is the path you want for camera frames, video frames, or anything coming out of a hardware producer.

We expose a single umbrella feature name, `"rnwebgpu/shared-texture-memory"`. Request it at device creation.
Like `importExternalTexture` on the web, this is **enabled by default**, there is nothing to request at device creation. The only thing to check is that the device supports it before importing. It always does on iOS/macOS; it can be missing on some Android drivers and emulators.

```tsx
import type { VideoFrame } from "react-native-wgpu";

const FEATURE = "rnwebgpu/shared-texture-memory" as GPUFeatureName;
import type { NativeVideoFrame } from "react-native-wgpu";

const adapter = await navigator.gpu.requestAdapter();
const requiredFeatures = adapter!.features.has(FEATURE) ? [FEATURE] : [];
const device = await adapter!.requestDevice({ requiredFeatures });
const device = await adapter!.requestDevice();

// On by default when supported; this is the only check you need.
if (!device.features.has("rnwebgpu/native-texture" as GPUFeatureName)) {
return; // rare: some Android drivers/emulators can't import native surfaces
}

// `frame` here is a VideoFrame whose .handle is the native surface
// (IOSurfaceRef / AHardwareBuffer*). VideoFrames are produced by helpers
// `frame` here is a NativeVideoFrame whose .handle is the native surface
// (IOSurfaceRef / AHardwareBuffer*). NativeVideoFrames are produced by helpers
// like RNWebGPU.createVideoPlayer or RNWebGPU.createTestVideoFrame, or by
// any third-party module that hands you a compatible native pointer.
const memory = device.importSharedTextureMemory({
Expand All @@ -257,6 +259,53 @@ frame.release();

`beginAccess`/`endAccess` bracket the GPU's read window on the shared surface. Pass `initialized: true` when the producer has already written meaningful pixels (the typical video/camera case) and `false` when the next pass will fully overwrite the texture.

### Importing External Textures

`GPUDevice.importExternalTexture` is the higher-level path for sampling a native surface. You hand it a `NativeVideoFrame` and get back a `GPUExternalTexture` that you bind as a `texture_external` and read with `textureSampleBaseClampToEdge`. It does two things for you on top of `SharedTextureMemory`:

- **Color conversion.** Camera and video surfaces are usually biplanar YUV (NV12), not RGB. An external texture carries the YUV→RGB matrix and the source/destination color-space transfer functions, so on the supported paths the sampler returns ready-to-use RGB in hardware. With raw `SharedTextureMemory` you would sample the luma/chroma planes and do that conversion by hand in WGSL. This is the main reason to prefer it for camera and video frames.
- **Lifecycle.** It owns the `SharedTextureMemory` + `createTexture` + `beginAccess`/`endAccess` sequence internally, so you just import the frame and `destroy()` the result.

It builds on the same default-on capability as Shared Texture Memory above, so feature-detect the device the same way before importing.

> **Android note:** the hardware YUV→RGB conversion is fully automatic on iOS (NV12 `IOSurface`). On Android, camera frames arrive as an _opaque_ YCbCr `AHardwareBuffer`, and Dawn's Vulkan path forces an identity (`RGB_IDENTITY`) sampler conversion, so the external sample comes back as raw `[Y, Cb, Cr]`. You still get the zero-copy import and the rotation/mirror transform, but you need to apply the YUV→RGB matrix yourself in the shader. See the `CAMERA_PRELUDE` in the [VisionCamera example](/apps/example/src/VisionCamera/shaders.ts) for a ready-made BT.709 decode.

```tsx
const adapter = await navigator.gpu.requestAdapter();
const device = await adapter!.requestDevice();
// Feature-detect as shown above before importing on unsupported hardware.

const render = () => {
// A GPUExternalTexture expires once the queue work that used it is submitted,
// so re-import one every frame.
const externalTexture = device.importExternalTexture({
source: frame, // a NativeVideoFrame
label: "video-frame",
});

const bindGroup = device.createBindGroup({
layout: pipeline.getBindGroupLayout(0),
entries: [
{ binding: 0, resource: externalTexture },
{ binding: 1, resource: sampler },
],
});

// ... encode a pass that samples `externalTexture`, then:
device.queue.submit([encoder.finish()]);

// Release the surface's access window right after the submit that sampled it.
externalTexture.destroy();
context.present();
};
```

Camera frames arrive in the sensor's native orientation, so `importExternalTexture` also accepts non-spec `rotation` (`0` | `90` | `180` | `270`, in degrees) and `mirrored` (horizontal flip) options. Dawn bakes them into the sampling transform, so the shader sees an upright frame. They map directly onto VisionCamera's `frame.orientation` / `frame.isMirrored`.

#### Calling `destroy()`

A `GPUExternalTexture` keeps an open access window on the underlying native surface until the wrapper is destroyed. On the Web `importExternalTexture` is core and the lifetime is handled for you; here the window is tied to the JavaScript object's lifetime. Call `externalTexture.destroy()` right after the `queue.submit()` that sampled it (never before) to release the surface back to its producer immediately. `destroy()` is idempotent, and the surface is also released when the object is garbage-collected, but relying on GC can starve a producer's buffer pool (e.g. an `AVPlayer`'s recycled `IOSurface`s) and pile up GPU resources, so prefer the explicit call in a render loop.

### Reanimated Integration

React Native WebGPU supports running WebGPU rendering on the UI thread using [React Native Reanimated](https://docs.swmansion.com/react-native-reanimated/) and [React Native Worklets](https://github.com/margelo/react-native-worklets).
Expand Down
9 changes: 7 additions & 2 deletions apps/example/src/App.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@ import { AsyncStarvation } from "./Diagnostics/AsyncStarvation";
import { DeviceLostHang } from "./Diagnostics/DeviceLostHang";
import { StorageBufferVertices } from "./StorageBufferVertices";
import { SharedTextureMemory } from "./SharedTextureMemory";
import { Camera } from "./Camera";
import { ImportExternalTexture } from "./ImportExternalTexture";
import { VisionCamera } from "./VisionCamera";

// The two lines below are needed by three.js
import "fast-text-encoding";
Expand Down Expand Up @@ -103,7 +104,11 @@ function App() {
name="SharedTextureMemory"
component={SharedTextureMemory}
/>
<Stack.Screen name="Camera" component={Camera} />
<Stack.Screen
name="ImportExternalTexture"
component={ImportExternalTexture}
/>
<Stack.Screen name="VisionCamera" component={VisionCamera} />
</Stack.Navigator>
</NavigationContainer>
</GestureHandlerRootView>
Expand Down
60 changes: 0 additions & 60 deletions apps/example/src/Camera/Camera.tsx

This file was deleted.

1 change: 0 additions & 1 deletion apps/example/src/Camera/index.ts

This file was deleted.

8 changes: 6 additions & 2 deletions apps/example/src/Home.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -132,8 +132,12 @@ export const examples = [
title: "🎞️ Shared Texture Memory",
},
{
screen: "Camera",
title: "📷 Camera",
screen: "ImportExternalTexture",
title: "🎬 Import External Texture",
},
{
screen: "VisionCamera",
title: "📷 VisionCamera integration",
},
];

Expand Down
Loading
Loading