feat: Add depth to Frame

[mrousavy]

What

TL;DR: Adds Frame.depth, a nullable CMSampleBuffer that holds depth data.

If enableDepthData is true (a prop passed to <Camera>), the Camera will be configured to set up a depth data output pipeline in addition to the video data output pipeline.
Ontop of that, a synchronizer output is created to sync depth data and video data buffers together for timestamps.

const device = Camera.getAvailableCameraDevices().find((d) => d.formats.some((f) => f.supportsDepthCapture))
const format = useCameraFormat(device, [{ depth: true }])

const fp = useFrameProcessor((frame) => {
  'worklet'
  console.log(frame.depth)
})

if (device == null) throw new Error(`Device does not support depth data!`)
return (
  <Camera
    device={device}
    format={format}
    enableDepthData={true}
    video={true}
    frameProcessor={fp}
  />
)

Changes

Tested on

Related issues

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
react-native-vision-camera	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Dec 13, 2024 1:27pm

[zigapk]

So glad to see this PR, just what I need. Would love to see this merged