Repository avatar
Multimedia
v0.1.0
active

depthpipe

io.github.raydeoliveira/depthpipe

Live 2D-to-3D stereo depth pipeline, Metal DIBR weaver, and MV-HEVC Spatial Video tool.

Documentation

DepthPipe

Live 2D-to-3D stereo depth pipeline, runtime-compiled Metal DIBR synthesizer, bidirectional MV-HEVC Spatial Video converter, and SteamOS Vulkan Z-buffer implicit layer CLI and stdio MCP server (877,424 bytes / 857 KB stripped macOS arm64 binary, 7.00 ms min / 9.19 ms median --version cold start on Apple M4 Max, zero external Swift dependencies).

What It Does and Does Not Do

CapabilityStatusNotes / Requirements
Hardware & media probing (probe, inspect)SupportedReports Metal GPU, VideoToolbox MV-HEVC encode/decode, ScreenCaptureKit TCC, and CoreML status; parses ISO BMFF hvcC, lhvC, and vexu boxes
Game Depth Profile catalog (profile, profiles)Supported13 deterministic profiles (near_plane, far_plane, disparity_px, convergence_px, depth_invert, reversed_z) keyed by Steam AppID or bundle ID
Metal DIBR & stereo weaving (render)SupportedRuntime-compiled MSL compute kernels (rgbd_dibr_synthesize, split_packed_stereo, shift_stereo_convergence, weave_stereo_pair) across 8 PNG output layouts (full-sbs, half-sbs, tab, row-interlaced, col-interlaced, checkerboard, lenticular, dubois-anaglyph)
Bidirectional MV-HEVC video (convert)SupportedAudio-preserving VideoToolbox MV-HEVC .mov <-> Full-SBS/Half-SBS/TAB/interlaced/anaglyph .mov/.mp4
Live capture & sim fixtures (capture)SupportedScreenCaptureKit 32BGRA capture (gated by CGPreflightScreenCaptureAccess(), exit 4 on denial) plus --sim --fixture (gradient-shapes, checker-fg, portrait-card, desktop-sim, desktop-pillar)
SteamOS Vulkan Z-buffer layer (vulkan-layer/)Linux source + macOS unit-testedBuilds libVkLayer_depthpipe_zbuf.so + VkLayer_depthpipe_zbuf.json + shaders/dibr_zbuf.comp on Linux/SteamOS; verified on macOS via clang++ -std=c++20 ASan+UBSan unit, CPU parity, and regression tests against minimal Khronos ABI types (not a macOS MoltenVK ICD layer)
Neural weights & FFmpegNot bundledOmit --model for built-in Metal synthetic depth, or supply --model <path.mlpackage> for CoreML ANE inference; zero bundled FFmpeg binaries

Install and Uninstall

# Build from source (Commercial edition)
cd projects/depthpipe


# Or via Homebrew tap
brew install raydeoliveira/tap/depthpipe

Build and test the SteamOS Vulkan layer on Linux/SteamOS (or run its C++20 ASan+UBSan tests on macOS):

make -C projects/depthpipe/vulkan-layer clean all test

Linux Vulkan layer environment variables: DEPTHPIPE_VK_ENABLE=1, DEPTHPIPE_VK_DISABLE=1, DEPTHPIPE_BYPASS=1, DEPTHPIPE_NEAR_PLANE, DEPTHPIPE_FAR_PLANE, DEPTHPIPE_DISPARITY_PX, DEPTHPIPE_CONVERGENCE_PX, DEPTHPIPE_REVERSED_Z, DEPTHPIPE_DEPTH_INVERT.

To uninstall:

brew uninstall depthpipe
rm -rf ~/.config/depthpipe

Quick Start (Human + --json Agent Usage)

$ depthpipe --version
depthpipe 0.1.0 (commercial, arm64)

$ depthpipe probe --json
{"data":{"capture_backend":"ScreenCaptureKit","coreml_available":true,"gpu_available":true,"gpu_backend":"Metal","gpu_name":"Apple M4 Max","metal_available":true,"mv_hevc_decode_supported":true,"mv_hevc_encode_supported":true,"screen_capture_permitted":true},"error":null,"ok":true,"schema_version":1}

$ depthpipe profile --app 1091500 --json
{"data":{"aliases":["com.cdprojektred.cyberpunk2077","cyberpunk2077"],"app_id":"1091500","convergence_px":0,"depth_invert":false,"disparity_px":26,"far_plane":1000,"id":"cyberpunk-2077","matched":true,"near_plane":0.1,"notes":"REDengine 4 reversed-Z float32 depth buffer; moderate convergence for indoor HUD.","reversed_z":true,"title":"Cyberpunk 2077"},"error":null,"ok":true,"schema_version":1}

$ depthpipe capture --sim --fixture gradient-shapes --max-frames 2 --out-layout full-sbs --output /tmp/dp_cap.png --dry-run --json
{"data":{"depth_source":"metal-synthetic","elapsed_ms":0,"fixture_name":"gradient-shapes","frames_captured":0,"frames_processed":0,"gpu_name":"Apple M4 Max","media_artifact":{"eye_height":240,"eye_width":320,"frames_processed":0,"out_layout":"full-sbs","output_height":240,"output_path":"/tmp/dp_cap.png","output_width":640},"source":{"bundle_id":"com.depthpipe.sim","height":240,"kind":"sim","source_id":1,"title":"Simulated Spatial Desktop (320x240)","width":320},"used_sim":true},"error":null,"ok":true,"schema_version":1}

# UNIX composability: chain edidsmith/xrctl -> depthpipe -> glassweave via --from-json -
$ edidsmith edid budget --width 3840 --height 1080 --hz 60 --link usbc-4lane --json \
  | depthpipe render --input frame.png --output frame_sbs.png --out-layout full-sbs --from-json - --json \
  | glassweave weave --output woven.png --from-json - --json

Safety and Rollback

  • Screen Recording TCC gating: capture without --sim or --fixture calls CGPreflightScreenCaptureAccess() first and exits 4 with System Settings > Privacy & Security > Screen Recording instructions when denied.
  • Overwrite and dimension guards: render, convert, and capture refuse overlapping input/output or colliding --output/--depth-out paths (including resolved symlinks and case-variants) with exit 2; HEVC 4:2:0 video outputs reject odd or < 2x2 dimensions with exit 2.
  • Non-TTY --yes guard: In non-interactive pipelines, overwriting an existing --output or --depth-out file fails closed (exit 2) unless --yes is passed; writing to a new output file in a pipe works without --yes.
  • Atomic writes & --from-json - fail-closed: Media files finalize through temporary staging files, and upstream ok: false or solved_mode.fits == false (BANDWIDTH_EXCEEDED) aborts immediately with exit 2.
  • Signals: Installs sigaction (sa_flags = 0) for SIGINT and SIGTERM, exiting 130.

Exit Codes and CLI Reference

  • 0: Success
  • 1: Runtime, GPU, capture, or media failure
  • 2: Invalid usage, NaN/Inf/odd dimensions, non-TTY overwrite without --yes, BANDWIDTH_EXCEEDED, or upstream --from-json ok: false
  • 3: Metal GPU or capture source unavailable
  • 4: Screen Recording TCC permission denied
  • 130: SIGINT or SIGTERM received

Run depthpipe --help for human CLI usage, depthpipe --describe for the JSON capability manifest, or depthpipe mcp for stdio JSON-RPC 2.0 (2024-11-05) MCP tools (probe_pipeline, lookup_depth_profile, inspect_media, render_stereo, convert_media, capture_frames).

License and Support

MCPB
https://github.com/raydeoliveira/depthpipe/releases/download/v0.1.0/depthpipe-0.1.0-macos-arm64.mcpb
Install Command
# mcpb: https://github.com/raydeoliveira/depthpipe/releases/download/v0.1.0/depthpipe-0.1.0-macos-arm64.mcpb