Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Visualize 3D Gaussian splatting #7154

Open
ssheorey opened this issue Jan 30, 2025 · 0 comments
Open

Visualize 3D Gaussian splatting #7154

ssheorey opened this issue Jan 30, 2025 · 0 comments
Milestone

Comments

@ssheorey
Copy link
Member

3D Gaussian splatting is a novel 3D representation that enables fast scene reconstruction and real time photorealistic rendering. We are adding support for 3DGS in Open3D. As part of that, we also want to add the ability to render 3DGS scenes.

Steps:

  1. Study the Open3D visualizer and GLSL shaders used.
  2. Update visualizer to call a SPIR-V shader directly with ARB_gl_SPIRV.
  3. Use 3DGS SYCL rendering kernels from the [isl-org gsplat fork](https://github.com/isl-org/gsplat) to render 3DGS scenes.
  4. Add Python binding.

Tests:

  • Load these Numpy files and display scene. Compare with reference gsplat viewer.

Open3D Tensor point cloud representation:

  • pcd.point[‘positions’] (N,3) – (x,y,z) for each splat.
  • pcd.point[‘opacity’] – (N,) opacity
  • pcd.point[‘rot’] (N, 4) - quaternion rotation of Gaussian.
  • pcd.point[‘scale'] (N, 3) - x, y, z scales for Gaussian.
  • pcd.point[‘f_dc’] (N, 3) – DC components for RGB colors.
  • pcd.point[‘f_rest’] (N, Nc, 3) – SH coeffs for RGB colors. (Nc = 4, 9 or 16)

Reference:

@ssheorey ssheorey converted this from a draft issue Jan 30, 2025
@ssheorey ssheorey added this to the v0.20 milestone Jan 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Backlog
Development

No branches or pull requests

1 participant