3D occupancy perception is gaining increasing attention due to its capability to offer detailed and precise environment representations. Previous weakly-supervised NeRF methods balance efficiency and accuracy, with mIoU varying by 5-10 points due to sampling count along camera rays. Recently, real-time Gaussian splatting has gained widespread popularity in 3D reconstruction, and the occupancy prediction task can also be viewed as a reconstruction task. Consequently, we propose GSRender, which naturally employs 3D Gaussian Splatting for occupancy prediction, simplifying the sampling process. In addition, the limitations of 2D supervision result in duplicate predictions along the same camera ray. We implemented the Ray Compensation (RC) module, which mitigates this issue by compensating for features from adjacent frames. Finally, we redesigned the loss to eliminate the impact of dynamic objects from adjacent frames. Extensive experiments demonstrate that our approach achieves SOTA (state-of-the-art) results in RayIoU (+6.0), while narrowing the gap with 3D supervision methods.
三维占用感知（3D Occupancy Perception）因其能够提供细致且精确的环境表示而备受关注。以往的弱监督 NeRF 方法在效率与精度之间取得了一定平衡，但沿相机射线的采样次数会导致 mIoU 存在 5-10 个点的波动。近年来，实时高斯点云技术（3D Gaussian Splatting）因其在三维重建中的出色表现广受欢迎，而占用预测任务同样可以视为一种重建任务。 基于此，我们提出了 GSRender，将三维高斯点云自然应用于占用预测任务，从而简化了采样过程。此外，由于二维监督的局限性，沿同一相机射线可能会出现重复预测。为解决这一问题，我们实现了 射线补偿模块（Ray Compensation, RC），通过补偿来自相邻帧的特征，缓解了这一问题。最后，我们重新设计了损失函数，消除了相邻帧中动态物体的影响。 大量实验结果表明，我们的方法在 RayIoU 上实现了 +6.0 的提升，同时缩小了与三维监督方法的差距，达到了当前最先进（SOTA）的结果。我们的代码将很快开源。