Retinal-NeRF: Real-time rendering of neural point fields across platform.

Abstract

Neural Radiance Fields (NeRFs) has emerged as a promising method for 3D reconstruction and novel view synthesis. However, NeRF-based methods rely on implicit encoding and heavy spatial sampling, which differ significantly from the widely used polygon mesh rasterization method. This discrepancy leads to a lack of support from common 3D software and hardware, resulting in inefficient rendering. In this work, we introduce a novel method called Retinal-NeRF, which combines point clouds and signed distance fields, for real-time rendering of 3D scenes. Our goal is achieved by converting NeRF into a representation that can be efficiently processed by standard graphics pipelines. To enhance the geometric accuracy of the scene, we propose two SDF regularization terms to improve the quality of the generated mesh. To ensure real-time rendering, we employ appearance decomposition technique to minimize the size of the MLP within the rendering pipeline. Notably, our findings indicate that the rendering speed of Retinal-NeRF is more than five times faster than existing real-time rendering techniques, without a significant loss in the quality of novel view synthesis.