GHNeRF

Recent advances in Neural Radiance Fields (NeRF) have demonstrated promising results in 3D scene representa- tions, including 3D human representations. However, these representations often lack crucial information on the un- derlying human pose and structure, which is crucial for AR/VR applications and games. In this paper, we intro- duce a novel approach, termed GHNeRF, designed to ad- dress these limitations by learning 2D/3D joint locations of human subjects with NeRF representation. GHNeRF uses a pre-trained 2D encoder streamlined to extract essential human features from 2D images, which are then incorpo- rated into the NeRF framework in order to encode human biomechanic features. This allows our network to simulta- neously learn biomechanic features, such as joint locations, along with human geometry and texture. To assess the effec- tiveness of our method, we conduct a comprehensive com- parison with state-of-the-art human NeRF techniques and joint estimation algorithms. Our results show that GHNeRF can achieve state-of-the-art results in near real-time.