Scene Dreamer

SceneDreamer is a generative model for 3D scenes which can create large-scale landscapes with no 3D annotations. It makes use of an efficient bird’s-eye-view representation generated from simplex noise, consisting of a height field and a semantic field. This representation enables the tool to represent a 3D scene with quadratic complexity, as well as disentangle geometry and semantics for training. Additionally, SceneDreamer has proposed a novel generative neural hash grid to parameterize the latent space given positions and scene semantics. This aims to produce generalizable features across scenes in order to align content. Finally, a neural volumetric renderer is used that has been trained through adversarial learning from 2D image collections.

The effectiveness of SceneDreamer has been demonstrated by its ability to generate vivid and diverse unbounded 3D worlds, surpassing state-of-the-art methods in this regard. The tool can be utilized for various applications including computer graphics, gaming, virtual reality and augmented reality. Its main advantages lie in its capability of synthesizing large scale 3D scenes without any 3D annotations and creating photorealistic images through its neural volumetric renderer.