K-Planes

Explicit Radiance Fields in Space, Time, and Appearance

Paper Website

Official Code

Nerfstudio add-on code

A unified model for static, dynamic and variable appearance NeRFs.

Installation

First, install nerfstudio and its dependencies. Then install the K-Planes add-on

pip install kplanes-nerfstudio

Running K-Planes

There are two default configurations provided which use the blender and DNeRF dataloaders. However, you can easily extend them to create a new configuration for different datasets.

The default configurations provided are

Method	Description	Scene type	Memory
kplanes	Tuned for blender scenes	static, synthetic	~4GB
kplanes-dynamic	Tuned for DNeRF dataset	dynamic (monocular), synthetic	~5GB

for training with these two configurations you should run

ns-train kplanes --data <data-folder>

or

ns-train kplanes-dynamic --data <data-folder>

Note

kplanes is set up to use blender data, (download it running ns-download-data blender), kplanes-dynamic is set up to use DNeRF data, (download it running ns-download-data dnerf).

Method

K-planes represents a scene in k dimensions – where k can be 3 for static 3-dimensional scenes or 4 for scenes which change in time – using k-choose-2 planes (or grids). After ray-sampling, the querying the field at a certain position consists in querying each plane (with interpolation), and combining the resulting features through multiplication. This factorization of space and time keeps memory usage low, and is very flexible in the kinds of priors and regularizers that can be added.

We support hybrid models with a small MLP (left) and fully explicit models (right), through the linear_decoder configuration key

The model also supports decomposing a scene into its space and time components. For more information on how to do this see the official code repo