Data#

Data Parsers#

Datasets#

Dataloaders#

Code for sampling images from a dataset of images.

class nerfstudio.data.utils.dataloaders.CacheDataloader(dataset: ~torch.utils.data.dataset.Dataset, num_images_to_sample_from: int = -1, num_times_to_repeat_images: int = -1, device: ~typing.Union[~torch.device, str] = 'cpu', collate_fn=<function nerfstudio_collate>, **kwargs)#

Bases: DataLoader

Collated image dataset that implements caching of default-pytorch-collatable data. Creates batches of the InputDataset return type.

Parameters:

dataset – Dataset to sample from.
num_samples_to_collate – How many images to sample rays for each batch. -1 for all images.
num_times_to_repeat_images – How often to collate new images. -1 to never pick new images.
device – Device to perform computation.
collate_fn – The function we will use to collate our training data

class nerfstudio.data.utils.dataloaders.EvalDataloader(input_dataset: InputDataset, device: Union[device, str] = 'cpu', **kwargs)#

Bases: DataLoader

Evaluation dataloader base class

Parameters:

input_dataset – InputDataset to load data from
device – Device to load data to

abstract __iter__()#: Iterates over the dataset

abstract __next__() → Tuple[RayBundle, Dict]#: Returns the next batch of data

get_camera(image_idx: int = 0) → Cameras#

Get camera for the given image index

Parameters:: image_idx – Camera image index

get_data_from_image_idx(image_idx: int) → Tuple[RayBundle, Dict]#

Returns the data for a specific image index.

Parameters:: image_idx – Camera image index

class nerfstudio.data.utils.dataloaders.FixedIndicesEvalDataloader(input_dataset: InputDataset, image_indices: Optional[Tuple[int]] = None, device: Union[device, str] = 'cpu', **kwargs)#

Bases: EvalDataloader

Dataloader that returns a fixed set of indices.

Parameters:

input_dataset – InputDataset to load data from
image_indices – List of image indices to load data from. If None, then use all images.
device – Device to load data to

class nerfstudio.data.utils.dataloaders.RandIndicesEvalDataloader(input_dataset: InputDataset, device: Union[device, str] = 'cpu', **kwargs)#

Bases: EvalDataloader

Dataloader that returns random images.

Parameters:

input_dataset – InputDataset to load data from
device – Device to load data to

Pixel Samplers#

Code for sampling pixels.

class nerfstudio.data.pixel_samplers.EquirectangularPixelSampler(num_rays_per_batch: int, keep_full_image: bool = False, **kwargs)#

Bases: PixelSampler

Samples ‘pixel_batch’s from ‘image_batch’s. Assumes images are equirectangular and the sampling is done uniformly on the sphere.

Parameters:

num_rays_per_batch – number of rays to sample per batch
keep_full_image – whether or not to include a reference to the full image in returned batch

sample_method(batch_size: int, num_images: int, image_height: int, image_width: int, mask: Optional[TensorType] = None, device: Union[device, str] = 'cpu') → Tensor#

Naive pixel sampler, uniformly samples across all possible pixels of all possible images.

Parameters:

batch_size – number of samples in a batch
num_images – number of images to sample over
mask – mask of possible pixels in an image to sample from.

class nerfstudio.data.pixel_samplers.PixelSampler(num_rays_per_batch: int, keep_full_image: bool = False, **kwargs)#

Bases: object

Samples ‘pixel_batch’s from ‘image_batch’s.

Parameters:

num_rays_per_batch – number of rays to sample per batch
keep_full_image – whether or not to include a reference to the full image in returned batch

collate_image_dataset_batch(batch: Dict, num_rays_per_batch: int, keep_full_image: bool = False)#

Operates on a batch of images and samples pixels to use for generating rays. Returns a collated batch which is input to the Graph. It will sample only within the valid ‘mask’ if it’s specified.

Parameters:

batch – batch of images to sample from
num_rays_per_batch – number of rays to sample per batch
keep_full_image – whether or not to include a reference to the full image in returned batch

collate_image_dataset_batch_list(batch: Dict, num_rays_per_batch: int, keep_full_image: bool = False)#

Does the same as collate_image_dataset_batch, except it will operate over a list of images / masks inside a list.

We will use this with the intent of DEPRECIATING it as soon as we find a viable alternative. The intention will be to replace this with a more efficient implementation that doesn’t require a for loop, but since pytorch’s ragged tensors are still in beta (this would allow for some vectorization), this will do.

Parameters:

batch – batch of images to sample from
num_rays_per_batch – number of rays to sample per batch
keep_full_image – whether or not to include a reference to the full image in returned batch

sample(image_batch: Dict)#

Sample an image batch and return a pixel batch.

Parameters:: image_batch – batch of images to sample from

sample_method(batch_size: int, num_images: int, image_height: int, image_width: int, mask: Optional[TensorType] = None, device: Union[device, str] = 'cpu') → Tensor#

Naive pixel sampler, uniformly samples across all possible pixels of all possible images.

Parameters:

batch_size – number of samples in a batch
num_images – number of images to sample over
mask – mask of possible pixels in an image to sample from.

set_num_rays_per_batch(num_rays_per_batch: int)#

Set the number of rays to sample per batch.

Parameters:: num_rays_per_batch – number of rays to sample per batch

Scene Box#

Dataset input structures.

class nerfstudio.data.scene_box.SceneBox(aabb: Optional[Tensor] = None)#

Bases: object

Data to represent the scene box.

aabb: Tensor = None#

axis-aligned bounding box. aabb[0] is the minimum (x,y,z) point. aabb[1] is the maximum (x,y,z) point.

Type:: aabb

static from_camera_poses(poses: Tensor, scale_factor: float) → SceneBox#

Returns the instance of SceneBox that fully envelopes a set of poses

Parameters:

poses – tensor of camera pose matrices
scale_factor – How much to scale the camera origins by.

static from_json(json_: Dict) → SceneBox#

Returns the an instance of SceneBox from a json dictionary.

Parameters:: json – the json dictionary containing scene box information

get_center()#: Returns the center of the box.

get_centered_and_scaled_scene_box(scale_factor: Union[float, Tensor] = 1.0)#

Returns a new box that has been shifted and rescaled to be centered about the origin.

Parameters:: scale_factor – How much to scale the camera origins by.

get_diagonal_length()#: Returns the longest diagonal length.

static get_normalized_positions(positions: Tensor, aabb: Tensor)#

Return normalized positions in range [0, 1] based on the aabb axis-aligned bounding box.

Parameters:

positions – the xyz positions
aabb – the axis-aligned bounding box

to_json() → Dict#: Returns a json object from the Python object.