Data#

Pixel Samplers#

Code for sampling pixels.

class nerfstudio.data.pixel_samplers.PairPixelSampler(config: PairPixelSamplerConfig, **kwargs)[source]#

Bases: PixelSampler

Samples pair of pixels from ‘image_batch’s. Samples pairs of pixels from: from the images randomly within a ‘radius’ distance apart. Useful for pair-based losses.

Parameters: config – the PairPixelSamplerConfig used to instantiate class

sample_method(batch_size: Optional[int], num_images: int, image_height: int, image_width: int, mask: Optional[Tensor] = None, device: Union[device, str] = 'cpu') → Int[Tensor, 'batch_size 3'][source]#

Naive pixel sampler, uniformly samples across all possible pixels of all possible images.

Parameters

batch_size – number of samples in a batch
num_images – number of images to sample over
mask – mask of possible pixels in an image to sample from.

class nerfstudio.data.pixel_samplers.PairPixelSamplerConfig(_target: ~typing.Type = <factory>, num_rays_per_batch: int = 4096, keep_full_image: bool = False, is_equirectangular: bool = False, ignore_mask: bool = False, fisheye_crop_radius: ~typing.Optional[float] = None, rejection_sample_mask: bool = True, max_num_iterations: int = 100, radius: int = 2)[source]#

Bases: PixelSamplerConfig

Config dataclass for PairPixelSampler.

radius: int = 2#: max distance between pairs of pixels.

class nerfstudio.data.pixel_samplers.PatchPixelSampler(config: PixelSamplerConfig, **kwargs)[source]#

Bases: PixelSampler

Samples ‘pixel_batch’s from ‘image_batch’s. Samples square patches from the images randomly. Useful for patch-based losses.

Parameters: config – the PatchPixelSamplerConfig used to instantiate class

sample_method(batch_size: int, num_images: int, image_height: int, image_width: int, mask: Optional[Tensor] = None, device: Union[device, str] = 'cpu') → Int[Tensor, 'batch_size 3'][source]#

Naive pixel sampler, uniformly samples across all possible pixels of all possible images.

Parameters

batch_size – number of samples in a batch
num_images – number of images to sample over
mask – mask of possible pixels in an image to sample from.

set_num_rays_per_batch(num_rays_per_batch: int)[source]#

Set the number of rays to sample per batch. Overridden to deal with patch-based sampling.

Parameters: num_rays_per_batch – number of rays to sample per batch

class nerfstudio.data.pixel_samplers.PatchPixelSamplerConfig(_target: ~typing.Type = <factory>, num_rays_per_batch: int = 4096, keep_full_image: bool = False, is_equirectangular: bool = False, ignore_mask: bool = False, fisheye_crop_radius: ~typing.Optional[float] = None, rejection_sample_mask: bool = True, max_num_iterations: int = 100, patch_size: int = 32)[source]#

Bases: PixelSamplerConfig

Config dataclass for PatchPixelSampler.

patch_size: int = 32#: Side length of patch. This must be consistent in the method config in order for samples to be reshaped into patches correctly.

class nerfstudio.data.pixel_samplers.PixelSampler(config: PixelSamplerConfig, **kwargs)[source]#

Bases: object

Samples ‘pixel_batch’s from ‘image_batch’s.

Parameters: config – the DataManagerConfig used to instantiate class

collate_image_dataset_batch(batch: Dict, num_rays_per_batch: int, keep_full_image: bool = False)[source]#

Operates on a batch of images and samples pixels to use for generating rays. Returns a collated batch which is input to the Graph. It will sample only within the valid ‘mask’ if it’s specified.

Parameters

batch – batch of images to sample from
num_rays_per_batch – number of rays to sample per batch
keep_full_image – whether or not to include a reference to the full image in returned batch

collate_image_dataset_batch_list(batch: Dict, num_rays_per_batch: int, keep_full_image: bool = False)[source]#

Does the same as collate_image_dataset_batch, except it will operate over a list of images / masks inside a list.

We will use this with the intent of DEPRECIATING it as soon as we find a viable alternative. The intention will be to replace this with a more efficient implementation that doesn’t require a for loop, but since pytorch’s ragged tensors are still in beta (this would allow for some vectorization), this will do.

Parameters

batch – batch of images to sample from
num_rays_per_batch – number of rays to sample per batch
keep_full_image – whether or not to include a reference to the full image in returned batch

sample(image_batch: Dict)[source]#

Sample an image batch and return a pixel batch.

Parameters: image_batch – batch of images to sample from

Naive pixel sampler, uniformly samples across all possible pixels of all possible images.

Parameters

batch_size – number of samples in a batch
num_images – number of images to sample over
mask – mask of possible pixels in an image to sample from.

set_num_rays_per_batch(num_rays_per_batch: int)[source]#

Set the number of rays to sample per batch.

Parameters: num_rays_per_batch – number of rays to sample per batch

class nerfstudio.data.pixel_samplers.PixelSamplerConfig(_target: ~typing.Type = <factory>, num_rays_per_batch: int = 4096, keep_full_image: bool = False, is_equirectangular: bool = False, ignore_mask: bool = False, fisheye_crop_radius: ~typing.Optional[float] = None, rejection_sample_mask: bool = True, max_num_iterations: int = 100)[source]#

Bases: InstantiateConfig

Configuration for pixel sampler instantiation.

fisheye_crop_radius: Optional[float] = None#: Set to the radius (in pixels) for fisheye cameras.

ignore_mask: bool = False#: Whether to ignore the masks when sampling.

is_equirectangular: bool = False#: List of whether or not camera i is equirectangular.

keep_full_image: bool = False#: Whether or not to include a reference to the full image in returned batch.

max_num_iterations: int = 100#: If rejection sampling masks, the maximum number of times to sample

num_rays_per_batch: int = 4096#: Number of rays to sample per batch.

rejection_sample_mask: bool = True#: Whether or not to use rejection sampling when sampling images with masks

Scene Box#

Dataset input structures.

class nerfstudio.data.scene_box.OrientedBox(R: jaxtyping.Float[Tensor, '3 3'], T: jaxtyping.Float[Tensor, '3'], S: jaxtyping.Float[Tensor, '3'])[source]#

Bases: object

R: Float[Tensor, '3 3']#

rotation matrix.

Type: R

S: Float[Tensor, '3']#

scale vector.

Type: S

T: Float[Tensor, '3']#

translation vector.

Type: T

static from_params(pos: Tuple[float, float, float], rpy: Tuple[float, float, float], scale: Tuple[float, float, float])[source]#: Construct a box from position, rotation, and scale parameters.

within(pts: Float[Tensor, 'n 3'])[source]#: Returns a boolean mask indicating whether each point is within the box.

class nerfstudio.data.scene_box.SceneBox(aabb: Float[Tensor, '2 3'])[source]#

Bases: object

Data to represent the scene box.

aabb: Float[Tensor, '2 3']#

axis-aligned bounding box. aabb[0] is the minimum (x,y,z) point. aabb[1] is the maximum (x,y,z) point.

Type: aabb

static from_camera_poses(poses: Float[Tensor, '*batch 3 4'], scale_factor: float) → SceneBox[source]#

Returns the instance of SceneBox that fully envelopes a set of poses

Parameters

poses – tensor of camera pose matrices
scale_factor – How much to scale the camera origins by.

get_center()[source]#: Returns the center of the box.

get_centered_and_scaled_scene_box(scale_factor: Union[float, Tensor] = 1.0)[source]#

Returns a new box that has been shifted and rescaled to be centered about the origin.

Parameters: scale_factor – How much to scale the camera origins by.

get_diagonal_length()[source]#: Returns the longest diagonal length.

static get_normalized_positions(positions: Float[Tensor, '*batch 3'], aabb: Float[Tensor, '2 3'])[source]#

Return normalized positions in range [0, 1] based on the aabb axis-aligned bounding box.

Parameters

positions – the xyz positions
aabb – the axis-aligned bounding box

within(pts: Float[Tensor, 'n 3'])[source]#: Returns a boolean mask indicating whether each point is within the box.