Ray Sampler#

Collection of sampling strategies

class nerfstudio.model_components.ray_samplers.DensityFn(*args, **kwargs)[source]#

Bases: Protocol

Function that evaluates density at a given point.

class nerfstudio.model_components.ray_samplers.LinearDisparitySampler(num_samples: Optional[int] = None, train_stratified=True, single_jitter=False)[source]#

Bases: SpacedSampler

Sample linearly in disparity along a ray

Parameters

  • num_samples – Number of samples per ray

  • train_stratified – Use stratified sampling during training. Defaults to True

  • single_jitter – Use a same random jitter for all samples along a ray. Defaults to False

class nerfstudio.model_components.ray_samplers.LogSampler(num_samples: Optional[int] = None, train_stratified=True, single_jitter=False)[source]#

Bases: SpacedSampler

Log sampler along a ray

Parameters

  • num_samples – Number of samples per ray

  • train_stratified – Use stratified sampling during training. Defaults to True

class nerfstudio.model_components.ray_samplers.NeuSSampler(num_samples: int = 64, num_samples_importance: int = 64, num_samples_outside: int = 32, num_upsample_steps: int = 4, base_variance: float = 64, single_jitter: bool = True)[source]#

Bases: Sampler

NeuS sampler that uses a sdf network to generate samples with fixed variance value in each iterations.

generate_ray_samples(ray_bundle: Optional[RayBundle] = None, sdf_fn: Optional[Callable[[RaySamples], Tensor]] = None, ray_samples: Optional[RaySamples] = None) → Union[Tuple[RaySamples, Tensor], RaySamples][source]#

Generate Ray Samples

static merge_ray_samples(ray_bundle: RayBundle, ray_samples_1: RaySamples, ray_samples_2: RaySamples)[source]#

Merge two set of ray samples and return sorted index which can be used to merge sdf values :param ray_samples_1: ray_samples to merge :param ray_samples_2: ray_samples to merge

static rendering_sdf_with_fixed_inv_s(ray_samples: RaySamples, sdf: Float[Tensor, 'num_samples 1'], inv_s: int) → Float[Tensor, 'num_samples 1'][source]#

rendering given a fixed inv_s as NeuS

Parameters

  • ray_samples – samples along ray

  • sdf – sdf values along ray

  • inv_s – fixed variance value

Returns

alpha value

class nerfstudio.model_components.ray_samplers.PDFSampler(num_samples: Optional[int] = None, train_stratified: bool = True, single_jitter: bool = False, include_original: bool = True, histogram_padding: float = 0.01)[source]#

Bases: Sampler

Sample based on probability distribution

Parameters

  • num_samples – Number of samples per ray

  • train_stratified – Randomize location within each bin during training.

  • single_jitter – Use a same random jitter for all samples along a ray. Defaults to False

  • include_original – Add original samples to ray.

  • histogram_padding – Amount to weights prior to computing PDF.

generate_ray_samples(ray_bundle: Optional[RayBundle] = None, ray_samples: Optional[RaySamples] = None, weights: Optional[Float[Tensor, '*batch num_samples 1']] = None, num_samples: Optional[int] = None, eps: float = 1e-05) → RaySamples[source]#

Generates position samples given a distribution.

Parameters

  • ray_bundle – Rays to generate samples for

  • ray_samples – Existing ray samples

  • weights – Weights for each bin

  • num_samples – Number of samples per ray

  • eps – Small value to prevent numerical issues.

Returns

Positions and deltas for samples along a ray

class nerfstudio.model_components.ray_samplers.ProposalNetworkSampler(num_proposal_samples_per_ray: ~typing.Tuple[int, ...] = (64, ), num_nerf_samples_per_ray: int = 32, num_proposal_network_iterations: int = 2, single_jitter: bool = False, update_sched: ~typing.Callable = <function ProposalNetworkSampler.<lambda>>, initial_sampler: ~typing.Optional[~nerfstudio.model_components.ray_samplers.Sampler] = None, pdf_sampler: ~typing.Optional[~nerfstudio.model_components.ray_samplers.PDFSampler] = None)[source]#

Bases: Sampler

Sampler that uses a proposal network to generate samples.

Parameters

  • num_proposal_samples_per_ray – Number of samples to generate per ray for each proposal step.

  • num_nerf_samples_per_ray – Number of samples to generate per ray for the NERF model.

  • num_proposal_network_iterations – Number of proposal network iterations to run.

  • single_jitter – Use a same random jitter for all samples along a ray.

  • update_sched – A function that takes the iteration number of steps between updates.

  • initial_sampler – Sampler to use for the first iteration. Uses UniformLinDispPiecewise if not set.

  • pdf_sampler – PDFSampler to use after the first iteration. Uses PDFSampler if not set.

generate_ray_samples(ray_bundle: Optional[RayBundle] = None, density_fns: Optional[List[Callable]] = None) → Tuple[RaySamples, List, List][source]#

Generate Ray Samples

set_anneal(anneal: float) → None[source]#

Set the anneal value for the proposal network.

step_cb(step)[source]#

Callback to register a training step has passed. This is used to keep track of the sampling schedule

class nerfstudio.model_components.ray_samplers.Sampler(num_samples: Optional[int] = None)[source]#

Bases: Module

Generate Samples

Parameters

num_samples – number of samples to take

forward(*args, **kwargs) → Any[source]#

Generate ray samples

abstract generate_ray_samples() → Any[source]#

Generate Ray Samples

class nerfstudio.model_components.ray_samplers.SpacedSampler(spacing_fn: Callable, spacing_fn_inv: Callable, num_samples: Optional[int] = None, train_stratified=True, single_jitter=False)[source]#

Bases: Sampler

Sample points according to a function.

Parameters

  • num_samples – Number of samples per ray

  • spacing_fn – Function that dictates sample spacing (ie lambda x : x is uniform).

  • spacing_fn_inv – The inverse of spacing_fn.

  • train_stratified – Use stratified sampling during training. Defaults to True

  • single_jitter – Use a same random jitter for all samples along a ray. Defaults to False

generate_ray_samples(ray_bundle: Optional[RayBundle] = None, num_samples: Optional[int] = None) → RaySamples[source]#

Generates position samples according to spacing function.

Parameters

  • ray_bundle – Rays to generate samples for

  • num_samples – Number of samples per ray

Returns

Positions and deltas for samples along a ray

class nerfstudio.model_components.ray_samplers.SqrtSampler(num_samples: Optional[int] = None, train_stratified=True, single_jitter=False)[source]#

Bases: SpacedSampler

Square root sampler along a ray

Parameters

  • num_samples – Number of samples per ray

  • train_stratified – Use stratified sampling during training. Defaults to True

class nerfstudio.model_components.ray_samplers.UniformLinDispPiecewiseSampler(num_samples: Optional[int] = None, train_stratified=True, single_jitter=False)[source]#

Bases: SpacedSampler

Piecewise sampler along a ray that allocates the first half of the samples uniformly and the second half using linearly in disparity spacing.

Parameters

  • num_samples – Number of samples per ray

  • train_stratified – Use stratified sampling during training. Defaults to True

  • single_jitter – Use a same random jitter for all samples along a ray. Defaults to False

class nerfstudio.model_components.ray_samplers.UniformSampler(num_samples: Optional[int] = None, train_stratified=True, single_jitter=False)[source]#

Bases: SpacedSampler

Sample uniformly along a ray

Parameters

  • num_samples – Number of samples per ray

  • train_stratified – Use stratified sampling during training. Defaults to True

  • single_jitter – Use a same random jitter for all samples along a ray. Defaults to False

class nerfstudio.model_components.ray_samplers.VolumetricSampler(occupancy_grid: OccGridEstimator, density_fn: Optional[DensityFn] = None)[source]#

Bases: Sampler

Sampler inspired by the one proposed in the Instant-NGP paper. Generates samples along a ray by sampling the occupancy field. Optionally removes occluded samples if the density_fn is provided.

Args: occupancy_grid: Occupancy grid to sample from. density_fn: Function that evaluates density at a given point. scene_aabb: Axis-aligned bounding box of the scene, should be set to None if the scene is unbounded.

forward(ray_bundle: RayBundle, render_step_size: float, near_plane: float = 0.0, far_plane: Optional[float] = None, alpha_thre: float = 0.01, cone_angle: float = 0.0) → Tuple[RaySamples, Float[Tensor, 'total_samples']][source]#

Generate ray samples in a bounding box.

Parameters

  • ray_bundle – Rays to generate samples for

  • render_step_size – Minimum step size to use for rendering

  • near_plane – Near plane for raymarching

  • far_plane – Far plane for raymarching

  • alpha_thre – Opacity threshold skipping samples.

  • cone_angle – Cone angle for raymarching, set to 0 for uniform marching.

Returns

a tuple of (ray_samples, packed_info, ray_indices) The ray_samples are packed, only storing the valid samples. The ray_indices contains the indices of the rays that each sample belongs to.

generate_ray_samples() → RaySamples[source]#

Generate Ray Samples

get_sigma_fn(origins, directions, times=None) → Optional[Callable][source]#

Returns a function that returns the density of a point.

Parameters

  • origins – Origins of rays

  • directions – Directions of rays

  • times – Times at which rays are sampled

Returns

Function that returns the density of a point or None if a density function is not provided.