Renderers#

Collection of renderers

Example:

field_outputs = field(ray_sampler)
weights = ray_sampler.get_weights(field_outputs[FieldHeadNames.DENSITY])

rgb_renderer = RGBRenderer()
rgb = rgb_renderer(rgb=field_outputs[FieldHeadNames.RGB], weights=weights)
class nerfstudio.model_components.renderers.AccumulationRenderer#

Bases: Module

Accumulated value along a ray.

classmethod forward(weights: Tensor, ray_indices: Optional[Tensor] = None, num_rays: Optional[int] = None) → Tensor#

Composite samples along ray and calculate accumulation.

Parameters:

  • weights – Weights for each sample

  • ray_indices – Ray index for each sample, used when samples are packed.

  • num_rays – Number of rays, used when samples are packed.

Returns:

Outputs of accumulated values.

class nerfstudio.model_components.renderers.DepthRenderer(method: Literal['median', 'expected'] = 'median')#

Bases: Module

Calculate depth along ray.

Depth Method:

  • median: Depth is set to the distance where the accumulated weight reaches 0.5.

  • expected: Expected depth along ray. Same procedure as rendering rgb, but with depth.

Parameters:

method – Depth calculation method.

forward(weights: Tensor, ray_samples: RaySamples, ray_indices: Optional[Tensor] = None, num_rays: Optional[int] = None) → Tensor#

Composite samples along ray and calculate depths.

Parameters:

  • weights – Weights for each sample.

  • ray_samples – Set of ray samples.

  • ray_indices – Ray index for each sample, used when samples are packed.

  • num_rays – Number of rays, used when samples are packed.

Returns:

Outputs of depth values.

class nerfstudio.model_components.renderers.NormalsRenderer#

Bases: Module

Calculate normals along the ray.

classmethod forward(normals: Tensor, weights: Tensor, normalize: bool = True) → Tensor#

Calculate normals along the ray.

Parameters:

  • normals – Normals for each sample.

  • weights – Weights of each sample.

  • normalize – Normalize normals.

class nerfstudio.model_components.renderers.RGBRenderer(background_color: Union[Literal['random', 'last_sample'], Tensor] = 'random')#

Bases: Module

Standard volumetric rendering.

Parameters:

background_color – Background color as RGB. Uses random colors if None.

classmethod combine_rgb(rgb: Tensor, weights: Tensor, background_color: Union[Literal['random', 'black', 'last_sample'], Tensor] = 'random', ray_indices: Optional[Tensor] = None, num_rays: Optional[int] = None) → Tensor#

Composite samples along ray and render color image

Parameters:

  • rgb – RGB for each sample

  • weights – Weights for each sample

  • background_color – Background color as RGB.

  • ray_indices – Ray index for each sample, used when samples are packed.

  • num_rays – Number of rays, used when samples are packed.

Returns:

Outputs rgb values.

forward(rgb: Tensor, weights: Tensor, ray_indices: Optional[Tensor] = None, num_rays: Optional[int] = None) → Tensor#

Composite samples along ray and render color image

Parameters:

  • rgb – RGB for each sample

  • weights – Weights for each sample

  • ray_indices – Ray index for each sample, used when samples are packed.

  • num_rays – Number of rays, used when samples are packed.

Returns:

Outputs of rgb values.

class nerfstudio.model_components.renderers.SHRenderer(background_color: Union[Literal['random', 'last_sample'], Tensor] = 'random', activation: Optional[Module] = Sigmoid())#

Bases: Module

Render RGB value from spherical harmonics.

Parameters:

  • background_color – Background color as RGB. Uses random colors if None

  • activation – Output activation.

forward(sh: Tensor, directions: Tensor, weights: Tensor) → Tensor#

Composite samples along ray and render color image

Parameters:

  • sh – Spherical harmonics coefficients for each sample

  • directions – Sample direction

  • weights – Weights for each sample

Returns:

Outputs of rgb values.

class nerfstudio.model_components.renderers.SemanticRenderer#

Bases: Module

Calculate semantics along the ray.

classmethod forward(semantics: Tensor, weights: Tensor) → Tensor#

Calculate semantics along the ray.

class nerfstudio.model_components.renderers.UncertaintyRenderer#

Bases: Module

Calculate uncertainty along the ray.

classmethod forward(betas: Tensor, weights: Tensor) → Tensor#

Calculate uncertainty along the ray.

Parameters:

  • betas – Uncertainty betas for each sample.

  • weights – Weights of each sample.

Returns:

Rendering of uncertainty.

nerfstudio.model_components.renderers.background_color_override_context(mode: Tensor) → Generator[None, None, None]#

Context manager for setting background mode.