Skip to content

xbarray.transformations.pointcloud.base

Functions:

Name Description
gather_pixel_value

Gather pixel values at given pixel coordinates.

bilinear_interpolate

Obtain bilinearly interpolated values at given pixel coordinates.

pixel_coordinate_and_depth_to_world

Convert pixel coordinates and depth to world coordinates.

depth_image_to_world

Convert a depth image to world coordinates.

world_to_pixel_coordinate_and_depth

Convert world coordinates to pixel coordinates and depth.

world_to_depth

Convert world coordinates to pixel coordinates and depth.

farthest_point_sampling

Perform farthest point sampling on a set of points.

random_point_sampling

Perform random point sampling on a set of points.

ball_query

Find, for every query point, up to num_neighbors points strictly within radius.

pointcloud_jitter

Apply jitter noise to point cloud.

gather_pixel_value

gather_pixel_value(
    backend: ComputeBackend[BArrayType, BDeviceType, BDtypeType, BRNGType],
    values: BArrayType,
    pixel_coordinates: BArrayType,
) -> BArrayType

Gather pixel values at given pixel coordinates. Args: backend (ComputeBackend): The compute backend to use. values (BArrayType): The input values of shape (..., H, W, C). pixel_coordinates (BArrayType): The pixel coordinates of shape (..., N, 2) in (x, y) order. Returns: BArrayType: The gathered values of shape (..., N, C).

bilinear_interpolate

bilinear_interpolate(
    backend: ComputeBackend[BArrayType, BDeviceType, BDtypeType, BRNGType],
    values: BArrayType,
    pixel_coordinates: BArrayType,
    index_dtype: Optional[BDtypeType] = None,
    uniform_weights: bool = False,
) -> BArrayType

Obtain bilinearly interpolated values at given pixel coordinates. Args: backend (ComputeBackend): The compute backend to use. values (BArrayType): The input values of shape (..., H, W, C). pixel_coordinates (BArrayType): The pixel coordinates of shape (..., N, 2) in (x, y) order. Returns: BArrayType: The interpolated values of shape (..., N, C).

pixel_coordinate_and_depth_to_world

pixel_coordinate_and_depth_to_world(
    backend: ComputeBackend[BArrayType, BDeviceType, BDtypeType, BRNGType],
    pixel_coordinates: BArrayType,
    depth: BArrayType,
    intrinsic_matrix: BArrayType,
    extrinsic_matrix: Optional[BArrayType] = None,
) -> BArrayType

Convert pixel coordinates and depth to world coordinates. Args: backend (ComputeBackend): The compute backend to use. pixel_coordinates (BArrayType): The pixel coordinates of shape (..., N, 2). depth (BArrayType): The depth values of shape (..., N). Assume invalid depth is either nan or <= 0. intrinsic_matrix (BArrayType): The camera intrinsic matrix of shape (..., 3, 3). extrinsic_matrix (BArrayType): The camera extrinsic matrix of shape (..., 3, 4) or (..., 4, 4). Returns: BArrayType: The world coordinates of shape (..., N, 4). The last dimension is (x, y, z, valid_mask).

depth_image_to_world

depth_image_to_world(
    backend: ComputeBackend[BArrayType, BDeviceType, BDtypeType, BRNGType],
    depth_image: BArrayType,
    intrinsic_matrix: BArrayType,
    extrinsic_matrix: Optional[BArrayType] = None,
) -> BArrayType

Convert a depth image to world coordinates. Args: backend (ComputeBackend): The compute backend to use. depth_image (BArrayType): The depth image of shape (..., H, W). intrinsic_matrix (BArrayType): The camera intrinsic matrix of shape (..., 3, 3). extrinsic_matrix (BArrayType): The camera extrinsic matrix of shape (..., 3, 4) or (..., 4, 4). Returns: BArrayType: The world coordinates of shape (..., H, W, 4). The last dimension is (x, y, z, valid_mask).

world_to_pixel_coordinate_and_depth

world_to_pixel_coordinate_and_depth(
    backend: ComputeBackend[BArrayType, BDeviceType, BDtypeType, BRNGType],
    world_coords: BArrayType,
    intrinsic_matrix: BArrayType,
    extrinsic_matrix: Optional[BArrayType] = None,
) -> Tuple[BArrayType, BArrayType]

Convert world coordinates to pixel coordinates and depth. Args: backend (ComputeBackend): The compute backend to use. world_coords (BArrayType): The world coordinates of shape (..., N, 3) or (..., N, 4). If the last dimension is 4, the last element is treated as a valid mask. intrinsic_matrix (BArrayType): The camera intrinsic matrix of shape (..., 3, 3). extrinsic_matrix (Optional[BArrayType]): The camera extrinsic matrix of shape (..., 3, 4) or (..., 4, 4). If None, assume identity matrix. Returns: BArrayType: The pixel coordinates xy of shape (..., N, 2). BArrayType: The depth values of shape (..., N). Invalid points (where valid mask is False) will have depth 0.

world_to_depth

world_to_depth(
    backend: ComputeBackend[BArrayType, BDeviceType, BDtypeType, BRNGType],
    world_coords: BArrayType,
    extrinsic_matrix: Optional[BArrayType] = None,
) -> BArrayType

Convert world coordinates to pixel coordinates and depth. Args: backend (ComputeBackend): The compute backend to use. world_coords (BArrayType): The world coordinates of shape (..., N, 3) or (..., N, 4). If the last dimension is 4, the last element is treated as a valid mask. extrinsic_matrix (Optional[BArrayType]): The camera extrinsic matrix of shape (..., 3, 4) or (..., 4, 4). If None, assume identity matrix. Returns: BArrayType: The depth values of shape (..., N). Invalid points (where valid mask is False) will have depth 0.

farthest_point_sampling

farthest_point_sampling(
    backend: ComputeBackend[BArrayType, BDeviceType, BDtypeType, BRNGType],
    points: BArrayType,
    num_samples: int,
    rng: BRNGType,
    points_valid: Optional[BArrayType] = None,
) -> Tuple[BRNGType, BArrayType, Optional[BArrayType]]

Perform farthest point sampling on a set of points. Args: backend (ComputeBackend): The compute backend to use. points (BArrayType): The input points of shape (..., N, D). num_samples (int): The number of points to sample. rng (BRNGType): The random number generator. points_valid (Optional[BArrayType]): A boolean mask of shape (..., N) indicating valid points. If None, all points are considered valid. Returns: BRNGType: The updated random number generator. BArrayType: The indices of the sampled points of shape (..., num_samples). Optional[BArrayType]: The valid mask of shape (..., num_samples). Returned only if points_valid is provided.

random_point_sampling

random_point_sampling(
    backend: ComputeBackend[BArrayType, BDeviceType, BDtypeType, BRNGType],
    points: BArrayType,
    num_samples: int,
    rng: BRNGType,
    points_valid: Optional[BArrayType] = None,
) -> Tuple[BRNGType, BArrayType, Optional[BArrayType]]

Perform random point sampling on a set of points. Args: backend (ComputeBackend): The compute backend to use. points (BArrayType): The input points of shape (..., N, D). num_samples (int): The number of points to sample. rng (BRNGType): The random number generator. points_valid (Optional[BArrayType]): A boolean mask of shape (..., N) indicating valid points. If None, all points are considered valid. Returns: BRNGType: The updated random number generator. BArrayType: The indices of the sampled points of shape (..., num_samples). Optional[BArrayType]: The valid mask of shape (..., num_samples). Returned only if points_valid is provided.

ball_query

ball_query(
    backend: ComputeBackend[BArrayType, BDeviceType, BDtypeType, BRNGType],
    query_points: BArrayType,
    points: BArrayType,
    num_neighbors: int,
    radius: float,
    query_valid: Optional[BArrayType] = None,
    points_valid: Optional[BArrayType] = None,
) -> Tuple[BArrayType, BArrayType, BArrayType]

Find, for every query point, up to num_neighbors points strictly within radius.

The neighbors are the first ones in index order, not the nearest ones. Args: backend (ComputeBackend): The compute backend to use. query_points (BArrayType): The query points of shape (..., Q, D). points (BArrayType): The points to search, of shape (..., N, D). num_neighbors (int): The most neighbors to return per query point. radius (float): The radius of the ball; a point exactly at this distance is excluded. query_valid (Optional[BArrayType]): A boolean mask of shape (..., Q); invalid queries have no neighbors. points_valid (Optional[BArrayType]): A boolean mask of shape (..., N); invalid points are never neighbors. Returns: BArrayType: The neighbor indices of shape (..., Q, num_neighbors), 0 where there is no neighbor. BArrayType: The squared distances of shape (..., Q, num_neighbors), 0 where there is no neighbor. BArrayType: The valid mask of shape (..., Q, num_neighbors); each query's neighbors come first.

pointcloud_jitter

pointcloud_jitter(
    backend: ComputeBackend[BArrayType, BDeviceType, BDtypeType, BRNGType],
    points: BArrayType,
    sigma: float,
    rng: BRNGType,
    clip: Optional[float] = None,
) -> Tuple[BRNGType, BArrayType]

Apply jitter noise to point cloud. Args: backend (ComputeBackend): The compute backend to use. points (BArrayType): The input points of shape (..., N, D) where D >= 3. sigma (float): The standard deviation of the jitter noise. rng (BRNGType): The random number generator. clip (Optional[float]): The maximum absolute value of the jitter noise. Returns: BRNGType: The updated random number generator. BArrayType: The jittered points of shape (..., N, D).