unienv_data.base.common¶
Classes:
| Name | Description |
|---|---|
BatchBase |
Base interface for dataset-like collections of structured samples. |
BatchSampler |
Read-only batch wrapper that samples mini-batches from another batch. |
Functions:
| Name | Description |
|---|---|
convert_index_to_backendarray |
Normalize a Python index into a 1-D backend integer array. |
Attributes:
| Name | Type | Description |
|---|---|---|
IndexableType |
|
|
BatchT |
|
BatchBase
¶
BatchBase(
single_space: Space[BatchT, BDeviceType, BDtypeType, BRNGType],
single_metadata_space: Optional[
DictSpace[BDeviceType, BDtypeType, BRNGType]
] = None,
)
Bases: ABC, Generic[BatchT, BArrayType, BDeviceType, BDtypeType, BRNGType]
Base interface for dataset-like collections of structured samples.
BatchBase stores the per-sample space definition and provides a uniform
API for retrieving either structured values or their flattened backend-array
representation. Concrete subclasses decide where data lives and which
mutation operations are supported.
Initialize the batch contract for one logical sample.
Methods:
| Name | Description |
|---|---|
get_flattened_at |
Fetch samples as flattened backend arrays. |
get_flattened_at_with_metadata |
Fetch flattened samples together with optional per-sample metadata. |
set_flattened_at |
Overwrite existing samples using flattened data. |
append_flattened |
Append one flattened sample to the batch. |
extend_flattened |
Append a batched block of flattened samples. |
get_at |
Fetch samples in their structured form. |
get_at_with_metadata |
Fetch structured samples together with optional metadata. |
set_at |
Overwrite existing samples using structured data. |
remove_at |
Remove one or more samples from the batch. |
append |
Append one structured sample to the batch. |
extend |
Append a batched block of structured samples. |
extend_from |
Copy data from another batch in bounded-size chunks. |
get_slice |
Create a lazy view over a subset of indices. |
get_column |
Create a lazy view over a nested field inside each sample. |
close |
|
Attributes:
| Name | Type | Description |
|---|---|---|
is_mutable |
bool
|
|
single_space |
|
|
single_metadata_space |
|
|
backend |
ComputeBackend[BArrayType, BDeviceType, BDtypeType, BRNGType]
|
|
device |
Optional[BDeviceType]
|
|
get_flattened_at
¶
get_flattened_at(idx: Union[IndexableType, BArrayType]) -> BArrayType
Fetch samples as flattened backend arrays.
get_flattened_at_with_metadata
¶
get_flattened_at_with_metadata(
idx: Union[IndexableType, BArrayType],
) -> Tuple[BArrayType, Optional[Dict[str, Any]]]
Fetch flattened samples together with optional per-sample metadata.
set_flattened_at
¶
set_flattened_at(idx: Union[IndexableType, BArrayType], value: BArrayType) -> None
Overwrite existing samples using flattened data.
append_flattened
¶
Append one flattened sample to the batch.
extend_flattened
¶
Append a batched block of flattened samples.
get_at
¶
get_at(idx: Union[IndexableType, BArrayType]) -> BatchT
Fetch samples in their structured form.
get_at_with_metadata
¶
get_at_with_metadata(
idx: Union[IndexableType, BArrayType],
) -> Tuple[BatchT, Optional[Dict[str, Any]]]
Fetch structured samples together with optional metadata.
set_at
¶
set_at(idx: Union[IndexableType, BArrayType], value: BatchT) -> None
Overwrite existing samples using structured data.
remove_at
¶
remove_at(idx: Union[IndexableType, BArrayType]) -> None
Remove one or more samples from the batch.
extend_from
¶
extend_from(
other: BatchBase[BatchT, BArrayType, BDeviceType, BDtypeType, BRNGType],
chunk_size: int = 8,
tqdm: bool = False,
) -> None
Copy data from another batch in bounded-size chunks.
get_slice
¶
get_slice(
idx: Union[IndexableType, BArrayType],
) -> BatchBase[BatchT, BArrayType, BDeviceType, BDtypeType, BRNGType]
Create a lazy view over a subset of indices.
BatchSampler
¶
BatchSampler(
single_space: Space[BatchT, BDeviceType, BDtypeType, BRNGType],
single_metadata_space: Optional[
DictSpace[BDeviceType, BDtypeType, BRNGType]
] = None,
batch_size: int = 1,
)
Bases: Generic[SamplerBatchT, SamplerArrayType, SamplerDeviceType, SamplerDtypeType, SamplerRNGType, BatchT, BArrayType, BDeviceType, BDtypeType, BRNGType], BatchBase[SamplerBatchT, SamplerArrayType, SamplerDeviceType, SamplerDtypeType, SamplerRNGType]
Read-only batch wrapper that samples mini-batches from another batch.
Configure the sampling batch shape without binding the source data yet.
Methods:
| Name | Description |
|---|---|
manual_seed |
Reset sampler RNG state, including the optional data-index RNG. |
sample_index |
Draw random indices for one batch. |
sample_flat |
Sample a batch and return it in flattened form. |
sample_flat_with_metadata |
Sample a flattened batch together with optional metadata. |
sample |
Sample a batch in its structured form. |
sample_with_metadata |
Sample a structured batch together with optional metadata. |
epoch_iter |
Iterate once over a random permutation of the source data. |
epoch_iter_with_metadata |
Iterate once over shuffled structured batches plus metadata. |
epoch_flat_iter |
Iterate once over shuffled batches in flattened form. |
epoch_flat_iter_with_metadata |
Iterate once over shuffled flattened batches plus metadata. |
close |
|
get_flattened_at |
Fetch samples as flattened backend arrays. |
get_flattened_at_with_metadata |
Fetch flattened samples together with optional per-sample metadata. |
set_flattened_at |
Overwrite existing samples using flattened data. |
append_flattened |
Append one flattened sample to the batch. |
extend_flattened |
Append a batched block of flattened samples. |
get_at |
Fetch samples in their structured form. |
get_at_with_metadata |
Fetch structured samples together with optional metadata. |
set_at |
Overwrite existing samples using structured data. |
remove_at |
Remove one or more samples from the batch. |
append |
Append one structured sample to the batch. |
extend |
Append a batched block of structured samples. |
extend_from |
Copy data from another batch in bounded-size chunks. |
get_slice |
Create a lazy view over a subset of indices. |
get_column |
Create a lazy view over a nested field inside each sample. |
Attributes:
| Name | Type | Description |
|---|---|---|
data |
BatchBase[BatchT, BArrayType, BDeviceType, BDtypeType, BRNGType]
|
|
rng |
Optional[SamplerRNGType]
|
|
data_rng |
Optional[BRNGType]
|
|
is_mutable |
bool
|
|
batch_size |
|
|
sampled_space |
Space[SamplerBatchT, SamplerDeviceType, SamplerDtypeType, SamplerRNGType]
|
|
sampled_metadata_space |
Optional[DictSpace[SamplerDeviceType, SamplerDtypeType, SamplerRNGType]]
|
|
single_space |
|
|
single_metadata_space |
|
|
backend |
ComputeBackend[BArrayType, BDeviceType, BDtypeType, BRNGType]
|
|
device |
Optional[BDeviceType]
|
|
sampled_space
property
¶
sampled_space: Space[SamplerBatchT, SamplerDeviceType, SamplerDtypeType, SamplerRNGType]
sampled_metadata_space
property
¶
sampled_metadata_space: Optional[
DictSpace[SamplerDeviceType, SamplerDtypeType, SamplerRNGType]
]
manual_seed
¶
Reset sampler RNG state, including the optional data-index RNG.
sample_flat_with_metadata
¶
sample_flat_with_metadata() -> Tuple[SamplerArrayType, Optional[Dict[str, Any]]]
Sample a flattened batch together with optional metadata.
sample_with_metadata
¶
sample_with_metadata() -> Tuple[SamplerBatchT, Optional[Dict[str, Any]]]
Sample a structured batch together with optional metadata.
epoch_iter
¶
epoch_iter() -> Iterator[SamplerBatchT]
Iterate once over a random permutation of the source data.
epoch_iter_with_metadata
¶
epoch_iter_with_metadata() -> Iterator[Tuple[SamplerBatchT, Optional[Dict[str, Any]]]]
Iterate once over shuffled structured batches plus metadata.
epoch_flat_iter
¶
epoch_flat_iter() -> Iterator[SamplerArrayType]
Iterate once over shuffled batches in flattened form.
epoch_flat_iter_with_metadata
¶
epoch_flat_iter_with_metadata() -> Iterator[
Tuple[SamplerArrayType, Optional[Dict[str, Any]]]
]
Iterate once over shuffled flattened batches plus metadata.
get_flattened_at
¶
get_flattened_at(idx: Union[IndexableType, BArrayType]) -> BArrayType
Fetch samples as flattened backend arrays.
get_flattened_at_with_metadata
¶
get_flattened_at_with_metadata(
idx: Union[IndexableType, BArrayType],
) -> Tuple[BArrayType, Optional[Dict[str, Any]]]
Fetch flattened samples together with optional per-sample metadata.
set_flattened_at
¶
set_flattened_at(idx: Union[IndexableType, BArrayType], value: BArrayType) -> None
Overwrite existing samples using flattened data.
append_flattened
¶
Append one flattened sample to the batch.
extend_flattened
¶
Append a batched block of flattened samples.
get_at
¶
get_at(idx: Union[IndexableType, BArrayType]) -> BatchT
Fetch samples in their structured form.
get_at_with_metadata
¶
get_at_with_metadata(
idx: Union[IndexableType, BArrayType],
) -> Tuple[BatchT, Optional[Dict[str, Any]]]
Fetch structured samples together with optional metadata.
set_at
¶
set_at(idx: Union[IndexableType, BArrayType], value: BatchT) -> None
Overwrite existing samples using structured data.
remove_at
¶
remove_at(idx: Union[IndexableType, BArrayType]) -> None
Remove one or more samples from the batch.
extend_from
¶
extend_from(
other: BatchBase[BatchT, BArrayType, BDeviceType, BDtypeType, BRNGType],
chunk_size: int = 8,
tqdm: bool = False,
) -> None
Copy data from another batch in bounded-size chunks.
get_slice
¶
get_slice(
idx: Union[IndexableType, BArrayType],
) -> BatchBase[BatchT, BArrayType, BDeviceType, BDtypeType, BRNGType]
Create a lazy view over a subset of indices.
convert_index_to_backendarray
¶
convert_index_to_backendarray(
backend: ComputeBackend[BArrayType, BDeviceType, BDtypeType, BRNGType],
index: IndexableType,
length: int,
device: Optional[BDeviceType] = None,
) -> BArrayType
Normalize a Python index into a 1-D backend integer array.
This helper lets batch implementations accept int, slice, and
... while still delegating storage access through a single backend-aware
indexing path.