unienv_data.storages.pytorch¶
Classes:
| Name | Description |
|---|---|
PytorchTensorStorage |
|
PytorchTensorStorage
¶
PytorchTensorStorage(
single_instance_space: Union[
BoxSpace[PyTorchArrayType, PyTorchDeviceType, PyTorchDtypeType, PyTorchRNGType],
BinarySpace[
PyTorchArrayType, PyTorchDeviceType, PyTorchDtypeType, PyTorchRNGType
],
],
data: Union[Tensor, MemoryMappedTensor],
mutable: bool = True,
)
Bases: SpaceStorage[PyTorchArrayType, PyTorchArrayType, PyTorchDeviceType, PyTorchDtypeType, PyTorchRNGType]
Methods:
| Name | Description |
|---|---|
create |
|
load_from |
|
get |
|
set |
|
append |
|
clear |
|
dumps |
|
close |
|
get_segments |
|
mark_segment_start |
|
mark_segment_end |
|
abort_segment |
|
extend_length |
This is used by capacity = None storages to extend the length of the storage |
shrink_length |
This is used by capacity = None storages to shrink the length of the storage |
open_segment_writer |
A writer that builds one segment of this storage off to the side; see |
Attributes:
| Name | Type | Description |
|---|---|---|
backend |
|
|
single_file_ext |
str
|
|
data |
|
|
device |
Optional[PyTorchDeviceType]
|
|
cache_filename |
Optional[Union[str, PathLike]]
|
|
is_mutable |
bool
|
|
is_multiprocessing_safe |
bool
|
|
capacity |
int
|
|
is_memmap |
bool
|
|
single_instance_space |
|
|
has_open_segment |
bool
|
|
pending_segment_length |
int
|
|
create
classmethod
¶
create(
single_instance_space: BoxSpace[
PyTorchArrayType, PyTorchDeviceType, PyTorchDtypeType, PyTorchRNGType
],
*,
capacity: Optional[int],
is_memmap: bool = True,
cache_path: Optional[str] = None,
multiprocessing: bool = False,
memmap_existok: bool = True,
) -> PytorchTensorStorage
load_from
classmethod
¶
load_from(
path: Union[str, PathLike],
single_instance_space: Union[
BoxSpace[PyTorchArrayType, PyTorchDeviceType, PyTorchDtypeType, PyTorchRNGType],
BinarySpace[
PyTorchArrayType, PyTorchDeviceType, PyTorchDtypeType, PyTorchRNGType
],
],
*,
is_memmap: bool = True,
capacity: Optional[int] = None,
read_only: bool = True,
multiprocessing: bool = False,
**kwargs,
) -> PytorchTensorStorage
extend_length
¶
This is used by capacity = None storages to extend the length of the storage If this is called on a storage with a fixed capacity, we will simply ignore the call.
shrink_length
¶
This is used by capacity = None storages to shrink the length of the storage If this is called on a storage with a fixed capacity, we will simply ignore the call.
open_segment_writer
¶
open_segment_writer() -> SegmentWriter
A writer that builds one segment of this storage off to the side; see SegmentWriter.
Unlike mark_segment_start() / append(), any number of writers can be open at a time.
The default keeps the samples in memory until the commit; storages override it to stream instead.