Robot environments, policies and data, unified.
Simulators, real robots and datasets describe their data with the same spaces. Policies, buffers and tools written against those spaces work with every source.
Three sources
One observation space
robot ├ arm │ ├ joint_position (7,)float32rad │ ├ joint_velocity (7,)float32rad/s │ ├ eef_position (3,)float32m │ └ eef_quaternion (4,)float32quat_wxyz └ gripper … external_cam_0 ├ rgb (H,W,3)uint8image · HWC ├ depth (H,W)float32depth · m └ …
Unchanged code
A Franka FR3 with a Robotiq gripper: simulated with unienv-genesis, driven live with unienv-droid, or replayed from the DROID dataset. Select a source.
What's inside
Five layers, each usable on its own.
Use only the layer you need. They share the same space definitions, so they compose.
Spaces
Typed, nested data contracts with metadata. Any space converts between NumPy, PyTorch and JAX, and between devices.
BoxSpaceDictSpacespace.to()
Environments & worlds
Stateful and functional envs, plus worlds assembled from reusable robot, sensor and scene nodes.
EnvFuncEnvWorldWorldNode
Policies
Stateful and functional policies that act against the env's spaces, with wrappers that stack like env wrappers.
PolicyFuncPolicy
Data
Replay buffers on Parquet, HDF5, image and video storages, with samplers and LeRobot / Hugging Face adapters.
ReplayBufferTransformedBatch
Remote
Host any env, world, node or policy over TCP, Unix sockets or WebSocket. Clients get proxies with the same interface.
RemoteServerRemoteClient
Quick start
From a space to a running loop.
- Describe your dataBuild the observation and action spaces once.
- Wrap the sourceA simulator, robot or dataset adaptor exposes those spaces.
- Write everything else oncePolicies, buffers and remote serving read the spaces, not the source.
import numpy as np from unienv_interface.backends.numpy import NumpyComputeBackend as B from unienv_interface.space.spaces import BoxSpace, DictSpace from unienv_data.replay_buffer import ReplayBuffer from unienv_data.storages.parquet import ParquetStorage space = DictSpace(B, { "obs": BoxSpace(B, 0.0, 1.0, np.float32, shape=(4,)), "action": BoxSpace(B, -1.0, 1.0, np.float32, shape=(2,)), }) buffer = ReplayBuffer.create( ParquetStorage, space, cache_path="cache/demo", capacity=10_000, )
import numpy as np from unienv_interface.backends.numpy import NumpyComputeBackend as B from unienv_interface.space import BoxSpace from unienv_interface.policy import Policy class RandomPolicy(Policy): # the policy declares what it reads and what it outputs backend, device = B, None observation_space = BoxSpace(B, -np.pi, np.pi, np.float32, shape=(7,)) action_space = BoxSpace(B, -1.0, 1.0, np.float32, shape=(7,)) def __init__(self, seed=0): self.rng = B.random.random_number_generator(seed) # state comes from reset / observe_transition; a random policy keeps none def reset(self, context, observation, **kwargs): pass def observe_transition(self, action_taken, new_observation): pass def get_action(self, **kwargs): return self.sample_action() policy = RandomPolicy() # or one served remotely: client.policy("pi") context, obs, info = env.reset(seed=0) policy.reset(context, obs) for _ in range(100): action = policy.get_action() obs, reward, terminated, truncated, info = env.step(action) policy.observe_transition(action, obs)
from unienv_interface.remote import RemoteServer, RemoteClient # on the robot with RemoteServer() as server: server.register("robot", make_env()) server.serve("tcp://0.0.0.0:7000") # on the workstation: a proxy with the same Env interface with RemoteClient.connect("tcp://robot-pc:7000") as client: env = client.env("robot") context, obs, info = env.reset(seed=42)
from unienv_interface.backends.pytorch import PyTorchComputeBackend # the same space, now producing torch tensors on the GPU torch_space = space.to(PyTorchComputeBackend, device="cuda") rng, sample = torch_space.sample(rng)
Performance
Zero-copy where data moves, tuned where it is stored.
Tensors cross frameworks and processes without being copied, and the storages are benchmarked on robot-sized data: camera frames, proprioception, long episodes.
Zero-copy across backends
space.data_to(x, backend=...) hands arrays between frameworks through DLPack: the result is a view of the same memory.
Shared-memory vectorized envs
AsyncVecEnv workers write observations into one shared buffer that the parent reads in place; CUDA envs share GPU memory over CUDA IPC.
Storage tuned on robot data
HDF5, Parquet, video and LeRobot storages are profiled on camera frames and long episodes, and read only the rows and columns a batch needs.
array_encoding="binary"Speedups are against UniEnv's previous implementation of the same feature, measured on one workstation; absolute times depend on the machine. Reproduce them with the scripts in benchmarks/.
Ecosystem
Adaptors for the tools you already use.
Every adaptor speaks UniEnv spaces, so it plugs into the same wrappers, buffers and remote layer.
Core libraries
Simulation
Node-based simulation framework for Genesis: robots, end effectors, scenes, reference policies.
ManiSkill 3 environments as UniEnv envs.
Environment adaptor for mjlab.
Environment adaptor for MuJoCo Playground.
LIBERO tasks and demonstration datasets in one observation layout.
RLBench tasks, with the language instruction as context.
Cite
Using UniEnv in research?
Please cite the software. XBArray is developed as part of UniEnv and shares this citation.