domains / science / rl-environment-training / pufferlib

pufferlib

Skill card

combined

The two skills cover the same library but target fundamentally different versions and workflows. The first skill (K-Dense-AI) is a version-aware, security-focused guide that distinguishes between the native 5.0 C/CUDA build, the 3.0.0 PyPI release, and a pinned 4.0 source, emphasizing safe defaults, provenance verification, and synthetic CPU-only checks. The second skill (Vibe-Skills) is a practical, high-performance guide focused on the 3.0/4.0 era Python API, providing concrete code examples for PuffeRL training, PufferEnv development, and vectorization. They disagree on the primary API (native C vs Python/Gymnasium) and the recommended entry points (safe synthetic checks vs direct training loops). Combining them is necessary to provide a complete picture that includes both the security/provenance constraints of the first and the practical implementation details of the second.

Repositories

Other copies

Combined skill

---
name: pufferlib
description: Version-aware guidance for PufferLib reinforcement-learning environments, covering safe provenance checks, native 5.0 C/CUDA builds, and Python 3.0/4.0 training workflows.
license: MIT
---

# PufferLib

## 1. Version Selection and Provenance

Before choosing an API, identify the target version profile:

| Profile | Status | Main Use | Installation |
|---|---|---|---|
| Native 5.0 | Current default | C/CUDA environments, native trainer | Source build (C/CUDA toolchain required) |
| PyPI 3.0.0 | Latest PyPI | Python/Gymnasium/PettingZoo, Torch PuffeRL | `uv add pufferlib==3.0.0` (sdist only) |
| Pinned 4.0 | Historical | C Ocean interface, Torch fallback | Git pin to specific commit |

**Safe Defaults:**
1. Verify official source, immutable revision, and checksums.
2. Sandbox native builds and first execution.
3. Do not import arbitrary environments by dotted path; use allowlisted built-ins.
4. Cap steps, environments, and memory.
5. Keep training and evaluation seeds separate.

## 2. Installation

### For PyPI 3.0.0 (Python API)
```bash
uv venv --python 3.11
uv add --exact --no-sync "pufferlib==3.0.0"
uv lock
uv sync --frozen
```
*Note: Requires NumPy <2 and Gymnasium <=0.29.1.*

### For Native 5.0
Build from source using the C/CUDA toolchain. The CLI is `./puffer train` after building the environment.

## 3. Environment Development (Python API)

Use the `PufferEnv` API for custom environments.

```python
import numpy as np
from pufferlib import PufferEnv

class MyEnvironment(PufferEnv):
    def __init__(self, buf=None):
        super().__init__(buf)
        self.observation_space = self.make_space((4,))
        self.action_space = self.make_discrete(4)
        self.reset()

    def reset(self):
        return np.zeros(4, dtype=np.float32)

    def step(self, action):
        obs = self._get_observation()
        reward = self._compute_reward()
        done = self._is_done()
        info = {}
        return obs, reward, done, info
```

*For 5.0 Native, refer to the C Ocean interface documentation.*

## 4. Training (PuffeRL)

### Python Training Loop (3.0/4.0)
```python
import pufferlib
from pufferlib import PuffeRL

env = pufferlib.make('procgen-coinrun', num_envs=256)
trainer = PuffeRL(
    env=env,
    policy=my_policy,
    device='cuda',
    learning_rate=3e-4,
    batch_size=32768
)

for iteration in range(num_iterations):
    trainer.evaluate()
    trainer.train()
    trainer.mean_and_log()
```

### CLI Training (3.0/4.0)
```bash
puffer train procgen-coinrun --train.device cuda --train.learning-rate 3e-4
```

## 5. Vectorization and Performance

- **Python Envs:** 100k-500k steps/sec.
- **C Envs:** 100M+ steps/sec.
- **Optimizations:** Use shared memory buffers, busy-wait flags, and surplus environments for async returns.

## 6. Safety and Checkpoints

- Hash checkpoint bytes before loading.
- Never pass W&B or Neptune credentials via CLI or environment variables.
- Default logging to local/none; external logging requires explicit opt-in.