from Hacker News

rl8 focuses on end-to-end reinforcement learning (all data and computations occur on one device) for infinite horizon tasks (tasks that don't finish within a finite number of steps). This niche gives a few benefits:

- End-to-end compute means there's no time wasted copying data between devices (more speed)

- Infinite horizon means learning buffers can be pre-allocated and written to directly (fixed memory, more speed)

- The API can be minimal (less cognitive load and more customizability)

Not all environments can be rewritten to be vectorized, and not all tasks can be reframed to be infinite horizon, so I imagine the number of people this would benefit is small. But, for those whose use cases do fit this niche like myself, it drastically reduces experiment times.

Any feedback would be much appreciated!