from
Hacker News
Top
New
Ask HN: How to store KV cache efficiently on disk?
by
ban-lan-gen
on 3/14/25, 8:22 PM with 0 comments
If I have a lot of precomputed KV cache values (basically just tensors), how do I store them efficiently on disk? Ideally, I would want to lookup their values by prefix quickly as well.