from Hacker News

WASM compression benchmarks and the cost of missing compression APIs

by comagoosie on 2/1/23, 11:23 AM with 21 comments

  • by miohtama on 2/2/23, 10:42 PM

    Related to compressing data before storing on SSD:

    Blosc - faster than memcpy()

    https://github.com/Blosc/c-blosc

    On right circumstances Blosc is so fast that even speed ups reading data from RAM (read less, decompress in L1 and L2 caches)

  • by dorfsmay on 2/2/23, 10:23 PM

    Note that both the x and the y axis for Chrome vs Firefox are significantly different.

  • by modeless on 2/2/23, 10:45 PM

    The web platform could really use zstd everywhere. As a content encoding for HTTP, and as an API available to JS/wasm. It's really clumsy to use a wasm version of zstd in a JS application because it's hard to get data in and out of wasm efficiently.

  • by royjacobs on 2/3/23, 9:21 AM

    Just to add a data point, I've written a tiny Rust library [0] that can compile to WASM, that is quite effective for smaller payloads. It is based on PAQ so memory usage explodes a bit once you start compressing large files, but on smaller files it is super competitive.

    As far as I can tell the blog author didn't include the 91MB file they used to test the compressors with, so I couldn't give it a try to see how it holds up. I guess 91MB would be too big anyway.

    [0] https://github.com/datatrash/mashi

  • by zokier on 2/2/23, 9:52 PM

    just for scale it would have been fun to see zstd/lz4 native (=non-wasm) performance

  • by Const-me on 2/2/23, 11:55 PM

    I wonder how this compares to the OS built-in NTFS compression?

    Windows shell has "Compress contents to save disk space" checkbox in folder properties. Usually, that compressed flag is inherited by new files created in a folder with that checkbox. OP can probably set the flag on Default\IndexedDB or Default\Service Worker folder and see whether this changes the results of that IO benchmark.

  • by Jap2-0 on 2/3/23, 6:41 AM

    A (hopefully) intriguing tangent: at the linked https://bench.nickb.dev/, a 10x increase in iterations on the allocation benchmark results in a change from 28ms to 280ms in Firefox but 44ms to 2800ms in Edge.

  • by mkesper on 2/3/23, 7:38 AM

    Note to author: 120ms is not 66% less than 200ms but 40%.