from Hacker News

Kraken: 3x faster decompression than zlib

by dahjelle on 4/27/16, 7:49 PM with 68 comments

• by rb2k_ on 4/27/16, 11:46 PM

  Another interesting compression project is "Zstandard"

  Open source, high performance, decent compression and a "dictionary mode" that's especially fun when it comes to compressing a lot of small data.

  http://www.zstd.net

  Previous discussion over at https://news.ycombinator.com/item?id=8941955

• by Johnie on 4/27/16, 9:37 PM

  There's too many projects named Kraken:

  * http://krakenjs.com/ - Javascript framework

  * https://www.kraken.com/ - BTC exchange

  * https://kraken.io/ - Image Optimizer

• by userbinator on 4/28/16, 3:42 AM

  I guess it's yet another optimised LZ variant, and it's no surprise that LZ-based/inspired algorithms are fast, because their main loop is very simple and does not involve the intense bit manipulation of Huffman/Arithmetic algorithms. That's how very carefully optimised LZ can end up being faster than memcpy() in some cases:

  http://www.oldskool.org/pc/lz4_8088

• by mmozeiko on 4/27/16, 10:05 PM

  Few benchmarks from @rygorous:

  https://gist.github.com/rygorous/67c78fde615e078117ea1870472... (on PC)

  https://gist.github.com/rygorous/9563760e24463a37984c117b572... (on PS4)

• by EvgeniyZh on 4/27/16, 10:42 PM

  How is it compared to Zstandard https://github.com/Cyan4973/zstd ?

• by muizelaar on 4/27/16, 9:15 PM

  Unfortunately not open source.

• by kazinator on 4/27/16, 10:50 PM

  Is this 3x faster than zlib at decompressing on zlib-compressed data, or 3x faster on samples compressed with its own format versus the same samples compressed with zlib?

• by Jerry2 on 4/28/16, 12:48 AM

  Does anyone know how Apple's lzfse [0] compares to some of these other compression algos?

  >Beginning with iOS 9 and OSX 10.11 El Capitan, we provide Apple’s proprietary compression algorithm, LZFSE. LZFSE is a new algorithm, matching the compression ratio of ZLIB level 5, but with much higher energy efficiency and speed (between 2x and 3x) for both encode and decode operations.

  I'm interested because of the low energy characteristics...

  [0] : https://developer.apple.com/library/tvos/documentation/Perfo...

• by MichaelGG on 4/27/16, 9:29 PM

  What about LZHAM; I thought that was the top "slow compress, good ratio, fast decompress" contender for games and such?

• by iso-8859-1 on 4/28/16, 8:56 AM

  It is no secret that vanilla zlib is slow. Would have been interesting if they compared with https://github.com/Dead2/zlib-ng

• by noir_lord on 4/27/16, 10:07 PM

  Is the speed difference algorithmic, implementation optimization or a mix of both?

• by stellar2 on 4/28/16, 6:30 AM

  Looking at the figures, it seems to have roughly the same performance as Zstandard in high compression mode.

• by Gratsby on 4/27/16, 10:05 PM

  But what is it's Weissman score?