Compressor "firn" improves ZSTD by 13% (opens in new tab)

(github.com)

3 pointsseccode1y ago3 comments

3 comments

This works by text substitution, I imagine that you could get similar or better gains using zstd custom dictionaries.

This is a direct comparison to zstd with a dictionary actually

Methodology for comparison: train zstd dictionary on enwik9. Then build my dictionary as most common words in enwik9. Mine does 13% better because of the way I discovered how you can generate dictionary replacement symbols

j / k navigate · click thread line to collapse

3 comments

themerone1y ago

This works by text substitution, I imagine that you could get similar or better gains using zstd custom dictionaries.

seccodeOP1y ago

This is a direct comparison to zstd with a dictionary actually

seccodeOP1y ago

j / k navigate · click thread line to collapse