Faster zlib alternatives
Solution 1:
Several years after this question was first asked some faster x86_64 zlib's appeared using the type of optimization BarsMonster suggested:
- Cloudflare zlib
- Intel zlib
- Intel's igzip
- Google's c-zlib
- libdeflate
The authors of samtools (a suite of tools for interacting with high-throughput sequencing data) produced a comparison of zlib speeds.
zlib-ng collects zlib optimizations not in mainline zlib but it may not be as stable as mainline zlib. Its issue tracker is also instructive as a reference to other known zlib speed ups.
Recently a pull request that claims to "speed up inflate_fast by up to 1.5x on x86_64" (by filling and copying 8 bytes at a time) has been made but not accepted into mainline zlib. The patch providing this work is also having a bumpy ride being accepted into Chromium (see this Chromium bug and the Chromium review) but hopefully the submitter will take a break to step away and recharge because from a distance it looks like progress was being very slowly made in a complex situation...
Even more recently, libdeflate is a heavily optimised rewrite that produces/confumes zlib compatible output/input but is does not provide a zlib compatible API. At the time of writing this has the zlib speed crown.
Solution 2:
Another alternative is the miniz library (public domain, unlicense.org), which implements most of the zlib API in a single C source file and reads/writes zlib-compatible compressed data streams. At compression level 1 it uses a real-time compressor which is extremely fast (a little slower than minilzo, but with higher compression).
Solution 3:
Not a rebuild, but two good alternatives to zlib are quicklz and fastlz. Both are very fast relative to gzip -1, but do not achieve as good of a compression ratio. For my application, size was going up 10-15% but compression speed was 6x, so a very good tradeoff.
Of course neither is compatible with zlib, so it may not work for you.