« Run MbUnit v2 tests under .NET 4 | Main | How to Crash Many WPF Applications (WPF 4 Edition) »

08 June 2012

Always wrap GZipStream with BufferedStream

GZipStream and DeflateStream don't buffer their input, so they will only compress the individual chunks that are passed to Write (or, worst-case, WriteByte). Unless you can guarantee that large blocks of data are being passed to GZipStream.Write, always wrap GZipStream in a BufferedStream when using CompressionMode.Compress.

In fact, if you call only WriteByte on a compressing GZipStream, it will create “compressed” output that's almost double the size of the input.

Experimentation determined that the optimal buffer size is 8K; above this, no further compression is gained. So, to create a GZipStream, use a method similar to the following:

public static Stream CreateCompressingGZipStream(Stream stream, bool leaveOpen)
    return new BufferedStream(
        new GZipStream(stream, CompressionMode.Compress, leaveOpen),

This chart shows how the compressed output keeps getting smaller as the buffer size increases (and how writing just one or two bytes at a time almost doubles the 1MB input):

GZipStream output size versus buffer size

// determine best buffer size
foreach (int bufferSize in new[] { 1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 1536,
    2048, 4096, 6144, 8192, 16384, 32768, 65536, 131072, 262144 })
    using (MemoryStream output = new MemoryStream())
        using (GZipStream compressor = new GZipStream(output, CompressionMode.Compress,
            leaveOpen : true))
        using (BufferedStream buffer = new BufferedStream(compressor, bufferSize))
            // write simple data that will compress easily
            for (int i = 0; i < 1000000; i++)

        Console.WriteLine("Buffer Size: {0:n0}, Compressed size: {1:n0}",
            bufferSize, output.Length);

Posted by Bradley Grainger at June 08, 2012 09:55 PM