« Run MbUnit v2 tests under .NET 4 | Main | How to Crash Many WPF Applications (WPF 4 Edition) »

08 June 2012

Always wrap GZipStream with BufferedStream

GZipStream and DeflateStream don't buffer their input, so they will only compress the individual chunks that are passed to Write (or, worst-case, WriteByte). Unless you can guarantee that large blocks of data are being passed to GZipStream.Write, always wrap GZipStream in a BufferedStream when using CompressionMode.Compress.

In fact, if you call only WriteByte on a compressing GZipStream, it will create “compressed” output that's almost double the size of the input.

Experimentation determined that the optimal buffer size is 8K; above this, no further compression is gained. So, to create a GZipStream, use a method similar to the following:

public static Stream CreateCompressingGZipStream(Stream stream, bool leaveOpen)
{
    return new BufferedStream(
        new GZipStream(stream, CompressionMode.Compress, leaveOpen),
        8192);
}

This chart shows how the compressed output keeps getting smaller as the buffer size increases (and how writing just one or two bytes at a time almost doubles the 1MB input):

GZipStream output size versus buffer size

// determine best buffer size
foreach (int bufferSize in new[] { 1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 1536,
    2048, 4096, 6144, 8192, 16384, 32768, 65536, 131072, 262144 })
{
    using (MemoryStream output = new MemoryStream())
    {
        using (GZipStream compressor = new GZipStream(output, CompressionMode.Compress,
            leaveOpen : true))
        using (BufferedStream buffer = new BufferedStream(compressor, bufferSize))
        {
            // write simple data that will compress easily
            for (int i = 0; i < 1000000; i++)
                buffer.WriteByte(0);
        }

        Console.WriteLine("Buffer Size: {0:n0}, Compressed size: {1:n0}",
            bufferSize, output.Length);
    }
}

Posted by Bradley Grainger at June 08, 2012 09:55 PM