Colt McAnlis' talk from SF Android

Colt McAnlis of Google and the Android Performance Patterns series dropped by SF Android to talk about data compression and how to make network-enabled applications better at transferring data over the wire.

Mobile applications are rapidly extending their reach into parts of the world where network data transfers run at 2G speeds and consumers pay for every byte of data transferred—larger transfers take longer and cost more. This is made worse by the fact that a longer transfer takes up more battery power because the radio is kept in a high power state for a longer period.

The video begins with a short discussion of data compression theory and how common data compression algorithms are composed together. This gives the audience context for how terms like entropy and skewing relate to the compressibility for a segment of data. Colt then breaks down compression into three practical areas where developers can focus to see gains in transfer performance:

Images

Images are inherently large, and some file formats provide better compression than others. PNG provides poor performance because it prioritizes quality over compressibility. Formats like JPG/WEBP allow preprocessing to introduce small amounts of quality loss as a trade-off for higher compression rates. Images are a format where the human eye (especially on a mobile device) cannot detect a small reduction in quality, and that results in a larger gain for the application overall.

Colt argues for having your servers and CDNs produce and provide multiple versions of the same image. Downloading a high-res version of an image and resizeing it on the device is a highly inefficient process for both memory and bandwidth. The reduced CDN delivery costs may even offset potential increases in storage/processing costs.

Serialized Data

Human readable formats, such as JSON or XML, are much too verbose and don't make sense when we send data over the wire for a machine to process. Using more efficient technologies like Protocol Buffers or Flatbuffers, which produces binary files that are smaller on the wire and faster to parse by the end application, will yield better performance.

Numerical Data

Numerical data is harder to compress because it has lower entropy than textual data. Colt describes some clever transforms that developers can apply to numerical data as a way of making is easier to compress for data transfer.