Page History

...

1. Data Format 
2. Index Prefix Compression
3. Page Compression
4. WAL compression
5. Column Compression
6. Columnar Store
7. Misc approaches

Data size could be reduced with efficient data format. Common row metadata could be skipped. Numeric values could be encoded to occupy less space. Fixed-length strings could be trimmed. NULL and zero values could be skipped with help of small row bitmaps.

Next compression could be applied specific columns, either manually or transparently.

Data pages could be compressed on per-page basis, what gives 2x-4x compression rate on typical workloads. Depending on concrete implementation pages could be stored in memory in compressed or uncompressed forms. File system specific features, such as hole punching, could be applied to reduce IO.

Indexes are compressed differently because it is necessary to keep index page structure for fast lookups. Prefix compression is a common mechanism.

Extreme compression rates up to 10x-15x are possible with column store formats. But it is only applicable to historical data with minimal update rate and is not suitable for lookups.

WAL could be compressed to decrease log size. In some cases it may improve overall system throughput.

Last, compression could be applied to special cases. Examples are compression of large values (LOBs, long strings) and per-page dictionary compression during data load.

Data Format

Efficient disk usage starts with proper data layout. Vendors strive to place data in pages in such a way that total overhead is kept as low as possible while still maintaining high read speed. Typically this is achieved as follows:

...