Btrfs/Compression
Compression[edit | edit source]
Btrfs supports three types of data compression methods. zlib, lzo and zstd.
| Algorithm | Compression levels | Default | Description |
|---|---|---|---|
| zlib | 1-9 | 3 | slow, good compression ratios, default method |
| LZO | N/A | N/A | very fast, low compression ratios |
| zstd | 1-15 | 3 | slow to very fast, good compression ratios at all levels. |
| Setting compression level is available since Linux Kernel 5.1. | |||
The compression speeds with zstd and lzo are generally real-time. This means that they are faster than the storage medium can read or write. This can increase the overall read/write throughput as well as increasing the life span of flash based storage such as NVME and SSD.
Take a look at Btrfs/Zstd benchmarks to see examples on how the compression level affects performance.
Enable/Disable Compression[edit | edit source]
Compression can be enabled with mount options, with chattr +c[1] (only zlib) or with btrfs property set[2].
Compression levels can currently only be set using mount[3] options
To see if a file has a compression flag set you can use lsattr[4] or getfattr[5]. Files compressed through the compress mount option do not get the compression flag set.
# lsattr
--------c----------- ./myfile.db
# getfattr -n btrfs.compression myfile.db
# file: myfile.db btrfs.compression="zstd:1"
zlib[edit | edit source]
Zlib is the default method used unless specified otherwise.
Enable/disable zlib with one of these options:
# chattr +c <filename> # chattr -c <filename> # btrfs property set <filename> compression zlib # btrfs property set <filename> compression none # setfattr -n btrfs.compression -v zstd <filename> # setfattr -n btrfs.compression -v "" <filename> # mount -o compress=zlib # mount -o compress=zlib:5 # mount -o compress-force=zlib # mount -o compress-force=zlib:5
LZO[edit | edit source]
LZO is extremely fast but offers worse compression ratios that zlib and zstd. It is available since the beginning of Btrfs.
Enable/disable LZO with one of these options:
# btrfs property set <filename> compression lzo # btrfs property set <filename> compression lzo # setfattr -n btrfs.compression -v lzo <filename> # setfattr -n btrfs.compression -v "" <filename> # chattr -c <filename> # mount -o compress=lzo # mount -o compress-force=lzo
Zstandard - Zstd[edit | edit source]
Zstd is the newest compression algorithm in Btrfs. It is available since Linux Kernel 4.14. It offers good compression ratios with very high speeds. By choosing an appropriate compression level you can tailor performance to your workloads.
Enable or disable Zstd with one these options:
# btrfs property set <filename> compression zstd # btrfs property set <filename> compression none # setfattr -n btrfs.compression -v zstd <filename> # setfattr -n btrfs.compression -v "" <filename> # chattr -c <filename> # mount -o compress=zstd # mount -o compress=zstd:5 # mount -o compress-force=zstd # mount -o compress-force=zstd:5
The compress-force mount option[edit | edit source]
Btrfs contains an internal heuristics that determines if some data is compressible so that it doesn't try to compress data that isn't compressible as this wastes CPU time. The compress-force mount option bypasses this heuristics in order to gain better compression ratios. A downside is that this increases fragmentation with non-compressible files.
Using defrag to compress files[edit | edit source]
Enabling compression does not re-compress existing files. Instead you have to use btrfs filesystem defrag to re-compress them. There is more information on defragmenting files at Btrfs/Defrag.
Use the -c<algo> to select zstd, zlib or lzo compression.
/media/vm/libvirt/images # btrfs fi defrag -v -czstd qBit_root.img
qBit_root.img
We see that the disk image was compressed from 4.1 to 3.3GiB.
/media/vm/libvirt/images # compsize qBit_root.img
Processed 1 file, 9460 regular extents (9471 refs), 0 inline. Type Perc Disk Usage Uncompressed Referenced TOTAL 81% 3.3G 4.1G 4.1G none 100% 3.0G 3.0G 3.0G zstd 29% 317M 1.0G 1.0G
If the filesystem is mounted with -o compress or -o compress-force, defrag will inherit compression mount option and compress all files, even without the -c<algo> option. Defrag will also inherit any compression levels from the mount options.
# mount /mnt/btrfs -o remount,compress-force=zstd:15 # btrfs fi defrag -v -r /mnt/btrfs/backups/ # mount /mnt/btrfs -o remount,compress=zstd:2
Benchmark[edit | edit source]
Best way to know what compression levels suit your needs is to simply try out the different compression levels and options with your intended workload.
The following benchmark tests the zstd compression using the zstd tool, not the btrfs internal compression, so it is not truly representing the internal filesystem compression. However, it can give an insight into how well your CPU handles different compression levels. There are many factors affecting read/write speeds and a quick benchmark does not give all answers.
A quick one-line benchmark for zstd would otherwise be:
# for i in {1..15}; do zstd -b$i -i3 <myfile>; done
This will benchmark zstd and iterate through all levels between 1 and 15.
See Btrfs/Zstd for more in depth results.
