Btrfs/Zstd

From Forza's ramblings

Zstandard (zstd)[edit | edit source]

Seagate ST1000LM014 - Laptop SSHD hard disk, 2.5", 1000 GB

Zstandard (zstd)[1] compression offers good compression ratio as well as good speeds. By using the compress=zstd:<num> mount option you can set the compression level to suit your specific use-case. For example you may want to use a low number (high speed) on your root filesystem, but a higher number for backup storage.

Mount options[edit | edit source]

mount -o compress=zstd:5 (sets the compression level to 5)

Default value is zstd:3. Setting compression level is available since Linux Kernel 5.1.

See Btrfs/Compression for more details on how to set and disable compression.

Benchmarks[edit | edit source]

In order to determine what performance different levels give you can do a simple benchmark using zstd -b<num> somefile where num is a value between 1 and 15. Se the zstd man page for detailed usage.

Compression on source file glibc-2.0.tar[edit | edit source]

# for i in {1..15}; do  zstd -b$i -i3 glibc-2.30.tar;done
comp.level         : src.size  ->  cmp.size (ratio), write/MB/s ,read   MB/s 
----------------------------------------------------------------------------
 1# glibc-2.30.tar : 225966080 ->  33910476 (6.664), 517.4 MB/s ,1258.1 MB/s
 2# glibc-2.30.tar : 225966080 ->  31949315 (7.073), 468.4 MB/s ,1232.4 MB/s
 3# glibc-2.30.tar : 225966080 ->  30286852 (7.461), 371.6 MB/s ,1269.0 MB/s
 4# glibc-2.30.tar : 225966080 ->  30054256 (7.519), 270.4 MB/s ,1191.7 MB/s
 5# glibc-2.30.tar : 225966080 ->  28134463 (8.032), 110.6 MB/s ,1011.7 MB/s
 6# glibc-2.30.tar : 225966080 ->  27031121 (8.359),  86.0 MB/s ,1060.0 MB/s
 7# glibc-2.30.tar : 225966080 ->  25812107 (8.754),  79.3 MB/s ,1418.1 MB/s
 8# glibc-2.30.tar : 225966080 ->  24800109 (9.111),  66.9 MB/s ,1535.2 MB/s
 9# glibc-2.30.tar : 225966080 ->  24364662 (9.274),  47.5 MB/s ,1510.1 MB/s
10# glibc-2.30.tar : 225966080 ->  23731258 (9.522),  40.5 MB/s ,1412.3 MB/s
11# glibc-2.30.tar : 225966080 ->  23557003 (9.592),  33.3 MB/s ,1475.4 MB/s
12# glibc-2.30.tar : 225966080 ->  23221788 (9.731),  23.0 MB/s ,1461.5 MB/s
13# glibc-2.30.tar : 225966080 ->  22920288 (9.859),  16.1 MB/s ,1596.2 MB/s
14# glibc-2.30.tar : 225966080 ->  22718877 (9.946),  13.9 MB/s ,1491.6 MB/s
15# glibc-2.30.tar : 225966080 ->  22510292 (10.04),  9.72 MB/s ,1500.2 MB/s

Compression on digikam4.db sqlite database[edit | edit source]

# for i in {1..15}; do  zstd -b$i -i3 digikam4.db;done
comp.level         : src.size  ->  cmp.size (ratio), write/MB/s ,read   MB/s 
------------------------------------------------------------------------------
 1# digikam4.db    :  24084480 ->   8108619 (2.970), 383.8 MB/s , 899.2 MB/s
 2# digikam4.db    :  24084480 ->   7577373 (3.178), 329.6 MB/s , 828.2 MB/s
 3# digikam4.db    :  24084480 ->   7376461 (3.265), 200.5 MB/s , 796.4 MB/s
 4# digikam4.db    :  24084480 ->   7431171 (3.241), 124.1 MB/s , 746.1 MB/s
 5# digikam4.db    :  24084480 ->   7313561 (3.293),  50.3 MB/s , 741.5 MB/s
 6# digikam4.db    :  24084480 ->   7270071 (3.313),  44.6 MB/s , 744.2 MB/s
 7# digikam4.db    :  24084480 ->   6652607 (3.620),  31.8 MB/s , 814.2 MB/s
 8# digikam4.db    :  24084480 ->   6581579 (3.659),  27.3 MB/s , 809.3 MB/s
 9# digikam4.db    :  24084480 ->   6571564 (3.665),  24.2 MB/s , 827.4 MB/s
10# digikam4.db    :  24084480 ->   6396986 (3.765),  18.6 MB/s , 830.2 MB/s
11# digikam4.db    :  24084480 ->   6388278 (3.770),  14.2 MB/s , 602.1 MB/s
12# digikam4.db    :  24084480 ->   6379188 (3.775), 10.37 MB/s , 787.5 MB/s
13# digikam4.db    :  24084480 ->   6375563 (3.778),  13.6 MB/s , 792.0 MB/s
14# digikam4.db    :  24084480 ->   6366966 (3.783),  13.0 MB/s , 821.7 MB/s
15# digikam4.db    :  24084480 ->   6357732 (3.788),  9.30 MB/s , 770.2 MB/s

Compression on Apache access-log[edit | edit source]

# for i in {1..15}; do  zstd -b$i -i3 access_log;done
comp.level         : src.size  ->  cmp.size (ratio), write/MB/s ,read   MB/s 
------------------------------------------------------------------------------
 1# access_log     :    283315 ->     39354 (7.199), 677.8 MB/s ,2424.4 MB/s
 2# access_log     :    283315 ->     39499 (7.173), 587.9 MB/s ,2277.7 MB/s
 3# access_log     :    283315 ->     40324 (7.026), 398.1 MB/s ,2150.3 MB/s
 4# access_log     :    283315 ->     40387 (7.015), 388.8 MB/s ,2025.2 MB/s
 5# access_log     :    283315 ->     40047 (7.075), 156.1 MB/s ,2146.5 MB/s
 6# access_log     :    283315 ->     40018 (7.080),  93.8 MB/s ,1704.2 MB/s
 7# access_log     :    283315 ->     39186 (7.230),  85.5 MB/s ,1926.9 MB/s
 8# access_log     :    283315 ->     38656 (7.329), 119.7 MB/s ,2343.6 MB/s
 9# access_log     :    283315 ->     38625 (7.335),  77.9 MB/s ,2307.2 MB/s
10# access_log     :    283315 ->     38625 (7.335),  57.5 MB/s ,1915.6 MB/s
11# access_log     :    283315 ->     38625 (7.335),  54.8 MB/s ,2345.7 MB/s
12# access_log     :    283315 ->     38606 (7.339),  66.8 MB/s ,2291.3 MB/s
13# access_log     :    283315 ->     38558 (7.348),  44.1 MB/s ,2386.1 MB/s
14# access_log     :    283315 ->     38558 (7.348),  43.2 MB/s ,2290.6 MB/s
15# access_log     :    283315 ->     38547 (7.350),  39.1 MB/s ,2304.7 MB/s

Compression on binary file /bin/bash[edit | edit source]

#  for i in {1..15}; do  zstd -b$i -i3 bash;done
comp.level         : src.size  ->  cmp.size (ratio), write/MB/s ,read   MB/s 
------------------------------------------------------------------------------
 1# bash           :    819800 ->    470027 (1.744), 342.8 MB/s , 859.1 MB/s
 2# bash           :    819800 ->    445975 (1.838), 263.7 MB/s , 798.4 MB/s
 3# bash           :    819800 ->    425997 (1.924), 168.2 MB/s , 757.2 MB/s
 4# bash           :    819800 ->    421671 (1.944), 138.5 MB/s , 732.5 MB/s
 5# bash           :    819800 ->    414218 (1.979),  79.8 MB/s , 709.5 MB/s
 6# bash           :    819800 ->    412324 (1.988),  70.5 MB/s , 720.2 MB/s
 7# bash           :    819800 ->    406171 (2.018),  53.2 MB/s , 742.2 MB/s
 8# bash           :    819800 ->    404584 (2.026),  45.2 MB/s , 763.1 MB/s
 9# bash           :    819800 ->    403488 (2.032),  36.0 MB/s , 774.8 MB/s
10# bash           :    819800 ->    403473 (2.032),  27.7 MB/s , 795.7 MB/s
11# bash           :    819800 ->    403473 (2.032),  27.8 MB/s , 777.0 MB/s
12# bash           :    819800 ->    402821 (2.035),  24.0 MB/s , 769.9 MB/s
13# bash           :    819800 ->    401840 (2.040),  14.8 MB/s , 783.6 MB/s
14# bash           :    819800 ->    401840 (2.040),  14.8 MB/s , 796.2 MB/s
15# bash           :    819800 ->    401835 (2.040),  14.5 MB/s , 782.9 MB/s

All tests made on an AMD Athlon 3000G 2 Cores - 3500 MHz - 35 watts CPU

  1. Zstandard official website[1]