Btrfs/Zstd
From Forza's ramblings
Zstandard (zstd)[edit | edit source]
Zstandard (zstd)[1] compression offers good compression ratio as well as good speeds. By using the compress=zstd:<num> mount option you can set the compression level to suit your specific use-case. For example you may want to use a low number (high speed) on your root filesystem, but a higher number for backup storage.
Mount options[edit | edit source]
# mount -o compress=zstd:5 (sets the compression level to 5)
Default value is zstd:3. Setting compression level is available since Linux Kernel 5.1.
See Btrfs/Compression for more details on how to set and disable compression.
Benchmarks[edit | edit source]
In order to determine what performance different levels give you can do a simple benchmark using zstd -b<num> somefile where num is a value between 1 and 15. Se the zstd man page for detailed usage.
Compression on source file glibc-2.0.tar[edit | edit source]
# for i in {1..15}; do zstd -b$i -i3 glibc-2.30.tar;done
comp.level : src.size -> cmp.size (ratio), write/MB/s ,read MB/s
----------------------------------------------------------------------------
1# glibc-2.30.tar : 225966080 -> 33910476 (6.664), 517.4 MB/s ,1258.1 MB/s
2# glibc-2.30.tar : 225966080 -> 31949315 (7.073), 468.4 MB/s ,1232.4 MB/s
3# glibc-2.30.tar : 225966080 -> 30286852 (7.461), 371.6 MB/s ,1269.0 MB/s
4# glibc-2.30.tar : 225966080 -> 30054256 (7.519), 270.4 MB/s ,1191.7 MB/s
5# glibc-2.30.tar : 225966080 -> 28134463 (8.032), 110.6 MB/s ,1011.7 MB/s
6# glibc-2.30.tar : 225966080 -> 27031121 (8.359), 86.0 MB/s ,1060.0 MB/s
7# glibc-2.30.tar : 225966080 -> 25812107 (8.754), 79.3 MB/s ,1418.1 MB/s
8# glibc-2.30.tar : 225966080 -> 24800109 (9.111), 66.9 MB/s ,1535.2 MB/s
9# glibc-2.30.tar : 225966080 -> 24364662 (9.274), 47.5 MB/s ,1510.1 MB/s
10# glibc-2.30.tar : 225966080 -> 23731258 (9.522), 40.5 MB/s ,1412.3 MB/s
11# glibc-2.30.tar : 225966080 -> 23557003 (9.592), 33.3 MB/s ,1475.4 MB/s
12# glibc-2.30.tar : 225966080 -> 23221788 (9.731), 23.0 MB/s ,1461.5 MB/s
13# glibc-2.30.tar : 225966080 -> 22920288 (9.859), 16.1 MB/s ,1596.2 MB/s
14# glibc-2.30.tar : 225966080 -> 22718877 (9.946), 13.9 MB/s ,1491.6 MB/s
15# glibc-2.30.tar : 225966080 -> 22510292 (10.04), 9.72 MB/s ,1500.2 MB/s
Compression on digikam4.db sqlite database[edit | edit source]
# for i in {1..15}; do zstd -b$i -i3 digikam4.db;done
comp.level : src.size -> cmp.size (ratio), write/MB/s ,read MB/s
------------------------------------------------------------------------------
1# digikam4.db : 24084480 -> 8108619 (2.970), 383.8 MB/s , 899.2 MB/s
2# digikam4.db : 24084480 -> 7577373 (3.178), 329.6 MB/s , 828.2 MB/s
3# digikam4.db : 24084480 -> 7376461 (3.265), 200.5 MB/s , 796.4 MB/s
4# digikam4.db : 24084480 -> 7431171 (3.241), 124.1 MB/s , 746.1 MB/s
5# digikam4.db : 24084480 -> 7313561 (3.293), 50.3 MB/s , 741.5 MB/s
6# digikam4.db : 24084480 -> 7270071 (3.313), 44.6 MB/s , 744.2 MB/s
7# digikam4.db : 24084480 -> 6652607 (3.620), 31.8 MB/s , 814.2 MB/s
8# digikam4.db : 24084480 -> 6581579 (3.659), 27.3 MB/s , 809.3 MB/s
9# digikam4.db : 24084480 -> 6571564 (3.665), 24.2 MB/s , 827.4 MB/s
10# digikam4.db : 24084480 -> 6396986 (3.765), 18.6 MB/s , 830.2 MB/s
11# digikam4.db : 24084480 -> 6388278 (3.770), 14.2 MB/s , 602.1 MB/s
12# digikam4.db : 24084480 -> 6379188 (3.775), 10.37 MB/s , 787.5 MB/s
13# digikam4.db : 24084480 -> 6375563 (3.778), 13.6 MB/s , 792.0 MB/s
14# digikam4.db : 24084480 -> 6366966 (3.783), 13.0 MB/s , 821.7 MB/s
15# digikam4.db : 24084480 -> 6357732 (3.788), 9.30 MB/s , 770.2 MB/s
Compression on Apache access-log[edit | edit source]
# for i in {1..15}; do zstd -b$i -i3 access_log;done
comp.level : src.size -> cmp.size (ratio), write/MB/s ,read MB/s
------------------------------------------------------------------------------
1# access_log : 283315 -> 39354 (7.199), 677.8 MB/s ,2424.4 MB/s
2# access_log : 283315 -> 39499 (7.173), 587.9 MB/s ,2277.7 MB/s
3# access_log : 283315 -> 40324 (7.026), 398.1 MB/s ,2150.3 MB/s
4# access_log : 283315 -> 40387 (7.015), 388.8 MB/s ,2025.2 MB/s
5# access_log : 283315 -> 40047 (7.075), 156.1 MB/s ,2146.5 MB/s
6# access_log : 283315 -> 40018 (7.080), 93.8 MB/s ,1704.2 MB/s
7# access_log : 283315 -> 39186 (7.230), 85.5 MB/s ,1926.9 MB/s
8# access_log : 283315 -> 38656 (7.329), 119.7 MB/s ,2343.6 MB/s
9# access_log : 283315 -> 38625 (7.335), 77.9 MB/s ,2307.2 MB/s
10# access_log : 283315 -> 38625 (7.335), 57.5 MB/s ,1915.6 MB/s
11# access_log : 283315 -> 38625 (7.335), 54.8 MB/s ,2345.7 MB/s
12# access_log : 283315 -> 38606 (7.339), 66.8 MB/s ,2291.3 MB/s
13# access_log : 283315 -> 38558 (7.348), 44.1 MB/s ,2386.1 MB/s
14# access_log : 283315 -> 38558 (7.348), 43.2 MB/s ,2290.6 MB/s
15# access_log : 283315 -> 38547 (7.350), 39.1 MB/s ,2304.7 MB/s
Compression on binary file /bin/bash[edit | edit source]
# for i in {1..15}; do zstd -b$i -i3 bash;done
comp.level : src.size -> cmp.size (ratio), write/MB/s ,read MB/s
------------------------------------------------------------------------------
1# bash : 819800 -> 470027 (1.744), 342.8 MB/s , 859.1 MB/s
2# bash : 819800 -> 445975 (1.838), 263.7 MB/s , 798.4 MB/s
3# bash : 819800 -> 425997 (1.924), 168.2 MB/s , 757.2 MB/s
4# bash : 819800 -> 421671 (1.944), 138.5 MB/s , 732.5 MB/s
5# bash : 819800 -> 414218 (1.979), 79.8 MB/s , 709.5 MB/s
6# bash : 819800 -> 412324 (1.988), 70.5 MB/s , 720.2 MB/s
7# bash : 819800 -> 406171 (2.018), 53.2 MB/s , 742.2 MB/s
8# bash : 819800 -> 404584 (2.026), 45.2 MB/s , 763.1 MB/s
9# bash : 819800 -> 403488 (2.032), 36.0 MB/s , 774.8 MB/s
10# bash : 819800 -> 403473 (2.032), 27.7 MB/s , 795.7 MB/s
11# bash : 819800 -> 403473 (2.032), 27.8 MB/s , 777.0 MB/s
12# bash : 819800 -> 402821 (2.035), 24.0 MB/s , 769.9 MB/s
13# bash : 819800 -> 401840 (2.040), 14.8 MB/s , 783.6 MB/s
14# bash : 819800 -> 401840 (2.040), 14.8 MB/s , 796.2 MB/s
15# bash : 819800 -> 401835 (2.040), 14.5 MB/s , 782.9 MB/s
All tests made on an AMD Athlon 3000G 2 Cores - 3500 MHz - 35 watts CPU
