Btrfs/ENOSPC

ENOSPC - Out of disk space

Graph showing btrfs usage divided by block groups.

Unlike most conventional filesystems, BTRFS uses a two-stage allocator. The first stage allocates large regions of space known as chunks for specific types of data, then the second stage allocates blocks like a regular (old-fashioned) filesystem within these larger regions.

Btrfs combines chunks into three types of block groups:

Type	Description
DATA	Stores normal user file data
METADATA	Stores internal metadata. Small files can also stored inline
SYSTEM	Stores mapping between physical devices and the logical space representing the filesystem
UNALLOCATED	Any unallocated space
Only the type of data that the chunk is allocated for can be stored in that block group.

The most common case these days when you get a -ENOSPC error on BTRFS is that the filesystem has run out of room for data or metadata in existing block groups, and that there is not enough unallocated space to allocate a new block group of correct type.

You can verify that this is the case by running btrfs filesystem usage on the filesystem that threw the error. If the Data or Metadata line shows a Total value that is significantly different from the Used value, then this is probably the cause.

# btrfs fi us /mnt/btrfs_vol
Overall:
    Device size:          14.01GiB
    Device allocated:      1.38GiB
    Device unallocated:   12.63GiB
    Device missing:        0.00B
    Used:                901.58MiB
    Free (estimated)      12.64GiB    (min: 6.33GiB)
    Data ratio:            1.00
    Metadata ratio:        2.00 
    Global reserve:        3.25MiB    (used: 0.00B)
    Multiple profiles:       no

Data,RAID0: Size:890.00MiB, Used:870.08MiB (97.76%)
   /dev/sdb1     445.00MiB
   /dev/sdc1     445.00MiB

Metadata,RAID1: Size:256.00MiB, Used:15.73MiB (6.15%)
   /dev/sdb1     256.00MiB
   /dev/sdc1     256.00MiB

System,RAID1: Size:8.00MiB, Used:16.00KiB (0.20%)
   /dev/sdb1       8.00MiB
   /dev/sdc1       8.00MiB

Unallocated:
   /dev/sdb1      21.00MiB  <== Not enough space for another chunk of data or metatada in RAID profile.
   /dev/sdc1      12.61GiB

Balance

What btrfs balance does is to send things back through the allocator, which results in space usage in the chunks being compacted. For example, if you have two metadata chunks that are both 40% full, a balance will result in them becoming one metadata chunk that's 80% full. By compacting space usage like this, the balance operation is then able to delete the now empty chunks, and thus frees up room for the allocation of new chunks.

It is important to run btrfs balance before you run out of unallocated space. A common way is to set up balance as a scheduled maintenance task.