{
    "content": [
        {
            "type": "text",
            "text": "# btrfs(5) (man)\n\n**Summary:** btrfs-man5 - topics about the BTRFS filesystem (mount options, supported file attributes and other)\n\n## See Also\n\n- acl(5)\n- chattr(1)\n- fstrim(8)\n- ioctl(2)\n- mkfs.btrfs(8)\n- mount(8)\n- swapon(8)\n- BTRFS-MAN5(5)\n\n## Section Outline\n\n- **NAME** (3 lines)\n- **DESCRIPTION** (35 lines)\n- **MOUNT OPTIONS** (204 lines) — 7 subsections\n  - degraded (145 lines)\n  - nologreplay (48 lines)\n  - norecovery (21 lines)\n  - rescue (157 lines)\n  - usebackuproot (33 lines)\n  - recovery (23 lines)\n  - noatime (16 lines)\n- **FILESYSTEM FEATURES** (159 lines) — 3 subsections\n  - raid1c34 (6 lines)\n  - raid56 (71 lines)\n  - zoned (84 lines)\n- **CHECKSUM ALGORITHMS** (102 lines)\n- **COMPRESSION** (100 lines)\n- **FILESYSTEM EXCLUSIVE OPERATIONS** (25 lines)\n- **FILESYSTEM LIMITS** (65 lines)\n- **BOOTLOADER SUPPORT** (13 lines)\n- **FILE ATTRIBUTES** (142 lines)\n- **ZONED MODE** (14 lines) — 1 subsections\n  - REQUIREMENTS, LIMITATIONS (55 lines)\n- **CONTROL DEVICE** (39 lines)\n- **FILESYSTEM WITH MULTIPLE PROFILES** (56 lines)\n- **SEEDING DEVICE** (74 lines)\n- **RAID56 STATUS AND RECOMMENDED PRACTICES** (5 lines) — 2 subsections\n  - Metadata (15 lines)\n  - Missing/incomplete support (12 lines)\n- **STORAGE MODEL** (37 lines) — 4 subsections\n  - No or partial atomicity of block reads/writes (1) (7 lines)\n  - The flush command does not flush (2) (14 lines)\n  - Data get silently changed on write (3) (10 lines)\n  - Data get silently written to another offset (3) (12 lines)\n- **HARDWARE CONSIDERATIONS** (39 lines) — 6 subsections\n  - DIRECT MEMORY ACCESS (DMA) (17 lines)\n  - ROTATIONAL DISKS (HDD) (22 lines)\n  - SOLID STATE DRIVES (SSD) (68 lines)\n  - NVM EXPRESS, NON-VOLATILE MEMORY (NVMe) (50 lines)\n  - HARDWARE AS THE MAIN SOURCE OF FILESYSTEM CORRUPTIONS (1 lines)\n  - tells you. (1 lines)\n- **SEE ALSO** (7 lines)\n\n## Full Content\n\n### NAME\n\nbtrfs-man5 - topics about the BTRFS filesystem (mount options, supported file attributes and\nother)\n\n### DESCRIPTION\n\nThis document describes topics related to BTRFS that are not specific to the tools. Currently\ncovers:\n\n\n1.  mount options\n\n2.  filesystem features\n\n3.  checksum algorithms\n\n4.  compression\n\n5.  filesystem exclusive operations\n\n6.  filesystem limits\n\n7.  bootloader support\n\n8.  file attributes\n\n9.  zoned mode\n\n10.  control device\n\n11.  filesystems with multiple block group profiles\n\n12.  seeding device\n\n13.  raid56 status and recommended practices\n\n14.  storage model\n\n15.  hardware considerations\n\n### MOUNT OPTIONS\n\nThis section describes mount options specific to BTRFS. For the generic mount options please\nrefer to mount(8) manpage. The options are sorted alphabetically (discarding the no prefix).\n\nNote\nmost mount options apply to the whole filesystem and only options in the first mounted\nsubvolume will take effect. This is due to lack of implementation and may change in the\nfuture. This means that (for example) you can’t set per-subvolume nodatacow, nodatasum,\nor compress using mount options. This should eventually be fixed, but it has proved to be\ndifficult to implement correctly within the Linux VFS framework.\n\nMount options are processed in order, only the last occurrence of an option takes effect and\nmay disable other options due to constraints (see eg. nodatacow and compress). The output of\nmount command shows which options have been applied.\n\n\n\nacl, noacl\n\n\n\n\n(default: on)\n\n\nEnable/disable support for Posix Access Control Lists (ACLs). See the acl(5) manual page\nfor more information about ACLs.\n\n\nThe support for ACL is build-time configurable (BTRFSFSPOSIXACL) and mount fails if\nacl is requested but the feature is not compiled in.\n\n\nautodefrag, noautodefrag\n\n\n\n\n(since: 3.0, default: off)\n\n\nEnable automatic file defragmentation. When enabled, small random writes into files (in a\nrange of tens of kilobytes, currently it’s 64K) are detected and queued up for the\ndefragmentation process. Not well suited for large database workloads.\n\n\nThe read latency may increase due to reading the adjacent blocks that make up the range\nfor defragmentation, successive write will merge the blocks in the new location.\n\nWarning\nDefragmenting with Linux kernel versions < 3.9 or ≥ 3.14-rc2 as well as with Linux\nstable kernel versions ≥ 3.10.31, ≥ 3.12.12 or ≥ 3.13.4 will break up the reflinks of\nCOW data (for example files copied with cp --reflink, snapshots or de-duplicated\ndata). This may cause considerable increase of space usage depending on the broken up\nreflinks.\n\nbarrier, nobarrier\n\n\n\n\n(default: on)\n\n\nEnsure that all IO write operations make it through the device cache and are stored\npermanently when the filesystem is at its consistency checkpoint. This typically means\nthat a flush command is sent to the device that will synchronize all pending data and\nordinary metadata blocks, then writes the superblock and issues another flush.\n\n\nThe write flushes incur a slight hit and also prevent the IO block scheduler to reorder\nrequests in a more effective way. Disabling barriers gets rid of that penalty but will\nmost certainly lead to a corrupted filesystem in case of a crash or power loss. The\nordinary metadata blocks could be yet unwritten at the time the new superblock is stored\npermanently, expecting that the block pointers to metadata were stored permanently\nbefore.\n\n\nOn a device with a volatile battery-backed write-back cache, the nobarrier option will\nnot lead to filesystem corruption as the pending blocks are supposed to make it to the\npermanent storage.\n\n\ncheckint, checkintdata, checkintprintmask=value\n\n\n\n\n\n(since: 3.0, default: off)\n\n\nThese debugging options control the behavior of the integrity checking module (the\nBTRFSFSCHECKINTEGRITY config option required). The main goal is to verify that all\nblocks from a given transaction period are properly linked.\n\n\ncheckint enables the integrity checker module, which examines all block write requests\nto ensure on-disk consistency, at a large memory and CPU cost.\n\n\ncheckintdata includes extent data in the integrity checks, and implies the checkint\noption.\n\n\ncheckintprintmask takes a bitmask of BTRFSICPRINTMASK* values as defined in\nfs/btrfs/check-integrity.c, to control the integrity checker module behavior.\n\n\nSee comments at the top of fs/btrfs/check-integrity.c for more information.\n\n\nclearcache\n\n\n\nForce clearing and rebuilding of the disk space cache if something has gone wrong. See\nalso: spacecache.\n\n\ncommit=seconds\n\n\n\n(since: 3.12, default: 30)\n\n\nSet the interval of periodic transaction commit when data are synchronized to permanent\nstorage. Higher interval values lead to larger amount of unwritten data, which has\nobvious consequences when the system crashes. The upper bound is not forced, but a\nwarning is printed if it’s more than 300 seconds (5 minutes). Use with care.\n\n\ncompress, compress=type[:level], compress-force, compress-force=type[:level]\n\n\n\n\n\n\n(default: off, level support since: 5.1)\n\n\nControl BTRFS file data compression. Type may be specified as zlib, lzo, zstd or no (for\nno compression, used for remounting). If no type is specified, zlib is used. If\ncompress-force is specified, then compression will always be attempted, but the data may\nend up uncompressed if the compression would make them larger.\n\n\nBoth zlib and zstd (since version 5.1) expose the compression level as a tunable knob\nwith higher levels trading speed and memory (zstd) for higher compression ratios. This\ncan be set by appending a colon and the desired level. Zlib accepts the range [1, 9] and\nzstd accepts [1, 15]. If no level is set, both currently use a default level of 3. The\nvalue 0 is an alias for the default level.\n\n\nOtherwise some simple heuristics are applied to detect an incompressible file. If the\nfirst blocks written to a file are not compressible, the whole file is permanently marked\nto skip compression. As this is too simple, the compress-force is a workaround that will\ncompress most of the files at the cost of some wasted CPU cycles on failed attempts.\nSince kernel 4.15, a set of heuristic algorithms have been improved by using frequency\nsampling, repeated pattern detection and Shannon entropy calculation to avoid that.\n\nNote\nIf compression is enabled, nodatacow and nodatasum are disabled.\n\ndatacow, nodatacow\n\n\n\n\n(default: on)\n\n\nEnable data copy-on-write for newly created files.  Nodatacow implies nodatasum, and\ndisables compression. All files created under nodatacow are also set the NOCOW file\nattribute (see chattr(1)).\n\nNote\nIf nodatacow or nodatasum are enabled, compression is disabled.\nUpdates in-place improve performance for workloads that do frequent overwrites, at the\ncost of potential partial writes, in case the write is interrupted (system crash, device\nfailure).\n\n\ndatasum, nodatasum\n\n\n\n\n(default: on)\n\n\nEnable data checksumming for newly created files.  Datasum implies datacow, ie. the\nnormal mode of operation. All files created under nodatasum inherit the \"no checksums\"\nproperty, however there’s no corresponding file attribute (see chattr(1)).\n\nNote\nIf nodatacow or nodatasum are enabled, compression is disabled.\nThere is a slight performance gain when checksums are turned off, the corresponding\nmetadata blocks holding the checksums do not need to updated. The cost of checksumming of\nthe blocks in memory is much lower than the IO, modern CPUs feature hardware support of\nthe checksumming algorithm.\n\n#### degraded\n\n(default: off)\n\n\nAllow mounts with less devices than the RAID profile constraints require. A read-write\nmount (or remount) may fail when there are too many devices missing, for example if a\nstripe member is completely missing from RAID0.\n\n\nSince 4.14, the constraint checks have been improved and are verified on the chunk level,\nnot at the device level. This allows degraded mounts of filesystems with mixed RAID\nprofiles for data and metadata, even if the device number constraints would not be\nsatisfied for some of the profiles.\n\n\nExample: metadata — raid1, data — single, devices — /dev/sda, /dev/sdb\n\n\nSuppose the data are completely stored on sda, then missing sdb will not prevent the\nmount, even if 1 missing device would normally prevent (any) single profile to mount. In\ncase some of the data chunks are stored on sdb, then the constraint of single/data is not\nsatisfied and the filesystem cannot be mounted.\n\n\ndevice=devicepath\n\n\n\nSpecify a path to a device that will be scanned for BTRFS filesystem during mount. This\nis usually done automatically by a device manager (like udev) or using the btrfs device\nscan command (eg. run from the initial ramdisk). In cases where this is not possible the\ndevice mount option can help.\n\nNote\nbooting eg. a RAID1 system may fail even if all filesystem’s device paths are\nprovided as the actual device nodes may not be discovered by the system at that\npoint.\n\ndiscard, discard=sync, discard=async, nodiscard\n\n\n\n\n\n\n(default: off, async support since: 5.6)\n\n\nEnable discarding of freed file blocks. This is useful for SSD devices, thinly\nprovisioned LUNs, or virtual machine images; however, every storage layer must support\ndiscard for it to work.\n\n\nIn the synchronous mode (sync or without option value), lack of asynchronous queued TRIM\non the backing device TRIM can severely degrade performance, because a synchronous TRIM\noperation will be attempted instead. Queued TRIM requires newer than SATA revision 3.1\nchipsets and devices.\n\n\nThe asynchronous mode (async) gathers extents in larger chunks before sending them to the\ndevices for TRIM. The overhead and performance impact should be negligible compared to\nthe previous mode and it’s supposed to be the preferred mode if needed.\n\n\nIf it is not necessary to immediately discard freed blocks, then the fstrim tool can be\nused to discard all free blocks in a batch. Scheduling a TRIM during a period of low\nsystem activity will prevent latent interference with the performance of other\noperations. Also, a device may ignore the TRIM command if the range is too small, so\nrunning a batch discard has a greater probability of actually discarding the blocks.\n\n\nenospcdebug, noenospcdebug\n\n\n\n\n(default: off)\n\n\nEnable verbose output for some ENOSPC conditions. It’s safe to use but can be noisy if\nthe system reaches near-full state.\n\n\nfatalerrors=action\n\n\n\n(since: 3.4, default: bug)\n\n\nAction to take when encountering a fatal error.\n\n\nbug\n\n\n\nBUG() on a fatal error, the system will stay in the crashed state and may be still\npartially usable, but reboot is required for full operation\n\n\npanic\n\n\n\npanic() on a fatal error, depending on other system configuration, this may be\nfollowed by a reboot. Please refer to the documentation of kernel boot parameters,\neg.  panic, oops or crashkernel.\n\n\n\n\nflushoncommit, noflushoncommit\n\n\n\n\n(default: off)\n\n\nThis option forces any data dirtied by a write in a prior transaction to commit as part\nof the current commit, effectively a full filesystem sync.\n\n\nThis makes the committed state a fully consistent view of the file system from the\napplication’s perspective (i.e. it includes all completed file system operations). This\nwas previously the behavior only when a snapshot was created.\n\n\nWhen off, the filesystem is consistent but buffered writes may last more than one\ntransaction commit.\n\n\nfragment=type\n\n\n\n(depends on compile-time option BTRFSDEBUG, since: 4.4, default: off)\n\n\nA debugging helper to intentionally fragment given type of block groups. The type can be\ndata, metadata or all. This mount option should not be used outside of debugging\nenvironments and is not recognized if the kernel config option BTRFSDEBUG is not\nenabled.\n\n#### nologreplay\n\n(default: off, even read-only)\n\n\nThe tree-log contains pending updates to the filesystem until the full commit. The log is\nreplayed on next mount, this can be disabled by this option. See also treelog. Note that\nnologreplay is the same as norecovery.\n\nWarning\ncurrently, the tree log is replayed even with a read-only mount! To disable that\nbehaviour, mount also with nologreplay.\n\nmaxinline=bytes\n\n\n\n(default: min(2048, page size) )\n\n\nSpecify the maximum amount of space, that can be inlined in a metadata B-tree leaf. The\nvalue is specified in bytes, optionally with a K suffix (case insensitive). In practice,\nthis value is limited by the filesystem block size (named sectorsize at mkfs time), and\nmemory page size of the system. In case of sectorsize limit, there’s some space\nunavailable due to leaf headers. For example, a 4k sectorsize, maximum size of inline\ndata is about 3900 bytes.\n\n\nInlining can be completely turned off by specifying 0. This will increase data block\nslack if file sizes are much smaller than block size but will reduce metadata consumption\nin return.\n\nNote\nthe default value has changed to 2048 in kernel 4.6.\n\nmetadataratio=value\n\n\n\n(default: 0, internal logic)\n\n\nSpecifies that 1 metadata chunk should be allocated after every value data chunks.\nDefault behaviour depends on internal logic, some percent of unused metadata space is\nattempted to be maintained but is not always possible if there’s not enough space left\nfor chunk allocation. The option could be useful to override the internal logic in favor\nof the metadata allocation if the expected workload is supposed to be metadata intense\n(snapshots, reflinks, xattrs, inlined files).\n\n#### norecovery\n\n(since: 4.5, default: off)\n\n\nDo not attempt any data recovery at mount time. This will disable logreplay and avoids\nother write operations. Note that this option is the same as nologreplay.\n\nNote\nThe opposite option recovery used to have different meaning but was changed for\nconsistency with other filesystems, where norecovery is used for skipping log replay.\nBTRFS does the same and in general will try to avoid any write operations.\n\nrescanuuidtree\n\n\n\n(since: 3.12, default: off)\n\n\nForce check and rebuild procedure of the UUID tree. This should not normally be needed.\n\n#### rescue\n\n(since: 5.9)\n\n\nModes allowing mount with damaged filesystem structures.\n\n\n\n•    usebackuproot (since: 5.9, replaces standalone option usebackuproot)\n\n•    nologreplay (since: 5.9, replaces standalone option nologreplay)\n\n•    ignorebadroots, ibadroots (since: 5.11)\n\n•    ignoredatacsums, idatacsums (since: 5.11)\n\n•    all (since: 5.9)\n\n\n\nskipbalance\n\n\n\n(since: 3.3, default: off)\n\n\nSkip automatic resume of an interrupted balance operation. The operation can later be\nresumed with btrfs balance resume, or the paused state can be removed with btrfs balance\ncancel. The default behaviour is to resume an interrupted balance immediately after a\nvolume is mounted.\n\n\nspacecache, spacecache=version, nospacecache\n\n\n\n\n\n(nospacecache since: 3.2, spacecache=v1 and spacecache=v2 since 4.5, default:\nspacecache=v1)\n\n\nOptions to control the free space cache. The free space cache greatly improves\nperformance when reading block group free space into memory. However, managing the space\ncache consumes some resources, including a small amount of disk space.\n\n\nThere are two implementations of the free space cache. The original one, referred to as\nv1, is the safe default. The v1 space cache can be disabled at mount time with\nnospacecache without clearing.\n\n\nOn very large filesystems (many terabytes) and certain workloads, the performance of the\nv1 space cache may degrade drastically. The v2 implementation, which adds a new B-tree\ncalled the free space tree, addresses this issue. Once enabled, the v2 space cache will\nalways be used and cannot be disabled unless it is cleared. Use\nclearcache,spacecache=v1 or clearcache,nospacecache to do so. If v2 is enabled,\nkernels without v2 support will only be able to mount the filesystem in read-only mode.\n\n\nThe btrfs-check(8) and mkfs.btrfs(8) commands have full v2 free space cache support since\nv4.19.\n\n\nIf a version is not explicitly specified, the default implementation will be chosen,\nwhich is v1.\n\n\nssd, ssdspread, nossd, nossdspread\n\n\n\n\n\n\n(default: SSD autodetected)\n\n\nOptions to control SSD allocation schemes. By default, BTRFS will enable or disable SSD\noptimizations depending on status of a device with respect to rotational or\nnon-rotational type. This is determined by the contents of\n/sys/block/DEV/queue/rotational). If it is 0, the ssd option is turned on. The option\nnossd will disable the autodetection.\n\n\nThe optimizations make use of the absence of the seek penalty that’s inherent for the\nrotational devices. The blocks can be typically written faster and are not offloaded to\nseparate threads.\n\nNote\nSince 4.14, the block layout optimizations have been dropped. This used to help with\nfirst generations of SSD devices. Their FTL (flash translation layer) was not\neffective and the optimization was supposed to improve the wear by better aligning\nblocks. This is no longer true with modern SSD devices and the optimization had no\nreal benefit. Furthermore it caused increased fragmentation. The layout tuning has\nbeen kept intact for the option ssdspread.\nThe ssdspread mount option attempts to allocate into bigger and aligned chunks of unused\nspace, and may perform better on low-end SSDs.  ssdspread implies ssd, enabling all\nother SSD heuristics as well. The option nossd will disable all SSD options while\nnossdspread only disables ssdspread.\n\n\nsubvol=path\n\n\n\nMount subvolume from path rather than the toplevel subvolume. The path is always treated\nas relative to the toplevel subvolume. This mount option overrides the default subvolume\nset for the given filesystem.\n\n\nsubvolid=subvolid\n\n\n\nMount subvolume specified by a subvolid number rather than the toplevel subvolume. You\ncan use btrfs subvolume list of btrfs subvolume show to see subvolume ID numbers. This\nmount option overrides the default subvolume set for the given filesystem.\n\nNote\nif both subvolid and subvol are specified, they must point at the same subvolume,\notherwise the mount will fail.\n\nthreadpool=number\n\n\n\n(default: min(NRCPUS + 2, 8) )\n\n\nThe number of worker threads to start. NRCPUS is number of on-line CPUs detected at the\ntime of mount. Small number leads to less parallelism in processing data and metadata,\nhigher numbers could lead to a performance hit due to increased locking contention,\nprocess scheduling, cache-line bouncing or costly data transfers between local CPU\nmemories.\n\n\ntreelog, notreelog\n\n\n\n\n(default: on)\n\n\nEnable the tree logging used for fsync and OSYNC writes. The tree log stores changes\nwithout the need of a full filesystem sync. The log operations are flushed at sync and\ntransaction commit. If the system crashes between two such syncs, the pending tree log\noperations are replayed during mount.\n\nWarning\ncurrently, the tree log is replayed even with a read-only mount! To disable that\nbehaviour, also mount with nologreplay.\nThe tree log could contain new files/directories, these would not exist on a mounted\nfilesystem if the log is not replayed.\n\n#### usebackuproot\n\n(since: 4.6, default: off)\n\n\nEnable autorecovery attempts if a bad tree root is found at mount time. Currently this\nscans a backup list of several previous tree roots and tries to use the first readable.\nThis can be used with read-only mounts as well.\n\nNote\nThis option has replaced recovery.\n\nusersubvolrmallowed\n\n\n\n(default: off)\n\n\nAllow subvolumes to be deleted by their respective owner. Otherwise, only the root user\ncan do that.\n\nNote\nhistorically, any user could create a snapshot even if he was not owner of the source\nsubvolume, the subvolume deletion has been restricted for that reason. The subvolume\ncreation has been restricted but this mount option is still required. This is a\nusability issue. Since 4.18, the rmdir(2) syscall can delete an empty subvolume just\nlike an ordinary directory. Whether this is possible can be detected at runtime, see\nrmdirsubvol feature in FILESYSTEM FEATURES.\n\nDEPRECATED MOUNT OPTIONS\nList of mount options that have been removed, kept for backward compatibility.\n\n#### recovery\n\n(since: 3.2, default: off, deprecated since: 4.5)\n\nNote\nthis option has been replaced by usebackuproot and should not be used but will work\non 4.5+ kernels.\n\ninodecache, noinodecache\n\n\n\n\n(removed in: 5.11, since: 3.0, default: off)\n\nNote\nthe functionality has been removed in 5.11, any stale data created by previous use of\nthe inodecache option can be removed by btrfs check --clear-ino-cache.\n\n\nNOTES ON GENERIC MOUNT OPTIONS\nSome of the general mount options from mount(8) that affect BTRFS and are worth mentioning.\n\n#### noatime\n\nunder read intensive work-loads, specifying noatime significantly improves performance\nbecause no new access time information needs to be written. Without this option, the\ndefault is relatime, which only reduces the number of inode atime updates in comparison\nto the traditional strictatime. The worst case for atime updates under relatime occurs\nwhen many files are read whose atime is older than 24 h and which are freshly\nsnapshotted. In that case the atime is updated and COW happens - for each file - in bulk.\nSee also https://lwn.net/Articles/499293/ - Atime and btrfs: a bad combination? (LWN,\n2012-05-31).\n\n\nNote that noatime may break applications that rely on atime uptimes like the venerable\nMutt (unless you use maildir mailboxes).\n\n### FILESYSTEM FEATURES\n\nThe basic set of filesystem features gets extended over time. The backward compatibility is\nmaintained and the features are optional, need to be explicitly asked for so accidental use\nwill not create incompatibilities.\n\nThere are several classes and the respective tools to manage the features:\n\n\n\nat mkfs time only\n\n\n\nThis is namely for core structures, like the b-tree nodesize or checksum algorithm, see\nmkfs.btrfs(8) for more details.\n\n\nafter mkfs, on an unmounted filesystem\n\n\n\nFeatures that may optimize internal structures or add new structures to support new\nfunctionality, see btrfstune(8). The command btrfs inspect-internal dump-super device\nwill dump a superblock, you can map the value of incompatflags to the features listed\nbelow\n\n\nafter mkfs, on a mounted filesystem\n\n\n\nThe features of a filesystem (with a given UUID) are listed in\n/sys/fs/btrfs/UUID/features/, one file per feature. The status is stored inside the file.\nThe value 1 is for enabled and active, while 0 means the feature was enabled at mount\ntime but turned off afterwards.\n\n\nWhether a particular feature can be turned on a mounted filesystem can be found in the\ndirectory /sys/fs/btrfs/features/, one file per feature. The value 1 means the feature\ncan be enabled.\n\n\nList of features (see also mkfs.btrfs(8) section FILESYSTEM FEATURES):\n\n\n\nbigmetadata\n\n\n\n(since: 3.4)\n\n\nthe filesystem uses nodesize for metadata blocks, this can be bigger than the page size\n\n\ncompresslzo\n\n\n\n(since: 2.6.38)\n\n\nthe lzo compression has been used on the filesystem, either as a mount option or via\nbtrfs filesystem defrag.\n\n\ncompresszstd\n\n\n\n(since: 4.14)\n\n\nthe zstd compression has been used on the filesystem, either as a mount option or via\nbtrfs filesystem defrag.\n\n\ndefaultsubvol\n\n\n\n(since: 2.6.34)\n\n\nthe default subvolume has been set on the filesystem\n\n\nextendediref\n\n\n\n(since: 3.7)\n\n\nincreased hardlink limit per file in a directory to 65536, older kernels supported a\nvarying number of hardlinks depending on the sum of all file name sizes that can be\nstored into one metadata block\n\n\nfreespacetree\n\n\n\n(since: 4.5)\n\n\nfree space representation using a dedicated b-tree, successor of v1 space cache\n\n\nmetadatauuid\n\n\n\n(since: 5.0)\n\n\nthe main filesystem UUID is the metadatauuid, which stores the new UUID only in the\nsuperblock while all metadata blocks still have the UUID set at mkfs time, see\nbtrfstune(8) for more\n\n\nmixedbackref\n\n\n\n(since: 2.6.31)\n\n\nthe last major disk format change, improved backreferences, now default\n\n\nmixedgroups\n\n\n\n(since: 2.6.37)\n\n\nmixed data and metadata block groups, ie. the data and metadata are not separated and\noccupy the same block groups, this mode is suitable for small volumes as there are no\nconstraints how the remaining space should be used (compared to the split mode, where\nempty metadata space cannot be used for data and vice versa)\n\n\non the other hand, the final layout is quite unpredictable and possibly highly\nfragmented, which means worse performance\n\n\nnoholes\n\n\n\n(since: 3.14)\n\n\nimproved representation of file extents where holes are not explicitly stored as an\nextent, saves a few percent of metadata if sparse files are used\n\n#### raid1c34\n\n(since: 5.5)\n\n\nextended RAID1 mode with copies on 3 or 4 devices respectively\n\n#### raid56\n\n(since: 3.9)\n\n\nthe filesystem contains or contained a raid56 profile of block groups\n\n\nrmdirsubvol\n\n\n\n(since: 4.18)\n\n\nindicate that rmdir(2) syscall can delete an empty subvolume just like an ordinary\ndirectory. Note that this feature only depends on the kernel version.\n\n\nskinnymetadata\n\n\n\n(since: 3.10)\n\n\nreduced-size metadata for extent references, saves a few percent of metadata\n\n\nsendstreamversion\n\n\n\n(since: 5.10)\n\n\nnumber of the highest supported send stream version\n\n\nsupportedchecksums\n\n\n\n(since: 5.5)\n\n\nlist of checksum algorithms supported by the kernel module, the respective modules or\nbuilt-in implementing the algorithms need to be present to mount the filesystem, see\nCHECKSUM ALGORITHMS\n\n\nsupportedsectorsizes\n\n\n\n(since: 5.13)\n\n\nlist of values that are accepted as sector sizes (mkfs.btrfs --sectorsize) by the running\nkernel\n\n\nsupportedrescueoptions\n\n\n\n(since: 5.11)\n\n\nlist of values for the mount option rescue that are supported by the running kernel, see\nbtrfs(5)\n\n#### zoned\n\n(since: 5.12)\n\n\nzoned mode is allocation/write friendly to host-managed zoned devices, allocation space\nis partitioned into fixed-size zones that must be updated sequentially, see ZONED MODE\n\n\nSWAPFILE SUPPORT\nThe swapfile is supported since kernel 5.0. Use swapon(8) to activate the swapfile. There are\nsome limitations of the implementation in btrfs and linux swap subsystem:\n\n\n•    filesystem - must be only single device\n\n•    filesystem - must have only single data profile\n\n•    swapfile - the containing subvolume cannot be snapshotted\n\n•    swapfile - must be preallocated\n\n•    swapfile - must be nodatacow (ie. also nodatasum)\n\n•    swapfile - must not be compressed\n\nThe limitations come namely from the COW-based design and mapping layer of blocks that allows\nthe advanced features like relocation and multi-device filesystems. However, the swap\nsubsystem expects simpler mapping and no background changes of the file blocks once they’ve\nbeen attached to swap.\n\nWith active swapfiles, the following whole-filesystem operations will skip swapfile extents\nor may fail:\n\n\n•    balance - block groups with swapfile extents are skipped and reported, the rest will be\nprocessed normally\n\n•    resize grow - unaffected\n\n•    resize shrink - works as long as the extents are outside of the shrunk range\n\n•    device add - a new device does not interfere with existing swapfile and this operation\nwill work, though no new swapfile can be activated afterwards\n\n•    device delete - if the device has been added as above, it can be also deleted\n\n•    device replace - ditto\n\nWhen there are no active swapfiles and a whole-filesystem exclusive operation is running (ie.\nbalance, device delete, shrink), the swapfiles cannot be temporarily activated. The operation\nmust finish first.\n\nTo create and activate a swapfile run the following commands:\n\n\n# truncate -s 0 swapfile\n# chattr +C swapfile\n# fallocate -l 2G swapfile\n# chmod 0600 swapfile\n# mkswap swapfile\n# swapon swapfile\n\nPlease note that the UUID returned by the mkswap utility identifies the swap \"filesystem\" and\nbecause it’s stored in a file, it’s not generally visible and usable as an identifier unlike\nif it was on a block device.\n\nThe file will appear in /proc/swaps:\n\n\n# cat /proc/swaps\nFilename          Type          Size           Used      Priority\n/path/swapfile    file          2097152        0         -2\n\nThe swapfile can be created as one-time operation or, once properly created, activated on\neach boot by the swapon -a command (usually started by the service manager). Add the\nfollowing entry to /etc/fstab, assuming the filesystem that provides the /path has been\nalready mounted at this point. Additional mount options relevant for the swapfile can be set\ntoo (like priority, not the btrfs mount options).\n\n\n/path/swapfile        none        swap        defaults      0 0\n\n### CHECKSUM ALGORITHMS\n\nThere are several checksum algorithms supported. The default and backward compatible is\ncrc32c. Since kernel 5.5 there are three more with different characteristics and trade-offs\nregarding speed and strength. The following list may help you to decide which one to select.\n\n\n\nCRC32C (32bit digest)\n\n\n\ndefault, best backward compatibility, very fast, modern CPUs have instruction-level\nsupport, not collision-resistant but still good error detection capabilities\n\n\nXXHASH (64bit digest)\n\n\n\ncan be used as CRC32C successor, very fast, optimized for modern CPUs utilizing\ninstruction pipelining, good collision resistance and error detection\n\n\nSHA256 (256bit digest)\n\n\n\na cryptographic-strength hash, relatively slow but with possible CPU instruction\nacceleration or specialized hardware cards, FIPS certified and in wide use\n\n\nBLAKE2b (256bit digest)\n\n\n\na cryptographic-strength hash, relatively fast with possible CPU acceleration using SIMD\nextensions, not standardized but based on BLAKE which was a SHA3 finalist, in wide use,\nthe algorithm used is BLAKE2b-256 that’s optimized for 64bit platforms\n\n\nThe digest size affects overall size of data block checksums stored in the filesystem. The\nmetadata blocks have a fixed area up to 256bits (32 bytes), so there’s no increase. Each data\nblock has a separate checksum stored, with additional overhead of the b-tree leaves.\n\nApproximate relative performance of the algorithms, measured against CRC32C using reference\nsoftware implementations on a 3.5GHz intel CPU:\n\n[ cols=\"^,>,>,>\",width=\"50%\" ]\n\n┌────────┬─────────────┬───────┬─────────────────┐\n│        │             │       │                 │\n│Digest  │ Cycles/4KiB │ Ratio │ Implementation  │\n├────────┼─────────────┼───────┼─────────────────┤\n│        │             │       │                 │\n│CRC32C  │ 1700        │ 1.00  │ CPU instruction │\n├────────┼─────────────┼───────┼─────────────────┤\n│        │             │       │                 │\n│XXHASH  │ 2500        │ 1.44  │ reference impl. │\n├────────┼─────────────┼───────┼─────────────────┤\n│        │             │       │                 │\n│SHA256  │ 105000      │ 61    │ reference impl. │\n├────────┼─────────────┼───────┼─────────────────┤\n│        │             │       │                 │\n│SHA256  │ 36000       │ 21    │ libgcrypt/AVX2  │\n├────────┼─────────────┼───────┼─────────────────┤\n│        │             │       │                 │\n│SHA256  │ 63000       │ 37    │ libsodium/AVX2  │\n├────────┼─────────────┼───────┼─────────────────┤\n│        │             │       │                 │\n│BLAKE2b │ 22000       │ 13    │ reference impl. │\n├────────┼─────────────┼───────┼─────────────────┤\n│        │             │       │                 │\n│BLAKE2b │ 19000       │ 11    │ libgcrypt/AVX2  │\n├────────┼─────────────┼───────┼─────────────────┤\n│        │             │       │                 │\n│BLAKE2b │ 19000       │ 11    │ libsodium/AVX2  │\n└────────┴─────────────┴───────┴─────────────────┘\n\nMany kernels are configured with SHA256 as built-in and not as a module. The accelerated\nversions are however provided by the modules and must be loaded explicitly (modprobe sha256)\nbefore mounting the filesystem to make use of them. You can check in\n/sys/fs/btrfs/FSID/checksum which one is used. If you see sha256-generic, then you may want\nto unmount and mount the filesystem again, changing that on a mounted filesystem is not\npossible. Check the file /proc/crypto, when the implementation is built-in, you’d find\n\n\nname         : sha256\ndriver       : sha256-generic\nmodule       : kernel\npriority     : 100\n...\n\nwhile accelerated implementation is e.g.\n\n\nname         : sha256\ndriver       : sha256-avx2\nmodule       : sha256ssse3\npriority     : 170\n...\n\n### COMPRESSION\n\nBtrfs supports transparent file compression. There are three algorithms available: ZLIB, LZO\nand ZSTD (since v4.14). Basically, compression is on a file by file basis. You can have a\nsingle btrfs mount point that has some files that are uncompressed, some that are compressed\nwith LZO, some with ZLIB, for instance (though you may not want it that way, it is\nsupported).\n\nTo enable compression, mount the filesystem with options compress or compress-force. Please\nrefer to section MOUNT OPTIONS. Once compression is enabled, all new writes will be subject\nto compression. Some files may not compress very well, and these are typically not\nrecompressed but still written uncompressed.\n\nEach compression algorithm has different speed/ratio trade offs. The levels can be selected\nby a mount option and affect only the resulting size (ie. no compatibility issues).\n\nBasic characteristics:\n\n\nZLIB   slower, higher compression ratio\n\n\n•    levels: 1 to 9,\nmapped directly,\ndefault level is 3\n\n•    good backward\ncompatibility\n\nLZO    faster compression and\ndecompression than zlib, worse\ncompression ratio, designed to be\nfast\n\n\n•    no levels\n\n•    good backward\ncompatibility\n\nZSTD   compression comparable to zlib\nwith higher\ncompression/decompression speeds\nand different ratio\n\n\n•    levels: 1 to 15\n\n•    since 4.14, levels\nsince 5.1\n\n\nThe differences depend on the actual data set and cannot be expressed by a single number or\nrecommendation. Higher levels consume more CPU time and may not bring a significant\nimprovement, lower levels are close to real time.\n\nThe algorithms could be mixed in one file as they’re stored per extent. The compression can\nbe changed on a file by btrfs filesystem defrag command, using the -c option, or by btrfs\nproperty set using the compression property. Setting compression by chattr +c utility will\nset it to zlib.\n\nINCOMPRESSIBLE DATA\nFiles with already compressed data or with data that won’t compress well with the CPU and\nmemory constraints of the kernel implementations are using a simple decision logic. If the\nfirst portion of data being compressed is not smaller than the original, the compression of\nthe file is disabled — unless the filesystem is mounted with compress-force. In that case\ncompression will always be attempted on the file only to be later discarded. This is not\noptimal and subject to optimizations and further development.\n\nIf a file is identified as incompressible, a flag is set (NOCOMPRESS) and it’s sticky. On\nthat file compression won’t be performed unless forced. The flag can be also set by chattr +m\n(since e2fsprogs 1.46.2) or by properties with value no or none. Empty value will reset it to\nthe default that’s currently applicable on the mounted filesystem.\n\nThere are two ways to detect incompressible data:\n\n\n•    actual compression attempt - data are compressed, if the result is not smaller, it’s\ndiscarded, so this depends on the algorithm and level\n\n•    pre-compression heuristics - a quick statistical evaluation on the data is performed and\nbased on the result either compression is performed or skipped, the NOCOMPRESS bit is not\nset just by the heuristic, only if the compression algorithm does not make an improvement\n\nPRE-COMPRESSION HEURISTICS\nThe heuristics aim to do a few quick statistical tests on the compressed data in order to\navoid probably costly compression that would turn out to be inefficient. Compression\nalgorithms could have internal detection of incompressible data too but this leads to more\noverhead as the compression is done in another thread and has to write the data anyway. The\nheuristic is read-only and can utilize cached memory.\n\nThe tests performed based on the following: data sampling, long repeated pattern detection,\nbyte frequency, Shannon entropy.\n\n\nCOMPATIBILITY WITH OTHER FEATURES\nCompression is done using the COW mechanism so it’s incompatible with nodatacow. Direct IO\nworks on compressed files but will fall back to buffered writes. Currently nodatasum and\ncompression don’t work together.\n\n### FILESYSTEM EXCLUSIVE OPERATIONS\n\nThere are several operations that affect the whole filesystem and cannot be run in parallel.\nAttempt to start one while another is running will fail.\n\nSince kernel 5.10 the currently running operation can be obtained from\n/sys/fs/UUID/exclusiveoperation with following values and operations:\n\n\n•    balance\n\n•    device add\n\n•    device delete\n\n•    device replace\n\n•    resize\n\n•    swapfile activate\n\n•    none\n\nEnqueuing is supported for several btrfs subcommands so they can be started at once and then\nserialized.\n\n### FILESYSTEM LIMITS\n\nmaximum file name length\n\n\n\n255\n\n\nmaximum symlink target length\n\n\n\ndepends on the nodesize value, for 4k it’s 3949 bytes, for larger nodesize it’s 4095 due\nto the system limit PATHMAX\n\n\nThe symlink target may not be a valid path, ie. the path name components can exceed the\nlimits (NAMEMAX), there’s no content validation at symlink(3) creation.\n\n\nmaximum number of inodes\n\n\n\n264 but depends on the available metadata space as the inodes are created dynamically\n\n\ninode numbers\n\n\n\nminimum number: 256 (for subvolumes), regular files and directories: 257\n\n\nmaximum file length\n\n\n\ninherent limit of btrfs is 264 (16 EiB) but the linux VFS limit is 263 (8 EiB)\n\n\nmaximum number of subvolumes\n\n\n\nthe subvolume ids can go up to 264 but the number of actual subvolumes depends on the\navailable metadata space, the space consumed by all subvolume metadata includes\nbookkeeping of shared extents can be large (MiB, GiB)\n\n\nmaximum number of hardlinks of a file in a directory\n\n\n\n65536 when the extref feature is turned on during mkfs (default), roughly 100 otherwise\n\n\nminimum filesystem size\n\n\n\nthe minimal size of each device depends on the mixed-bg feature, without that (the\ndefault) it’s about 109MiB, with mixed-bg it’s is 16MiB\n\n### BOOTLOADER SUPPORT\n\nGRUB2 (https://www.gnu.org/software/grub) has the most advanced support of booting from BTRFS\nwith respect to features.\n\nU-boot (https://www.denx.de/wiki/U-Boot/) has decent support for booting but not all BTRFS\nfeatures are implemented, check the documentation.\n\nEXTLINUX (from the https://syslinux.org project) can boot but does not support all features.\nPlease check the upstream documentation before you use it.\n\nThe first 1MiB on each device is unused with the exception of primary superblock that is on\nthe offset 64KiB and spans 4KiB.\n\n### FILE ATTRIBUTES\n\nThe btrfs filesystem supports setting file attributes or flags. Note there are old and new\ninterfaces, with confusing names. The following list should clarify that:\n\n\n•    attributes: chattr(1) or lsattr(1) utilities (the ioctls are FSIOCGETFLAGS and\nFSIOCSETFLAGS), due to the ioctl names the attributes are also called flags\n\n•    xflags: to distinguish from the previous, it’s extended flags, with tunable bits similar\nto the attributes but extensible and new bits will be added in the future (the ioctls are\nFSIOCFSGETXATTR and FSIOCFSSETXATTR but they are not related to extended attributes\nthat are also called xattrs), there’s no standard tool to change the bits, there’s\nsupport in xfsio(8) as command xfsio -c chattr\n\nATTRIBUTES\na\n\n\n\nappend only, new writes are always written at the end of the file\n\n\nA\n\n\n\nno atime updates\n\n\nc\n\n\n\ncompress data, all data written after this attribute is set will be compressed. Please\nnote that compression is also affected by the mount options or the parent directory\nattributes.\n\n\nWhen set on a directory, all newly created files will inherit this attribute. This\nattribute cannot be set with m at the same time.\n\n\nC\n\n\n\nno copy-on-write, file data modifications are done in-place\n\n\nWhen set on a directory, all newly created files will inherit this attribute.\n\nNote\ndue to implementation limitations, this flag can be set/unset only on empty files.\n\nd\n\n\n\nno dump, makes sense with 3rd party tools like dump(8), on BTRFS the attribute can be\nset/unset but no other special handling is done\n\n\nD\n\n\n\nsynchronous directory updates, for more details search open(2) for OSYNC and ODSYNC\n\n\ni\n\n\n\nimmutable, no file data and metadata changes allowed even to the root user as long as\nthis attribute is set (obviously the exception is unsetting the attribute)\n\n\nm\n\n\n\nno compression, permanently turn off compression on the given file. Any compression mount\noptions will not affect this file. (chattr support added in 1.46.2)\n\n\nWhen set on a directory, all newly created files will inherit this attribute. This\nattribute cannot be set with c at the same time.\n\n\nS\n\n\n\nsynchronous updates, for more details search open(2) for OSYNC and ODSYNC\n\n\nNo other attributes are supported. For the complete list please refer to the chattr(1) manual\npage.\n\n\nXFLAGS\nThere’s overlap of letters assigned to the bits with the attributes, this list refers to what\nxfsio(8) provides:\n\n\n\ni\n\n\n\nimmutable, same as the attribute\n\n\na\n\n\n\nappend only, same as the attribute\n\n\ns\n\n\n\nsynchronous updates, same as the attribute S\n\n\nA\n\n\n\nno atime updates, same as the attribute\n\n\nd\n\n\n\nno dump, same as the attribute\n\n### ZONED MODE\n\nSince version 5.12 btrfs supports so called zoned mode. This is a special on-disk format and\nallocation/write strategy that’s friendly to zoned devices. In short, a device is partitioned\ninto fixed-size zones and each zone can be updated by append-only manner, or reset. As btrfs\nhas no fixed data structures, except the super blocks, the zoned mode only requires block\nplacement that follows the device constraints. You can learn about the whole architecture at\nhttps://zonedstorage.io .\n\nThe devices are also called SMR/ZBC/ZNS, in host-managed mode. Note that there are devices\nthat appear as non-zoned but actually are, this is drive-managed and using zoned mode won’t\nhelp.\n\nThe zone size depends on the device, typical sizes are 256MiB or 1GiB. In general it must be\na power of two. Emulated zoned devices like nullblk allow to set various zone sizes.\n\n#### REQUIREMENTS, LIMITATIONS\n\n•    all devices must have the same zone size\n\n•    maximum zone size is 8GiB\n\n•    mixing zoned and non-zoned devices is possible, the zone writes are emulated, but this\nis namely for testing\n\n•    the super block is handled in a special way and is at different locations than on a\nnon-zoned filesystem:\n\n•    primary: 0B (and the next two zones)\n\n•    secondary: 512G (and the next two zones)\n\n•    tertiary: 4TiB (4096GiB, and the next two zones)\n\nINCOMPATIBLE FEATURES\nThe main constraint of the zoned devices is lack of in-place update of the data. This is\ninherently incompatbile with some features:\n\n\n•    nodatacow - overwrite in-place, cannot create such files\n\n•    fallocate - preallocating space for in-place first write\n\n•    mixed-bg - unordered writes to data and metadata, fixing that means using separate data\nand metadata block groups\n\n•    booting - the zone at offset 0 contains superblock, resetting the zone would destroy the\nbootloader data\n\nInitial support lacks some features but they’re planned:\n\n\n•    only single profile is supported\n\n•    fstrim - due to dependency on free space cache v1\n\nSUPER BLOCK\nAs said above, super block is handled in a special way. In order to be crash safe, at least\none zone in a known location must contain a valid superblock. This is implemented as a ring\nbuffer in two consecutive zones, starting from known offsets 0, 512G and 4TiB. The values are\ndifferent than on non-zoned devices. Each new super block is appended to the end of the zone,\nonce it’s filled, the zone is reset and writes continue to the next one. Looking up the\nlatest super block needs to read offsets of both zones and determine the last written\nversion.\n\nThe amount of space reserved for super block depends on the zone size. The secondary and\ntertiary copies are at distant offsets as the capacity of the devices is expected to be\nlarge, tens of terabytes. Maximum zone size supported is 8GiB, which would mean that eg.\noffset 0-16GiB would be reserved just for the super block on a hypothetical device of that\nzone size. This is wasteful but required to guarantee crash safety.\n\n### CONTROL DEVICE\n\nThere’s a character special device /dev/btrfs-control with major and minor numbers 10 and 234\n(the device can be found under the misc category).\n\n\n$ ls -l /dev/btrfs-control\ncrw------- 1 root root 10, 234 Jan  1 12:00 /dev/btrfs-control\n\nThe device accepts some ioctl calls that can perform following actions on the filesystem\nmodule:\n\n\n•    scan devices for btrfs filesystem (ie. to let multi-device filesystems mount\nautomatically) and register them with the kernel module\n\n•    similar to scan, but also wait until the device scanning process is finished for a given\nfilesystem\n\n•    get the supported features (can be also found under /sys/fs/btrfs/features)\n\nThe device is created when btrfs is initialized, either as a module or a built-in\nfunctionality and makes sense only in connection with that. Running eg. mkfs without the\nmodule loaded will not register the device and will probably warn about that.\n\nIn rare cases when the module is loaded but the device is not present (most likely\naccidentally deleted), it’s possible to recreate it by\n\n\n# mknod --mode=600 /dev/btrfs-control c 10 234\n\nor (since 5.11) by a convenience command\n\n\n# btrfs rescue create-control-device\n\nThe control device is not strictly required but the device scanning will not work and a\nworkaround would need to be used to mount a multi-device filesystem. The mount option device\ncan trigger the device scanning during mount, see also btrfs device scan.\n\n### FILESYSTEM WITH MULTIPLE PROFILES\n\nIt is possible that a btrfs filesystem contains multiple block group profiles of the same\ntype. This could happen when a profile conversion using balance filters is interrupted (see\nbtrfs-balance(8)). Some btrfs commands perform a test to detect this kind of condition and\nprint a warning like this:\n\n\nWARNING: Multiple block group profiles detected, see 'man btrfs(5)'.\nWARNING:   Data: single, raid1\nWARNING:   Metadata: single, raid1\n\nThe corresponding output of btrfs filesystem df might look like:\n\n\nWARNING: Multiple block group profiles detected, see 'man btrfs(5)'.\nWARNING:   Data: single, raid1\nWARNING:   Metadata: single, raid1\nData, RAID1: total=832.00MiB, used=0.00B\nData, single: total=1.63GiB, used=0.00B\nSystem, single: total=4.00MiB, used=16.00KiB\nMetadata, single: total=8.00MiB, used=112.00KiB\nMetadata, RAID1: total=64.00MiB, used=32.00KiB\nGlobalReserve, single: total=16.25MiB, used=0.00B\n\nThere’s more than one line for type Data and Metadata, while the profiles are single and\nRAID1.\n\nThis state of the filesystem OK but most likely needs the user/administrator to take an\naction and finish the interrupted tasks. This cannot be easily done automatically, also the\nuser knows the expected final profiles.\n\nIn the example above, the filesystem started as a single device and single block group\nprofile. Then another device was added, followed by balance with convert=raid1 but for some\nreason hasn’t finished. Restarting the balance with convert=raid1 will continue and end up\nwith filesystem with all block group profiles RAID1.\n\nNote\nIf you’re familiar with balance filters, you can use convert=raid1,profiles=single,soft,\nwhich will take only the unconverted single profiles and convert them to raid1. This may\nspeed up the conversion as it would not try to rewrite the already convert raid1\nprofiles.\n\nHaving just one profile is desired as this also clearly defines the profile of newly\nallocated block groups, otherwise this depends on internal allocation policy. When there are\nmultiple profiles present, the order of selection is RAID6, RAID5, RAID10, RAID1, RAID0 as\nlong as the device number constraints are satisfied.\n\nCommands that print the warning were chosen so they’re brought to user attention when the\nfilesystem state is being changed in that regard. This is: device add, device delete, balance\ncancel, balance pause. Commands that report space usage: filesystem df, device usage. The\ncommand filesystem usage provides a line in the overall summary:\n\n\nMultiple profiles:                 yes (data, metadata)\n\n### SEEDING DEVICE\n\nThe COW mechanism and multiple devices under one hood enable an interesting concept, called a\nseeding device: extending a read-only filesystem on a single device filesystem with another\ndevice that captures all writes. For example imagine an immutable golden image of an\noperating system enhanced with another device that allows to use the data from the golden\nimage and normal operation. This idea originated on CD-ROMs with base OS and allowing to use\nthem for live systems, but this became obsolete. There are technologies providing similar\nfunctionality, like unionmount, overlayfs or qcow2 image snapshot.\n\nThe seeding device starts as a normal filesystem, once the contents is ready, btrfstune -S 1\nis used to flag it as a seeding device. Mounting such device will not allow any writes,\nexcept adding a new device by btrfs device add. Then the filesystem can be remounted as\nread-write.\n\nGiven that the filesystem on the seeding device is always recognized as read-only, it can be\nused to seed multiple filesystems, at the same time. The UUID that is normally attached to a\ndevice is automatically changed to a random UUID on each mount.\n\nOnce the seeding device is mounted, it needs the writable device. After adding it, something\nlike remount -o remount,rw /path makes the filesystem at /path ready for use. The simplest\nusecase is to throw away all changes by unmounting the filesystem when convenient.\n\nAlternatively, deleting the seeding device from the filesystem can turn it into a normal\nfilesystem, provided that the writable device can also contain all the data from the seeding\ndevice.\n\nThe seeding device flag can be cleared again by btrfstune -f -s 0, eg. allowing to update\nwith newer data but please note that this will invalidate all existing filesystems that use\nthis particular seeding device. This works for some usecases, not for others, and a forcing\nflag to the command is mandatory to avoid accidental mistakes.\n\nExample how to create and use one seeding device:\n\n\n# mkfs.btrfs /dev/sda\n# mount /dev/sda /mnt/mnt1\n# ... fill mnt1 with data\n# umount /mnt/mnt1\n# btrfstune -S 1 /dev/sda\n# mount /dev/sda /mnt/mnt1\n# btrfs device add /dev/sdb /mnt\n# mount -o remount,rw /mnt/mnt1\n# ... /mnt/mnt1 is now writable\n\nNow /mnt/mnt1 can be used normally. The device /dev/sda can be mounted again with a another\nwritable device:\n\n\n# mount /dev/sda /mnt/mnt2\n# btrfs device add /dev/sdc /mnt/mnt2\n# mount -o remount,rw /mnt/mnt2\n# ... /mnt/mnt2 is now writable\n\nThe writable device (/dev/sdb) can be decoupled from the seeding device and used\nindependently:\n\n\n# btrfs device delete /dev/sda /mnt/mnt1\n\nAs the contents originated in the seeding device, it’s possible to turn /dev/sdb to a seeding\ndevice again and repeat the whole process.\n\nA few things to note:\n\n\n•    it’s recommended to use only single device for the seeding device, it works for multiple\ndevices but the single profile must be used in order to make the seeding device deletion\nwork\n\n•    block group profiles single and dup support the usecases above\n\n•    the label is copied from the seeding device and can be changed by btrfs filesystem label\n\n•    each new mount of the seeding device gets a new random UUID\n\n### RAID56 STATUS AND RECOMMENDED PRACTICES\n\nThe RAID56 feature provides striping and parity over several devices, same as the traditional\nRAID5/6. There are some implementation and design deficiencies that make it unreliable for\nsome corner cases and the feature should not be used in production, only for evaluation or\ntesting. The power failure safety for metadata with RAID56 is not 100%.\n\n#### Metadata\n\nDo not use raid5 nor raid6 for metadata. Use raid1 or raid1c3 respectively.\n\nThe substitute profiles provide the same guarantees against loss of 1 or 2 devices, and in\nsome respect can be an improvement. Recovering from one missing device will only need to\naccess the remaining 1st or 2nd copy, that in general may be stored on some other devices due\nto the way RAID1 works on btrfs, unlike on a striped profile (similar to raid0) that would\nneed all devices all the time.\n\nThe space allocation pattern and consumption is different (eg. on N devices): for raid5 as an\nexample, a 1GiB chunk is reserved on each device, while with raid1 there’s each 1GiB chunk\nstored on 2 devices. The consumption of each 1GiB of used metadata is then N * 1GiB for vs 2\n* 1GiB. Using raid1 is also more convenient for balancing/converting to other profile due to\nlower requirement on the available chunk space.\n\n#### Missing/incomplete support\n\nWhen RAID56 is on the same filesystem with different raid profiles, the space reporting is\ninaccurate, eg. df, btrfs filesystem df or btrfs filesystem usge. When there’s only a one\nprofile per block group type (eg. raid5 for data) the reporting is accurate.\n\nWhen scrub is started on a RAID56 filesystem, it’s started on all devices that degrade the\nperformance. The workaround is to start it on each device separately. Due to that the device\nstats may not match the actual state and some errors might get reported multiple times.\n\nThe write hole problem.\n\n### STORAGE MODEL\n\nA storage model is a model that captures key physical aspects of data structure in a data\nstore. A filesystem is the logical structure organizing data on top of the storage device.\n\nThe filesystem assumes several features or limitations of the storage device and utilizes\nthem or applies measures to guarantee reliability. BTRFS in particular is based on a COW\n(copy on write) mode of writing, ie. not updating data in place but rather writing a new copy\nto a different location and then atomically switching the pointers.\n\nIn an ideal world, the device does what it promises. The filesystem assumes that this may not\nbe true so additional mechanisms are applied to either detect misbehaving hardware or get\nvalid data by other means. The devices may (and do) apply their own detection and repair\nmechanisms but we won’t assume any.\n\nThe following assumptions about storage devices are considered (sorted by importance, numbers\nare for further reference):\n\n\n1.  atomicity of reads and writes of blocks/sectors (the smallest unit of data the device\npresents to the upper layers)\n\n2.  there’s a flush command that instructs the device to forcibly order writes before and\nafter the command; alternatively there’s a barrier command that facilitates the ordering\nbut may not flush the data\n\n3.  data sent to write to a given device offset will be written without further changes to\nthe data and to the offset\n\n4.  writes can be reordered by the device, unless explicitly serialized by the flush command\n\n5.  reads and writes can be freely reordered and interleaved\n\nThe consistency model of BTRFS builds on these assumptions. The logical data updates are\ngrouped, into a generation, written on the device, serialized by the flush command and then\nthe super block is written ending the generation. All logical links among metadata comprising\na consistent view of the data may not cross the generation boundary.\n\nWHEN THINGS GO WRONG\n\n#### No or partial atomicity of block reads/writes (1)\n\n•    Problem: a partial block contents is written (torn write), eg. due to a power glitch or\nother electronics failure during the read/write\n\n•    Detection: checksum mismatch on read\n\n•    Repair: use another copy or rebuild from multiple blocks using some encoding scheme\n\n#### The flush command does not flush (2)\n\nThis is perhaps the most serious problem and impossible to mitigate by filesystem without\nlimitations and design restrictions. What could happen in the worst case is that writes from\none generation bleed to another one, while still letting the filesystem consider the\ngenerations isolated. Crash at any point would leave data on the device in an inconsistent\nstate without any hint what exactly got written, what is missing and leading to stale\nmetadata link information.\n\nDevices usually honor the flush command, but for performance reasons may do internal caching,\nwhere the flushed data are not yet persistently stored. A power failure could lead to a\nsimilar scenario as above, although it’s less likely that later writes would be written\nbefore the cached ones. This is beyond what a filesystem can take into account. Devices or\ncontrollers are usually equipped with batteries or capacitors to write the cache contents\neven after power is cut. (Battery backed write cache)\n\n#### Data get silently changed on write (3)\n\nSuch thing should not happen frequently, but still can happen spuriously due the complex\ninternal workings of devices or physical effects of the storage media itself.\n\n\n•    Problem: while the data are written atomically, the contents get changed\n\n•    Detection: checksum mismatch on read\n\n•    Repair: use another copy or rebuild from multiple blocks using some encoding scheme\n\n#### Data get silently written to another offset (3)\n\nThis would be another serious problem as the filesystem has no information when it happens.\nFor that reason the measures have to be done ahead of time. This problem is also commonly\ncalled ghost write.\n\nThe metadata blocks have the checksum embedded in the blocks, so a correct atomic write would\nnot corrupt the checksum. It’s likely that after reading such block the data inside would not\nbe consistent with the rest. To rule that out there’s embedded block number in the metadata\nblock. It’s the logical block number because this is what the logical structure expects and\nverifies.\n\n### HARDWARE CONSIDERATIONS\n\nThe following is based on information publicly available, user feedback, community\ndiscussions or bug report analyses. It’s not complete and further research is encouraged when\nin doubt.\n\nMAIN MEMORY\nThe data structures and raw data blocks are temporarily stored in computer memory before they\nget written to the device. It is critical that memory is reliable because even simple bit\nflips can have vast consequences and lead to damaged structures, not only in the filesystem\nbut in the whole operating system.\n\nBased on experience in the community, memory bit flips are more common than one would think.\nWhen it happens, it’s reported by the tree-checker or by a checksum mismatch after reading\nblocks. There are some very obvious instances of bit flips that happen, e.g. in an ordered\nsequence of keys in metadata blocks. We can easily infer from the other data what values get\ndamaged and how. However, fixing that is not straightforward and would require\ncross-referencing data from the entire filesystem to see the scope.\n\nIf available, ECC memory should lower the chances of bit flips, but this type of memory is\nnot available in all cases. A memory test should be performed in case there’s a visible bit\nflip pattern, though this may not detect a faulty memory module because the actual load of\nthe system could be the factor making the problems appear. In recent years attacks on how the\nmemory modules operate have been demonstrated (rowhammer) achieving specific bits to be\nflipped. While these were targeted, this shows that a series of reads or writes can affect\nunrelated parts of memory.\n\nFurther reading:\n\n\n•    https://en.wikipedia.org/wiki/Rowhammer\n\nWhat to do:\n\n\n•    run memtest, note that sometimes memory errors happen only when the system is under\nheavy load that the default memtest cannot trigger\n\n•    memory errors may appear as filesystem going read-only due to \"pre write\" check, that\nverify meta data before they get written but fail some basic consistency checks\n\n#### DIRECT MEMORY ACCESS (DMA)\n\nAnother class of errors is related to DMA (direct memory access) performed by device drivers.\nWhile this could be considered a software error, the data transfers that happen without CPU\nassistance may accidentally corrupt other pages. Storage devices utilize DMA for performance\nreasons, the filesystem structures and data pages are passed back and forth, making errors\npossible in case page life time is not properly tracked.\n\nThere are lots of quirks (device-specific workarounds) in Linux kernel drivers (regarding not\nonly DMA) that are added when found. The quirks may avoid specific errors or disable some\nfeatures to avoid worse problems.\n\nWhat to do:\n\n\n•    use up-to-date kernel (recent releases or maintained long term support versions)\n\n•    as this may be caused by faulty drivers, keep the systems up-to-date\n\n#### ROTATIONAL DISKS (HDD)\n\nRotational HDDs typically fail at the level of individual sectors or small clusters. Read\nfailures are caught on the levels below the filesystem and are returned to the user as EIO -\nInput/output error. Reading the blocks repeatedly may return the data eventually, but this is\nbetter done by specialized tools and filesystem takes the result of the lower layers.\nRewriting the sectors may trigger internal remapping but this inevitably leads to data loss.\n\nDisk firmware is technically software but from the filesystem perspective is part of the\nhardware. IO requests are processed, and caching or various other optimizations are\nperformed, which may lead to bugs under high load or unexpected physical conditions or\nunsupported use cases.\n\nDisks are connected by cables with two ends, both of which can cause problems when not\nattached properly. Data transfers are protected by checksums and the lower layers try hard to\ntransfer the data correctly or not at all. The errors from badly-connecting cables may\nmanifest as large amount of failed read or write requests, or as short error bursts depending\non physical conditions.\n\nWhat to do:\n\n\n•    check smartctl for potential issues\n\n#### SOLID STATE DRIVES (SSD)\n\nThe mechanism of information storage is different from HDDs and this affects the failure mode\nas well. The data are stored in cells grouped in large blocks with limited number of resets\nand other write constraints. The firmware tries to avoid unnecessary resets and performs\noptimizations to maximize the storage media lifetime. The known techniques are deduplication\n(blocks with same fingerprint/hash are mapped to same physical block), compression or\ninternal remapping and garbage collection of used memory cells. Due to the additional\nprocessing there are measures to verity the data e.g. by ECC codes.\n\nThe observations of failing SSDs show that the whole electronic fails at once or affects a\nlot of data (eg. stored on one chip). Recovering such data may need specialized equipment and\nreading data repeatedly does not help as it’s possible with HDDs.\n\nThere are several technologies of the memory cells with different characteristics and price.\nThe lifetime is directly affected by the type and frequency of data written. Writing \"too\nmuch\" distinct data (e.g. encrypted) may render the internal deduplication ineffective and\nlead to a lot of rewrites and increased wear of the memory cells.\n\nThere are several technologies and manufacturers so it’s hard to describe them but there are\nsome that exhibit similar behaviour:\n\n\n•    expensive SSD will use more durable memory cells and is optimized for reliability and\nhigh load\n\n•    cheap SSD is projected for a lower load (\"desktop user\") and is optimized for cost, it\nmay employ the optimizations and/or extended error reporting partially or not at all\n\nIt’s not possible to reliably determine the expected lifetime of an SSD due to lack of\ninformation about how it works or due to lack of reliable stats provided by the device.\n\nMetadata writes tend to be the biggest component of lifetime writes to a SSD, so there is\nsome value in reducing them. Depending on the device class (high end/low end) the features\nlike DUP block group profiles may affect the reliability in both ways:\n\n\n•    high end are typically more reliable and using single for data and metadata could be\nsuitable to reduce device wear\n\n•    low end could lack ability to identify errors so an additional redundancy at the\nfilesystem level (checksums, DUP) could help\n\nOnly users who consume 50 to 100% of the SSD’s actual lifetime writes need to be concerned by\nthe write amplification of btrfs DUP metadata. Most users will be far below 50% of the actual\nlifetime, or will write the drive to death and discover how many writes 100% of the actual\nlifetime was. SSD firmware often adds its own write multipliers that can be arbitrary and\nunpredictable and dependent on application behavior, and these will typically have far\ngreater effect on SSD lifespan than DUP metadata. It’s more or less impossible to predict\nwhen a SSD will run out of lifetime writes to within a factor of two, so it’s hard to justify\nwear reduction as a benefit.\n\nFurther reading:\n\n\n•    https://www.snia.org/educational-library/ssd-and-deduplication-end-spinning-disk-2012\n\n•    https://www.snia.org/educational-library/realities-solid-state-storage-2013-2013\n\n•    https://www.snia.org/educational-library/ssd-performance-primer-2013\n\n•    https://www.snia.org/educational-library/how-controllers-maximize-ssd-life-2013\n\nWhat to do:\n\n\n•    run smartctl or self-tests to look for potential issues\n\n•    keep the firmware up-to-date\n\n#### NVM EXPRESS, NON-VOLATILE MEMORY (NVMe)\n\nNVMe is a type of persistent memory usually connected over a system bus (PCIe) or similar\ninterface and the speeds are an order of magnitude faster than SSD. It is also a non-rotating\ntype of storage, and is not typically connected by a cable. It’s not a SCSI type device\neither but rather a complete specification for logical device interface.\n\nIn a way the errors could be compared to a combination of SSD class and regular memory.\nErrors may exhibit as random bit flips or IO failures. There are tools to access the internal\nlog (nvme log and nvme-cli) for a more detailed analysis.\n\nThere are separate error detection and correction steps performed e.g. on the bus level and\nin most cases never making in to the filesystem level. Once this happens it could mean\nthere’s some systematic error like overheating or bad physical connection of the device. You\nmay want to run self-tests (using smartctl).\n\n\n•    https://en.wikipedia.org/wiki/NVMExpress\n\n•    https://www.smartmontools.org/wiki/NVMeSupport\n\nDRIVE FIRMWARE\nFirmware is technically still software but embedded into the hardware. As all software has\nbugs, so does firmware. Storage devices can update the firmware and fix known bugs. In some\ncases the it’s possible to avoid certain bugs by quirks (device-specific workarounds) in\nLinux kernel.\n\nA faulty firmware can cause wide range of corruptions from small and localized to large\naffecting lots of data. Self-repair capabilities may not be sufficient.\n\nWhat to do:\n\n\n•    check for firmware updates in case there are known problems, note that updating firmware\ncan be risky on itself\n\n•    use up-to-date kernel (recent releases or maintained long term support versions)\n\nSD FLASH CARDS\nThere are a lot of devices with low power consumption and thus using storage media based on\nlow power consumption too, typically flash memory stored on a chip enclosed in a detachable\ncard package. An improperly inserted card may be damaged by electrical spikes when the device\nis turned on or off. The chips storing data in turn may be damaged permanently. All types of\nflash memory have a limited number of rewrites, so the data are internally translated by FTL\n(flash translation layer). This is implemented in firmware (technically a software) and prone\nto bugs that manifest as hardware errors.\n\nAdding redundancy like using DUP profiles for both data and metadata can help in some cases\nbut a full backup might be the best option once problems appear and replacing the card could\nbe required as well.\n\n#### HARDWARE AS THE MAIN SOURCE OF FILESYSTEM CORRUPTIONS\n\nIf you use unreliable hardware and don’’t know about that, don’’t blame the filesystem when it\n\n#### tells you.\n\n### SEE ALSO\n\nacl(5), btrfs(8), chattr(1), fstrim(8), ioctl(2), mkfs.btrfs(8), mount(8), swapon(8)\n\n\n\n\n\n2022-02-24                                BTRFS-MAN5(5)\n\n"
        }
    ],
    "structuredContent": {
        "command": "btrfs",
        "section": "5",
        "mode": "man",
        "summary": "btrfs-man5 - topics about the BTRFS filesystem (mount options, supported file attributes and other)",
        "synopsis": null,
        "flags": [],
        "examples": [],
        "see_also": [
            {
                "name": "acl",
                "section": "5",
                "url": "https://www.chedong.com/phpMan.php/man/acl/5/json"
            },
            {
                "name": "chattr",
                "section": "1",
                "url": "https://www.chedong.com/phpMan.php/man/chattr/1/json"
            },
            {
                "name": "fstrim",
                "section": "8",
                "url": "https://www.chedong.com/phpMan.php/man/fstrim/8/json"
            },
            {
                "name": "ioctl",
                "section": "2",
                "url": "https://www.chedong.com/phpMan.php/man/ioctl/2/json"
            },
            {
                "name": "mkfs.btrfs",
                "section": "8",
                "url": "https://www.chedong.com/phpMan.php/man/mkfs.btrfs/8/json"
            },
            {
                "name": "mount",
                "section": "8",
                "url": "https://www.chedong.com/phpMan.php/man/mount/8/json"
            },
            {
                "name": "swapon",
                "section": "8",
                "url": "https://www.chedong.com/phpMan.php/man/swapon/8/json"
            },
            {
                "name": "BTRFS-MAN5",
                "section": "5",
                "url": "https://www.chedong.com/phpMan.php/man/BTRFS-MAN5/5/json"
            }
        ],
        "section_outline": [
            {
                "name": "NAME",
                "lines": 3,
                "subsections": []
            },
            {
                "name": "DESCRIPTION",
                "lines": 35,
                "subsections": []
            },
            {
                "name": "MOUNT OPTIONS",
                "lines": 204,
                "subsections": [
                    {
                        "name": "degraded",
                        "lines": 145
                    },
                    {
                        "name": "nologreplay",
                        "lines": 48
                    },
                    {
                        "name": "norecovery",
                        "lines": 21
                    },
                    {
                        "name": "rescue",
                        "lines": 157
                    },
                    {
                        "name": "usebackuproot",
                        "lines": 33
                    },
                    {
                        "name": "recovery",
                        "lines": 23
                    },
                    {
                        "name": "noatime",
                        "lines": 16
                    }
                ]
            },
            {
                "name": "FILESYSTEM FEATURES",
                "lines": 159,
                "subsections": [
                    {
                        "name": "raid1c34",
                        "lines": 6
                    },
                    {
                        "name": "raid56",
                        "lines": 71
                    },
                    {
                        "name": "zoned",
                        "lines": 84
                    }
                ]
            },
            {
                "name": "CHECKSUM ALGORITHMS",
                "lines": 102,
                "subsections": []
            },
            {
                "name": "COMPRESSION",
                "lines": 100,
                "subsections": []
            },
            {
                "name": "FILESYSTEM EXCLUSIVE OPERATIONS",
                "lines": 25,
                "subsections": []
            },
            {
                "name": "FILESYSTEM LIMITS",
                "lines": 65,
                "subsections": []
            },
            {
                "name": "BOOTLOADER SUPPORT",
                "lines": 13,
                "subsections": []
            },
            {
                "name": "FILE ATTRIBUTES",
                "lines": 142,
                "subsections": []
            },
            {
                "name": "ZONED MODE",
                "lines": 14,
                "subsections": [
                    {
                        "name": "REQUIREMENTS, LIMITATIONS",
                        "lines": 55
                    }
                ]
            },
            {
                "name": "CONTROL DEVICE",
                "lines": 39,
                "subsections": []
            },
            {
                "name": "FILESYSTEM WITH MULTIPLE PROFILES",
                "lines": 56,
                "subsections": []
            },
            {
                "name": "SEEDING DEVICE",
                "lines": 74,
                "subsections": []
            },
            {
                "name": "RAID56 STATUS AND RECOMMENDED PRACTICES",
                "lines": 5,
                "subsections": [
                    {
                        "name": "Metadata",
                        "lines": 15
                    },
                    {
                        "name": "Missing/incomplete support",
                        "lines": 12
                    }
                ]
            },
            {
                "name": "STORAGE MODEL",
                "lines": 37,
                "subsections": [
                    {
                        "name": "No or partial atomicity of block reads/writes (1)",
                        "lines": 7
                    },
                    {
                        "name": "The flush command does not flush (2)",
                        "lines": 14
                    },
                    {
                        "name": "Data get silently changed on write (3)",
                        "lines": 10
                    },
                    {
                        "name": "Data get silently written to another offset (3)",
                        "lines": 12
                    }
                ]
            },
            {
                "name": "HARDWARE CONSIDERATIONS",
                "lines": 39,
                "subsections": [
                    {
                        "name": "DIRECT MEMORY ACCESS (DMA)",
                        "lines": 17
                    },
                    {
                        "name": "ROTATIONAL DISKS (HDD)",
                        "lines": 22
                    },
                    {
                        "name": "SOLID STATE DRIVES (SSD)",
                        "lines": 68
                    },
                    {
                        "name": "NVM EXPRESS, NON-VOLATILE MEMORY (NVMe)",
                        "lines": 50
                    },
                    {
                        "name": "HARDWARE AS THE MAIN SOURCE OF FILESYSTEM CORRUPTIONS",
                        "lines": 1
                    },
                    {
                        "name": "tells you.",
                        "lines": 1
                    }
                ]
            },
            {
                "name": "SEE ALSO",
                "lines": 7,
                "subsections": []
            }
        ]
    }
}