{ "content": [ { "type": "text", "text": "# lvmraid (man)\n\n## NAME\n\nlvmraid — LVM RAID\n\n## DESCRIPTION\n\nlvm(8) RAID is a way to create a Logical Volume (LV) that uses multiple physical devices to\nimprove performance or tolerate device failures. In LVM, the physical devices are Physical\nVolumes (PVs) in a single Volume Group (VG).\n\n## Sections\n\n- **NAME**\n- **DESCRIPTION** (9 subsections)\n- **Synchronization** (2 subsections)\n- **SubLVs** (23 subsections)\n- **Monitoring** (20 subsections)\n- **History**\n\nUse structuredContent.sections for detailed options, examples, and full documentation.\n" } ], "structuredContent": { "command": "lvmraid", "section": "", "mode": "man", "summary": "lvmraid — LVM RAID", "synopsis": null, "tldr_summary": null, "tldr_examples": [], "tldr_source": null, "flags": [ { "flag": "", "long": "--raidintegritymode", "arg": null, "description": "Use a journal (default) or bitmap for keeping integrity checksums consistent in case of a crash. The bitmap areas are recalculated after a crash, so corruption in those areas would not be detected. A journal does not have this problem. The journal mode doubles writes to storage, but can improve performance for scattered writes packed into a single journal write. bitmap mode can in theory achieve full write throughput of the device, but would not benefit from the potential scattered write optimization." }, { "flag": "", "long": "--raidintegrityblocksize", "arg": null, "description": "The block size to use for dm-integrity on raid images. The integrity block size should usu‐ ally match the device logical block size, or the file system sector/block sizes. It may be less than the file system sector/block size, but not less than the device logical block size. Possible values: 512, 1024, 2048, 4096." } ], "examples": [], "see_also": [], "section_outline": [ { "name": "NAME", "lines": 3, "subsections": [] }, { "name": "DESCRIPTION", "lines": 20, "subsections": [ { "name": "Create a RAID LV", "lines": 2 }, { "name": "raid10.", "lines": 11 }, { "name": "raid0", "lines": 16 }, { "name": "raid1", "lines": 14 }, { "name": "raid4", "lines": 22 }, { "name": "raid5", "lines": 26 }, { "name": "raid6", "lines": 25 }, { "name": "raid10", "lines": 4 }, { "name": "lvcreate --type raid10", "lines": 28 } ] }, { "name": "Synchronization", "lines": 31, "subsections": [ { "name": "Scrubbing", "lines": 54 }, { "name": "Scrubbing Limitations", "lines": 22 } ] }, { "name": "SubLVs", "lines": 5, "subsections": [ { "name": "lvs -a -o name,segtype,devices", "lines": 19 }, { "name": "Examples", "lines": 1 }, { "name": "raid0", "lines": 9 }, { "name": "raid1", "lines": 12 }, { "name": "raid4", "lines": 16 }, { "name": "raid5", "lines": 16 }, { "name": "raid6", "lines": 25 }, { "name": "raid10", "lines": 21 }, { "name": "Device Failure", "lines": 30 }, { "name": "partial", "lines": 3 }, { "name": "refresh needed", "lines": 6 }, { "name": "mismatches exist", "lines": 10 }, { "name": "Activating an LV with missing devices", "lines": 5 }, { "name": "complete", "lines": 2 }, { "name": "degraded", "lines": 3 }, { "name": "partial", "lines": 10 }, { "name": "Replacing Devices", "lines": 15 }, { "name": "Refreshing an LV", "lines": 19 }, { "name": "Automatic repair", "lines": 7 }, { "name": "warn", "lines": 3 }, { "name": "allocate", "lines": 9 }, { "name": "Corrupted Data", "lines": 6 }, { "name": "Rebuild specific PVs", "lines": 9 } ] }, { "name": "Monitoring", "lines": 12, "subsections": [ { "name": "Configuration Options", "lines": 11 }, { "name": "Data Integrity", "lines": 14 }, { "name": "lvcreate --type raidN --raidintegrity y", "lines": 9 }, { "name": "Integrity options", "lines": 1 }, { "name": "--raidintegritymode journal|bitmap", "lines": 7, "long": "--raidintegritymode" }, { "name": "--raidintegrityblocksize 512|1024|2048|4096", "lines": 6, "long": "--raidintegrityblocksize" }, { "name": "Integrity initialization", "lines": 7 }, { "name": "Integrity limitations", "lines": 19 }, { "name": "RAID1 Tuning", "lines": 18 }, { "name": "lvs -a -o name,attr", "lines": 15 }, { "name": "RAID Takeover", "lines": 64 }, { "name": "Indirect conversions", "lines": 3 }, { "name": "linear to raid6", "lines": 18 }, { "name": "linear to striped", "lines": 19 }, { "name": "raid6 to striped", "lines": 10 }, { "name": "Examples", "lines": 70 }, { "name": "RAID Reshaping", "lines": 13 }, { "name": "Examples", "lines": 449 }, { "name": "RAID5 Variants", "lines": 22 }, { "name": "RAID6 Variants", "lines": 51 } ] }, { "name": "History", "lines": 12, "subsections": [] } ], "sections": { "NAME": { "content": "lvmraid — LVM RAID\n\n", "subsections": [] }, "DESCRIPTION": { "content": "lvm(8) RAID is a way to create a Logical Volume (LV) that uses multiple physical devices to\nimprove performance or tolerate device failures. In LVM, the physical devices are Physical\nVolumes (PVs) in a single Volume Group (VG).\n\nHow LV data blocks are placed onto PVs is determined by the RAID level. RAID levels are com‐\nmonly referred to as 'raid' followed by a number, e.g. raid1, raid5 or raid6. Selecting a\nRAID level involves making tradeoffs among: physical device requirements, fault tolerance,\nand performance. A description of the RAID levels can be found at\nwww.snia.org/sites/default/files/SNIADDFTechnicalPositionv2.0.pdf\n\nLVM RAID uses both Device Mapper (DM) and Multiple Device (MD) drivers from the Linux kernel.\nDM is used to create and manage visible LVM devices, and MD is used to place data on physical\ndevices.\n\nLVM creates hidden LVs (dm devices) layered between the visible LV and physical devices. LVs\nin the middle layers are called sub LVs. For LVM raid, a sub LV pair to store data and meta‐\ndata (raid superblock and write intent bitmap) is created per raid image/leg (see lvs command\nexamples below).\n\n", "subsections": [ { "name": "Create a RAID LV", "content": "To create a RAID LV, use lvcreate and specify an LV type. The LV type corresponds to a RAID\nlevel. The basic RAID levels that can be used are: raid0, raid1, raid4, raid5, raid6," }, { "name": "raid10.", "content": "lvcreate --type RaidLevel [OPTIONS] --name Name --size Size VG [PVs]\n\nTo display the LV type of an existing LV, run:\n\nlvs -o name,segtype LV\n\n(The LV type is also referred to as \"segment type\" or \"segtype\".)\n\nLVs can be created with the following types:\n\n" }, { "name": "raid0", "content": "Also called striping, raid0 spreads LV data across multiple devices in units of stripe size.\nThis is used to increase performance. LV data will be lost if any of the devices fail.\n\nlvcreate --type raid0 [--stripes Number --stripesize Size] VG [PVs]\n\n\n--stripes specifies the number of devices to spread the LV across.\n\n\n--stripesize specifies the size of each stripe in kilobytes. This is the amount of data that\nis written to one device before moving to the next.\n\nPVs specifies the devices to use. If not specified, lvm will choose Number devices, one for\neach stripe based on the number of PVs available or supplied.\n\n" }, { "name": "raid1", "content": "Also called mirroring, raid1 uses multiple devices to duplicate LV data. The LV data remains\navailable if all but one of the devices fail. The minimum number of devices (i.e. sub LV\npairs) required is 2.\n\nlvcreate --type raid1 [--mirrors Number] VG [PVs]\n\n\n--mirrors specifies the number of mirror images in addition to the original LV image, e.g.\n--mirrors 1 means there are two images of the data, the original and one mirror image.\n\nPVs specifies the devices to use. If not specified, lvm will choose Number devices, one for\neach image.\n\n" }, { "name": "raid4", "content": "raid4 is a form of striping that uses an extra, first device dedicated to storing parity\nblocks. The LV data remains available if one device fails. The parity is used to recalcu‐\nlate data that is lost from a single device. The minimum number of devices required is 3.\n\nlvcreate --type raid4 [--stripes Number --stripesize Size] VG [PVs]\n\n\n--stripes specifies the number of devices to use for LV data. This does not include the ex‐\ntra device lvm adds for storing parity blocks. A raid4 LV with Number stripes re‐\nquires Number+1 devices. Number must be 2 or more.\n\n\n--stripesize specifies the size of each stripe in kilobytes. This is the amount of data that\nis written to one device before moving to the next.\n\nPVs specifies the devices to use. If not specified, lvm will choose Number+1 separate de‐\nvices.\n\nraid4 is called non-rotating parity because the parity blocks are always stored on the same\ndevice.\n\n" }, { "name": "raid5", "content": "raid5 is a form of striping that uses an extra device for storing parity blocks. LV data and\nparity blocks are stored on each device, typically in a rotating pattern for performance rea‐\nsons. The LV data remains available if one device fails. The parity is used to recalculate\ndata that is lost from a single device. The minimum number of devices required is 3 (unless\nconverting from 2 legged raid1 to reshape to more stripes; see reshaping).\n\nlvcreate --type raid5 [--stripes Number --stripesize Size] VG [PVs]\n\n\n--stripes specifies the number of devices to use for LV data. This does not include the ex‐\ntra device lvm adds for storing parity blocks. A raid5 LV with Number stripes re‐\nquires Number+1 devices. Number must be 2 or more.\n\n\n--stripesize specifies the size of each stripe in kilobytes. This is the amount of data that\nis written to one device before moving to the next.\n\nPVs specifies the devices to use. If not specified, lvm will choose Number+1 separate de‐\nvices.\n\nraid5 is called rotating parity because the parity blocks are placed on different devices in\na round-robin sequence. There are variations of raid5 with different algorithms for placing\nthe parity blocks. The default variant is raid5ls (raid5 left symmetric, which is a rotat‐\ning parity 0 with data restart.) See RAID5 variants below.\n\n" }, { "name": "raid6", "content": "raid6 is a form of striping like raid5, but uses two extra devices for parity blocks. LV\ndata and parity blocks are stored on each device, typically in a rotating pattern for perfom‐\nramce reasons. The LV data remains available if up to two devices fail. The parity is used\nto recalculate data that is lost from one or two devices. The minimum number of devices re‐\nquired is 5.\n\nlvcreate --type raid6 [--stripes Number --stripesize Size] VG [PVs]\n\n\n--stripes specifies the number of devices to use for LV data. This does not include the ex‐\ntra two devices lvm adds for storing parity blocks. A raid6 LV with Number stripes\nrequires Number+2 devices. Number must be 3 or more.\n\n\n--stripesize specifies the size of each stripe in kilobytes. This is the amount of data that\nis written to one device before moving to the next.\n\nPVs specifies the devices to use. If not specified, lvm will choose Number+2 separate de‐\nvices.\n\nLike raid5, there are variations of raid6 with different algorithms for placing the parity\nblocks. The default variant is raid6zr (raid6 zero restart, aka left symmetric, which is a\nrotating parity 0 with data restart.) See RAID6 variants below.\n\n" }, { "name": "raid10", "content": "raid10 is a combination of raid1 and raid0, striping data across mirrored devices. LV data\nremains available if one or more devices remains in each mirror set. The minimum number of\ndevices required is 4.\n" }, { "name": "lvcreate --type raid10", "content": "[--mirrors NumberMirrors]\n[--stripes NumberStripes --stripesize Size]\nVG [PVs]\n\n\n--mirrors specifies the number of mirror images within each stripe. e.g. --mirrors 1 means\nthere are two images of the data, the original and one mirror image.\n\n\n--stripes specifies the total number of devices to use in all raid1 images (not the number of\nraid1 devices to spread the LV across, even though that is the effective result). The\nnumber of devices in each raid1 mirror will be NumberStripes/(NumberMirrors+1), e.g.\nmirrors 1 and stripes 4 will stripe data across two raid1 mirrors, where each mirror\nis devices.\n\n\n--stripesize specifies the size of each stripe in kilobytes. This is the amount of data that\nis written to one device before moving to the next.\n\nPVs specifies the devices to use. If not specified, lvm will choose the necessary devices.\nDevices are used to create mirrors in the order listed, e.g. for mirrors 1, stripes 2, list‐\ning PV1 PV2 PV3 PV4 results in mirrors PV1/PV2 and PV3/PV4.\n\nRAID10 is not mirroring on top of stripes, which would be RAID01, which is less tolerant of\ndevice failures.\n\n\n" } ] }, "Synchronization": { "content": "Synchronization is the process that makes all the devices in a RAID LV consistent with each\nother.\n\nIn a RAID1 LV, all mirror images should have the same data. When a new mirror image is\nadded, or a mirror image is missing data, then images need to be synchronized. Data blocks\nare copied from an existing image to a new or outdated image to make them match.\n\nIn a RAID 4/5/6 LV, parity blocks and data blocks should match based on the parity calcula‐\ntion. When the devices in a RAID LV change, the data and parity blocks can become inconsis‐\ntent and need to be synchronized. Correct blocks are read, parity is calculated, and recal‐\nculated blocks are written.\n\nThe RAID implementation keeps track of which parts of a RAID LV are synchronized. When a\nRAID LV is first created and activated the first synchronization is called initialization. A\npointer stored in the raid metadata keeps track of the initialization process thus allowing\nit to be restarted after a deactivation of the RaidLV or a crash. Any writes to the RaidLV\ndirties the respective region of the write intent bitmap which allow for fast recovery of the\nregions after a crash. Without this, the entire LV would need to be synchronized every time\nit was activated.\n\nAutomatic synchronization happens when a RAID LV is activated, but it is usually partial be‐\ncause the bitmaps reduce the areas that are checked. A full sync becomes necessary when de‐\nvices in the RAID LV are replaced.\n\nThe synchronization status of a RAID LV is reported by the following command, where\n\"Cpy%Sync\" = \"100%\" means sync is complete:\n\nlvs -a -o name,syncpercent\n\n\n", "subsections": [ { "name": "Scrubbing", "content": "Scrubbing is a full scan of the RAID LV requested by a user. Scrubbing can find problems\nthat are missed by partial synchronization.\n\nScrubbing assumes that RAID metadata and bitmaps may be inaccurate, so it verifies all RAID\nmetadata, LV data, and parity blocks. Scrubbing can find inconsistencies caused by hardware\nerrors or degradation. These kinds of problems may be undetected by automatic synchroniza‐\ntion which excludes areas outside of the RAID write-intent bitmap.\n\nThe command to scrub a RAID LV can operate in two different modes:\n\nlvchange --syncaction check|repair LV\n\n\ncheck Check mode is read-only and only detects inconsistent areas in the RAID LV, it does not\ncorrect them.\n\n\nrepair Repair mode checks and writes corrected blocks to synchronize any inconsistent areas.\n\n\nScrubbing can consume a lot of bandwidth and slow down application I/O on the RAID LV. To\ncontrol the I/O rate used for scrubbing, use:\n\n\n--maxrecoveryrate Size[k|UNIT]\nSets the maximum recovery rate for a RAID LV. Size is specified as an amount per sec‐\nond for each device in the array. If no suffix is given, then KiB/sec/device is used.\nSetting the recovery rate to 0 means it will be unbounded.\n\n\n--minrecoveryrate Size[k|UNIT]\nSets the minimum recovery rate for a RAID LV. Size is specified as an amount per sec‐\nond for each device in the array. If no suffix is given, then KiB/sec/device is used.\nSetting the recovery rate to 0 means it will be unbounded.\n\n\nTo display the current scrubbing in progress on an LV, including the syncaction mode and per‐\ncent complete, run:\n\nlvs -a -o name,raidsyncaction,syncpercent\n\nAfter scrubbing is complete, to display the number of inconsistent blocks found, run:\n\nlvs -o name,raidmismatchcount\n\nAlso, if mismatches were found, the lvs attr field will display the letter \"m\" (mismatch) in\nthe 9th position, e.g.\n\n# lvs -o name,vgname,segtype,attr vg/lv\nLV VG Type Attr\nlv vg raid1 Rwi-a-r-m-\n\n\n" }, { "name": "Scrubbing Limitations", "content": "The check mode can only report the number of inconsistent blocks, it cannot report which\nblocks are inconsistent. This makes it impossible to know which device has errors, or if the\nerrors affect file system data, metadata or nothing at all.\n\nThe repair mode can make the RAID LV data consistent, but it does not know which data is cor‐\nrect. The result may be consistent but incorrect data. When two different blocks of data\nmust be made consistent, it chooses the block from the device that would be used during RAID\nintialization. However, if the PV holding corrupt data is known, lvchange --rebuild can be\nused in place of scrubbing to reconstruct the data on the bad device.\n\nFuture developments might include:\n\nAllowing a user to choose the correct version of data during repair.\n\nUsing a majority of devices to determine the correct version of data to use in a 3-way RAID1\nor RAID6 LV.\n\nUsing a checksumming device to pin-point when and where an error occurs, allowing it to be\nrewritten.\n\n\n" } ] }, "SubLVs": { "content": "An LV is often a combination of other hidden LVs called SubLVs. The SubLVs either use physi‐\ncal devices, or are built from other SubLVs themselves. SubLVs hold LV data blocks, RAID\nparity blocks, and RAID metadata. SubLVs are generally hidden, so the lvs -a option is re‐\nquired to display them:\n", "subsections": [ { "name": "lvs -a -o name,segtype,devices", "content": "SubLV names begin with the visible LV name, and have an automatic suffix indicating its role:\n\n\n• SubLVs holding LV data or parity blocks have the suffix rimage#. These SubLVs are some‐\ntimes referred to as DataLVs.\n\n\n• SubLVs holding RAID metadata have the suffix rmeta#. RAID metadata includes superblock\ninformation, RAID type, bitmap, and device health information. These SubLVs are sometimes\nreferred to as MetaLVs.\n\n\nSubLVs are an internal implementation detail of LVM. The way they are used, constructed and\nnamed may change.\n\nThe following examples show the SubLV arrangement for each of the basic RAID LV types, using\nthe fewest number of devices allowed for each.\n\n" }, { "name": "Examples", "content": "" }, { "name": "raid0", "content": "Each rimage SubLV holds a portion of LV data. No parity is used. No RAID metadata is used.\n\n# lvcreate --type raid0 --stripes 2 --name lvr0 ...\n\n# lvs -a -o name,segtype,devices\nlvr0 raid0 lvr0rimage0(0),lvr0rimage1(0)\n[lvr0rimage0] linear /dev/sda(...)\n[lvr0rimage1] linear /dev/sdb(...)\n" }, { "name": "raid1", "content": "Each rimage SubLV holds a complete copy of LV data. No parity is used. Each rmeta SubLV\nholds RAID metadata.\n\n# lvcreate --type raid1 --mirrors 1 --name lvr1 ...\n\n# lvs -a -o name,segtype,devices\nlvr1 raid1 lvr1rimage0(0),lvr1rimage1(0)\n[lvr1rimage0] linear /dev/sda(...)\n[lvr1rimage1] linear /dev/sdb(...)\n[lvr1rmeta0] linear /dev/sda(...)\n[lvr1rmeta1] linear /dev/sdb(...)\n" }, { "name": "raid4", "content": "At least three rimage SubLVs each hold a portion of LV data and one rimage SubLV holds par‐\nity. Each rmeta SubLV holds RAID metadata.\n\n# lvcreate --type raid4 --stripes 2 --name lvr4 ...\n\n# lvs -a -o name,segtype,devices\nlvr4 raid4 lvr4rimage0(0),\\\nlvr4rimage1(0),\\\nlvr4rimage2(0)\n[lvr4rimage0] linear /dev/sda(...)\n[lvr4rimage1] linear /dev/sdb(...)\n[lvr4rimage2] linear /dev/sdc(...)\n[lvr4rmeta0] linear /dev/sda(...)\n[lvr4rmeta1] linear /dev/sdb(...)\n[lvr4rmeta2] linear /dev/sdc(...)\n" }, { "name": "raid5", "content": "At least three rimage SubLVs each typcially hold a portion of LV data and parity (see section\non raid5) Each rmeta SubLV holds RAID metadata.\n\n# lvcreate --type raid5 --stripes 2 --name lvr5 ...\n\n# lvs -a -o name,segtype,devices\nlvr5 raid5 lvr5rimage0(0),\\\nlvr5rimage1(0),\\\nlvr5rimage2(0)\n[lvr5rimage0] linear /dev/sda(...)\n[lvr5rimage1] linear /dev/sdb(...)\n[lvr5rimage2] linear /dev/sdc(...)\n[lvr5rmeta0] linear /dev/sda(...)\n[lvr5rmeta1] linear /dev/sdb(...)\n[lvr5rmeta2] linear /dev/sdc(...)\n" }, { "name": "raid6", "content": "At least five rimage SubLVs each typically hold a portion of LV data and parity. (see sec‐\ntion on raid6) Each rmeta SubLV holds RAID metadata.\n\n# lvcreate --type raid6 --stripes 3 --name lvr6\n\n# lvs -a -o name,segtype,devices\nlvr6 raid6 lvr6rimage0(0),\\\nlvr6rimage1(0),\\\nlvr6rimage2(0),\\\nlvr6rimage3(0),\\\nlvr6rimage4(0),\\\nlvr6rimage5(0)\n[lvr6rimage0] linear /dev/sda(...)\n[lvr6rimage1] linear /dev/sdb(...)\n[lvr6rimage2] linear /dev/sdc(...)\n[lvr6rimage3] linear /dev/sdd(...)\n[lvr6rimage4] linear /dev/sde(...)\n[lvr6rimage5] linear /dev/sdf(...)\n[lvr6rmeta0] linear /dev/sda(...)\n[lvr6rmeta1] linear /dev/sdb(...)\n[lvr6rmeta2] linear /dev/sdc(...)\n[lvr6rmeta3] linear /dev/sdd(...)\n[lvr6rmeta4] linear /dev/sde(...)\n[lvr6rmeta5] linear /dev/sdf(...)\n" }, { "name": "raid10", "content": "At least four rimage SubLVs each hold a portion of LV data. No parity is used. Each rmeta\nSubLV holds RAID metadata.\n\n# lvcreate --type raid10 --stripes 2 --mirrors 1 --name lvr10\n\n# lvs -a -o name,segtype,devices\nlvr10 raid10 lvr10rimage0(0),\\\nlvr10rimage1(0),\\\nlvr10rimage2(0),\\\nlvr10rimage3(0)\n[lvr10rimage0] linear /dev/sda(...)\n[lvr10rimage1] linear /dev/sdb(...)\n[lvr10rimage2] linear /dev/sdc(...)\n[lvr10rimage3] linear /dev/sdd(...)\n[lvr10rmeta0] linear /dev/sda(...)\n[lvr10rmeta1] linear /dev/sdb(...)\n[lvr10rmeta2] linear /dev/sdc(...)\n[lvr10rmeta3] linear /dev/sdd(...)\n\n\n" }, { "name": "Device Failure", "content": "Physical devices in a RAID LV can fail or be lost for multiple reasons. A device could be\ndisconnected, permanently failed, or temporarily disconnected. The purpose of RAID LVs (lev‐\nels 1 and higher) is to continue operating in a degraded mode, without losing LV data, even\nafter a device fails. The number of devices that can fail without the loss of LV data de‐\npends on the RAID level:\n\n\n• RAID0 (striped) LVs cannot tolerate losing any devices. LV data will be lost if any de‐\nvices fail.\n\n\n• RAID1 LVs can tolerate losing all but one device without LV data loss.\n\n\n• RAID4 and RAID5 LVs can tolerate losing one device without LV data loss.\n\n\n• RAID6 LVs can tolerate losing two devices without LV data loss.\n\n\n• RAID10 is variable, and depends on which devices are lost. It stripes across multiple\nmirror groups with raid1 layout thus it can tolerate losing all but one device in each of\nthese groups without LV data loss.\n\n\nIf a RAID LV is missing devices, or has other device-related problems, lvs reports this in\nthe healthstatus (and attr) fields:\n\nlvs -o name,lvhealthstatus\n" }, { "name": "partial", "content": "Devices are missing from the LV. This is also indicated by the letter \"p\" (partial) in the\n9th position of the lvs attr field.\n" }, { "name": "refresh needed", "content": "A device was temporarily missing but has returned. The LV needs to be refreshed to use the\ndevice again (which will usually require partial synchronization). This is also indicated by\nthe letter \"r\" (refresh needed) in the 9th position of the lvs attr field. See Refreshing an\nLV. This could also indicate a problem with the device, in which case it should be be re‐\nplaced, see Replacing Devices.\n" }, { "name": "mismatches exist", "content": "See Scrubbing.\n\nMost commands will also print a warning if a device is missing, e.g.\nWARNING: Device for PV uItL3Z-wBME-DQy0-... not found or rejected ...\n\nThis warning will go away if the device returns or is removed from the VG (see vgreduce --re‐‐\nmovemissing).\n\n\n" }, { "name": "Activating an LV with missing devices", "content": "A RAID LV that is missing devices may be activated or not, depending on the \"activation mode\"\nused in lvchange:\n\nlvchange -ay --activationmode complete|degraded|partial LV\n" }, { "name": "complete", "content": "The LV is only activated if all devices are present.\n" }, { "name": "degraded", "content": "The LV is activated with missing devices if the RAID level can tolerate the number of missing\ndevices without LV data loss.\n" }, { "name": "partial", "content": "The LV is always activated, even if portions of the LV data are missing because of the miss‐\ning device(s). This should only be used to perform extreme recovery or repair operations.\n\nlvm.conf(5) activation/activationmode\ncontrols the activation mode when not specified by the command.\n\nThe default value is printed by:\nlvmconfig --type default activation/activationmode\n\n" }, { "name": "Replacing Devices", "content": "Devices in a RAID LV can be replaced by other devices in the VG. When replacing devices that\nare no longer visible on the system, use lvconvert --repair. When replacing devices that are\nstill visible, use lvconvert --replace. The repair command will attempt to restore the same\nnumber of data LVs that were previously in the LV. The replace option can be repeated to re‐\nplace multiple PVs. Replacement devices can be optionally listed with either option.\n\nlvconvert --repair LV [NewPVs]\n\nlvconvert --replace OldPV LV [NewPV]\n\nlvconvert --replace OldPV1 --replace OldPV2 LV [NewPVs]\n\nNew devices require synchronization with existing devices, see Synchronization.\n\n" }, { "name": "Refreshing an LV", "content": "Refreshing a RAID LV clears any transient device failures (device was temporarily discon‐\nnected) and returns the LV to its fully redundant mode. Restoring a device will usually re‐\nquire at least partial synchronization (see Synchronization). Failure to clear a transient\nfailure results in the RAID LV operating in degraded mode until it is reactivated. Use the\nlvchange command to refresh an LV:\n\nlvchange --refresh LV\n\n# lvs -o name,vgname,segtype,attr,size vg\nLV VG Type Attr LSize\nlv vg raid1 Rwi-a-r-r- 100.00g\n\n# lvchange --refresh vg/lv\n\n# lvs -o name,vgname,segtype,attr,size vg\nLV VG Type Attr LSize\nlv vg raid1 Rwi-a-r--- 100.00g\n\n" }, { "name": "Automatic repair", "content": "If a device in a RAID LV fails, device-mapper in the kernel notifies the dmeventd(8) monitor‐\ning process (see Monitoring). dmeventd can be configured to automatically respond using:\n\nlvm.conf(5) activation/raidfaultpolicy\n\nPossible settings are:\n" }, { "name": "warn", "content": "A warning is added to the system log indicating that a device has failed in the RAID LV. It\nis left to the user to repair the LV, e.g. replace failed devices.\n" }, { "name": "allocate", "content": "dmeventd automatically attempts to repair the LV using spare devices in the VG. Note that\neven a transient failure is treated as a permanent failure under this setting. A new device\nis allocated and full synchronization is started.\n\nThe specific command run by dmeventd to warn or repair is:\nlvconvert --repair --use-policies LV\n\n\n" }, { "name": "Corrupted Data", "content": "Data on a device can be corrupted due to hardware errors without the device ever being dis‐\nconnected or there being any fault in the software. This should be rare, and can be detected\n(see Scrubbing).\n\n\n" }, { "name": "Rebuild specific PVs", "content": "If specific PVs in a RAID LV are known to have corrupt data, the data on those PVs can be re‐\nconstructed with:\n\nlvchange --rebuild PV LV\n\nThe rebuild option can be repeated with different PVs to replace the data on multiple PVs.\n\n\n" } ] }, "Monitoring": { "content": "When a RAID LV is activated the dmeventd(8) process is started to monitor the health of the\nLV. Various events detected in the kernel can cause a notification to be sent from device-\nmapper to the monitoring process, including device failures and synchronization completion\n(e.g. for initialization or scrubbing).\n\nThe LVM configuration file contains options that affect how the monitoring process will re‐\nspond to failure events (e.g. raidfaultpolicy). It is possible to turn on and off monitor‐\ning with lvchange, but it is not recommended to turn this off unless you have a thorough\nknowledge of the consequences.\n\n\n", "subsections": [ { "name": "Configuration Options", "content": "There are a number of options in the LVM configuration file that affect the behavior of RAID\nLVs. The tunable options are listed below. A detailed description of each can be found in\nthe LVM configuration file itself.\nmirrorsegtypedefault\nraid10segtypedefault\nraidregionsize\nraidfaultpolicy\nactivationmode\n\n\n" }, { "name": "Data Integrity", "content": "The device mapper integrity target can be used in combination with RAID levels 1,4,5,6,10 to\ndetect and correct data corruption in RAID images. A dm-integrity layer is placed above each\nRAID image, and an extra sub LV is created to hold integrity metadata (data checksums) for\neach RAID image. When data is read from an image, integrity checksums are used to detect\ncorruption. If detected, dm-raid reads the data from another (good) image to return to the\ncaller. dm-raid will also automatically write the good data back to the image with bad data\nto correct the corruption.\n\nWhen creating a RAID LV with integrity, or adding integrity, space is required for integrity\nmetadata. Every 500MB of LV data requires an additional 4MB to be allocated for integrity\nmetadata, for each RAID image.\n\nCreate a RAID LV with integrity:\n" }, { "name": "lvcreate --type raidN --raidintegrity y", "content": "Add integrity to an existing RAID LV:\n\nlvconvert --raidintegrity y LV\n\nRemove integrity from a RAID LV:\n\nlvconvert --raidintegrity n LV\n\n" }, { "name": "Integrity options", "content": "" }, { "name": "--raidintegritymode journal|bitmap", "content": "Use a journal (default) or bitmap for keeping integrity checksums consistent in case of a\ncrash. The bitmap areas are recalculated after a crash, so corruption in those areas would\nnot be detected. A journal does not have this problem. The journal mode doubles writes to\nstorage, but can improve performance for scattered writes packed into a single journal write.\nbitmap mode can in theory achieve full write throughput of the device, but would not benefit\nfrom the potential scattered write optimization.\n", "long": "--raidintegritymode" }, { "name": "--raidintegrityblocksize 512|1024|2048|4096", "content": "The block size to use for dm-integrity on raid images. The integrity block size should usu‐\nally match the device logical block size, or the file system sector/block sizes. It may be\nless than the file system sector/block size, but not less than the device logical block size.\nPossible values: 512, 1024, 2048, 4096.\n\n", "long": "--raidintegrityblocksize" }, { "name": "Integrity initialization", "content": "When integrity is added to an LV, the kernel needs to initialize the integrity metadata\n(checksums) for all blocks in the LV. The data corruption checking performed by dm-integrity\nwill only operate on areas of the LV that are already initialized. The progress of integrity\ninitialization is reported by the \"syncpercent\" LV reporting field (and under the Cpy%Sync\nlvs column.)\n\n" }, { "name": "Integrity limitations", "content": "To work around some limitations, it is possible to remove integrity from the LV, make the\nchange, then add integrity again. (Integrity metadata would need to initialized when added\nagain.)\n\nLVM must be able to allocate the integrity metadata sub LV on a single PV that is already in\nuse by the associated RAID image. This can potentially cause a problem during lvextend if the\noriginal PV holding the image and integrity metadata is full. To work around this limita‐\ntion, remove integrity, extend the LV, and add integrity again.\n\nAdditional RAID images can be added to raid1 LVs, but not to other raid levels.\n\nA raid1 LV with integrity cannot be converted to linear (remove integrity to do this.)\n\nRAID LVs with integrity cannot yet be used as sub LVs with other LV types.\n\nThe following are not yet permitted on RAID LVs with integrity: lvreduce, pvmove, snapshots,\nsplitmirror, raid syncaction commands, raid rebuild.\n\n" }, { "name": "RAID1 Tuning", "content": "A RAID1 LV can be tuned so that certain devices are avoided for reading while all devices are\nstill written to.\n\nlvchange --[raid]writemostly PV[:y|n|t] LV\n\nThe specified device will be marked as \"write mostly\", which means that reading from this de‐\nvice will be avoided, and other devices will be preferred for reading (unless no other de‐\nvices are available.) This minimizes the I/O to the specified device.\n\nIf the PV name has no suffix, the write mostly attribute is set. If the PV name has the suf‐\nfix :n, the write mostly attribute is cleared, and the suffix :t toggles the current setting.\n\nThe write mostly option can be repeated on the command line to change multiple devices at\nonce.\n\nTo report the current write mostly setting, the lvs attr field will show the letter \"w\" in\nthe 9th position when write mostly is set:\n" }, { "name": "lvs -a -o name,attr", "content": "When a device is marked write mostly, the maximum number of outstanding writes to that device\ncan be configured. Once the maximum is reached, further writes become synchronous. When\nsynchronous, a write to the LV will not complete until writes to all the mirror images are\ncomplete.\n\nlvchange --[raid]writebehind Number LV\n\nTo report the current write behind setting, run:\n\nlvs -o name,raidwritebehind\n\nWhen write behind is not configured, or set to 0, all LV writes are synchronous.\n\n\n" }, { "name": "RAID Takeover", "content": "RAID takeover is converting a RAID LV from one RAID level to another, e.g. raid5 to raid6.\nChanging the RAID level is usually done to increase or decrease resilience to device failures\nor to restripe LVs. This is done using lvconvert and specifying the new RAID level as the LV\ntype:\n\nlvconvert --type RaidLevel LV [PVs]\n\nThe most common and recommended RAID takeover conversions are:\n\n\nlinear to raid1\nLinear is a single image of LV data, and converting it to raid1 adds a mirror image\nwhich is a direct copy of the original linear image.\n\n\nstriped/raid0 to raid4/5/6\nAdding parity devices to a striped volume results in raid4/5/6.\n\n\nUnnatural conversions that are not recommended include converting between striped and non-\nstriped types. This is because file systems often optimize I/O patterns based on device\nstriping values. If those values change, it can decrease performance.\n\nConverting to a higher RAID level requires allocating new SubLVs to hold RAID metadata, and\nnew SubLVs to hold parity blocks for LV data. Converting to a lower RAID level removes the\nSubLVs that are no longer needed.\n\nConversion often requires full synchronization of the RAID LV (see Synchronization). Con‐\nverting to RAID1 requires copying all LV data blocks to N new images on new devices. Con‐\nverting to a parity RAID level requires reading all LV data blocks, calculating parity, and\nwriting the new parity blocks. Synchronization can take a long time depending on the\nthroughpout of the devices used and the size of the RaidLV. It can degrade performance (rate\ncontrols also apply to conversion; see --minrecoveryrate and --maxrecoveryrate.)\n\nWarning: though it is possible to create striped LVs with up to 128 stripes, a maximum of 64\nstripes can be converted to raid0, 63 to raid4/5 and 62 to raid6 because of the added parity\nSubLVs. A striped LV with a maximum of 32 stripes can be converted to raid10.\n\n\nThe following takeover conversions are currently possible:\n\n• between striped and raid0.\n\n• between linear and raid1.\n\n• between mirror and raid1.\n\n• between raid1 with two images and raid4/5.\n\n• between striped/raid0 and raid4.\n\n• between striped/raid0 and raid5.\n\n• between striped/raid0 and raid6.\n\n• between raid4 and raid5.\n\n• between raid4/raid5 and raid6.\n\n• between striped/raid0 and raid10.\n\n• between striped and raid4.\n\n" }, { "name": "Indirect conversions", "content": "Converting from one raid level to another may require multiple steps, converting first to in‐\ntermediate raid levels.\n" }, { "name": "linear to raid6", "content": "To convert an LV from linear to raid6:\n1. convert to raid1 with two images\n2. convert to raid5 (internally raid5ls) with two images\n3. convert to raid5 with three or more stripes (reshape)\n4. convert to raid6 (internally raid6ls6)\n5. convert to raid6 (internally raid6zr, reshape)\n\nThe commands to perform the steps above are:\n1. lvconvert --type raid1 --mirrors 1 LV\n2. lvconvert --type raid5 LV\n3. lvconvert --stripes 3 LV\n4. lvconvert --type raid6 LV\n5. lvconvert --type raid6 LV\n\nThe final conversion from raid6ls6 to raid6zr is done to avoid the potential write/recov‐\nery performance reduction in raid6ls6 because of the dedicated parity device. raid6zr ro‐\ntates data and parity blocks to avoid this.\n" }, { "name": "linear to striped", "content": "To convert an LV from linear to striped:\n1. convert to raid1 with two images\n2. convert to raid5n\n3. convert to raid5n with five 128k stripes (reshape)\n4. convert raid5n to striped\n\nThe commands to perform the steps above are:\n1. lvconvert --type raid1 --mirrors 1 LV\n2. lvconvert --type raid5n LV\n3. lvconvert --stripes 5 --stripesize 128k LV\n4. lvconvert --type striped LV\n\nThe raid5n type in step 2 is used because it has dedicated parity SubLVs at the end, and can\nbe converted to striped directly. The stripe size is increased in step 3 to add extra space\nfor the conversion process. This step grows the LV size by a factor of five. After conver‐\nsion, this extra space can be reduced (or used to grow the file system using the LV).\n\nReversing these steps will convert a striped LV to linear.\n" }, { "name": "raid6 to striped", "content": "To convert an LV from raid6nr to striped:\n1. convert to raid6n6\n2. convert to striped\n\nThe commands to perform the steps above are:\n1. lvconvert --type raid6n6 LV\n2. lvconvert --type striped LV\n\n\n" }, { "name": "Examples", "content": "Converting an LV from linear to raid1.\n\n# lvs -a -o name,segtype,size vg\nLV Type LSize\nlv linear 300.00g\n\n# lvconvert --type raid1 --mirrors 1 vg/lv\n\n# lvs -a -o name,segtype,size vg\nLV Type LSize\nlv raid1 300.00g\n[lvrimage0] linear 300.00g\n[lvrimage1] linear 300.00g\n[lvrmeta0] linear 3.00m\n[lvrmeta1] linear 3.00m\n\nConverting an LV from mirror to raid1.\n\n# lvs -a -o name,segtype,size vg\nLV Type LSize\nlv mirror 100.00g\n[lvmimage0] linear 100.00g\n[lvmimage1] linear 100.00g\n[lvmlog] linear 3.00m\n\n# lvconvert --type raid1 vg/lv\n\n# lvs -a -o name,segtype,size vg\nLV Type LSize\nlv raid1 100.00g\n[lvrimage0] linear 100.00g\n[lvrimage1] linear 100.00g\n[lvrmeta0] linear 3.00m\n[lvrmeta1] linear 3.00m\n\nConverting an LV from linear to raid1 (with 3 images).\n\n# lvconvert --type raid1 --mirrors 2 vg/lv\n\nConverting an LV from striped (with 4 stripes) to raid6n6.\n\n# lvcreate --stripes 4 -L64M -n lv vg\n\n# lvconvert --type raid6 vg/lv\n\n# lvs -a -o lvname,segtype,syncpercent,datacopies\nLV Type Cpy%Sync #Cpy\nlv raid6n6 100.00 3\n[lvrimage0] linear\n[lvrimage1] linear\n[lvrimage2] linear\n[lvrimage3] linear\n[lvrimage4] linear\n[lvrimage5] linear\n[lvrmeta0] linear\n[lvrmeta1] linear\n[lvrmeta2] linear\n[lvrmeta3] linear\n[lvrmeta4] linear\n[lvrmeta5] linear\n\nThis convert begins by allocating MetaLVs (rmeta#) for each of the existing stripe devices.\nIt then creates 2 additional MetaLV/DataLV pairs (rmeta#/rimage#) for dedicated raid6 par‐\nity.\n\nIf rotating data/parity is required, such as with raid6nr, it must be done by reshaping (see\nbelow).\n\n\n" }, { "name": "RAID Reshaping", "content": "RAID reshaping is changing attributes of a RAID LV while keeping the same RAID level. This\nincludes changing RAID layout, stripe size, or number of stripes.\n\nWhen changing the RAID layout or stripe size, no new SubLVs (MetaLVs or DataLVs) need to be\nallocated, but DataLVs are extended by a small amount (typically 1 extent). The extra space\nallows blocks in a stripe to be updated safely, and not be corrupted in case of a crash. If\na crash occurs, reshaping can just be restarted.\n\n(If blocks in a stripe were updated in place, a crash could leave them partially updated and\ncorrupted. Instead, an existing stripe is quiesced, read, changed in layout, and the new\nstripe written to free space. Once that is done, the new stripe is unquiesced and used.)\n\n" }, { "name": "Examples", "content": "(Command output shown in examples may change.)\n\nConverting raid6n6 to raid6nr with rotating data/parity.\n\nThis conversion naturally follows a previous conversion from striped/raid0 to raid6n6\n(shown above). It completes the transition to a more traditional RAID6.\n\n# lvs -o lvname,segtype,syncpercent,datacopies\nLV Type Cpy%Sync #Cpy\nlv raid6n6 100.00 3\n[lvrimage0] linear\n[lvrimage1] linear\n[lvrimage2] linear\n[lvrimage3] linear\n[lvrimage4] linear\n[lvrimage5] linear\n[lvrmeta0] linear\n[lvrmeta1] linear\n[lvrmeta2] linear\n[lvrmeta3] linear\n[lvrmeta4] linear\n[lvrmeta5] linear\n\n# lvconvert --type raid6nr vg/lv\n\n# lvs -a -o lvname,segtype,syncpercent,datacopies\nLV Type Cpy%Sync #Cpy\nlv raid6nr 100.00 3\n[lvrimage0] linear\n[lvrimage0] linear\n[lvrimage1] linear\n[lvrimage1] linear\n[lvrimage2] linear\n[lvrimage2] linear\n[lvrimage3] linear\n[lvrimage3] linear\n[lvrimage4] linear\n[lvrimage5] linear\n[lvrmeta0] linear\n[lvrmeta1] linear\n[lvrmeta2] linear\n[lvrmeta3] linear\n[lvrmeta4] linear\n[lvrmeta5] linear\n\nThe DataLVs are larger (additional segment in each) which provides space for out-of-place re‐\nshaping. The result is:\n\n# lvs -a -o lvname,segtype,segperanges,dataoffset\nLV Type PE Ranges DOff\nlv raid6nr lvrimage0:0-32 \\\nlvrimage1:0-32 \\\nlvrimage2:0-32 \\\nlvrimage3:0-32\n[lvrimage0] linear /dev/sda:0-31 2048\n[lvrimage0] linear /dev/sda:33-33\n[lvrimage1] linear /dev/sdaa:0-31 2048\n[lvrimage1] linear /dev/sdaa:33-33\n[lvrimage2] linear /dev/sdab:1-33 2048\n[lvrimage3] linear /dev/sdac:1-33 2048\n[lvrmeta0] linear /dev/sda:32-32\n[lvrmeta1] linear /dev/sdaa:32-32\n[lvrmeta2] linear /dev/sdab:0-0\n[lvrmeta3] linear /dev/sdac:0-0\n\nAll segments with PE ranges '33-33' provide the out-of-place reshape space. The dataoffset\ncolumn shows that the data was moved from initial offset 0 to 2048 sectors on each component\nDataLV.\n\nFor performance reasons the raid6nr RaidLV can be restriped. Convert it from 3-way striped\nto 5-way-striped.\n\n# lvconvert --stripes 5 vg/lv\nUsing default stripesize 64.00 KiB.\nWARNING: Adding stripes to active logical volume vg/lv will \\\ngrow it from 99 to 165 extents!\nRun \"lvresize -l99 vg/lv\" to shrink it or use the additional \\\ncapacity.\nLogical volume vg/lv successfully converted.\n\n# lvs vg/lv\nLV VG Attr LSize Cpy%Sync\nlv vg rwi-a-r-s- 652.00m 52.94\n\n# lvs -a -o lvname,attr,segtype,segperanges,dataoffset vg\nLV Attr Type PE Ranges DOff\nlv rwi-a-r--- raid6nr lvrimage0:0-33 \\\nlvrimage1:0-33 \\\nlvrimage2:0-33 ... \\\nlvrimage5:0-33 \\\nlvrimage6:0-33 0\n[lvrimage0] iwi-aor--- linear /dev/sda:0-32 0\n[lvrimage0] iwi-aor--- linear /dev/sda:34-34\n[lvrimage1] iwi-aor--- linear /dev/sdaa:0-32 0\n[lvrimage1] iwi-aor--- linear /dev/sdaa:34-34\n[lvrimage2] iwi-aor--- linear /dev/sdab:0-32 0\n[lvrimage2] iwi-aor--- linear /dev/sdab:34-34\n[lvrimage3] iwi-aor--- linear /dev/sdac:1-34 0\n[lvrimage4] iwi-aor--- linear /dev/sdad:1-34 0\n[lvrimage5] iwi-aor--- linear /dev/sdae:1-34 0\n[lvrimage6] iwi-aor--- linear /dev/sdaf:1-34 0\n[lvrmeta0] ewi-aor--- linear /dev/sda:33-33\n[lvrmeta1] ewi-aor--- linear /dev/sdaa:33-33\n[lvrmeta2] ewi-aor--- linear /dev/sdab:33-33\n[lvrmeta3] ewi-aor--- linear /dev/sdac:0-0\n[lvrmeta4] ewi-aor--- linear /dev/sdad:0-0\n[lvrmeta5] ewi-aor--- linear /dev/sdae:0-0\n[lvrmeta6] ewi-aor--- linear /dev/sdaf:0-0\n\nStripes also can be removed from raid5 and 6. Convert the 5-way striped raid6nr LV to\n4-way-striped. The force option needs to be used, because removing stripes (i.e. image Sub‐\nLVs) from a RaidLV will shrink its size.\n\n# lvconvert --stripes 4 vg/lv\nUsing default stripesize 64.00 KiB.\nWARNING: Removing stripes from active logical volume vg/lv will \\\nshrink it from 660.00 MiB to 528.00 MiB!\nTHIS MAY DESTROY (PARTS OF) YOUR DATA!\nIf that leaves the logical volume larger than 206 extents due \\\nto stripe rounding,\nyou may want to grow the content afterwards (filesystem etc.)\nWARNING: to remove freed stripes after the conversion has finished,\\\nyou have to run \"lvconvert --stripes 4 vg/lv\"\nLogical volume vg/lv successfully converted.\n\n# lvs -a -o lvname,attr,segtype,segperanges,dataoffset vg\nLV Attr Type PE Ranges DOff\nlv rwi-a-r-s- raid6nr lvrimage0:0-33 \\\nlvrimage1:0-33 \\\nlvrimage2:0-33 ... \\\nlvrimage5:0-33 \\\nlvrimage6:0-33 0\n[lvrimage0] Iwi-aor--- linear /dev/sda:0-32 0\n[lvrimage0] Iwi-aor--- linear /dev/sda:34-34\n[lvrimage1] Iwi-aor--- linear /dev/sdaa:0-32 0\n[lvrimage1] Iwi-aor--- linear /dev/sdaa:34-34\n[lvrimage2] Iwi-aor--- linear /dev/sdab:0-32 0\n[lvrimage2] Iwi-aor--- linear /dev/sdab:34-34\n[lvrimage3] Iwi-aor--- linear /dev/sdac:1-34 0\n[lvrimage4] Iwi-aor--- linear /dev/sdad:1-34 0\n[lvrimage5] Iwi-aor--- linear /dev/sdae:1-34 0\n[lvrimage6] Iwi-aor-R- linear /dev/sdaf:1-34 0\n[lvrmeta0] ewi-aor--- linear /dev/sda:33-33\n[lvrmeta1] ewi-aor--- linear /dev/sdaa:33-33\n[lvrmeta2] ewi-aor--- linear /dev/sdab:33-33\n[lvrmeta3] ewi-aor--- linear /dev/sdac:0-0\n[lvrmeta4] ewi-aor--- linear /dev/sdad:0-0\n[lvrmeta5] ewi-aor--- linear /dev/sdae:0-0\n[lvrmeta6] ewi-aor-R- linear /dev/sdaf:0-0\n\nThe 's' in column 9 of the attribute field shows the RaidLV is still reshaping. The 'R' in\nthe same column of the attribute field shows the freed image Sub LVs which will need removing\nonce the reshaping finished.\n\n# lvs -o lvname,attr,segtype,segperanges,dataoffset vg\nLV Attr Type PE Ranges DOff\nlv rwi-a-r-R- raid6nr lvrimage0:0-33 \\\nlvrimage1:0-33 \\\nlvrimage2:0-33 ... \\\nlvrimage5:0-33 \\\nlvrimage6:0-33 8192\n\nNow that the reshape is finished the 'R' atribute on the RaidLV shows images can be removed.\n\n# lvs -o lvname,attr,segtype,segperanges,dataoffset vg\nLV Attr Type PE Ranges DOff\nlv rwi-a-r-R- raid6nr lvrimage0:0-33 \\\nlvrimage1:0-33 \\\nlvrimage2:0-33 ... \\\nlvrimage5:0-33 \\\nlvrimage6:0-33 8192\n\nThis is achieved by repeating the command (\"lvconvert --stripes 4 vg/lv\" would be suffi‐\ncient).\n\n# lvconvert --stripes 4 vg/lv\nUsing default stripesize 64.00 KiB.\nLogical volume vg/lv successfully converted.\n\n# lvs -a -o lvname,attr,segtype,segperanges,dataoffset vg\nLV Attr Type PE Ranges DOff\nlv rwi-a-r--- raid6nr lvrimage0:0-33 \\\nlvrimage1:0-33 \\\nlvrimage2:0-33 ... \\\nlvrimage5:0-33 8192\n[lvrimage0] iwi-aor--- linear /dev/sda:0-32 8192\n[lvrimage0] iwi-aor--- linear /dev/sda:34-34\n[lvrimage1] iwi-aor--- linear /dev/sdaa:0-32 8192\n[lvrimage1] iwi-aor--- linear /dev/sdaa:34-34\n[lvrimage2] iwi-aor--- linear /dev/sdab:0-32 8192\n[lvrimage2] iwi-aor--- linear /dev/sdab:34-34\n[lvrimage3] iwi-aor--- linear /dev/sdac:1-34 8192\n[lvrimage4] iwi-aor--- linear /dev/sdad:1-34 8192\n[lvrimage5] iwi-aor--- linear /dev/sdae:1-34 8192\n[lvrmeta0] ewi-aor--- linear /dev/sda:33-33\n[lvrmeta1] ewi-aor--- linear /dev/sdaa:33-33\n[lvrmeta2] ewi-aor--- linear /dev/sdab:33-33\n[lvrmeta3] ewi-aor--- linear /dev/sdac:0-0\n[lvrmeta4] ewi-aor--- linear /dev/sdad:0-0\n[lvrmeta5] ewi-aor--- linear /dev/sdae:0-0\n\n# lvs -a -o lvname,attr,segtype,reshapelen vg\nLV Attr Type RSize\nlv rwi-a-r--- raid6nr 24.00m\n[lvrimage0] iwi-aor--- linear 4.00m\n[lvrimage0] iwi-aor--- linear\n[lvrimage1] iwi-aor--- linear 4.00m\n[lvrimage1] iwi-aor--- linear\n[lvrimage2] iwi-aor--- linear 4.00m\n[lvrimage2] iwi-aor--- linear\n[lvrimage3] iwi-aor--- linear 4.00m\n[lvrimage4] iwi-aor--- linear 4.00m\n[lvrimage5] iwi-aor--- linear 4.00m\n[lvrmeta0] ewi-aor--- linear\n[lvrmeta1] ewi-aor--- linear\n[lvrmeta2] ewi-aor--- linear\n[lvrmeta3] ewi-aor--- linear\n[lvrmeta4] ewi-aor--- linear\n[lvrmeta5] ewi-aor--- linear\n\nFuture developments might include automatic removal of the freed images.\n\nIf the reshape space shall be removed any lvconvert command not changing the layout can be\nused:\n\n# lvconvert --stripes 4 vg/lv\nUsing default stripesize 64.00 KiB.\nNo change in RAID LV vg/lv layout, freeing reshape space.\nLogical volume vg/lv successfully converted.\n\n# lvs -a -o lvname,attr,segtype,reshapelen vg\nLV Attr Type RSize\nlv rwi-a-r--- raid6nr 0\n[lvrimage0] iwi-aor--- linear 0\n[lvrimage0] iwi-aor--- linear\n[lvrimage1] iwi-aor--- linear 0\n[lvrimage1] iwi-aor--- linear\n[lvrimage2] iwi-aor--- linear 0\n[lvrimage2] iwi-aor--- linear\n[lvrimage3] iwi-aor--- linear 0\n[lvrimage4] iwi-aor--- linear 0\n[lvrimage5] iwi-aor--- linear 0\n[lvrmeta0] ewi-aor--- linear\n[lvrmeta1] ewi-aor--- linear\n[lvrmeta2] ewi-aor--- linear\n[lvrmeta3] ewi-aor--- linear\n[lvrmeta4] ewi-aor--- linear\n[lvrmeta5] ewi-aor--- linear\n\nIn case the RaidLV should be converted to striped:\n\n# lvconvert --type striped vg/lv\nUnable to convert LV vg/lv from raid6nr to striped.\nConverting vg/lv from raid6nr is directly possible to the \\\nfollowing layouts:\nraid6nc\nraid6zr\nraid6la6\nraid6ls6\nraid6ra6\nraid6rs6\nraid6n6\n\nA direct conversion isn't possible thus the command informed about the possible ones.\nraid6n6 is suitable to convert to striped so convert to it first (this is a reshape chang‐\ning the raid6 layout from raid6nr to raid6n6).\n\n# lvconvert --type raid6n6\nUsing default stripesize 64.00 KiB.\nConverting raid6nr LV vg/lv to raid6n6.\nAre you sure you want to convert raid6nr LV vg/lv? [y/n]: y\nLogical volume vg/lv successfully converted.\n\nWait for the reshape to finish.\n\n# lvconvert --type striped vg/lv\nLogical volume vg/lv successfully converted.\n\n# lvs -o lvname,attr,segtype,segperanges,dataoffset vg\nLV Attr Type PE Ranges DOff\nlv -wi-a----- striped /dev/sda:2-32 \\\n/dev/sdaa:2-32 \\\n/dev/sdab:2-32 \\\n/dev/sdac:3-33\nlv -wi-a----- striped /dev/sda:34-35 \\\n/dev/sdaa:34-35 \\\n/dev/sdab:34-35 \\\n/dev/sdac:34-35\n\nFrom striped we can convert to raid10\n\n# lvconvert --type raid10 vg/lv\nUsing default stripesize 64.00 KiB.\nLogical volume vg/lv successfully converted.\n\n# lvs -o lvname,attr,segtype,segperanges,dataoffset vg\nLV Attr Type PE Ranges DOff\nlv rwi-a-r--- raid10 lvrimage0:0-32 \\\nlvrimage4:0-32 \\\nlvrimage1:0-32 ... \\\nlvrimage3:0-32 \\\nlvrimage7:0-32 0\n\n# lvs -a -o lvname,attr,segtype,segperanges,dataoffset vg\nWARNING: Cannot find matching striped segment for vg/lvrimage3.\nLV Attr Type PE Ranges DOff\nlv rwi-a-r--- raid10 lvrimage0:0-32 \\\nlvrimage4:0-32 \\\nlvrimage1:0-32 ... \\\nlvrimage3:0-32 \\\nlvrimage7:0-32 0\n[lvrimage0] iwi-aor--- linear /dev/sda:2-32 0\n[lvrimage0] iwi-aor--- linear /dev/sda:34-35\n[lvrimage1] iwi-aor--- linear /dev/sdaa:2-32 0\n[lvrimage1] iwi-aor--- linear /dev/sdaa:34-35\n[lvrimage2] iwi-aor--- linear /dev/sdab:2-32 0\n[lvrimage2] iwi-aor--- linear /dev/sdab:34-35\n[lvrimage3] iwi-XXr--- linear /dev/sdac:3-35 0\n[lvrimage4] iwi-aor--- linear /dev/sdad:1-33 0\n[lvrimage5] iwi-aor--- linear /dev/sdae:1-33 0\n[lvrimage6] iwi-aor--- linear /dev/sdaf:1-33 0\n[lvrimage7] iwi-aor--- linear /dev/sdag:1-33 0\n[lvrmeta0] ewi-aor--- linear /dev/sda:0-0\n[lvrmeta1] ewi-aor--- linear /dev/sdaa:0-0\n[lvrmeta2] ewi-aor--- linear /dev/sdab:0-0\n[lvrmeta3] ewi-aor--- linear /dev/sdac:0-0\n[lvrmeta4] ewi-aor--- linear /dev/sdad:0-0\n[lvrmeta5] ewi-aor--- linear /dev/sdae:0-0\n[lvrmeta6] ewi-aor--- linear /dev/sdaf:0-0\n[lvrmeta7] ewi-aor--- linear /dev/sdag:0-0\n\nraid10 allows to add stripes but can't remove them.\n\n\nA more elaborate example to convert from linear to striped with interim conversions to raid1\nthen raid5 followed by restripe (4 steps).\n\nWe start with the linear LV.\n\n# lvs -a -o name,size,segtype,syncpercent,datastripes,\\\nstripesize,reshapelenle,devices vg\nLV LSize Type Cpy%Sync #DStr Stripe RSize Devices\nlv 128.00m linear 1 0 /dev/sda(0)\n\nThen convert it to a 2-way raid1.\n\n# lvconvert --mirrors 1 vg/lv\nLogical volume vg/lv successfully converted.\n\n# lvs -a -o name,size,segtype,datastripes,\\\nstripesize,reshapelenle,devices vg\nLV LSize Type #DStr Stripe RSize Devices\nlv 128.00m raid1 2 0 lvrimage0(0),\\\nlvrimage1(0)\n[lvrimage0] 128.00m linear 1 0 /dev/sda(0)\n[lvrimage1] 128.00m linear 1 0 /dev/sdhx(1)\n[lvrmeta0] 4.00m linear 1 0 /dev/sda(32)\n[lvrmeta1] 4.00m linear 1 0 /dev/sdhx(0)\n\nOnce the raid1 LV is fully synchronized we convert it to raid5n (only 2-way raid1 LVs can be\nconverted to raid5). We select raid5n here because it has dedicated parity SubLVs at the\nend and can be converted to striped directly without any additional conversion.\n\n# lvconvert --type raid5n vg/lv\nUsing default stripesize 64.00 KiB.\nLogical volume vg/lv successfully converted.\n\n# lvs -a -o name,size,segtype,syncpercent,datastripes,\\\nstripesize,reshapelenle,devices vg\nLV LSize Type #DStr Stripe RSize Devices\nlv 128.00m raid5n 1 64.00k 0 lvrimage0(0),\\\nlvrimage1(0)\n[lvrimage0] 128.00m linear 1 0 0 /dev/sda(0)\n[lvrimage1] 128.00m linear 1 0 0 /dev/sdhx(1)\n[lvrmeta0] 4.00m linear 1 0 /dev/sda(32)\n[lvrmeta1] 4.00m linear 1 0 /dev/sdhx(0)\n\nNow we'll change the number of data stripes from 1 to 5 and request 128K stripe size in one\ncommand. This will grow the size of the LV by a factor of 5 (we add 4 data stripes to the\none given). That additonal space can be used by e.g. growing any contained filesystem or the\nLV can be reduced in size after the reshaping conversion has finished.\n\n# lvconvert --stripesize 128k --stripes 5 vg/lv\nConverting stripesize 64.00 KiB of raid5n LV vg/lv to 128.00 KiB.\nWARNING: Adding stripes to active logical volume vg/lv will grow \\\nit from 32 to 160 extents!\nRun \"lvresize -l32 vg/lv\" to shrink it or use the additional capacity.\nLogical volume vg/lv successfully converted.\n\n# lvs -a -o name,size,segtype,datastripes,\\\nstripesize,reshapelenle,devices\nLV LSize Type #DStr Stripe RSize Devices\nlv 640.00m raid5n 5 128.00k 6 lvrimage0(0),\\\nlvrimage1(0),\\\nlvrimage2(0),\\\nlvrimage3(0),\\\nlvrimage4(0),\\\nlvrimage5(0)\n[lvrimage0] 132.00m linear 1 0 1 /dev/sda(33)\n[lvrimage0] 132.00m linear 1 0 /dev/sda(0)\n[lvrimage1] 132.00m linear 1 0 1 /dev/sdhx(33)\n[lvrimage1] 132.00m linear 1 0 /dev/sdhx(1)\n[lvrimage2] 132.00m linear 1 0 1 /dev/sdhw(33)\n[lvrimage2] 132.00m linear 1 0 /dev/sdhw(1)\n[lvrimage3] 132.00m linear 1 0 1 /dev/sdhv(33)\n[lvrimage3] 132.00m linear 1 0 /dev/sdhv(1)\n[lvrimage4] 132.00m linear 1 0 1 /dev/sdhu(33)\n[lvrimage4] 132.00m linear 1 0 /dev/sdhu(1)\n[lvrimage5] 132.00m linear 1 0 1 /dev/sdht(33)\n[lvrimage5] 132.00m linear 1 0 /dev/sdht(1)\n[lvrmeta0] 4.00m linear 1 0 /dev/sda(32)\n[lvrmeta1] 4.00m linear 1 0 /dev/sdhx(0)\n[lvrmeta2] 4.00m linear 1 0 /dev/sdhw(0)\n[lvrmeta3] 4.00m linear 1 0 /dev/sdhv(0)\n[lvrmeta4] 4.00m linear 1 0 /dev/sdhu(0)\n[lvrmeta5] 4.00m linear 1 0 /dev/sdht(0)\n\nOnce the conversion has finished we can can convert to striped.\n\n# lvconvert --type striped vg/lv\nLogical volume vg/lv successfully converted.\n\n# lvs -a -o name,size,segtype,datastripes,\\\nstripesize,reshapelenle,devices vg\nLV LSize Type #DStr Stripe RSize Devices\nlv 640.00m striped 5 128.00k /dev/sda(33),\\\n/dev/sdhx(33),\\\n/dev/sdhw(33),\\\n/dev/sdhv(33),\\\n/dev/sdhu(33)\nlv 640.00m striped 5 128.00k /dev/sda(0),\\\n/dev/sdhx(1),\\\n/dev/sdhw(1),\\\n/dev/sdhv(1),\\\n/dev/sdhu(1)\n\nReversing these steps will convert a given striped LV to linear.\n\nMind the facts that stripes are removed thus the capacity of the RaidLV will shrink and that\nchanging the RaidLV layout will influence its performance.\n\n\"lvconvert --stripes 1 vg/lv\" for converting to 1 stripe will inform upfront about the re‐\nduced size to allow for resizing the content or growing the RaidLV before actually converting\nto 1 stripe. The --force option is needed to allow stripe removing conversions to prevent\ndata loss.\n\nOf course any interim step can be the intended last one (e.g. striped-> raid1).\n\n" }, { "name": "RAID5 Variants", "content": "raid5ls\n• RAID5 left symmetric\n• Rotating parity N with data restart\n\nraid5la\n• RAID5 left symmetric\n• Rotating parity N with data continuation\n\nraid5rs\n• RAID5 right symmetric\n• Rotating parity 0 with data restart\n\nraid5ra\n• RAID5 right asymmetric\n• Rotating parity 0 with data continuation\n\nraid5n\n• RAID5 parity n\n• Dedicated parity device n used for striped/raid0 conversions\n• Used for RAID Takeover\n\n" }, { "name": "RAID6 Variants", "content": "raid6\n• RAID6 zero restart (aka left symmetric)\n• Rotating parity 0 with data restart\n• Same as raid6zr\n\nraid6zr\n• RAID6 zero restart (aka left symmetric)\n• Rotating parity 0 with data restart\n\nraid6nr\n• RAID6 N restart (aka right symmetric)\n• Rotating parity N with data restart\n\nraid6nc\n• RAID6 N continue\n• Rotating parity N with data continuation\n\nraid6n6\n• RAID6 last parity devices\n• Fixed dedicated last devices (P-Syndrome N-1 and Q-Syndrome N)\nwith striped data used for striped/raid0 conversions\n• Used for RAID Takeover\n\nraid6{ls,rs,la,ra}6\n• RAID6 last parity device\n• Dedicated last parity device used for conversions from/to\nraid5{ls,rs,la,ra}\n\nraid6ls6\n• RAID6 N continue\n• Same as raid5ls for N-1 devices with fixed Q-Syndrome N\n• Used for RAID Takeover\n\nraid6la6\n• RAID6 N continue\n• Same as raid5la for N-1 devices with fixed Q-Syndrome N\n• Used forRAID Takeover\n\nraid6rs6\n• RAID6 N continue\n• Same as raid5rs for N-1 devices with fixed Q-Syndrome N\n• Used for RAID Takeover\n\nraid6ra6\n• RAID6 N continue\n• ame as raid5ra for N-1 devices with fixed Q-Syndrome N\n• Used for RAID Takeover\n\n\n\n" } ] }, "History": { "content": "The 2.6.38-rc1 version of the Linux kernel introduced a device-mapper target to interface\nwith the software RAID (MD) personalities. This provided device-mapper with RAID 4/5/6 capa‐\nbilities and a larger development community. Later, support for RAID1, RAID10, and RAID1E\n(RAID 10 variants) were added. Support for these new kernel RAID targets was added to LVM\nversion 2.02.87. The capabilities of the LVM raid1 type have surpassed the old mirror type.\nraid1 is now recommended instead of mirror. raid1 became the default for mirroring in LVM\nversion 2.02.100.\n\n\n\n\nRed Hat, Inc LVM TOOLS 2.03.11(2) (2021-01-08) LVMRAID(7)", "subsections": [] } } } }