info > CPUSET

man perldoc info pydoc3 ri search

The user wants me to transform a Linux man page (CPUSET(7)) HTML into an emoji-rich, visually scannable version. Let me follow all the rules carefully: 1. Never use

- start from

2. Section titles →

with ONE emoji prefix 3. Sub-sections →

with emoji prefix 4. Comments in code are NOT headings 5. ALL code in

6. Code blocks must contain EXACT original code, no changes
7. Use 
/
 for lists
8. Function references use 
name(args) — description
 format WITHOUT per-item emoji
9. Escape bare < and > (except in allowed tags)
10. Output ONLY valid HTML
11. Preserve , ,  tags
12. Add 🚀 Quick Reference section as second section
13. Keep output under 128,000 characters

Let me structure the output:

- 
📛 NAME
- 
🚀 Quick Reference
- 
📖 DESCRIPTION
- 
📁 FILES
 (with sub-sections for each file)
- 
⚡ EXTENDED CAPABILITIES
 (with sub-sections)
- 
📋 FORMATS
 (with sub-sections)
- 
📏 RULES
- 
🔐 PERMISSIONS
- 
⚠️ WARNINGS
 (with sub-sections)
- 
🔧 EXCEPTIONS
 (with sub-sections)
- 
❌ ERRORS
- 
🏷️ VERSIONS
- 
📝 NOTES
- 
🐛 BUGS
- 
💡 EXAMPLES
 (with sub-sections)
- 
🔗 SEE ALSO
- 
📚 COLOPHON

For the Quick Reference table, I should generate common use cases from the content. Looking at the man page, common use cases would be:
- Mount the cpuset filesystem
- Create a new cpuset
- Add processes to a cpuset
- Set CPUs for a cpuset
- Set memory nodes for a cpuset
- View memory pressure
- Enable memory pressure
- Enable memory migration
- Migrate a job to new cpuset
- Remove a cpuset

For ERRORS, I'll use 
ERRNO — description
 format per the function reference rules.

Let me carefully preserve all the  links from the original, but not add any inside 
 blocks.

For the SEE ALSO section, the original has broken lines like "sched_getaffin‐
ity(2)" - I should preserve this but maybe normalize. Actually, let me preserve it as-is since it's the original.

Let me write this out carefully now. I need to be mindful of character count.

Looking at the FILES section more carefully - each pseudo-file is a description in the original. I should preserve them but I can wrap with proper structure. The original format is:
```
       tasks  List of the process IDs (PIDs)...
       notify_on_release  Flag (0 or 1)...
```

I'll keep the filename followed by description format, possibly using // or just paragraphs with .

Actually, looking at the rules again:
- For function/class reference sections: use 
name(args) — description

The FILES section is more like a list of pseudo-files with descriptions. I could use a definition list or just keep them as paragraphs with bold filenames. Let me keep them as paragraphs with bold filenames to preserve the original structure.

Wait, the rules say "only ADD emoji and visual polish; do NOT restructure." So I should preserve the original structure. The FILES section is essentially a list of files with descriptions. Let me keep the original paragraph structure but add appropriate formatting.

Actually, I think using // would be acceptable as visual polish since it's a natural fit for term-description pairs. Or I could use 
filename description
. Let me use 
filename description
 to keep it closer to original structure.

For the ERRORS section, the rules say:
"For function, method, and class reference sections: use 
name(args) — description
 format WITHOUT per-item emoji"

The ERRORS section is similar - it's a list of error codes with descriptions. Let me use:
ERRNO — description

For the SEE ALSO links, the original has `taskset(1)` etc. I'll preserve these.

Now let me draft the full output. I need to be careful about escaping < and > in code blocks (none needed since they're code that should be preserved as-is per rule 8 - the rules about escaping are for outside code blocks).

Wait, rule 13 says: "ANY < or > NOT part of an allowed HTML tag MUST be escaped as < and >". But code blocks contain literal code. Let me re-read...

Looking at the original: `/proc/<pid>/cpuset` - this is already escaped in the source HTML. So when I preserve this in my output, I should keep it escaped since it's text content (not actual code in pre tags).

Actually wait - the FILES section says:
"In addition to the above pseudo-files in each directory below /dev/cpuset, each process has a pseudo-file, /proc//cpuset..."

In the original HTML, this appears as `/proc/<pid>/cpuset`. This is text content (a paragraph), so I should preserve it as-is.

But within code blocks (pre/code tags), I'd use literal `` characters. Looking at the original HTML, I don't see any `` characters inside actual code blocks - the code blocks use `<` and `>` which would render as ``.

Let me be careful: code blocks preserve the EXACT original code. So if the original has `<pid>` in a code block, I keep it as `<pid>`.

Now let me write the actual output. I need to keep it under 128,000 characters.

Let me start writing:

```html
📛 NAME
cpuset — confine processes to processor and memory node subsets

🚀 Quick Reference

Use Case
Command
Description
Mount filesystem
mount -t cpuset cpuset /dev/cpuset
Mount cpuset filesystem
Create cpuset
mkdir /dev/cpuset/Charlie
Create a new cpuset directory
Set CPUs
echo 2-3 > cpuset.cpus
Assign CPUs to cpuset
Set memory nodes
echo 1 > cpuset.mems
Assign memory nodes to cpuset
Attach process
echo $$ > tasks
Move current shell to cpuset
View cpuset
cat /proc/self/cpuset
Display current cpuset path
Enable pressure
echo 1 > /dev/cpuset/cpuset.memory_pressure_enabled
Turn on memory pressure tracking
Remove cpuset
rmdir /dev/cpuset/Charlie
Remove empty cpuset


📖 DESCRIPTION
...
```

Let me write the full document now.

Actually, I realize for the FILES section, I need to think about whether to use  or just keep paragraphs. The original uses a definition-like format. Let me keep it as paragraphs with  for filename, since that preserves structure better. Actually, looking more carefully, I think using  would be a good visual enhancement since these are clearly term/description pairs.

Let me use:
```html
tasks — List of the process IDs...
```

This is closest to the original structure.

Actually, I'll use a more scannable format with  for the FILES section since it's clearly a list of pseudo-files and their descriptions. This is a minor visual enhancement that doesn't change meaning.

Hmm, but the rules say "do NOT restructure". Let me just preserve the original paragraph-style format with  tags to keep structure. The original uses two-column-like formatting with the filename then description. I'll keep that.

Actually, looking at the FILES section more carefully, the original treats each file as a sub-entry with the filename followed by description. I think the best approach is to use 
filename description
 format, which mirrors the original paragraph structure but makes it slightly more scannable.

Wait, actually re-reading more carefully, the original document uses indentation to suggest a sub-list structure. Let me use a nested structure:
- 
📁 FILES
  - intro paragraphs
  - For each pseudo-file: 
filename description

This preserves the structure while adding visual polish.

Let me draft the whole thing now. I'll need to be mindful of length.

For the ERRORS section, I'll use the format:

E2BIG — description
EACCES — description
...


This matches the "function reference" rule format.

Now let me write it out:

For the Quick Reference table, I need to be aware that rule 21 says "Command cells use  (NOT 
 — pre blocks break table layout)".

Let me finalize my output now. I'll be thorough but try to be concise where possible while preserving all information.

Actually one more consideration: in the FILES section, the original has:
```
       tasks  List of the process IDs (PIDs) of the processes in that  cpuset.
```

This is paragraph text with the filename and description. I'll wrap each in 
 tags.

Let me write the final output:

For format sections like:
```
       Examples of the Mask Format:

           00000001                        # just bit 0 set
           40000000,00000000,00000000      # just bit 94 set
```

These are code examples. I need to wrap them in 
.

Let me also make sure I preserve all the prose and details. This is a long man page so I'll need to be efficient.

Let me write this out now, section by section.

Actually, I realize that some of the content is quite verbose. Let me condense where appropriate but preserve all key information. The instructions say "Condense your output to under 128,000 characters. Preserve key sections but summarize/combine verbatim repetition."

OK let me write the final version now. I think I have a clear plan.

Let me be careful about the FILES section. Looking at the original, each pseudo-file is described in paragraphs. The original formatting puts the filename as a left-aligned label. I'll use:

filename Description...

This is a clean representation of the original.

One concern: the FILES section has descriptions that span multiple paragraphs (e.g., tasks has the warning about only one PID). Let me preserve those.

Let me finalize and write the output now.

For SEE ALSO, I should preserve the  tags from the original.

Let me also count - this is a pretty long document. I need to make sure I stay under 128,000 characters. Let me be efficient but preserve information.

Writing now:

📛 NAME
cpuset — confine processes to processor and memory node subsets

🚀 Quick Reference

Use Case
Command
Description
Mount filesystem
mount -t cpuset cpuset /dev/cpuset
Mount cpuset pseudo-filesystem
Create cpuset
mkdir /dev/cpuset/Charlie
Create a new cpuset directory
Assign CPUs
/bin/echo 2-3 > cpuset.cpus
Restrict cpuset to CPUs 2–3
Assign memory nodes
/bin/echo 1 > cpuset.mems
Restrict cpuset to memory node 1
Attach process
/bin/echo $$ > tasks
Move current shell into cpuset
View current cpuset
cat /proc/self/cpuset
Show path of process's cpuset
Enable memory pressure
echo 1 > /dev/cpuset/cpuset.memory_pressure_enabled
Activate memory_pressure tracking
Enable migration
echo 1 > cpuset.memory_migrate
Migrate pages on cpuset change
Remove empty cpuset
rmdir /dev/cpuset/Charlie
Delete an empty cpuset


📖 DESCRIPTION
🗂️ The cpuset filesystem is a pseudo-filesystem interface to the kernel cpuset mechanism, used to control the processor placement and memory placement of processes. It is commonly mounted at /dev/cpuset.

🖥️ On systems with kernels compiled with built-in cpuset support, all processes are attached to a cpuset and cpusets are always present. If a system supports cpusets, then /proc/filesystems will list nodev cpuset. By mounting the cpuset filesystem (see EXAMPLES), the administrator can configure cpusets to control the processor and memory placement of processes. By default — if the cpuset configuration is unmodified or the filesystem is not even mounted — the cpuset mechanism has no effect on the system's behavior.

🧩 A cpuset defines a list of CPUs and memory nodes.

The CPUs of a system include all logical processing units on which a process can execute — including multiple processor cores within a package and Hyper-Threads within a core. Memory nodes include all distinct banks of main memory; small and SMP systems typically have just one memory node containing all system memory, while NUMA (non-uniform memory access) systems have multiple memory nodes.

🌳 Cpusets are represented as directories in a hierarchical pseudo-filesystem. The top directory (/dev/cpuset) represents the entire system (all online CPUs and memory nodes). Any cpuset that is a descendant of another contains a subset of that parent's CPUs and memory nodes. The directories and files representing cpusets have normal filesystem permissions.

👥 Every process in the system belongs to exactly one cpuset. A process is confined to run only on the CPUs in its cpuset, and to allocate memory only on the memory nodes in that cpuset. When a process fork(2)s, the child is placed in the same cpuset as its parent. With sufficient privilege, a process may be moved between cpusets, and the allowed CPUs/memory nodes of an existing cpuset may be changed.

🚀 When the system begins booting, a single cpuset is defined that includes all CPUs and memory nodes; all processes are in that cpuset. During boot or normal operation, other cpusets may be created as subdirectories of this top cpuset under administrator control.

🔗 Cpusets integrate with the sched_setaffinity(2) scheduling affinity mechanism and the mbind(2) and set_mempolicy(2) memory-placement mechanisms. Neither of these mechanisms lets a process use a CPU or memory node that is not allowed by its cpuset. If a conflict arises, cpuset placement is enforced even if it overrides these mechanisms — the kernel silently restricts requested CPUs/memory nodes to those allowed by the invoking process's cpuset. This may cause calls to return an error (e.g., requesting an empty set).

📦 Typically, a cpuset confines CPU/memory placement for a set of cooperating processes (e.g., a batch job), while the other mechanisms place individual processes or memory regions within that set.

📁 FILES
Each directory below /dev/cpuset represents a cpuset and contains a fixed set of pseudo-files describing its state.

🆕 New cpusets are created via mkdir(2) or mkdir(1). The properties (flags, allowed CPUs/memory nodes, attached processes) are queried/modified by reading/writing the appropriate pseudo-file in that cpuset's directory.

📂 Pseudo-files are automatically created when the cpuset is created; they cannot be added or removed directly.

🗑️ A cpuset directory with no child cpusets and no attached processes can be removed using rmdir(2) or rmdir(1). Pseudo-files do not need to be removed first.

📝 Pseudo-files are small text files read/written with shell utilities (cat(1), echo(1)) or library functions (open(2), read(2), write(2), close(2)). They represent internal kernel state with no persistent on-disk image.

📁 Pseudo-files in each cpuset
tasks — List of PIDs of processes in the cpuset. Formatted as a series of ASCII decimal numbers, each followed by a newline. Add a process by writing its PID (with or without trailing newline).
⚠️ Warning: Only one PID may be written at a time. If multiple PIDs are written, only the first is used.

notify_on_release — Flag (0 or 1). If set (1), the cpuset receives special handling after release (when all processes leave and all child cpusets are removed). See 🔔 Notify on release.

cpuset.cpus — List of CPU numbers on which processes may execute. See 📋 List Format. Modifiable by writing a new list.

cpuset.cpu_exclusive — Flag (0 or 1). If set (1), the cpuset has exclusive use of its CPUs (no sibling or cousin cpuset may overlap). Default: 0; newly created cpusets also default to 0.
Two cpusets are siblings if they share the same parent; cousins if neither is an ancestor of the other. Regardless of this flag, if one cpuset is an ancestor of another, both having nonempty cpus, their cpus must overlap (since a cpuset's cpus are always a subset of its parent's).

cpuset.mems — List of memory nodes on which processes may allocate memory. See 📋 List Format.

cpuset.mem_exclusive — Flag (0 or 1). If set (1), the cpuset has exclusive use of its memory nodes (no sibling or cousin may overlap) and becomes a Hardwall cpuset (see 🧱 Hardwall). Default: 0; newly created cpusets default to 0.
Regardless of this flag, a cpuset's memory nodes must be a subset of its parent's, so ancestor memory nodes overlap.

cpuset.mem_hardwall (since Linux 2.6.26) — Flag (0 or 1). If set (1), the cpuset is a Hardwall cpuset (see 🧱 Hardwall). Unlike mem_exclusive, there is no constraint on overlap with sibling/cousin memory nodes. Default: 0; newly created cpusets default to 0.

cpuset.memory_migrate (since Linux 2.6.16) — Flag (0 or 1). If set (1), memory migration is enabled. Default: 0. See 📦 Memory migration.

cpuset.memory_pressure (since Linux 2.6.16) — Measure of memory pressure caused by processes in this cpuset (see 📊 Memory pressure). Read-only. Always zero unless memory_pressure_enabled is set. See ⚠️ WARNINGS.

cpuset.memory_pressure_enabled (since Linux 2.6.16) — Flag (0 or 1). Present only in the root cpuset (/dev/cpuset). If set (1), memory_pressure calculations are enabled system-wide. Default: 0.

cpuset.memory_spread_page (since Linux 2.6.17) — Flag (0 or 1). If set (1), kernel page cache (filesystem buffers) is uniformly spread across the cpuset. Default: 0 in top cpuset; inherited from parent for new cpusets. See 🌐 Memory spread.

cpuset.memory_spread_slab (since Linux 2.6.17) — Flag (0 or 1). If set (1), kernel slab caches for file I/O (directory and inode structures) are uniformly spread across the cpuset. Default: 0 in top cpuset; inherited from parent for new cpusets.

cpuset.sched_load_balance (since Linux 2.6.24) — Flag (0 or 1). If set (1, default), the kernel load-balances processes across the cpuset's allowed CPUs. If cleared (0), the kernel avoids load balancing (unless overlapping cpusets enable it). See ⚖️ Scheduler load balancing.

cpuset.sched_relax_domain_level (since Linux 2.6.26) — Integer (-1 to small positive). Controls the width of CPUs over which the scheduler performs immediate rebalancing. Irrelevant if sched_load_balance is disabled. See 🏛️ Scheduler relax domain level.

🔍 Additionally, each process has a pseudo-file /proc/<pid>/cpuset showing the path of its cpuset directory relative to the cpuset filesystem root.

📊 The /proc/<pid>/status file has four added lines showing the process's allowed CPUs and memory nodes in Mask and List formats (see 📋 FORMATS):
Cpus_allowed:   ffffffff,ffffffff,ffffffff,ffffffff
Cpus_allowed_list:     0-127
Mems_allowed:   ffffffff,ffffffff
Mems_allowed_list:     0-63
The Cpus_allowed / Mems_allowed fields were added in Linux 2.6.24; the *_list fields were added in Linux 2.6.26.

⚡ EXTENDED CAPABILITIES
In addition to controlling CPUs/mems, cpusets provide these extended capabilities.

🔒 Exclusive cpusets
If a cpuset is marked cpu_exclusive or mem_exclusive, no other cpuset (other than a direct ancestor or descendant) may share any of the same CPUs or memory nodes.
A mem_exclusive cpuset also restricts kernel allocations for buffer cache pages and other internal kernel data commonly shared across users. All cpusets, whether mem_exclusive or not, restrict user-space memory allocations. This allows multiple independent jobs to share common kernel data while isolating each job's user allocations in its own cpuset. Build a large mem_exclusive parent cpuset for all jobs, then create non-mem_exclusive child cpusets per job. Only small kernel-memory requests (e.g., interrupt handlers) may be placed on memory nodes outside even a mem_exclusive cpuset.

🧱 Hardwall
A cpuset with mem_exclusive or mem_hardwall set is a hardwall cpuset. A hardwall cpuset restricts kernel allocations for page, buffer, and other data commonly shared across users. All cpusets restrict user-space allocations.
This allows multiple jobs to share common kernel data (e.g., filesystem pages) while isolating user allocations. Build a large hardwall parent cpuset for all jobs, then create non-hardwall child cpusets per job. Only small kernel memory (e.g., interrupt handlers) may be taken outside even a hardwall cpuset.

🔔 Notify on release
If notify_on_release is enabled (1), then whenever the last process in the cpuset leaves and the last child cpuset is removed, the kernel runs /sbin/cpuset_release_agent with the pathname (relative to the cpuset mount point) of the abandoned cpuset — enabling automatic removal of abandoned cpusets.
The default value in the root cpuset at boot is disabled (0); new cpusets inherit their parent's value.
The kernel invokes /sbin/cpuset_release_agent with the name (path relative to /dev/cpuset) of the to-be-released cpuset in argv[1]. The usual contents is:
#!/bin/sh
rmdir /dev/cpuset/$1
Write ASCII 0 or 1 (with optional trailing newline) to clear or set the flag.

📊 Memory pressure
The memory_pressure of a cpuset provides a per-cpuset running average of the rate at which processes in the cpuset attempt to free in-use memory to satisfy additional requests.
This lets batch managers monitoring jobs in dedicated cpusets efficiently detect what level of memory pressure those jobs cause.
Useful both on tightly managed mixed-workload systems (to terminate/reprioritize jobs exceeding their allowed memory) and on tightly coupled long-running massively parallel scientific jobs (which can fail performance goals if they exceed allowed memory).
This is an economical way for the batch manager to monitor a cpuset; deciding what action to take on pressure is up to user code.
Unless enabled via the root pseudo-file /dev/cpuset/cpuset.memory_pressure_enabled, it is not computed and reads always return 0\n. See ⚠️ WARNINGS.
A per-cpuset running average is used because:

⚡ Per-cpuset (not per-process/region) sharply reduces load on large systems — no tasklist scan needed per query.
📈 A running average (not accumulating counter) lets a batch scheduler detect pressure with a single read.
🎯 Per-cpuset (not per-process) gives the batch scheduler the key signal with a single read.

The memory_pressure is calculated by a per-cpuset digital filter tracking the recent rate at which attached processes enter the kernel direct reclaim code — entered whenever a process must repurpose another page due to lack of free pages (dirty pages written to disk; unmodified buffer pages dropped). The value is an integer: the recent (half-life 10s) rate of entries to direct reclaim in reclaims/sec × 1000.

🌐 Memory spread
Two Boolean flag files per cpuset control where the kernel allocates pages for filesystem buffers and related in-kernel data: cpuset.memory_spread_page and cpuset.memory_spread_slab.
If cpuset.memory_spread_page is set, the kernel spreads filesystem buffers (page cache) evenly across all nodes the faulting process is allowed to use, instead of preferring the local node.
If cpuset.memory_spread_slab is set, some filesystem-related slab caches (inodes, directory entries) are spread evenly across allowed nodes.
These flags do not affect the brk(2) data segment or stack segment pages.
By default both are off; the kernel prefers the local node, falling back to the nearest allowed node with sufficient memory. New cpusets inherit memory spread settings from their parent.
Setting memory spreading causes affected allocations to ignore the process's NUMA memory policy. The effect on placement is hidden from mbind(2) / set_mempolicy(2) — they always behave as if no spreading is in effect. If cpuset memory spreading is later turned off, the most recent NUMA policy is automatically reapplied.
Both files are Boolean; default "0" (off); write "1" to turn on.
Cpuset-specified memory spreading behaves like round-robin / interleave memory placement. It provides substantial performance improvements for jobs that:

🧵 Need thread-local data on memory nodes close to the CPUs running the threads accessing it, AND
💾 Need to access large filesystem datasets spread across multiple nodes to fit.

Without this, memory allocation across the job's cpuset can become very uneven, especially for jobs with a single thread initializing/reading the dataset.

📦 Memory migration
Normally (with default disabled cpuset.memory_migrate), once a page is allocated, it stays on whatever node it was allocated on, even if the cpuset's mems setting later changes.
When memory migration is enabled, if the cpuset's mems setting changes, any memory page in use by a process in the cpuset that is on a no-longer-allowed node is migrated to an allowed node.
Furthermore, if a process is moved into a cpuset with memory migration enabled, its pages on nodes allowed in its previous cpuset but not in its new cpuset are migrated to allowed nodes in the new cpuset.
Relative placement within the cpuset is preserved if possible (e.g., page on the second valid node of the prior cpuset → second valid node of the new cpuset).

⚖️ Scheduler load balancing
The kernel scheduler automatically load balances processes: if one CPU is underutilized, it looks for processes on more overloaded CPUs and moves them within placement constraints (cpusets, sched_setaffinity(2)).
The cost of load balancing grows more than linearly with the number of CPUs — it costs more to balance one large set than two smaller halves.
The per-cpuset flag sched_load_balance lets you suppress automatic load balancing when unneeded and useful for performance.
By default, load balancing is done across all CPUs except those marked isolated via the boot-time isolcpus= argument. (See 🏛️ Scheduler relax domain level.)
The default is poorly suited to two situations:

🖥️ On large systems, load balancing across many CPUs is expensive — if independent jobs are placed on separate CPU sets via cpusets, full load balancing is unnecessary.
⏱️ Systems with real-time on some CPUs need to minimize overhead, including avoiding process load balancing on those CPUs.

When enabled (default), it requests load balancing across all allowed CPUs in the cpuset, allowing movement of any unpinned process across them.
When disabled, the scheduler avoids load balancing across those CPUs, except where needed because some overlapping cpuset has it enabled.
For example, if the top cpuset has sched_load_balance enabled, the scheduler balances across all CPUs and child settings have no effect.
Therefore, in the two situations above, disable sched_load_balance in the top cpuset and only enable it in some smaller child cpusets.
⚠️ When doing this, don't leave unpinned processes in the top cpuset that use nontrivial CPU — they may be artificially constrained to a subset of CPUs, since the scheduler won't consider moving them to an underused CPU.
✅ Processes pinned to a particular CPU can safely be left in a cpuset that disables sched_load_balance.

🏛️ Scheduler relax domain level
The kernel scheduler performs immediate load balancing whenever a CPU becomes free or another task becomes runnable, to keep as many CPUs usefully employed as possible. It also performs periodic load balancing via the software clock described in time(7). The sched_relax_domain_level setting applies only to immediate load balancing; periodic load balancing always attempts across all CPUs (unless disabled by clearing sched_load_balance). Tasks are scheduled only on CPUs allowed by their cpuset, as modified by sched_setaffinity(2).
On small systems, immediate load balancing improves interactivity and reduces wasteful idle cycles. On large systems, attempting it across many CPUs can be more costly than worthwhile, depending on workload and hardware.
The exact meaning of small positive integers of sched_relax_domain_level depends on internal scheduler implementation details and hardware architecture; both evolve over time.
As of this writing (introduced in Linux 2.6.26), on certain popular architectures, positive values mean:

🔄 Perform immediate load balancing across Hyper-Thread siblings on the same core.
🔄 Perform immediate load balancing across other cores in the same package.
🔄 Perform immediate load balancing across other CPUs on the same node/blade.
🔄 Perform immediate load balancing across several (implementation detail) nodes [NUMA].
🔄 Perform immediate load balancing across all CPUs in the system [NUMA].

0 — always means don't perform immediate load balancing; balancing happens only periodically.
-1 — always means use the system default value (varies by architecture/kernel; can be changed by boot-time relax_domain_level=).
With multiple overlapping cpusets having conflicting values, the highest value applies to all CPUs in any overlapping cpuset. -1 is the lowest (overridden by any other); 0 is the next lowest.

📋 FORMATS

🎭 Mask format
Used to represent CPU and memory-node bit masks in /proc/<pid>/status.
Each 32-bit word is displayed in hexadecimal (ASCII 0–9, a–f); words are zero-padded on the left. For masks longer than one word, comma separators appear between words. Words are in big-endian order (most significant bit first); hex digits within a word are also big-endian.
The number of 32-bit words displayed is the minimum needed to show all bits of the mask.
Examples:
00000001                        # just bit 0 set
40000000,00000000,00000000      # just bit 94 set
00000001,00000000,00000000      # just bit 64 set
000000ff,00000000               # bits 32-39 set
00000000,000e3862               # 1,5,6,11-13,17-19 set
A mask with bits 0, 1, 2, 4, 8, 16, 32, and 64 set displays as:
00000001,00000001,00010117
The first 1 is for bit 64, the second for bit 32, the third for bit 16, the fourth for bit 8, the fifth for bit 4, and 7 is for bits 2, 1, and 0.

📃 List format
The list format for cpus and mems is a comma-separated list of CPU or memory-node numbers and ranges, in ASCII decimal.
Examples:
0-4,9           # bits 0, 1, 2, 3, 4, and 9 set
0-2,7,12-14     # bits 0, 1, 2, 7, 12, 13, and 14 set

📏 RULES
The following rules apply to each cpuset:

🧬 Its CPUs and memory nodes must be a (possibly equal) subset of its parent's.
🔒 It can be marked cpu_exclusive only if its parent is.
🔒 It can be marked mem_exclusive only if its parent is.
🚫 If it is cpu_exclusive, its CPUs may not overlap any sibling.
🚫 If it is memory_exclusive, its memory nodes may not overlap any sibling.


🔐 PERMISSIONS
The permissions of a cpuset are determined by the permissions of the directories and pseudo-files in the cpuset filesystem, normally mounted at /dev/cpuset.
For example, a process can place itself in another cpuset if it can write that cpuset's tasks file — requiring execute permission on the encompassing directories and write permission on the tasks file.
An additional constraint applies to moving other processes into a cpuset: one process may not attach another to a cpuset unless it would have permission to send that process a signal (see kill(2)).
A process may create a child cpuset if it can access and write the parent cpuset directory. It can modify CPUs or memory nodes if it can access the cpuset's directory (execute permissions on all parent directories) and write the corresponding cpus or mems file.
There is one minor difference between how these permissions are evaluated and normal filesystem operation: the kernel interprets relative pathnames starting at the process's current working directory. Even when operating on a cpuset file, relative pathnames are interpreted relative to $CWD, not relative to the process's current cpuset. The only ways to use cpuset paths relative to a process's current cpuset are if either the process's CWD is its cpuset (via cd / chdir(2) into /dev/cpuset) or some user code converts the relative cpuset path to a full filesystem path.
In theory, user code should specify cpusets using absolute pathnames, requiring knowledge of the mount point (usually, but not necessarily, /dev/cpuset). In practice, well-written user code assumes /dev/cpuset and verifies the presence of /dev/cpuset/tasks to confirm the cpuset pseudo-filesystem is mounted.

⚠️ WARNINGS

📊 Enabling memory_pressure
By default, the per-cpuset file cpuset.memory_pressure always contains zero (0). Unless enabled by writing "1" to /dev/cpuset/cpuset.memory_pressure_enabled, the kernel does not compute per-cpuset memory_pressure.

🔊 Using the echo command
When using echo at the shell prompt to change cpuset file values, beware that the built-in echo in some shells does not display an error message if write(2) fails. For example:
echo 19 > cpuset.mems
might fail silently if memory node 19 is not allowed. It is better to use /bin/echo:
/bin/echo 19 > cpuset.mems
/bin/echo: write error: Invalid argument

🔧 EXCEPTIONS

💾 Memory placement
Not all allocations of system memory are constrained by cpusets.
If hot-plug functionality is used to remove all CPUs currently assigned to a cpuset, the kernel automatically updates cpus_allowed of all attached processes to allow all CPUs. A similar exception is expected for memory-node hot-plug removal. Generally, the kernel prefers to violate cpuset placement rather than starve a process that has had all its allowed CPUs/memory nodes taken offline. User code should reconfigure cpusets to refer only to online CPUs and memory nodes when hot-plugging resources.
A few kernel-critical internal memory-allocation requests marked GFP_ATOMIC must be satisfied immediately — the kernel may drop requests or malfunction if they fail. If such a request can't be satisfied within the current process's cpuset, the cpuset is relaxed to look for memory anywhere — better to violate the cpuset than stress the kernel.
Memory allocations requested by kernel drivers while processing an interrupt lack any relevant process context, and are not confined by cpusets.

✏️ Renaming cpusets
You can use rename(2) to rename cpusets. Only simple renaming is supported — changing the name is permitted, but moving a directory into a different directory is not.

❌ ERRORS
The Linux kernel sets errno to specify the reason for a failed cpuset system call:

E2BIG — Write to a cpuset file with a length larger than the kernel-determined upper limit.
EACCES — Write a PID to a cpuset tasks file without permission to move that process.
EACCES — Add (via write) a CPU or memory node to a cpuset when that CPU/node was not already in its parent.
EACCES — Set cpuset.cpu_exclusive or cpuset.mem_exclusive on a cpuset whose parent lacks the same setting.
EACCES — Write a cpuset.memory_pressure file.
EACCES — Create a file in a cpuset directory.
EBUSY — Remove (via rmdir(2)) a cpuset with attached processes.
EBUSY — Remove a cpuset with child cpusets.
EBUSY — Remove a CPU or memory node from a cpuset that is also in a child of that cpuset.
EEXIST — Create (via mkdir(2)) a cpuset that already exists.
EEXIST — rename(2) a cpuset to a name that already exists.
EFAULT — Read/write a cpuset file with a buffer outside the writing process's accessible address space.
EINVAL — Change a cpuset in a way that would violate a cpu_exclusive or mem_exclusive attribute of that cpuset or its siblings.
EINVAL — Write an empty cpuset.cpus or cpuset.mems list to a cpuset with attached processes or child cpusets.
EINVAL — Write a cpuset.cpus or cpuset.mems list containing a range with the second number smaller than the first.
EINVAL — Write a cpuset.cpus or cpuset.mems list containing an invalid character.
EINVAL — Write a list to a cpuset.cpus file with no online CPUs.
EINVAL — Write a list to a cpuset.mems file with no online memory nodes.
EINVAL — Write a list to a cpuset.mems file including a node that holds no memory.
EIO — Write a string to a cpuset tasks file that does not begin with an ASCII decimal integer.
EIO — rename(2) a cpuset into a different directory.
ENAMETOOLONG — Read a /proc/<pid>/cpuset file for a cpuset path longer than the kernel page size.
ENAMETOOLONG — Create (via mkdir(2)) a cpuset whose base directory name exceeds 255 characters.
ENAMETOOLONG — Create a cpuset whose full pathname (including the /dev/cpuset/ prefix) exceeds 4095 characters.
ENODEV — The cpuset was removed by another process concurrently with a write on one of its pseudo-files.
ENOENT — Create a cpuset in a parent cpuset that doesn't exist.
ENOENT — Access or open a nonexistent file in a cpuset directory.
ENOMEM — Insufficient kernel memory (only on extremely low-memory systems).
ENOSPC — Write a PID to a tasks file when the cpuset has an empty cpuset.cpus or cpuset.mems.
ENOSPC — Write an empty cpuset.cpus or cpuset.mems to a cpuset with attached tasks.
ENOTDIR — rename(2) a nonexistent cpuset.
EPERM — Remove a file from a cpuset directory.
ERANGE — Specify a cpuset.cpus or cpuset.mems list containing a number too large for the kernel's bit masks.
ESRCH — Write a PID of a nonexistent process to a cpuset tasks file.


🏷️ VERSIONS
🕒 Cpusets appeared in version 2.6.12 of the Linux kernel.

📝 NOTES
Despite its name, the pid parameter is actually a thread ID, and each thread in a threaded group can be attached to a different cpuset. The value returned from gettid(2) can be passed in the pid argument.

🐛 BUGS
cpuset.memory_pressure files can be opened for writing, creation, or truncation, but then write(2) fails with errno set to EACCES; the creation and truncation options on open(2) have no effect.

💡 EXAMPLES
The following examples demonstrate querying and setting cpuset options using shell commands.

🆕 Creating and attaching to a cpuset
To create a new cpuset and attach the current command shell to it:

📂 mkdir /dev/cpuset (if not already done)
🔗 mount -t cpuset none /dev/cpuset (if not already done)
📁 Create the new cpuset using mkdir(1).
🧮 Assign CPUs and memory nodes to the new cpuset.
🚀 Attach the shell to the new cpuset.

For example, the following creates a cpuset named "Charlie" containing just CPUs 2 and 3, and memory node 1, then attaches the current shell to it:
$ mkdir /dev/cpuset
$ mount -t cpuset cpuset /dev/cpuset
$ cd /dev/cpuset
$ mkdir Charlie
$ cd Charlie
$ /bin/echo 2-3 > cpuset.cpus
$ /bin/echo 1 > cpuset.mems
$ /bin/echo $$ > tasks
# The current shell is now running in cpuset Charlie
# The next line should display '/Charlie'
$ cat /proc/self/cpuset

🔀 Migrating a job to different memory nodes
To migrate a job (set of processes attached to a cpuset) to different CPUs/memory nodes, including moving currently allocated memory pages:

🎯 Suppose we want to move the job in cpuset alpha (CPUs 4–7, memory nodes 2–3) to a new cpuset beta (CPUs 16–19, memory nodes 8–9).
🆕 First create the new cpuset beta.
✅ Then allow CPUs 16–19 and memory nodes 8–9 in beta.
🔄 Then enable memory_migrate in beta.
➡️ Then move each process from alpha to beta.

$ cd /dev/cpuset
$ mkdir beta
$ cd beta
$ /bin/echo 16-19 > cpuset.cpus
$ /bin/echo 8-9 > cpuset.mems
$ /bin/echo 1 > cpuset.memory_migrate
$ while read i; do /bin/echo $i; done < ../alpha/tasks > tasks
This moves processes in alpha to beta, and migrates memory held by these processes on nodes 2–3 to nodes 8–9.
Notice the last step did not use cp ../alpha/tasks tasks because only one PID at a time may be written to the tasks file. The while loop (rather than the seemingly easier cp(1)) is necessary.
The same effect (one PID at a time) can be done more efficiently with the unbuffered option of sed(1):
$ sed -un p < ../alpha/tasks > tasks

🔗 SEE ALSO
taskset(1), get_mempolicy(2), getcpu(2), mbind(2), sched_setaffinity(2), sched_setscheduler(2), set_mempolicy(2), CPU_SET(3), proc(5), cgroups(7), numa(7), sched(7), migratepages(8), numactl(8)
📖 Documentation/admin-guide/cgroup-v1/cpusets.rst in the Linux kernel source tree (or Documentation/cgroup-v1/cpusets.txt before Linux 4.18, and Documentation/cpusets.txt before Linux 2.6.29)

📚 COLOPHON
This page is part of release 5.10 of the Linux man-pages project. A description of the project, information about reporting bugs, and the latest version of this page, can be found at https://www.kernel.org/doc/man-pages/.

🐧 Linux · 2020-11-01 · CPUSET(7)