info > sed(1)

man perldoc info pydoc3 ri search

📖 NAME

GNU sed – a stream editor, version 4.8. This document is the GNU sed manual.

🚀 Quick Reference

Use Case	Command	Description
Basic substitution	sed 's/old/new/' file	✅ Replace first occurrence of 'old' with 'new' on each line
Global substitution	sed 's/old/new/g' file	🔄 Replace all occurrences
In-place edit	sed -i 's/old/new/' file	✏️ Edit file directly (creates backup with -i.bak)
Print specific line	sed -n '45p' file	📄 Print only line 45
Delete lines	sed '3d' file	🗑️ Delete line 3
Delete range	sed '30,35d' file	🗑️ Delete lines 30-35
Case-insensitive match	sed '/pattern/Id' file	🔍 Delete lines matching 'pattern' case-insensitively
Extended regex	sed -E 's/[0-9]+/N/g' file	🔧 Use extended regular expressions (fewer backslashes)
Print line numbers	sed -n '=' file	🔢 Print line numbers only
Quit after match	sed '/pattern/q' file	🚪 Print lines until pattern, then exit
Read file into stream	sed '2r /path/to/file' input	📂 Insert file content after line 2
Write matched lines to file	sed -n '/pattern/w output' input	💾 Write lines matching pattern to output file
Multiple commands	sed -e 's/foo/bar/' -e 's/baz/qux/' file	🔗 Chain multiple operations
Use sed script file	sed -f script.sed input	📜 Execute commands from a file

📝 1 Introduction

sed is a stream editor. It performs basic text transformations on an input stream (file or pipeline). Unlike editors like ed, sed makes only one pass over the input, making it efficient. Its ability to filter text in a pipeline distinguishes it from other editors.

⚙️ 2 Running sed

This chapter covers how to run sed.

2.1 Overview

Normally invoked as:

sed SCRIPT INPUTFILE...

Example: replace all 'hello' with 'world' in input.txt:

sed 's/hello/world/' input.txt > output.txt

If no input file specified or - is given, reads from stdin. Use -i for in-place editing. Use -n to suppress automatic printing; use p to print specific lines. Multiple input files are treated as one long stream (use -s to treat separately).

2.2 Command-Line Options

Full format: sed OPTIONS... [SCRIPT] [INPUTFILE...]

• --version – 📟 Print version and copyright.
• --help – ❓ Print usage summary.
• -n, --quiet, --silent – 🔇 Disable automatic printing (only explicit p commands output).
• --debug – 🐛 Print canonical sed program and annotate execution.
• -e SCRIPT, --expression=SCRIPT – ➕ Add commands to the script.
• -f SCRIPT-FILE, --file=SCRIPT-FILE – 📄 Add commands from file.
• -i[SUFFIX], --in-place[=SUFFIX] – ✏️ Edit files in-place (creates backup with suffix).
• -l N, --line-length=N – 📏 Set line-wrap length for l command (default 70).
• --posix – 🏛️ Disable GNU extensions, make behavior POSIX-compliant.
• -b, --binary – 💾 Open files in binary mode (no CR-LF conversion).
• --follow-symlinks – 🔗 Follow symbolic links when using -i.
• -E, -r, --regexp-extended – 🔧 Use extended regular expressions.
• -s, --separate – 🧩 Treat input files as separate entities.
• --sandbox – 🏖️ Reject e/w/r commands (safe mode).
• -u, --unbuffered – ⚡ Minimize buffering (useful for tail -f).
• -z, --null-data, --zero-terminated – 🧵 Treat input as NUL-terminated lines.

2.3 Exit status

Exit codes: 0 = success, 1 = invalid command/syntax/regex, 2 = input file open failure, 4 = I/O or runtime error. The q and Q commands can return custom exit codes.

📜 3 sed scripts

3.1 sed script overview

A sed program consists of one or more commands, passed via -e, -f, or first non-option argument. Command syntax: [addr]X[options]. Commands can be separated by ; or newlines. Some commands (a, c, i) cannot be followed by semicolons.

3.2 sed commands summary

• a\TEXT – ➕ Append text after a line.
• b LABEL – 🔀 Branch unconditionally to label.
• c\TEXT – 🔄 Replace (change) lines with text.
• d – 🗑️ Delete pattern space, start next cycle.
• D – 🗑️ Delete up to first newline in pattern space.
• e – 🏃 Execute command in pattern space, replace with output.
• e COMMAND – 🏃 Execute COMMAND and send output to stdout.
• F – 📁 Print current input file name.
• g – ♻️ Replace pattern space with hold space.
• G – ♻️ Append hold space to pattern space.
• h – ♻️ Replace hold space with pattern space.
• H – ♻️ Append pattern space to hold space.
• i\TEXT – 📌 Insert text before a line.
• l – 🔍 Print pattern space unambiguously (C-style escapes).
• n – ⏭️ Print pattern space (if auto-print enabled), then read next line.
• N – ⏭️ Append next line to pattern space (with newline).
• p – 🖨️ Print pattern space.
• P – 🖨️ Print pattern space up to first newline.
• q[EXIT-CODE] – 🚪 Quit (print pattern space, exit).
• Q[EXIT-CODE] – 🚪 Quit silently (do not print pattern space).
• r filename – 📂 Read file contents into output.
• R filename – 📂 Queue a line from file to be read.
• s/REGEXP/REPLACEMENT/[FLAGS] – 🔄 Substitute matched text.
• t LABEL – 🔀 Branch if a successful substitution occurred.
• T LABEL – 🔀 Branch if no successful substitution.
• v [VERSION] – ✅ Verify GNU sed version (fails if not supported).
• w filename – 💾 Write pattern space to file.
• W filename – 💾 Write first line of pattern space to file.
• x – 🔄 Exchange hold and pattern spaces.
• y/src/dst/ – 🔄 Transliterate characters.
• z – 🧹 Empty pattern space (zap).
• # – 💬 Comment (to end of line).
• { CMD ; CMD ... } – 📦 Group commands.
• = – 🔢 Print current line number.
• : LABEL – 🏷️ Define label for branching.

3.3 The 's' Command

Syntax: s/REGEXP/REPLACEMENT/FLAGS. Replaces matched portion with REPLACEMENT. Supports backreferences \1–\9, & for whole match. Case conversion: \L, \l, \U, \u, \E. Flags: g (global), NUMBER (replace Nth match), p (print), w FILENAME (write), e (execute result as command), I (case-insensitive), M (multi-line mode).

3.4 Often-Used Commands

• # – 💬 Comment (no addresses allowed). Warning: #n at start forces -n.
• q [EXIT-CODE] – 🚪 Quit after printing current line.
• d – 🗑️ Delete current line.
• p – 🖨️ Print line (usually with -n).
• n – ⏭️ Skip to next line.
• { COMMANDS } – 📦 Group commands for a single address.

3.5 Less Frequently-Used Commands

• y/SOURCE/DEST/ – 🔄 Transliterate characters.
• a TEXT – ➕ Append text after a line (GNU extension).
• a\TEXT – ➕ Append text with line continuation.
• i TEXT – 📌 Insert text before a line.
• i\TEXT – 📌 Insert with line continuation.
• c TEXT – 🔄 Replace line(s) with text.
• c\TEXT – 🔄 Replace with line continuation.
• = – 🔢 Print line number.
• l N – 🔍 Unambiguous print (with line-wrap length).
• r FILENAME – 📂 Read file contents.
• w FILENAME – 💾 Write pattern space to file.
• D – 🗑️ Delete first line of pattern space.
• N – ⏭️ Append next line to pattern space.
• P – 🖨️ Print first line of pattern space.
• h, H, g, G, x – ♻️ Hold space operations.

3.6 Commands for sed gurus

• : LABEL – 🏷️ Define label.
• b LABEL – 🔀 Unconditional branch.
• t LABEL – 🔀 Conditional branch (if substitution succeeded).

3.7 Commands Specific to GNU sed

• e [COMMAND] – 🏃 Execute command; output replaces pattern space or goes to stdout.
• F – 📁 Print filename.
• Q [EXIT-CODE] – 🚪 Quit silently.
• R FILENAME – 📂 Queue a line from file.
• T LABEL – 🔀 Branch if substitution failed.
• v VERSION – ✅ Verify GNU sed version.
• W FILENAME – 💾 Write first line of pattern space.
• z – 🧹 Empty pattern space.

3.8 Multiple commands syntax

Commands can be separated by newlines, -e options, or semicolons (for most commands). Commands a, c, i, #, r, R, w, W, e, and s///[we] require newlines to terminate text/filenames.

🎯 4 Addresses: selecting lines

4.1 Addresses overview

Addresses determine which lines commands apply to. They can be a line number, a regular expression, or a range. Negation with !.

4.2 Selecting lines by numbers

• NUMBER – Match specific line.
• $ – Last line.
• FIRST~STEP – GNU extension: match every STEPth line starting from FIRST.

4.3 Selecting lines by text matching

• /REGEXP/ – Match lines containing REGEXP.
• \%REGEXP% – Use alternative delimiter.
• /REGEXP/I – Case-insensitive match.
• /REGEXP/M – Multi-line mode (^ and $ match line boundaries).

4.4 Range Addresses

Two addresses separated by comma. Special forms: 0,/REGEXP/ (includes first line), ADDR1,+N (N lines after), ADDR1,~N (up to next multiple of N).

🔍 5 Regular Expressions: selecting text

5.1 Overview

Regular expressions are patterns matched against subject strings. Most characters are ordinary; special characters include ^, $, ., *, etc.

5.2 BRE vs ERE

Basic (default) vs Extended (-E). In BRE, +, ?, {}, (), | need backslash to be special. In ERE, they are special without backslash.

5.3 BRE syntax

• CHAR – Literal character.
• * – Zero or more of preceding.
• . – Any character (including newline).
• ^ – Start of line.
• $ – End of line.
• [LIST] – Character class.
• \+ – One or more (GNU extension).
• \? – Zero or one (GNU extension).
• \{I\} – Exactly I times.
• \(REGEXP\) – Grouping.
• REGEXP1\|REGEXP2 – Alternation.
• \DIGIT – Backreference.
• \n – Newline.

5.4 ERE syntax

Same as BRE but with opposite escaping: +, ?, {}, (), | are special without backslash.

5.5 Character Classes and Bracket Expressions

Bracket expressions [...] match one character. Named classes: [:alnum:], [:alpha:], [:blank:], [:cntrl:], [:digit:], [:graph:], [:lower:], [:print:], [:punct:], [:space:], [:upper:], [:xdigit:].

5.6 Regular expression extensions

• \w – Word character (letter, digit, underscore).
• \W – Non-word character.
• \b – Word boundary.
• \B – Non-word boundary.
• \s – Whitespace (space, tab, newline).
• \S – Non-whitespace.
• \< – Beginning of word.
• \> – End of word.
• \` – Start of pattern space.
• \' – End of pattern space.

5.7 Back-references and Subexpressions

Back-references \1–\9 refer to parenthesized subexpressions. Used in both patterns and replacement.

5.8 Escape Sequences

Non-printable characters: \a (BEL), \f (form feed), \n (newline), \r (CR), \t (tab), \v (vertical tab), \cX (control), \dXXX (decimal), \oXXX (octal), \xXX (hex).

5.9 Multibyte and Locale Considerations

GNU sed handles UTF-8 multibyte characters. Invalid sequences are not matched by .. Use z command to clear pattern space. Upper/lower conversion works with multibyte. Use LC_ALL=C for bytewise processing.

🧠 6 Advanced sed: cycles and buffers

6.1 How sed Works

Two buffers: pattern space (active) and hold space (auxiliary). For each line: read, execute commands, print (unless -n), then next cycle.

6.2 Hold and Pattern Buffers

Use h, H, g, G, x to move data between buffers.

6.3 Multiline techniques

Use N, D, P, G, H to process multiple lines. Example: process paragraphs with sed '/./{H;$!d} ; x ; s/REGEXP/REPLACEMENT/'.

6.4 Branching and Flow Control

Commands b (unconditional), t (conditional on success), T (conditional on failure) change flow. Labels defined with :LABEL. Branching can create loops, join lines, etc.

📚 7 Some Sample Scripts

7.1 Joining lines

Join specific lines: sed '2{N;s/\n//;}'. Join backslash-continued lines: sed -e ':x /\\$/ { N; s/\\\n//g ; bx }'. Join lines starting with whitespace: sed -E ':a ; $!N ; s/\n\s+/ / ; ta ; P ; D'.

7.2 Centering lines

Center text to 80 columns using hold space and pattern manipulations.

7.3 Increment a number

Replace trailing 9s with underscores, increment last digit, then replace underscores with zeros.

7.4 Rename files to lower case

Generate shell commands to rename files using sed and y command.

7.5 Print bash environment

Strip function definitions from set output.

7.6 Reverse characters of lines

Using markers and loops to reverse character order.

7.7 Text search across multiple lines

Find doubled words using N and D with backreferences.

7.8 Line length adjustment

Wrap lines at 40 characters using nested loops.

7.9 Reverse lines of files (tac)

Accumulate lines in hold space and print at end.

7.10 Numbering lines (cat -n)

Complex script using hold space and y for incrementing numbers.

7.11 Numbering non-blank lines (cat -b)

Similar but skip blank lines.

7.12 Counting characters (wc -c)

Map characters to letters, propagate carries, convert back to decimal.

7.13 Counting words (wc -w)

Convert words to 'a's, then count like characters.

7.14 Counting lines (wc -l)

Just $=.

7.15 Printing the first lines (head)

10q.

7.16 Printing the last lines (tail)

Sliding window technique using N and D.

7.17 Make duplicate lines unique (uniq)

Using N, P, D to compare adjacent lines.

7.18 Print duplicated lines (uniq -d)

Print only lines that appear more than once.

7.19 Remove all duplicated lines (uniq -u)

Print only unique lines.

7.20 Squeezing blank lines (cat -s)

Various scripts to reduce multiple blank lines to one.

⚠️ 8 Limitations

GNU sed has no built-in line length limit (only memory). Recursion may limit stack for complex patterns.

📎 9 Other Resources

• GNU sed website: https://www.gnu.org/software/sed/
• Mailing list: sed-devel@gnu.org
• sed FAQ: http://sed.sf.net/sedfaq.html
• sed $HOME: http://sed.sf.net

🐛 10 Reporting Bugs

Email to bug-sed@gnu.org with sed --version output. Provide a minimal test case. Known non-bugs include: N on last line, regex backslash semantics, -i on read-only files, 0a not working, [a-z] locale sensitivity, s/.*// not clearing multibyte patterns.

📖 NAME 🚀 Quick Reference 📝 1 Introduction ⚙️ 2 Running sed 📜 3 sed scripts 🎯 4 Addresses: selecting lines 🔍 5 Regular Expressions: selecting text 🧠 6 Advanced sed: cycles and buffers 📚 7 Some Sample Scripts ⚠️ 8 Limitations 📎 9 Other Resources 🐛 10 Reporting Bugs