Working with text files is a fundamental task in computing, especially on Linux and Unix-like systems. Whether you’re cleaning up log files, processing data, or managing configuration settings, you often need to remove specific lines based on their content. While many tools exist, the sed command (Stream Editor) is incredibly powerful and efficient for this purpose.
sed processes text input stream by stream, or more commonly, file by file, applying specified operations to each line. One of its most common uses is deleting lines that match a particular pattern.
Understanding the Basic sed Deletion Command
The core of deleting lines with sed involves using the d command. You tell sed which lines to apply this command to using an address, which in this case, is a pattern defined by a regular expression.
The basic syntax looks like this:
sed '/pattern/d' your_file.txt
Let’s break this down:
sed: Invokes the Stream Editor command./pattern/: This is the address.sedwill look for lines that containpattern. The pattern is enclosed in forward slashes (/). Patterns are powerful regular expressions.d: This is the command to execute on the addressed lines. Thedstands for delete.your_file.txt: The name of the file you are processing.
Key Point: By default, sed prints the modified output to standard output (your terminal). It does not change the original file.
Examples:
Deleting lines containing a specific word:
To remove all lines that include the word “DEBUG” from a log file:sed '/DEBUG/d' application.logThe result (the log file without the DEBUG lines) will be shown in your terminal.
Deleting lines that start with a specific character:
To remove comment lines (those starting with#) from a configuration file:
bash
sed '/^#/d' my.conf
Here,^is a regular expression anchor meaning the beginning of the line.
Modifying the File Directly (Use Caution!)
Printing to standard output is great for testing or piping the output to another command. However, you often want to save the changes directly back to the original file. This is done using the -i option:
sed -i '/pattern/d' your_file.txt
Critical Security/Safety Tip: Using the -i option modifies the file in place. There is no undo. Before using -i on important files, always test your sed command first without -i to ensure it does exactly what you expect, or make a backup of your file. A common practice is cp your_file.txt your_file.txt.bak before running the sed -i command.
Deleting Lines Not Matching a Pattern
Sometimes, the goal is the opposite: you want to keep only the lines that match a pattern and delete everything else. You can achieve this by combining the pattern address with the negation operator (!).
sed '/pattern/!d' your_file.txt
Here, the ! negates the address. The d command is applied to all lines that do not match /pattern/.
Example:
To keep only the lines containing “ERROR” and delete all other lines from a log file:
sed '/ERROR/!d' application.log
Again, add -i if you want to modify the file in place, exercising the same caution.
Conclusion
sed is an indispensable tool for manipulating text files from the command line. Mastering the simple /pattern/d syntax, along with the crucial -i option for in-place editing (used with care!), allows you to quickly and efficiently clean up files by deleting unwanted lines based on powerful pattern matching. Incorporating the negation operator ! provides even more flexibility, letting you isolate specific data while discarding the rest. With a little practice, sed can significantly streamline your text processing workflows.
Source: https://kifarunix.com/delete-lines-matching-a-specific-pattern-in-a-file-using-sed/
