File manipulation is a fundamental skill for anyone working with the command line. In this blog, we'll explore how to compare and manipulate file content using find, grep, awk, and sed. Regular expressions will be our trusty companion throughout, enhancing the power and flexibility of these commands.

The Role of Regular Expressions

Before diving into specific commands, let's understand the importance of regular expressions. Regular expressions, often abbreviated as regex, are patterns that define a search. They enable us to express flexible and complex search criteria, making them invaluable for text processing.

Find Command

The find command is excellent for locating files based on various attributes. When combined with regex, it becomes a potent tool for precision searches.

Example 1: Find all text files in the current directory

find . -type f -regex ".*\.txt"

Example 2: Find files modified within the last 7 days

find . -type f -mtime -7

Example 3: Find files starting with "report"

find . -type f -regex "./report.*"

Example 4: Find all directories containing "data"

find . -type d -name "*data*"

Example 5: Find files with names matching a specific pattern

find . -type f -name "file[0-9].txt"

Grep Command

grep is a versatile tool for searching text patterns within files. It's an indispensable command for quickly isolating relevant information.

Example 1: Search for lines containing "error" in a log file

grep "error" logfile.txt

Example 2: Search for lines starting with "warning" in multiple files

grep "^warning" *.log

Example 3: Search for lines not containing "success" in a file

grep -v "success" data.txt

Example 4: Search for lines with words starting with "cat" in a file

grep "\<cat" animals.txt

Example 5: Search for lines with exactly 8 characters in a file

grep -E "^.{8}$" passwords.txt

Awk Command

awk is a powerful text processing tool that operates on a per-line basis. Regular expressions enhance its ability to extract and manipulate data.

Example 1: Print the second column of a CSV file

awk -F, '{print $2}' data.csv

Example 2: Print lines containing more than 3 fields in a file

awk 'NF > 3' report.txt

Example 3: Replace "apple" with "orange" in the third column of a file

awk '{gsub(/apple/, "orange", $3); print}' fruits.txt

Example 4: Print lines where the first column starts with "A"

awk '$1 ~ /^A/' names.txt

Example 5: Calculate and print the average of the numbers in the second column

awk '{sum+=$2} END {print sum/NR}' numbers.txt

Sed Command

sed is a stream editor used for text transformations. Regular expressions make it a robust tool for search and replace operations.

Example 1: Replace all occurrences of "apple" with "orange" in a file

sed 's/apple/orange/g' filename.txt

Example 2: Delete lines containing the word "obsolete"

sed '/obsolete/d' data.txt

Example 3: Append " - Processed" to lines starting with "Task:"

sed '/^Task:/ s/$/ - Processed/' tasks.txt

Example 4: Replace the first occurrence of "blue" with "red" on each line

sed 's/blue/red/' colors.txt

Example 5: Insert a new line after lines containing "important"

sed '/important/a\New Line Inserted' notes.txt

Regular expressions, when coupled with these commands, provide a powerful toolkit for anyone dealing with file content manipulation in the command line. Mastering these concepts will significantly enhance your efficiency and effectiveness in handling textual data.