Sorting/filtering/counting text data
From Tech-Wiki
More actions
Simple shell pipelines for filtering, sorting and counting delimited text data.
Validation status
Reviewed for migration on 27 September 2026. The examples remain useful for small text files; use a format-aware parser for complex or untrusted input.
Original technical reference
Let's assume we'd like to count the most hit firewall rules parsing the log:
$ cat firewall.log | grep -v -E '^$' | grep -v date | cut -d\; -f4 | cut -d\= -f2 | sort -n | uniq -c | sort
remove blank lines: grep -v -E '^$'
remove lines with 'date' in them: grep -v date
cut out field "rule=XX": cut -d\; -f4 (delimiter: ";" and print field 4)
cut out rule numbers: cut -d\= -f2 (delimiter: "=" and print field 2)
sort lines numerically: sort -n
count number of times the rule number appears: uniq -c
For reference, this is the firewall log's content:
$ cat firewall.log date:25-mar source=10.1.1.1;destination=192.168.1.1;port=80;rule=3 source=10.1.1.1;destination=192.168.1.1;port=80;rule=3 source=10.1.1.1;destination=192.168.1.1;port=21;rule=2 source=10.1.1.1;destination=192.168.1.1;port=22;rule=1