I usually do text processing in Bash, Notepad++ and Excel. Each has its own pros and cons, that's why I usually combine them.
Here you have the tools I use in Bash:
grep, tail, head, cat, cut, less, awk, sed, sort, uniq, wc, xargs, watch ...
Here you have the tools I use in Bash:
grep, tail, head, cat, cut, less, awk, sed, sort, uniq, wc, xargs, watch ...
cat file.txt | awk '!x[$0]++'