Been looking for one a while a go and it feels like it’s something sound be built in.
Been looking for one a while a go and it feels like it’s something sound be built in.
> Return the unique lines in the file words.txt that don't exist in countries.txt
comm -23 <(sort words.txt | uniq) <(sort countries.txt | uniq)
>Return the lines that are in both words.txt and countries.txt: comm -12 <(sort words.txt | uniq) <(sort countries.txt | uniq) ~/bin/union
===========
#! /usr/bin/awk -f
!acc[$0]++
~/bin/intersection
==================
#! /usr/bin/awk -f
!buf[$0]++ {acc[$0] += 1}
ENDFILE {
delete buf;
files++
}
END {
for (k in acc) if (acc[k] == files) print k
}
~/bin/set-diff
==============
#! /usr/bin/awk -f
! filenum { acc[$0] = 1 }
filenum { delete acc[$0] }
ENDFILE { filenum++ }
END {
for (k in acc) print k
}It's basically meant to look up spellings in /usr/share/dict/words, but it can work on any file. It will match any line that your pattern is a prefix of, so you'd have to add logic to eliminate longer matches.
But if you had some huge file to search and you wanted to do it from a shell script, that would be one way. Caveat: although it's fairly standard, 'look' might not be installed on every system.
Also, you have to be sure to maintain your file in sorted order. So no adding things by appending to the end, and checking if something is in the set is much quicker at the expense of adding things being much slower.