UNIX Power Tools

UNIX Power ToolsSearch this book
Previous: 35.19 Joining Lines with join Chapter 35
You Can't Quite Call This Editing
Next: 35.21 Using IFS to Split Strings
 

35.20 Quick Reference: uniq

uniq is used to remove duplicate adjacent lines from a sorted file, sending one copy of each line to standard output or to a second file, if one is specified on the command line.

NOTE: Be warned:

% uniq file1 file2

will not print the unique lines from both file1 and file2 to standard output! It will replace the contents of file2 with the unique lines from file1.

uniq is often used as a filter (1.30). See also comm (28.12), sort (36.1), and especially sort -u (36.6).

options

-c       Print each line once, counting instances of each.
-d       Print duplicate lines once, but no unique lines.
-u       Print only unique lines (no copy of duplicate entries is kept).
-n       Ignore first n fields of a line. Fields are separated by spaces
or by tabs.
+n       Ignore first n characters of a field.

You can specify only one of the options -c, -d, or -u.

Examples

To send only one copy of each line from list to output file list.new:

uniq list list.new

To show which names appear more than once:

sort names | uniq -d

To show which lines appear exactly three times:

grep 
sort names | uniq -c | grep " 3 "

- DG from O'Reilly & Associates' UNIX in a Nutshell (SVR4/Solaris)


Previous: 35.19 Joining Lines with join UNIX Power ToolsNext: 35.21 Using IFS to Split Strings
35.19 Joining Lines with join Book Index35.21 Using IFS to Split Strings

The UNIX CD Bookshelf NavigationThe UNIX CD BookshelfUNIX Power ToolsUNIX in a NutshellLearning the vi Editorsed & awkLearning the Korn ShellLearning the UNIX Operating System