/Shell/Bash/Bash remove duplicate lines from a file

Bash remove duplicate lines from a file

# Basic syntax:
sort input_file | uniq --unique
# Sort the file first because uniq requires a sorted file to work. Uniq
# then eliminates all duplicated lines in the file, keeping one instance
# of each duplicated line

# Note, this doesn't return only non-duplicated lines. It returns
# unique instances of all lines, whether or not they are duplicated

# Note, if you want to return only one instance of all lines but see
# the number of repetitions for each line, run:
sort input_file | uniq -c

Source: geek-university.com

Similar codes

25 votes331 views4 answers
Install chrome on linux
Shell/Bash
0 vote309 views1 answer
The current numpy installation fails to pass a sanity check due to a bug in the windows runtime
Shell/Bash
1 vote328 views2 answers
Change wallpaper command line linux
Shell/Bash
1 vote296 views1 answer
Whats home directory variable linux
Shell/Bash
32 votes326 views2 answers
Install postman ubuntu
Shell/Bash
0 vote369 views1 answer
Manjaro mirrors update
Shell/Bash
6 votes326 views2 answers
Rename computer ubuntu 20.04
Shell/Bash
0 vote339 views1 answer
Bash add all numbers
Shell/Bash
0 vote312 views1 answer
Git fetch upstream from master
Shell/Bash
0 vote328 views1 answer
Add more changes to same commit
Shell/Bash