/Shell/Bash/Bash remove duplicate lines from a file

Bash remove duplicate lines from a file

# Basic syntax:
sort input_file | uniq --unique
# Sort the file first because uniq requires a sorted file to work. Uniq
# then eliminates all duplicated lines in the file, keeping one instance
# of each duplicated line

# Note, this doesn't return only non-duplicated lines. It returns
# unique instances of all lines, whether or not they are duplicated

# Note, if you want to return only one instance of all lines but see
# the number of repetitions for each line, run:
sort input_file | uniq -c

Source: geek-university.com

Similar codes

0 vote245 views1 answer
Rkunter after get ubuntu new release
Shell/Bash
0 vote264 views1 answer
Change last commit date
Shell/Bash
0 vote247 views6 answers
How to remove a file from staging area in git
Shell/Bash
1 vote297 views1 answer
Awk count if
Shell/Bash
2 votes250 views1 answer
Jupyter notebook conda install
Shell/Bash
0 vote251 views1 answer
Install caprine linux
Shell/Bash
0 vote268 views2 answers
Signing key android
Shell/Bash
0 vote262 views1 answer
Change nohup name
Shell/Bash
1 vote295 views2 answers
How to close terminal
Shell/Bash
3 votes261 views2 answers
Npm ERR! Maximum call stack size exceeded ubuntu
Shell/Bash