/Shell/Bash/Bash remove duplicate lines from a file

Bash remove duplicate lines from a file

# Basic syntax:
sort input_file | uniq --unique
# Sort the file first because uniq requires a sorted file to work. Uniq
# then eliminates all duplicated lines in the file, keeping one instance
# of each duplicated line

# Note, this doesn't return only non-duplicated lines. It returns
# unique instances of all lines, whether or not they are duplicated

# Note, if you want to return only one instance of all lines but see
# the number of repetitions for each line, run:
sort input_file | uniq -c

Source: geek-university.com

Similar codes

1 vote281 views2 answers
How to update google chrome in ubuntu 20.04
Shell/Bash
0 vote258 views1 answer
Asynchronous shell commands
Shell/Bash
1 vote243 views3 answers
Bash run a command in the background
Shell/Bash
0 vote251 views1 answer
Unattended-upgr
Shell/Bash
0 vote212 views1 answer
Not able to start conda navigator ubuntu 18.04
Shell/Bash
0 vote224 views1 answer
Terminal italics test sequence
Shell/Bash
5 votes216 views2 answers
Ubuntu install latest vim
Shell/Bash
7 votes225 views5 answers
How to install dependencies from package.json
Shell/Bash
1 vote233 views1 answer
How to install open media vault on raspberry pi
Shell/Bash
0 vote206 views1 answer
Check user of windows
Shell/Bash