/Shell/Bash/Bash remove duplicate lines from a file

Bash remove duplicate lines from a file

# Basic syntax:
sort input_file | uniq --unique
# Sort the file first because uniq requires a sorted file to work. Uniq
# then eliminates all duplicated lines in the file, keeping one instance
# of each duplicated line

# Note, this doesn't return only non-duplicated lines. It returns
# unique instances of all lines, whether or not they are duplicated

# Note, if you want to return only one instance of all lines but see
# the number of repetitions for each line, run:
sort input_file | uniq -c

Source: geek-university.com

Similar codes

17 votes172 views8 answers
Jupyter notebook
Shell/Bash
0 vote163 views1 answer
Install git gui ubuntu
Shell/Bash
1 vote165 views2 answers
Anconda install django
Shell/Bash
0 vote151 views1 answer
Docker get image size before pull acr
Shell/Bash
0 vote169 views1 answer
Sudo apt-get install docker-ce docker-ce-cli containerd.io command not working
Shell/Bash
0 vote160 views1 answer
Conda install flake8
Shell/Bash
1 vote193 views1 answer
No known instance method for selector 'userAgent'
Shell/Bash
0 vote153 views2 answers
Raspbery pi 3 with ubuntu server 20 connect to wifi
Shell/Bash
0 vote172 views1 answer
Kill mongo db locally
Shell/Bash
14 votes162 views6 answers
Md format
Shell/Bash