/Shell/Bash/Bash remove duplicate lines from a file

Bash remove duplicate lines from a file

# Basic syntax:
sort input_file | uniq --unique
# Sort the file first because uniq requires a sorted file to work. Uniq
# then eliminates all duplicated lines in the file, keeping one instance
# of each duplicated line

# Note, this doesn't return only non-duplicated lines. It returns
# unique instances of all lines, whether or not they are duplicated

# Note, if you want to return only one instance of all lines but see
# the number of repetitions for each line, run:
sort input_file | uniq -c

Source: geek-university.com

Similar codes

39 votes185 views4 answers
How to install homebrew on macOS
Shell/Bash
0 vote192 views2 answers
Bash count files in directory recursively matchingattern
Shell/Bash
-1 vote192 views3 answers
How to push newly created branch in git bash
Shell/Bash
0 vote178 views1 answer
Cannot find module 'nativescript-ui-sidedrawer/angular' or its corresponding type declarations
Shell/Bash
8 votes204 views3 answers
Powershell make directory
Shell/Bash
0 vote181 views1 answer
Kali virtualbox guest additions check version
Shell/Bash
0 vote149 views1 answer
Could not get batched bridge
Shell/Bash
1 vote191 views1 answer
Grep specific file
Shell/Bash
0 vote175 views1 answer
Apache2 connection refused ubuntu
Shell/Bash
0 vote155 views1 answer
Ubuntu vs kali linux
Shell/Bash