/Python/Txt file duplicate line remover python

Txt file duplicate line remover python

lines_seen = set() # holds lines already seen

with open("file.txt", "r+") as f:
    d = f.readlines()
    f.seek(0)
    for i in d:
        if i not in lines_seen:
            f.write(i)
            lines_seen.add(i)
    f.truncate()

Source: stackoverflow.com

Similar codes

1 vote198 views1 answer
Handler.setLevel(logging.DEBUG) not working python
Python
2 votes199 views2 answers
Fibonacci sequence python 2.7
Python
0 vote141 views1 answer
Form validation examples
Python
0 vote187 views1 answer
TypeError: add() got an unexpected keyword argument 'kernel_regularizer'
Python
0 vote183 views1 answer
Python array set key value
Python
217 votes309 views6 answers
Python random number generator
Python
1 vote187 views1 answer
Install chromedriver ubuntu python
Python
9 votes199 views5 answers
Threading python
Python
0 vote196 views1 answer
Convert the below Series to pandas datetime : DoB = pd.Series(["07Sep59","01Jan55","15Dec47","11Jul42"])
Python
0 vote211 views1 answer
Length of number without using len function
Python