/Python/Python delete duplicate lines in file

Python delete duplicate lines in file

with open("file.txt", "r") as txt_file:
  new_data = list(set(txt_file))
  return new_data

Txt file duplicate line remover python

lines_seen = set() # holds lines already seen

with open("file.txt", "r+") as f:
    d = f.readlines()
    f.seek(0)
    for i in d:
        if i not in lines_seen:
            f.write(i)
            lines_seen.add(i)
    f.truncate()

Source: stackoverflow.com

Similar codes

0 vote313 views1 answer
Python __getattr__ geeksforgeeks
Python
17 votes269 views1 answer
Strip all spaces python
Python
24 votes322 views3 answers
Tkinter image
Python
0 vote315 views1 answer
Circular heatmap python
Python
11 votes310 views6 answers
How to add to a list python
Python
1 vote279 views2 answers
Assign multiline string to variable in python
Python
1 vote322 views2 answers
Middle value of a list in python
Python
1 vote303 views1 answer
How to get the mouse input in pygame
Python
0 vote296 views1 answer
Tensorboard dev upload in background colab
Python
1 vote302 views1 answer
How to split a string by colon in python
Python