/Python/Txt file duplicate line remover python

Txt file duplicate line remover python

lines_seen = set() # holds lines already seen

with open("file.txt", "r+") as f:
    d = f.readlines()
    f.seek(0)
    for i in d:
        if i not in lines_seen:
            f.write(i)
            lines_seen.add(i)
    f.truncate()

Source: stackoverflow.com

Similar codes

4 votes282 views3 answers
Read csv without header pandas
Python
24 votes286 views3 answers
Get last n characters of string python
Python
5 votes263 views1 answer
Window size cv2
Python
4 votes301 views2 answers
Loginrequiredmixin django
Python Django
0 vote261 views1 answer
?: (admin.W411) 'django.template.context_processors.request' must be enabled in DjangoTemplates (TEMPLATES) in order to use the admin navigation sidebar.
Python
1 vote250 views2 answers
Download pdf from url python
Python
0 vote269 views1 answer
Sns linear regression
Python
0 vote280 views1 answer
Padnas get list of rows
Python
0 vote250 views1 answer
Django template date format yyyy-mm-dd
Python Django
0 vote256 views1 answer
Dataframe move row up one
Python