/Python/Remove duplicate rows in csv file python

Remove duplicate rows in csv file python

# credit to the Stack Overflow user in the source link

with open('file_with_duplicates.csv','r') as in_file, open('ouput.csv','w') as out_file:
  
    seen = set() # set for fast O(1) amortized lookup
    
    for line in in_file:
        if line in seen: 
          continue # skip duplicate

        seen.add(line)
        out_file.write(line)

Source: stackoverflow.com

Similar codes

2 votes229 views2 answers
Calculate mean on python
Python
0 vote243 views1 answer
Add variable to print python
Python
-1 vote216 views2 answers
Print tables in python
Python
1 vote220 views1 answer
How to get pygame key
Python
0 vote217 views1 answer
Set difference in multidimensional array numpy
Python
0 vote228 views1 answer
C# script for download music from telegram channel
Python
0 vote255 views1 answer
Cbv uk django
Python Django
0 vote245 views1 answer
Flask admin forgeign keys show literal
Python Flask
0 vote229 views1 answer
Convert unit dynamo revit
Python
0 vote233 views1 answer
(for in) printing in python
Python