/Python/Remove duplicate rows in csv file python

Remove duplicate rows in csv file python

# credit to the Stack Overflow user in the source link

with open('file_with_duplicates.csv','r') as in_file, open('ouput.csv','w') as out_file:
  
    seen = set() # set for fast O(1) amortized lookup
    
    for line in in_file:
        if line in seen: 
          continue # skip duplicate

        seen.add(line)
        out_file.write(line)

Source: stackoverflow.com

Similar codes

0 vote194 views1 answer
Zero crossing rate python
Python
2 votes254 views3 answers
Play mp3 file python
Python
0 vote190 views2 answers
How to read mysql table in python
Python
0 vote183 views2 answers
Numpy index of first true value
Python
0 vote190 views1 answer
Python how to invert an array
Python
10 votes209 views2 answers
Max columns in python
Python
0 vote203 views1 answer
How to pull images from android device from usb in python
Python
0 vote199 views2 answers
How to play a mp3 file in python
Python
0 vote248 views1 answer
Multiplication table for number python codewars
Python
0 vote186 views1 answer
Set value through serializer django
Python