/Python/Remove duplicate rows in csv file python

Remove duplicate rows in csv file python

# credit to the Stack Overflow user in the source link

with open('file_with_duplicates.csv','r') as in_file, open('ouput.csv','w') as out_file:
  
    seen = set() # set for fast O(1) amortized lookup
    
    for line in in_file:
        if line in seen: 
          continue # skip duplicate

        seen.add(line)
        out_file.write(line)

Source: stackoverflow.com

Similar codes

0 vote312 views1 answer
Frogenset ito list pandas
Python
2 votes311 views1 answer
Python IRR calculation
Python
1 vote347 views1 answer
Python try then change something and try again if fails
Python
0 vote292 views1 answer
Plt force axis numbers
Python
63 votes267 views3 answers
Iterate through a list
Python
-1 vote287 views1 answer
Ovh minecraft
Python
3 votes319 views1 answer
Pyautogui moveTo overtime
Python
6 votes316 views1 answer
Python soap
Python
47 votes275 views3 answers
Convert to string in python
Python
87 votes287 views2 answers
How to write a while statement in python
Python