/Python/Remove duplicate rows in csv file python

Remove duplicate rows in csv file python

# credit to the Stack Overflow user in the source link

with open('file_with_duplicates.csv','r') as in_file, open('ouput.csv','w') as out_file:
  
    seen = set() # set for fast O(1) amortized lookup
    
    for line in in_file:
        if line in seen: 
          continue # skip duplicate

        seen.add(line)
        out_file.write(line)

Source: stackoverflow.com

Similar codes

0 vote263 views1 answer
Round to nearest multiple of 5 python
Python
1 vote308 views1 answer
Use incognito in selenium webdriver
Python
0 vote279 views1 answer
Python plot lines with dots
Python
0 vote283 views1 answer
Jetson nx unable to install matplotlib
Python
1 vote324 views1 answer
Hide particular attribute in django admin
Python Django
0 vote265 views2 answers
Examples of function decorators in Python
Python
0 vote238 views1 answer
Python how to move multiple values from one list to another at once
Python
0 vote245 views1 answer
Locate certs path for python
Python
0 vote282 views1 answer
Except Exception, e: suds python
Python
0 vote281 views1 answer
Python search resultset from finadall
Python