Duplicate finder python modules

def findDup(parentFolder):
    # Dups in format {hash:[names]}
    dups = {}
    for dirName, subdirs, fileList in os.walk(parentFolder):
        print('Scanning %s...' % dirName)
        for filename in fileList:
            # Get the path to the file
            path = os.path.join(dirName, filename)
            # Calculate hash
            file_hash = hashfile(path)
            # Add or append the file path
            if file_hash in dups:
                dups[file_hash].append(path)
            else:
                dups[file_hash] = [path]
    return dups

Source: pythoncentral.io

Similar codes

2 votes336 views1 answer
Import validation error in django
Python Django
30 votes296 views2 answers
For loop example python 3
Python
2 votes300 views1 answer
How to save a model fast ai
Python
8 votes316 views2 answers
How to read excel file in jupyter notebook
Python
0 vote314 views1 answer
Generator comprehension python
Python
-1 vote303 views2 answers
Get tuple value python
Python
0 vote297 views1 answer
Python tensorflow don't show information
Python
0 vote288 views1 answer
Python lists as dataframe rows
Python
5 votes294 views1 answer
Pandas for loop after loc reset_index
Python
0 vote329 views1 answer
"must be called with either an object pk or a slug in the URLconf"
Python