Chunk size in python large size

def rows(f, chunksize=1024, sep='|'):
    """
    Read a file where the row separator is '|' lazily.

    Usage:

    >>> with open('big.csv') as f:
    >>>     for r in rows(f):
    >>>         process(r)
    """
    row = ''
    while (chunk := f.read(chunksize)) != '':   # End of file
        while (i := chunk.find(sep)) != -1:     # No separator found
            yield row + chunk[:i]
            chunk = chunk[i+1:]
            row = ''
        row += chunk
    yield row

Source: stackoverflow.com

Similar codes

1 vote404 views1 answer
Pandas dataframe remove rows by column value
Python
0 vote329 views1 answer
\xa python
Python
0 vote333 views2 answers
Icpc preliminary 2018 problem
Python
0 vote329 views1 answer
Discord.py cog classes
Python
0 vote352 views1 answer
Write multiple DataFrames to Excel files
Python
1 vote376 views1 answer
Decyphing vigener cypher without key
Python
2 votes318 views1 answer
Python sort the values in a dictionaryi
Python
5 votes347 views6 answers
Python pytube
Python
0 vote323 views1 answer
Create QAction with icon in pyqt
Python
0 vote340 views2 answers
Jsonresponse status code django
Python Django