Chunk size in python large size

def rows(f, chunksize=1024, sep='|'):
    """
    Read a file where the row separator is '|' lazily.

    Usage:

    >>> with open('big.csv') as f:
    >>>     for r in rows(f):
    >>>         process(r)
    """
    row = ''
    while (chunk := f.read(chunksize)) != '':   # End of file
        while (i := chunk.find(sep)) != -1:     # No separator found
            yield row + chunk[:i]
            chunk = chunk[i+1:]
            row = ''
        row += chunk
    yield row

Source: stackoverflow.com

Similar codes

1 vote238 views1 answer
Algorithms for Determine the sum of al digits of n
Python
0 vote234 views1 answer
Discord.py get server id
Python
1 vote256 views1 answer
Flask send client to another web page
Python
5 votes235 views1 answer
Upload excel file into pycharm
Python
9 votes217 views1 answer
How to check if datapoint is in pandas column
Python
0 vote224 views1 answer
Pvm python
Python
0 vote253 views1 answer
Django python base 64 encode
Python Django
8 votes251 views3 answers
Python clone object
Python
1 vote227 views1 answer
How to get the user ip in djagno
Python
1 vote251 views1 answer
Matplotlib wrap title
Python