Chunk size in python large size

def rows(f, chunksize=1024, sep='|'):
    """
    Read a file where the row separator is '|' lazily.

    Usage:

    >>> with open('big.csv') as f:
    >>>     for r in rows(f):
    >>>         process(r)
    """
    row = ''
    while (chunk := f.read(chunksize)) != '':   # End of file
        while (i := chunk.find(sep)) != -1:     # No separator found
            yield row + chunk[:i]
            chunk = chunk[i+1:]
            row = ''
        row += chunk
    yield row

Source: stackoverflow.com

Similar codes

0 vote369 views1 answer
Python dictionary key in range
Python
0 vote346 views3 answers
OpenCV(3.4.11) Error: Assertion failed (_img.rows * _img.cols == vecSize) in CvCascadeImageReader::PosReader::get
Python
1 vote340 views1 answer
Django proper capitalization case jinja
Python
6 votes364 views2 answers
Python gzip a file
Python
1 vote343 views1 answer
Print last output python
Python
0 vote357 views1 answer
Can we pickle pyspark dataframe using python
Python
0 vote361 views1 answer
Django multi column index
Python
0 vote373 views1 answer
Python word cloud
Python
0 vote345 views2 answers
Python generate secret key
Python
0 vote410 views1 answer
Save datetime day in seperate column
Python