Chunk size in python large size

def rows(f, chunksize=1024, sep='|'):
    """
    Read a file where the row separator is '|' lazily.

    Usage:

    >>> with open('big.csv') as f:
    >>>     for r in rows(f):
    >>>         process(r)
    """
    row = ''
    while (chunk := f.read(chunksize)) != '':   # End of file
        while (i := chunk.find(sep)) != -1:     # No separator found
            yield row + chunk[:i]
            chunk = chunk[i+1:]
            row = ''
        row += chunk
    yield row

Source: stackoverflow.com

Similar codes

0 vote297 views1 answer
How to use annoy python
Python
0 vote272 views1 answer
Networkx draw edge description
Python
0 vote294 views1 answer
Regularization pytorch
Python
0 vote304 views1 answer
Open tkinter and cli
Python
0 vote325 views1 answer
Mechanize python #12
Python
0 vote273 views1 answer
Store in a variable the ocntent of a file python
Python
15 votes301 views6 answers
Python last element of list
Python
1 vote335 views1 answer
Flask post vs get
Python Flask
0 vote280 views1 answer
Heroku python heroku port issue
Python
0 vote266 views3 answers
Captcha.image install in python
Python