Chunk size in python large size

def rows(f, chunksize=1024, sep='|'):
    """
    Read a file where the row separator is '|' lazily.

    Usage:

    >>> with open('big.csv') as f:
    >>>     for r in rows(f):
    >>>         process(r)
    """
    row = ''
    while (chunk := f.read(chunksize)) != '':   # End of file
        while (i := chunk.find(sep)) != -1:     # No separator found
            yield row + chunk[:i]
            chunk = chunk[i+1:]
            row = ''
        row += chunk
    yield row

Source: stackoverflow.com

Similar codes

0 vote191 views1 answer
Python jointly shuffle list
Python
40 votes175 views6 answers
Reset index pandas
Python
0 vote174 views1 answer
Gevent with flask
Python
14 votes169 views6 answers
Find last element in list python
Python
0 vote214 views1 answer
The requested resource was not found on this server. django
Python
0 vote179 views1 answer
Add variable in text python
Python
0 vote219 views1 answer
Swapping upper case and lower case string python
Python
1 vote161 views1 answer
Dataframe sort values descending
Python
7 votes186 views4 answers
Qr code in python
Python
0 vote181 views1 answer
Python group groupe of 2
Python