Chunk size in python large size

def rows(f, chunksize=1024, sep='|'):
    """
    Read a file where the row separator is '|' lazily.

    Usage:

    >>> with open('big.csv') as f:
    >>>     for r in rows(f):
    >>>         process(r)
    """
    row = ''
    while (chunk := f.read(chunksize)) != '':   # End of file
        while (i := chunk.find(sep)) != -1:     # No separator found
            yield row + chunk[:i]
            chunk = chunk[i+1:]
            row = ''
        row += chunk
    yield row

Source: stackoverflow.com

Similar codes

0 vote279 views1 answer
Save impt
Python
0 vote299 views1 answer
How to check if object is of file type in python3
Python
0 vote298 views1 answer
How to copy list python
Python
10 votes355 views1 answer
Pandas slice based on column value
Python
1 vote338 views1 answer
Flask return error response
Python Flask
0 vote337 views1 answer
Get date only from datetimefiel django
Python Django
3 votes286 views2 answers
How to create a network scanner in python
Python
0 vote325 views1 answer
How to count the appearance of number or string in a list python
Python
3 votes290 views2 answers
Pyperclip copy paste
Python
2 votes275 views1 answer
Django group with permission
Python