Chunk size in python large size

def rows(f, chunksize=1024, sep='|'):
    """
    Read a file where the row separator is '|' lazily.

    Usage:

    >>> with open('big.csv') as f:
    >>>     for r in rows(f):
    >>>         process(r)
    """
    row = ''
    while (chunk := f.read(chunksize)) != '':   # End of file
        while (i := chunk.find(sep)) != -1:     # No separator found
            yield row + chunk[:i]
            chunk = chunk[i+1:]
            row = ''
        row += chunk
    yield row

Source: stackoverflow.com

Similar codes

4 votes192 views5 answers
AttributeError: module 'datetime' has no attribute 'now'
Python
1 vote184 views4 answers
Python ip camera
Python
0 vote173 views1 answer
Lines = paths.read().splitlines()
Python
2 votes175 views1 answer
Python find specific file in directory
Python
0 vote181 views1 answer
Method for format age in python
Python
0 vote173 views1 answer
Python get time executed by function
Python
0 vote209 views1 answer
Sum of all substrings of a number
Python
1 vote177 views4 answers
Bogo Sort
Python
1 vote183 views3 answers
Create list of days python
Python
-3 votes163 views5 answers
Multiplication table in python
Python