Chunk size in python large size

def rows(f, chunksize=1024, sep='|'):
    """
    Read a file where the row separator is '|' lazily.

    Usage:

    >>> with open('big.csv') as f:
    >>>     for r in rows(f):
    >>>         process(r)
    """
    row = ''
    while (chunk := f.read(chunksize)) != '':   # End of file
        while (i := chunk.find(sep)) != -1:     # No separator found
            yield row + chunk[:i]
            chunk = chunk[i+1:]
            row = ''
        row += chunk
    yield row

Source: stackoverflow.com

Similar codes

0 vote153 views1 answer
Initialize dictionary to zero in python
Python
23 votes211 views2 answers
Django rest
Python Django
12 votes208 views5 answers
Python async await
Python
0 vote165 views1 answer
How to bacome michael reeves in python
Python
31 votes159 views5 answers
How to loop through files in a directory python
Python
9 votes203 views2 answers
Is in array python
Python
1 vote183 views1 answer
Sum values in django models
Python
-1 vote166 views3 answers
List
Python
0 vote175 views6 answers
ORing two cv mat objects
Python
2 votes149 views2 answers
Head first python
Python