Chunk size in python large size

def rows(f, chunksize=1024, sep='|'):
    """
    Read a file where the row separator is '|' lazily.

    Usage:

    >>> with open('big.csv') as f:
    >>>     for r in rows(f):
    >>>         process(r)
    """
    row = ''
    while (chunk := f.read(chunksize)) != '':   # End of file
        while (i := chunk.find(sep)) != -1:     # No separator found
            yield row + chunk[:i]
            chunk = chunk[i+1:]
            row = ''
        row += chunk
    yield row

Source: stackoverflow.com

Similar codes

1 vote135 views1 answer
Keras import optimizer adam
Python
0 vote115 views1 answer
Assign exec function to variable python
Python
0 vote148 views2 answers
How to use django-filters with viewset
Python Django
0 vote121 views1 answer
Python turtle triangle
Python
0 vote114 views1 answer
Dictionary lookup python
Python
0 vote112 views1 answer
Gui radio button python
Python
0 vote119 views1 answer
Importando todo o pacote em python
Python
0 vote122 views1 answer
Compare lists element wise python
Python
1 vote122 views2 answers
Python delete key if exists
Python
0 vote120 views1 answer
Python change file location
Python