Chunk size in python large size

def rows(f, chunksize=1024, sep='|'):
    """
    Read a file where the row separator is '|' lazily.

    Usage:

    >>> with open('big.csv') as f:
    >>>     for r in rows(f):
    >>>         process(r)
    """
    row = ''
    while (chunk := f.read(chunksize)) != '':   # End of file
        while (i := chunk.find(sep)) != -1:     # No separator found
            yield row + chunk[:i]
            chunk = chunk[i+1:]
            row = ''
        row += chunk
    yield row

Source: stackoverflow.com

Similar codes

0 vote214 views1 answer
Aggregation with f() in django rest api
Python
7 votes231 views2 answers
Ckeditor django
Python Django
0 vote213 views1 answer
Rdflib check if graph is empty
Python
0 vote223 views1 answer
IntegrityError import in django
Python Django
0 vote199 views1 answer
Delete dataframe from memory python
Python
25 votes197 views5 answers
Create an env in python
Python
1 vote220 views1 answer
Add self role with discord bot python
Python
0 vote206 views1 answer
Matplotlib rc params
Python
0 vote268 views1 answer
Django default template location
Python
1 vote170 views1 answer
Python if boolean
Python