Chunk size in python large size

def rows(f, chunksize=1024, sep='|'):
    """
    Read a file where the row separator is '|' lazily.

    Usage:

    >>> with open('big.csv') as f:
    >>>     for r in rows(f):
    >>>         process(r)
    """
    row = ''
    while (chunk := f.read(chunksize)) != '':   # End of file
        while (i := chunk.find(sep)) != -1:     # No separator found
            yield row + chunk[:i]
            chunk = chunk[i+1:]
            row = ''
        row += chunk
    yield row

Source: stackoverflow.com

Similar codes

20 votes354 views6 answers
Python append to list
Python
23 votes334 views6 answers
Check tensorflow version
Python
5 votes314 views4 answers
Python get last element in dict
Python
18 votes362 views4 answers
Reset index python
Python
6 votes332 views3 answers
How to remove all zeros from a list in python
Python
1 vote286 views1 answer
Does range start at 0 python
Python
3 votes341 views2 answers
How to connect python to html
Python
1 vote317 views1 answer
Kaaba python tutorial
Python
2 votes285 views2 answers
Python threading return value
Python
0 vote330 views1 answer
Western school district
Python