Chunk size in python large size

def rows(f, chunksize=1024, sep='|'):
    """
    Read a file where the row separator is '|' lazily.

    Usage:

    >>> with open('big.csv') as f:
    >>>     for r in rows(f):
    >>>         process(r)
    """
    row = ''
    while (chunk := f.read(chunksize)) != '':   # End of file
        while (i := chunk.find(sep)) != -1:     # No separator found
            yield row + chunk[:i]
            chunk = chunk[i+1:]
            row = ''
        row += chunk
    yield row

Source: stackoverflow.com

Similar codes

0 vote270 views1 answer
Geodataframe get crs
Python
0 vote294 views1 answer
Python pass string with spaces as argument
Python
0 vote259 views2 answers
Random.randint(0 1)
Python
10 votes326 views5 answers
Zip python
Python
6 votes281 views4 answers
Gamestop
Python
3 votes263 views2 answers
Count how many vowels in a string python
Python
0 vote274 views2 answers
How to make a gui with kivy
Python
3 votes284 views2 answers
Date object into date format python
Python
0 vote277 views1 answer
Pandas snippets
Python
0 vote269 views1 answer
How to access item in list private in python
Python