Chunk size in python large size

def rows(f, chunksize=1024, sep='|'):
    """
    Read a file where the row separator is '|' lazily.

    Usage:

    >>> with open('big.csv') as f:
    >>>     for r in rows(f):
    >>>         process(r)
    """
    row = ''
    while (chunk := f.read(chunksize)) != '':   # End of file
        while (i := chunk.find(sep)) != -1:     # No separator found
            yield row + chunk[:i]
            chunk = chunk[i+1:]
            row = ''
        row += chunk
    yield row

Source: stackoverflow.com

Similar codes

2 votes234 views2 answers
Tkinter mainloop
Python
3 votes233 views2 answers
Pandas remove time from date
Python
0 vote225 views1 answer
Pyspark pivot max aggregation
Python
0 vote224 views1 answer
Can Selenium python Web driver helps to extract data from DB
Python
0 vote217 views1 answer
Python remove table widget numbers
Python
0 vote224 views1 answer
Python missing characters hackerrank solution
Python
0 vote218 views1 answer
Compare two data frames in assert
Python
0 vote233 views1 answer
Python-Generating numbers according to a corellation matrix
Python
7 votes219 views5 answers
Python get parent directory
Python
0 vote226 views1 answer
Django python base 64 encode
Python