primeqa.util.file_utils.block_shuffle

primeqa.util.file_utils.block_shuffle(iter, *, block_size=20000, rand=<module 'random' from '/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/random.py'>)

shuffle the possibly endless iterator by blocks Good shuffling over multiple files: block_shuffle(read_lines(files, shuffled_files=rand), rand=rand, block_size=100000) :param iter: the iterator we will yield shuffled items from :param block_size: size of memory to use for block shuffling :param rand: rand.shuffle will be used on the list block :return: