primeqa.util.dataloader.distloader_seq_pair.SeqPairLoader#

class primeqa.util.dataloader.distloader_seq_pair.SeqPairLoader(hypers, per_gpu_batch_size: int, tokenizer, data_dir, *, files_per_dataloader=1, checkpoint_info=None, is_separate=False, is_single=False, json_mapper=<function standard_json_mapper>, teacher_labels=None)#

Bases: primeqa.util.dataloader.distloader_base.MultiFileLoader

Methods

all_batches

batch_dict

display_batch

get_checkpoint_info

get_dataloader

quick_test

reset