primeqa.qg.processors.passage_qg.qg_processor.QGProcessor

class primeqa.qg.processors.passage_qg.qg_processor.QGProcessor(tokenizer: transformers.tokenization_utils.PreTrainedTokenizer, input_max_len: int, target_max_len: int)

Bases: object

Class for qg processing, contains methods to preprocess data execute sql etc.

Methods

convert_to_features
preprocess_data

Attributes

tokenizer
input_max_len
target_max_len

__call__(dataset) → datasets.arrow_dataset.Dataset

Call self as a function.