primeqa.util.dataloader.distloader_base.sentence_to_inputs

primeqa.util.dataloader.distloader_base.sentence_to_inputs(sentence: str, tokenizer: transformers.tokenization_utils.PreTrainedTokenizer, max_seq_length: int, start_end_tok_ids=None)

Parameters

• sentence –

• tokenizer –

• max_seq_length – we don’t pad to this length, only truncate

• start_end_tok_ids – if not None, then start_id, end_id = start_end_tok_ids

Returns