primeqa.ir.dense.colbert_top.colbert.modeling.tokenization.doc_tokenization_xlmr.DocTokenizerXLMR#

class primeqa.ir.dense.colbert_top.colbert.modeling.tokenization.doc_tokenization_xlmr.DocTokenizerXLMR(doc_maxlen, model_type)#

Bases: object

Methods

encode

tensorize

tokenize