primeqa.mrc.processors.postprocessors.natural_questions.NaturalQuestionsPostProcessor#

class primeqa.mrc.processors.postprocessors.natural_questions.NaturalQuestionsPostProcessor(*args, **kwargs)#

Bases: primeqa.mrc.processors.postprocessors.extractive.ExtractivePostProcessor

Post processor for NQ.

Methods

`prepare_examples_as_references`	Prepare reference for each example.
`process`	Adjust answer start/end positions to original document html.
`process_references_and_predictions`	Convert data and model predictions into MRC answers and references for use in metrics.

prepare_examples_as_references(examples: datasets.arrow_dataset.Dataset) → List[Dict[str, Any]]#

Prepare reference for each example. document_plaintext and question are not included to comply with the definition of NQLabel in primeqa.mrc.metrics.nq_f1.eval_utils. :param examples: Dataset examples generated by process_train and process_eval of preprocessoor.

Returns: List of answer labels in the format of NQLabel.

process(examples: datasets.arrow_dataset.Dataset, features: datasets.arrow_dataset.Dataset, predictions: Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray])#

Adjust answer start/end positions to original document html. The start/end positions return from super().process() point to the context of document tokens. :param examples: Dataset examples generated by process_train and process_eval of preprocessoor. :param features: Features generated by process_train and process_eval of preprocessoor. :param predictions: Prediction output generated by task head.

Returns: Predictions with adjusted answer offset.

process_references_and_predictions(examples, features, predictions) → primeqa.mrc.data_models.eval_prediction_with_processing.EvalPredictionWithProcessing#: Convert data and model predictions into MRC answers and references for use in metrics.