primeqa.mrc.metrics.tydi_f1.tydi_eval.score_answers#

primeqa.mrc.metrics.tydi_f1.tydi_eval.score_answers(gold_annotation_dict, pred_dict, passage_non_null_threshold, span_non_null_threshold, verbose, skip_missing_example_ids=True, minimal_offsets_per_passage=False)#

Scores all answers for all documents.

Parameters

  • gold_annotation_dict – a dict from example id to list of `TyDiLabel`s.

  • pred_dict – a dict from example id to list of `TyDiLabel`s.

  • passage_non_null_threshold –

  • span_non_null_threshold – minimal number of non-null annotations per example to be considered non-null

  • verbose – whether to enable verbose logging

  • skip_missing_example_ids – skip missing examples

  • minimal_offsets_per_passage – whether minimal answer offsets are per passage (as opposed to per document)

Returns

List of scores for passage answers. minimal_answer_stats: List of scores for minimal answers.

Return type

passage_answer_stats