primeqa.mrc.metrics.tydi_f1.tydi_eval.score_passage_answer#

primeqa.mrc.metrics.tydi_f1.tydi_eval.score_passage_answer(gold_label_list, pred_label, passage_non_null_threshold)#

Scores a passage answer as correct or not.

  1. First decide if there is a gold passage answer with FLAGS.passage_non_null_threshold.

  2. The prediction will get a match if: a. There is a gold passage answer. b. The prediction span match exactly with one of the non-null gold

    passage answer index.

Parameters

  • gold_label_list – A list of TyDiLabel, could be None.

  • pred_label – A single TyDiLabel, could be None.

  • passage_non_null_threshold – See FLAGS.passage_non_null_threshold.

Returns

gold_has_answer, pred_has_answer, is_correct, score