primeqa.mrc.metrics.nq_f1.nq_eval.score_long_answer#

primeqa.mrc.metrics.nq_f1.nq_eval.score_long_answer(gold_label_list, pred_label, long_non_null_threshold=None)#

Scores a long answer as correct or not.

  1. First decide if there is a gold long answer with LONG_NO_NULL_THRESHOLD.

  2. The prediction will get a match if: a. There is a gold long answer. b. The prediction span match exactly with one of the non-null gold

    long answer span.

Parameters

  • gold_label_list – A list of NQLabel, could be None.

  • pred_label – A single NQLabel, could be None.

  • long_non_null_threshold – Min number of non null spans in the annotations to consider the question as having a non null answer

Returns

gold_has_answer, pred_has_answer, is_correct, score