primeqa.mrc.metrics.tydi_f1.tydi_eval.score_minimal_answer#

primeqa.mrc.metrics.tydi_f1.tydi_eval.score_minimal_answer(gold_label_list, pred_label, span_non_null_threshold)#

Scores a minimal answer.

Outputs score against gold label that gives max F1.

First decide if there is a gold minimal answer with FLAGS.span_non_null_threshold. If any of the gold label has “yes”, or “no”, and pred label predicted it correctly, than precision, recall, f1 is all 1.0.

Parameters

  • gold_label_list – A list of TyDiLabel.

  • pred_label – A single TyDiLabel.

  • span_non_null_threshold – See FLAGS.span_non_null_threshold.

Returns

gold_has_answer, pred_has_answer, (precision, recall, f1), score