primeqa.mrc.metrics.tydi_f1.tydi_eval.compute_pr_curves#

primeqa.mrc.metrics.tydi_f1.tydi_eval.compute_pr_curves(answer_stats, targets=None)#

Computes PR curve and returns R@P for specific targets.

The values are computed as follows: find the (precision, recall) point with maximum recall and where precision > target.

This is only relevant if you return the system scores in your predictions. You may find this useful when attempting to tune the threshold for your system on the dev set before requesting an evaluation on the test set via the leaderboard.

Parameters

  • answer_stats – List of statistic tuples from the answer scores.

  • targets (None) – List of precision thresholds to target.

Returns

[target, r, p, score].

Return type

List of table with rows