primeqa.mrc.metrics.nq_f1.eval_utils.gold_has_short_answer

primeqa.mrc.metrics.nq_f1.eval_utils.gold_has_short_answer(gold_label_list: list, short_non_null_threshold: int = 2) → bool

Gets vote from multi-annotators for judging if there is a short answer. :param gold_label_list: list of gold labels :param short_non_null_threshold: Require this many non-null short answer annotations

to count gold as containing a short answer. Defaults to 2 like the original paper.