primeqa.mrc.metrics.nq_f1.nq_eval.get_metrics_as_dict

primeqa.mrc.metrics.nq_f1.nq_eval.get_metrics_as_dict(gold_path, prediction_path, num_threads=10)

Library version of the end-to-end evaluation.

Parameters

• gold_path – Path to the gzip JSON data. For multiple files, should be a glob pattern (e.g. “/path/to/files-*”)

• prediction_path – Path to the JSON prediction data.

• num_threads (10) – Number of threads to use when parsing multiple files.

Returns

A dictionary mapping string names to metric scores.

Return type

metrics