AVAMeanAP

class mmeval.metrics.AVAMeanAP(ann_file: str, label_file: str, exclude_file: Optional[str] = None, num_classes: int = 81, custom_classes: Optional[List[int]] = None, verbose: bool = True, **kwargs)

AVA evaluation metric.

AVA(Atomic Visual Action): https://research.google.com/ava.

This metric computes mAP using the ava evaluation toolkit provided by the author.

参数

• ann_file (str) – The annotation file path.

• label_file (str) – The label file path.

• exclude_file (str, optional) – The excluded timestamp file path. Defaults to None.

• num_classes (int) – Number of classes. Defaults to 81.

• custom_classes (list(int), optional) – A subset of class ids from origin dataset. Defaults to None.

• verbose (bool) – Whether to print messages in the evaluation process. Defaults to True.

• **kwargs – Keyword parameters passed to BaseMetric.

实际案例

>>> from mmeval import AVAMeanAP
>>> import numpy as np
>>>
>>> ann_file = 'tests/test_metrics/ava_detection_gt.csv'
>>> label_file = 'tests/test_metrics/ava_action_list.txt'
>>> num_classes = 4
>>> ava_metric = AVAMeanAP(ann_file=ann_file, label_file=label_file,
>>>                        num_classes=4)
>>>
>>> predictions = [
>>> {
>>>     'video_id': '3reY9zJKhqN',
>>>     'timestamp': 1774,
>>>     'outputs': [
>>>         np.array([[0.362, 0.156, 0.969, 0.666, 0.106],
>>>                   [0.442, 0.083, 0.721, 0.947, 0.162]]),
>>>         np.array([[0.288, 0.365, 0.766, 0.551, 0.706],
>>>                   [0.178, 0.296, 0.707, 0.995, 0.223]]),
>>>         np.array([[0.417, 0.167, 0.843, 0.939, 0.015],
>>>                   [0.35, 0.421, 0.57, 0.689, 0.427]]) ]
>>> },
>>> {
>>>     'video_id': 'HmR8SmNIoxu',
>>>     'timestamp': 1384,
>>>     'outputs': [
>>>         np.array([[0.256, 0.338, 0.726, 0.799, 0.563],
>>>                   [0.071, 0.256, 0.64, 0.75, 0.297]]),
>>>         np.array([[0.326, 0.036, 0.513, 0.991, 0.405],
>>>                   [0.351, 0.035, 0.729, 0.936, 0.945]]),
>>>         np.array([[0.051, 0.005, 0.975, 0.942, 0.424],
>>>                   [0.347, 0.05, 0.97, 0.944, 0.396]])]
>>> },
>>> {
>>>     'video_id': '5HNXoce1raG',
>>>     'timestamp': 1097,
>>>     'outputs': [
>>>         np.array([[0.39, 0.087, 0.833, 0.616, 0.447],
>>>                   [0.461, 0.212, 0.627, 0.527, 0.036]]),
>>>         np.array([[0.022, 0.394, 0.93, 0.527, 0.109],
>>>                   [0.208, 0.462, 0.874, 0.948, 0.954]]),
>>>         np.array([[0.206, 0.456, 0.564, 0.725, 0.685],
>>>                   [0.106, 0.445, 0.782, 0.673, 0.367]])]}
>>> ]
>>> ava_metric(predictions)
{'mAP@0.5IOU': 0.027777778}

add(predictions: Sequence[dict]) → None

Add detection results to the results list.

参数

predictions (Sequence[dict]) –

A list of prediction dict which contains the following keys:

• video_id: The id of the video, e.g., 3reY9zJKhqN.

• timestamp: The timestamp of the video e.g., 1774.

• outputs: A list bbox results of each class with the format of [x1, y1, x2, y2, score].

ava_eval(result_file: str) → dict

Perform ava evaluation.

参数

result_file (str) – The dumped results file path.

返回

The evaluation results.

返回类型

dict

compute_metric(results: list) → dict

Compute the AVA MeanAP.

参数

results (list) – A list of detection results.

返回

The computed ava metric.

返回类型

dict

read_csv(csv_file: str, class_whitelist: Optional[set] = None) → tuple

Loads boxes and class labels from a CSV file in the AVA format.

CSV file format described at https://research.google.com/ava/download.html.

参数

• csv_file (str) – A csv file path.

• class_whitelist (set, optional) – If provided, boxes corresponding to (integer) class labels not in this set are skipped.

返回

• boxes (dict): A dictionary mapping each unique image key (string) to a list of boxes, given as coordinates [y1, x1, y2, x2].

• labels (dict): A dictionary mapping each unique image key (string) to a list of integer class labels, matching the corresponding box in boxes.

• scores (dict): A dictionary mapping each unique image key (string) to a list of score values labels, matching the corresponding label in labels. If scores are not provided in the csv, then they will default to 1.0.

返回类型

tuple (boxes, labels, scores)

read_exclusions(exclude_file: str) → set

Reads a CSV file of excluded timestamps.

参数

exclude_file (str) – The path of exclude file.

返回

A set of strings containing excluded image keys, e.g. “aaaaaaaaaaa,0904” or an empty set if exclusions file is None.

返回类型

excluded (set)

read_label(label_file: str) → tuple

Reads a label mapping file.

参数

label_file (str) – The path of label file.

返回

• labelmap (list): The label map in the form used by the object_detection_evaluation module - a list of {“id”: integer, “name”: classname } dicts.

• class_ids (set): A set containing all of the valid class id integers.

返回类型

tuple (labelmap, class_ids)

results2csv(results: List[dict], out_file: str) → None

Dump the results to a csv file.

参数

• results (list[dict]) – A list of detection results.

• out_file (str) – The output csv file path.