Shortcuts

OIDMeanAP

class mmeval.metrics.OIDMeanAP(iof_thrs: Union[float, List[float]] = 0.5, use_group_of: bool = True, get_supercategory: bool = True, filter_labels: bool = True, class_relation_matrix: Optional[numpy.ndarray] = None, **kwargs)[source]

Open Images Dataset detection evaluation metric.

The Open Images Dataset detection evaluation uses a variant of the standard PASCAL VOC 2010 mean Average Precision (mAP) at IoU > 0.5. There are some key features of Open Images annotations, which are addressed by the new metric.

For more see: https://storage.googleapis.com/openimages/web/evaluation.html

Parameters
  • iof_thrs (float | List[float]) – IoF thresholds. Defaults to 0.5.

  • use_group_of (bool) – Whether consider group of groud truth bboxes during evaluating. Defaults to True.

  • get_supercategory (bool, optional) – Whether to get parent class of the current class. Defaults to True.

  • filter_labels (bool, optional) – Whether filter unannotated classes. Defaults to True.

  • class_relation_matrix (numpy.ndarray, optional) – The matrix of the corresponding relationship between the parent class and the child class. If None, it will be obtained from the ‘relation_matrix’ field in self.dataset_meta. Defaults to None.

  • **kwargs – Keyword parameters passed to VOCMeanAP.

Examples

>>> import numpy as np
>>> from mmeval import OIDMeanAP
>>> num_classes = 4
>>> # use a fake relation_matrix
>>> relation_matrix = np.eye(num_classes, num_classes)
>>> oid_map = OIDMeanAP(
...     num_classes=4, class_relation_matrix=relation_matrix)
>>>
>>> def _gen_bboxes(num_bboxes, img_w=256, img_h=256):
...     # random generate bounding boxes in 'xyxy' formart.
...     x = np.random.rand(num_bboxes, ) * img_w
...     y = np.random.rand(num_bboxes, ) * img_h
...     w = np.random.rand(num_bboxes, ) * (img_w - x)
...     h = np.random.rand(num_bboxes, ) * (img_h - y)
...     return np.stack([x, y, x + w, y + h], axis=1)
>>>
>>> prediction = {
...     'bboxes': _gen_bboxes(10),
...     'scores': np.random.rand(10, ),
...     'labels': np.random.randint(0, num_classes, size=(10, ))
... }
>>> instances = []
>>> for bbox in _gen_bboxes(20):
...     instances.append({
...         'bbox': bbox.tolist(),
...         'bbox_label': random.randint(0, num_classes - 1),
...         'is_group_of': random.randint(0, 1) == 0,
...     })
>>> groundtruth = {
...     'instances': instances,
...     'image_level_labels': np.random.randint(0, num_classes, size=(10, )),  # noqa: E501
... }
>>> oid_map(predictions=[prediction, ], groundtruths=[groundtruth, ])  
{'AP@50': ..., 'mAP': ...}
add(predictions: Sequence[dict], groundtruths: Sequence[dict])None[source]

Add the intermediate results to self._results.

Parameters
  • predictions (Sequence[dict]) –

    A sequence of dict. Each dict representing a detection result for an image, with the following keys:

    • bboxes (numpy.ndarray): Shape (N, 4), the predicted bounding bboxes of this image, in ‘xyxy’ foramrt.

    • scores (numpy.ndarray): Shape (N, 1), the predicted scores of bounding boxes.

    • labels (numpy.ndarray): Shape (N, 1), the predicted labels of bounding boxes.

  • groundtruths (Sequence[dict]) –

    A sequence of dict. Each dict representing a groundtruths for an image, with the following keys:

    • instances (List[dict]): Each dict representing a bounding box with the following keys:

      • bbox (list): Containing box location in ‘xyxy’ foramrt.

      • bbox_label (int): Box label index.

      • is_group_of (bool): Whether the box is group or not.

    • image_level_labels (numpy.ndarray): The image level labels.

calculate_class_tpfp(predictions: List[dict], groundtruths: List[dict], class_index: int, pool: Optional[multiprocessing.pool.Pool])Tuple[source]

Calculate the tp and fp of the given class index.

This is an overridden method of VOCMeanAP.

Parameters
  • predictions (List[dict]) – A list of dict. Each dict is the detection result of an image. Same as VOCMeanAP.add.

  • groundtruths (List[dict]) – A list of dict. Each dict is the ground truth of an image. Same as VOCMeanAP.add.

  • class_index (int) – The class index.

  • pool (Pool, optional) – An instance of class:multiprocessing.Pool. If None, do not use multiprocessing.

Returns

  • tp (numpy.ndarray): Shape (num_ious, num_scales, num_pred), the true positive flag of each predict bbox.

  • fp (numpy.ndarray): Shape (num_ious, num_scales, num_pred), the false positive flag of each predict bbox.

  • num_gts (numpy.ndarray): Shape (num_ious, num_scales), the number of ground truths.

Return type

tuple (tp, fp, num_gts)

property class_relation_matrix: numpy.ndarray

Returns the class relation matrix.

The class relation matrix should be set during initialization, otherwise it will be obtained from the ‘relation_matrix’ field in self.dataset_meta.

Returns

The class relation matrix.

Return type

numpy.ndarray

Raises

RuntimeError – If the class relation matrix is not set.

get_class_gts(groundtruths: List[dict], class_index: int)Tuple[source]

Get prediciton gt information of a certain class index.

This is an overridden method of VOCMeanAP.

Parameters
  • groundtruths (list[dict]) – Same as self.add.

  • class_index (int) – Index of a specific class.

Returns

gt bboxes.

Return type

List[np.ndarray]

Read the Docs v: latest
Versions
latest
stable
Downloads
pdf
html
epub
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.