COCODetection¶
- class mmeval.metrics.COCODetection(ann_file: Optional[str] = None, metric: Union[str, List[str]] = 'bbox', iou_thrs: Optional[Union[float, Sequence[float]]] = None, classwise: bool = False, proposal_nums: Sequence[int] = (1, 10, 100), metric_items: Optional[Sequence[str]] = None, format_only: bool = False, outfile_prefix: Optional[str] = None, gt_mask_area: bool = True, backend_args: Optional[dict] = None, print_results: bool = True, **kwargs)[源代码]¶
COCO object detection task evaluation metric.
Evaluate AR, AP, and mAP for detection tasks including proposal/box detection and instance segmentation. Please refer to https://cocodataset.org/#detection-eval for more details.
- 参数
ann_file (str, optional) – Path to the coco format annotation file. If not specified, ground truth annotations from the dataset will be converted to coco format. Defaults to None.
metric (str | List[str]) – Metrics to be evaluated. Valid metrics include ‘bbox’, ‘segm’, and ‘proposal’. Defaults to ‘bbox’.
iou_thrs (float | List[float], optional) – IoU threshold to compute AP and AR. If not specified, IoUs from 0.5 to 0.95 will be used. Defaults to None.
classwise (bool) – Whether to return the computed results of each class. Defaults to False.
proposal_nums (Sequence[int]) – Numbers of proposals to be evaluated. Defaults to (1, 10, 100).
metric_items (List[str], optional) – Metric result names to be recorded in the evaluation result. Defaults to None.
format_only (bool) – Format the output results without perform evaluation. It is useful when you want to format the result to a specific format and submit it to the test server. Defaults to False.
outfile_prefix (str, optional) – The prefix of json files. It includes the file path and the prefix of filename, e.g., “a/b/prefix”. If not specified, a temp file will be created. Defaults to None.
gt_mask_area (bool) – Whether to calculate GT mask area when not loading ann_file. If True, the GT instance area will be the mask area, else the bounding box area. It will not be used when loading ann_file. Defaults to True.
backend_args (dict, optional) – Arguments to instantiate the preifx of uri corresponding backend. Defaults to None.
print_results (bool) – Whether to print the results. Defaults to True.
logger (Logger, optional) – logger used to record messages. When set to
None
, the default logger will be used. Defaults to None.**kwargs – Keyword parameters passed to
BaseMetric
.
实际案例
>>> import numpy as np >>> from mmeval import COCODetection >>> try: >>> from mmeval.metrics.utils.coco_wrapper import mask_util >>> except ImportError as e: >>> mask_util = None >>> >>> num_classes = 4 >>> fake_dataset_metas = { ... 'classes': tuple([str(i) for i in range(num_classes)]) ... } >>> >>> coco_det_metric = COCODetection( ... dataset_meta=fake_dataset_metas, ... metric=['bbox', 'segm'] ... ) >>> def _gen_bboxes(num_bboxes, img_w=256, img_h=256): ... # random generate bounding boxes in 'xyxy' formart. ... x = np.random.rand(num_bboxes, ) * img_w ... y = np.random.rand(num_bboxes, ) * img_h ... w = np.random.rand(num_bboxes, ) * (img_w - x) ... h = np.random.rand(num_bboxes, ) * (img_h - y) ... return np.stack([x, y, x + w, y + h], axis=1) >>> >>> def _gen_masks(bboxes, img_w=256, img_h=256): ... if mask_util is None: ... raise ImportError( ... 'Please try to install official pycocotools by ' ... '"pip install pycocotools"') ... masks = [] ... for i, bbox in enumerate(bboxes): ... mask = np.zeros((img_h, img_w)) ... bbox = bbox.astype(np.int32) ... box_mask = (np.random.rand( ... bbox[3] - bbox[1], ... bbox[2] - bbox[0]) > 0.3).astype(np.int32) ... mask[bbox[1]:bbox[3], bbox[0]:bbox[2]] = box_mask ... masks.append( ... mask_util.encode( ... np.array(mask[:, :, np.newaxis], order='F', ... dtype='uint8'))[0]) # encoded with RLE ... return masks >>> >>> img_id = 1 >>> img_w, img_h = 256, 256 >>> num_bboxes = 10 >>> pred_boxes = _gen_bboxes( ... num_bboxes=num_bboxes, ... img_w=img_w, ... img_h=img_h) >>> pred_masks = _gen_masks( ... bboxes=pred_boxes, ... img_w=img_w, ... img_h=img_h) >>> prediction = { ... 'img_id': img_id, ... 'bboxes': pred_boxes, ... 'scores': np.random.rand(num_bboxes, ), ... 'labels': np.random.randint(0, num_classes, size=(num_bboxes, )), ... 'masks': pred_masks ... } >>> gt_boxes = _gen_bboxes( ... num_bboxes=num_bboxes, ... img_w=img_w, ... img_h=img_h) >>> gt_masks = _gen_masks( ... bboxes=pred_boxes, ... img_w=img_w, ... img_h=img_h) >>> groundtruth = { ... 'img_id': img_id, ... 'width': img_w, ... 'height': img_h, ... 'bboxes': gt_boxes, ... 'labels': np.random.randint(0, num_classes, size=(num_bboxes, )), ... 'masks': gt_masks, ... 'ignore_flags': np.zeros(num_bboxes) ... } >>> coco_det_metric(predictions=[prediction, ], groundtruths=[groundtruth, ]) {'bbox_mAP': ..., 'bbox_mAP_50': ..., ..., 'segm_mAP': ..., 'segm_mAP_50': ..., ..., 'bbox_result': ..., 'segm_result': ..., ...}
- add(predictions: Sequence[Dict], groundtruths: Sequence[Dict]) → None[源代码]¶
Add the intermediate results to self._results.
- 参数
predictions (Sequence[dict]) –
A sequence of dict. Each dict representing a detection result for an image, with the following keys:
img_id (int): Image id.
bboxes (numpy.ndarray): Shape (N, 4), the predicted bounding bboxes of this image, in ‘xyxy’ foramrt.
scores (numpy.ndarray): Shape (N, ), the predicted scores of bounding boxes.
labels (numpy.ndarray): Shape (N, ), the predicted labels of bounding boxes.
masks (list[RLE], optional): The predicted masks.
mask_scores (np.array, optional): Shape (N, ), the predicted scores of masks.
groundtruths (Sequence[dict]) –
A sequence of dict. If load from ann_file, the dict inside can be empty. Else, each dict represents a groundtruths for an image, with the following keys:
img_id (int): Image id.
width (int): The width of the image.
height (int): The height of the image.
bboxes (numpy.ndarray): Shape (K, 4), the ground truth bounding bboxes of this image, in ‘xyxy’ foramrt.
labels (numpy.ndarray): Shape (K, ), the ground truth labels of bounding boxes.
masks (list[RLE], optional): The predicted masks.
ignore_flags (numpy.ndarray, optional): Shape (K, ), the ignore flags.
- add_predictions(predictions: Sequence[Dict]) → None[源代码]¶
Add predictions only.
If the ann_file has been passed, we can add predictions only.
- 参数
predictions (Sequence[dict]) – Refer to
COCODetection.add
.
- property classes: list¶
Get classes from self.dataset_meta.
- compute_metric(results: list) → dict[源代码]¶
Compute the COCO metrics.
- 参数
results (List[tuple]) – A list of tuple. Each tuple is the prediction and ground truth of an image. This list has already been synced across all ranks.
- 返回
The computed metric. The keys are the names of the metrics, and the values are corresponding results.
- 返回类型
dict
- gt_to_coco_json(gt_dicts: Sequence[dict], outfile_prefix: str) → str[源代码]¶
Convert ground truth to coco format json file.
- 参数
gt_dicts (Sequence[dict]) – Ground truth of the dataset.
outfile_prefix (str) – The filename prefix of the json files. If the prefix is “somepath/xxx”, the json file will be named “somepath/xxx.gt.json”.
- 返回
The filename of the json file.
- 返回类型
str
- results2json(results: Sequence[dict], outfile_prefix: str) → dict[源代码]¶
Dump the detection results to a COCO style json file.
There are 3 types of results: proposals, bbox predictions, mask predictions, and they have different data types. This method will automatically recognize the type, and dump them to json files.
- 参数
results (Sequence[dict]) – Testing results of the dataset.
outfile_prefix (str) – The filename prefix of the json files. If the prefix is “somepath/xxx”, the json files will be named “somepath/xxx.bbox.json”, “somepath/xxx.segm.json”, “somepath/xxx.proposal.json”.
- 返回
Possible keys are “bbox”, “segm”, “proposal”, and values are corresponding filenames.
- 返回类型
dict