MAP

class composer.metrics.MAP(class_metrics=False, compute_on_step=True, dist_sync_on_step=False, process_group=None, dist_sync_fn=None)

Computes the Mean-Average-Precision (mAP) and Mean-Average-Recall (mAR) for object detection predictions.

Optionally, the mAP and mAR values can be calculated per class. Predicted boxes and targets have to be in Pascal VOC format (xmin-top left, ymin-top left, xmax-bottom right, ymax-bottom right). See the update() method for more information about the input format to this metric. See this blog for more details on (mAP) and (mAR).

Warning

This metric is a wrapper for the pycocotools, which is a standard implementation for the mAP metric for object detection. Using this metric therefore requires you to have pycocotools installed. Please install with pip install pycocotools

Warning

As the pycocotools library cannot deal with tensors directly, all results have to be transfered to the CPU, this may have an performance impact on your training.

Parameters

• class_metrics (bool, optional) – Option to enable per-class metrics for mAP and mAR_100. Has a performance impact. Default: False.

• compute_on_step (bool, optional) – Forward only calls update() and return None if this is set to False. Default: False.

• dist_sync_on_step (bool, optional) – Synchronize metric state across processes at each forward() before returning the value at the step. Default: False.

• process_group (any, optional) – Specify the process group on which synchronization is called. Default: None (which selects the entire world).

• dist_sync_fn (callable, optional) – Callback that performs the allgather operation on the metric state. When None, DDP will be used to perform the all_gather. Default: None.

Raises

ValueError – If class_metrics is not a boolean.

compute()

Compute the Mean-Average-Precision (mAP) and Mean-Average-Recall (mAR) scores.

All detections added in the update() method are included.

Note

Main map score is calculated with @[ IoU=0.50:0.95 | area=all | maxDets=100 ]

Returns

MAPMetricResults (dict) – containing:

map (torch.Tensor): map at 95 iou.

map_50 (torch.Tensor): map at 50 iou.

map_75 (torch.Tensor): map at 75 iou.

map_small (torch.Tensor): map at 95 iou for small objects.

map_medium (torch.Tensor): map at 95 iou for medium objects.

map_large (torch.Tensor): map at 95 iou for large objects.

mar_1 (torch.Tensor): mar at 1 max detection.

mar_10 (torch.Tensor): mar at 10 max detections.

mar_100 (torch.Tensor): mar at 100 max detections.

mar_small (torch.Tensor): mar at 100 max detections for small objects.

mar_medium (torch.Tensor): mar at 100 max detections for medium objects.

mar_large (torch.Tensor): mar at 100 max detections for large objects.

map_per_class (torch.Tensor) (-1 if class metrics are disabled): map value for each class.

mar_100_per_class (torch.Tensor) (-1 if class metrics are disabled): mar at 100 detections for each class.

update(preds, target)

Add detections and groundtruth to the metric.

Parameters

• preds (list[Dict[str, Tensor]]) –

  A list of dictionaries containing the key-values:

  boxes (torch.FloatTensor): [num_boxes, 4] predicted boxes of the format [xmin, ymin, xmax, ymax] in absolute image coordinates.

  scores (torch.FloatTensor): of shape [num_boxes] containing detection scores for the boxes.

  labels (torch.IntTensor): of shape [num_boxes] containing 0-indexed detection classes for the boxes.

• target (list[Dict[str, Tensor]]) –

  A list of dictionaries containing the key-values:

  boxes (torch.FloatTensor): [num_boxes, 4] ground truth boxes of the format [xmin, ymin, xmax, ymax] in absolute image coordinates.

  labels (torch.IntTensor): of shape [num_boxes] containing 1-indexed groundtruth classes for the boxes.

Raises

• ValueError – If preds and target are not of the same length.

• ValueError – If any of preds.boxes, preds.scores and preds.labels are not of the same length.

• ValueError – If any of target.boxes and target.labels are not of the same length.

• ValueError – If any box is not type float and of length 4.

• ValueError – If any class is not type int and of length 1.

• ValueError – If any score is not type float and of length 1.