mAP (mean Average Precision) for Object Detection

mAP is the metric to measure the accuracy of object detectors. It is the average of the maximum precisions at different recall values.

Precision & recall

Precision measures how accurate is your predictions. i.e. the percentage of your positive predictions are correct. If you have a precision score of close to 1.0 then there is a high likelihood that whatever the classifier predicts as a positive detection is in fact a correct prediction.
Recall measures how good you find all the positives. For example, we can find 80% of the possible positive cases in our top K predictions. If you have a recall score close to 1.0 then almost all objects that are in your dataset will be positively detected by the model.
Here are their mathematical definitions:

IoU (Intersection over union)

IoU measures how much overlap between 2 regions, This measures how good is our prediction in the object detector with the ground truth.


AP (Average Precision)

AP calculation with precision-recall curve

Precision-Recall curve for an example detector.

The calculation if the AP score is to take the average value of the precision across all recall values (see explanation in section 4.2 of the Pascal Challenge paper).
From section 4.2 in paper

However the actual calculation is a little different.

precision-recall curve

We set the precision for recall r to the maximum precision obtained for any recall r' > r. Then we compute the AP as the area under this curve (shown in light blue) by numerical integration. No approximation is involved since the curve is piecewise constant.

Code for AP calculation

def compute_ap(recall, precision):
    """ Compute the average precision, given the recall and precision curves.
    Code originally from https://github.com/rbgirshick/py-faster-rcnn.

    # Arguments
        recall:    The recall curve (list).
        precision: The precision curve (list).
    # Returns
        The average precision as computed in py-faster-rcnn.
    """
    # correct AP calculation
    # first append sentinel values at the end
    mrec = np.concatenate(([0.], recall, [1.]))
    mpre = np.concatenate(([0.], precision, [0.]))

    # compute the precision envelope
    for i in range(mpre.size - 1, 0, -1):
        mpre[i - 1] = np.maximum(mpre[i - 1], mpre[i])

    # to calculate area under PR curve, look for points
    # where X axis (recall) changes value
    i = np.where(mrec[1:] != mrec[:-1])[0]

    # and sum (\Delta recall) * prec
    ap = np.sum((mrec[i + 1] - mrec[i]) * mpre[i + 1])
    return ap
import numpy as np

recall = [0.2,0.4,0.4,0.4,0.4,0.6,0.8,0.8,0.8,1.0]
precision = [1.0,1.0,0.67,0.5,0.4,0.5,0.57,0.5,0.44,0.5]
AP = compute_ap(recall, precision)
print('AP = {}'.format(AP))
# AP = 0.728
# If we calculate using 11 spaced recall values, 
# AP = (5 × 1.0 + 4 × 0.57 + 2 × 0.5)/11 = 0.753. 
# Check the this blog: mAP for Object Detection

# Intermediate results
def compute_ap(recall, precision):
    # correct AP calculation
    # first append sentinel values at the end
    mrec = np.concatenate(([0.], recall, [1.]))
    mpre = np.concatenate(([0.], precision, [0.]))
    # mrec = [0.  0.2 0.4 0.4 0.4 0.4 0.6 0.8 0.8 0.8 1.  1. ]
    # mpre = [0.   1.   1.   0.67 0.5  0.4  0.5  0.57 0.5  0.44 0.5  0.  ]
    
    # compute the precision envelope
    for i in range(mpre.size - 1, 0, -1):
        mpre[i - 1] = np.maximum(mpre[i - 1], mpre[i])
    # mpre = [1.   1.   1.   0.67 0.57 0.57 0.57 0.57 0.5  0.5  0.5  0.  ]

    # to calculate area under PR curve, look for points
    # where X axis (recall) changes value
    i = np.where(mrec[1:] != mrec[:-1])[0]
    # i = [0 1 5 6 9]
    # mrec[1:] = [0.2 0.4 0.4 0.4 0.4 0.6 0.8 0.8 0.8 1.  1. ]
    # mrec[:-1] = [0.  0.2 0.4 0.4 0.4 0.4 0.6 0.8 0.8 0.8 1. ]

    # and sum (\Delta recall) * prec
    ap = np.sum((mrec[i + 1] - mrec[i]) * mpre[i + 1])
    return ap     

Reference

mean Average Precision-github
mAP for Object Detection
Evaluation 12: mean average precision
Understanding the mAP Evaluation Metric for Object Detection
Discussion on fast.ai

最后编辑于
©著作权归作者所有,转载或内容合作请联系作者
平台声明:文章内容(如有图片或视频亦包括在内)由作者上传并发布,文章内容仅代表作者本人观点,简书系信息发布平台,仅提供信息存储服务。

推荐阅读更多精彩内容