NMS是one-stage和two-stage目标检测任务中常用的一种后处理方法，用来过滤无效重叠的检测框。

NMS

NMS全称非极大值抑制，出自ICPR2006的论文《Efficient Non-Maximum Suppression》。其基本思想很简单，就是保留局部最大值而去除局部非最大值。

NMS对所有的类别的检测框进行循环过滤。对于某个类别C，首先对这些矩形框按照概率降序排列，选中概率最大的框作为候选框，对于剩下的框，依次与候选框求IOU，如果IOU大于某个阈值（超参），则将这些框丢弃（置0），并标记保留最大概率框。

以此类推，最终所有的框相互之间的IOU都是小于超参阈值的，或者概率被置为0了。剩下的所有概率非0的框就是最终的检测框。

基于这种计算逻辑的NMS有两个缺点。首先，NMS算法需要一个超参即IOU Threshold，这个阈值在不同任务中很难平衡。其次，NMS会将相邻或者重叠的两个物体对应的两个大概率目标框去掉一个，造成漏检。

实现：

def nms(dets, threshold):
    x1 = dets[:, 0]
    y1 = dets[:, 1]
    x2 = dets[:, 2]
    y2 = dets[:, 3]

    areas = (y2 - y1 + 1) * (x2 - x1 + 1)
    scores = dets[:, 4]
    keep = []
    index = scores.argsort()[::-1]

    while index.size > 0:
        i = index[0]       # every time the first is the biggst, and add it directly
        keep.append(i)

        x11 = np.maximum(x1[i], x1[index[1:]])    # calculate the points of overlap 
        y11 = np.maximum(y1[i], y1[index[1:]])
        x22 = np.minimum(x2[i], x2[index[1:]])
        y22 = np.minimum(y2[i], y2[index[1:]])
        
        w = np.maximum(0, x22 - x11 + 1)    # the weights of overlap
        h = np.maximum(0, y22 - y11 + 1)    # the height of overlap
       
        overlaps = w * h
        ious = overlaps / (areas[i] + areas[index[1:]] - overlaps)
 
        idx = np.where(ious <= threshold)[0]
        index = index[idx + 1]   # because index start from 1
 
    return keep

Soft-NMS

Soft-NMS出自CVPR2017的论文《Improving Object Detection With One Line of Code》，对NMS做了一些改进。

Soft-NMS总体算法流程同NMS相同，主要差别循环过程中对阈值的判断部分。NMS是简单的对IOU大于阈值的检测框进行删除出来，而Soft-NMS则是通过权重来降低检测框原有的置信度。对于有重叠的框，重叠区域越大，置信度衰减越严重。

soft-nms

Soft-NMS计算降低置信度的权重常用两种方法：线性法和高斯法。

线性法：

liner

高斯法：

Gaussian

实现如下，方法1为线性法，方法2为高斯法，其他参数的话Soft-NMS退化为NMS：

def soft_nms(dets, sigma=0.5, threshold1=0.7, threshold2=0.1, method=1):
    n = dets.shape[0]

    x1 = dets[:, 0]
    y1 = dets[:, 1]
    x2 = dets[:, 2]
    y2 = dets[:, 3]
    scores = dets[:, 4]
    areas = (y2 - y1 + 1) * (x2 - x1 + 1)

    new_scores = scores.copy()
    index = [i for i in range(n)]
    keep = []

    while len(index) > 0:
        # get max box position of current based new scores
        max_score = 0
        max_pos = -1

        for i in index:
            if new_scores[i] >= max_score:
                max_pos = i
                max_score = new_scores[i]

        if max_pos == -1:
            break

        keep.append(max_pos)
        index.remove(max_pos)

        # calculate ious between current max box and others
        x11 = np.maximum(x1[max_pos], x1[index])
        y11 = np.maximum(y1[max_pos], y1[index])
        x22 = np.minimum(x2[max_pos], x2[index])
        y22 = np.minimum(y2[max_pos], y2[index])

        w = np.maximum(0, x22 - x11 + 1)
        h = np.maximum(0, y22 - y11 + 1)

        overlaps = w * h
        ious = overlaps / (areas[max_pos] + areas[index] - overlaps)

        # adjust score of others
        new_index = []
        for i, ids in enumerate(index):
            iou = ious[i]
            weight = 1
 
            if method == 1:
                # linear
                if iou >= threshold1:
                    weight = 1 - iou
            elif method == 2:
                # gaussian
                weight = np.exp(-(iou * iou) / sigma)
            else:
                # normal nms
                if iou >= threshold1:
                    weight = 0

            new_scores[ids] = new_scores[ids] * weight

            if new_scores[ids] > threshold2:
                new_index.append(ids)
        index = new_index

    return keep

实验

模拟的5个候选框：

box

if __name__ == '__main__':
    boxes = np.array([[100, 100, 210, 210, 0.72],
                  [250, 250, 420, 420, 0.8],
                  [220, 220, 320, 330, 0.92],
                  [100, 100, 210, 210, 0.72],
                  [230, 240, 325, 330, 0.81],
                  [220, 230, 315, 340, 0.9]])  # (x1,y1,x2,y2,score)

    keep = nms(boxes, threshold=0.7)
    print(keep)
 
    keep = soft_nms(boxes, threshold1=0.7, threshold2=0.2, method=0)
    print(keep)

    keep = soft_nms(boxes, threshold1=0.7, threshold2=0.2, method=1)
    print(keep)

    keep = soft_nms(boxes, threshold1=0.7, threshold2=0.2, method=2)
    print(keep)

[2, 1, 3]
[2, 1, 3]
[2, 1, 3, 4]
[2, 1, 3, 4]

实验可以看出，在相同IOU阈值的情况下，Soft-NMS相比NMS保留了一个检测结果，这在同类别物体重叠的情况下能够提升其召回率。但是Soft-NMS又多引入了一个超参，这个参数的设置也会显著影响后处理的结果；而且，由于Soft-NMS在每次迭代都会修改score值，其最大值是在动态变化的需要在每次迭代都寻找一次，因此Soft-NMS相比NMS计算效率有所降低。