前面一篇Keras 基础学习的文章说的是多分类问题，稍微提及了如果一个样本属于多个类，比如在CelebA数据集中一张人脸照片是椭圆形的，戴眼镜的... 这个时候样本的label标注是multi-hot-encoding的，也就是会出现多个标签1。如[0,1,1,0]， 1代表属性出现，0代表属性不出现。当然其实这种问题也可以分解为多个二分类/多分类问题，比如人脸形状训练一个模型，有无戴眼镜训练一个模型，头发颜色训练一个模型，然后使用几个模型一起预测。缺点的话就是成本太高，一个模型可以进行多标签分类就容易多了。这里面最典型的问题应该是就CelebA数据集衍生出的一系列模型，电影类型分类等。

这篇文章我们会简单介绍一个多标签分类问题，便于大家理解。
我们使用Adrian Rosebrock 提供的衣服数据集, 这篇文章也是在他的基础上做了点修改。

先看下数据:

keras_multi_label_dataset.jpg

整个数据集有两种属性，一种是颜色(blue, red, black)，另一种是衣服的类型(dress, jeans, shirt) 。说明我们对每个衣服的label应该是长度为6的vector，其中两个值为1,其它为0。如假设one-hot-vector编码顺序是(blue, red, black, dress, jeans, shirt)则black jeans的 label就是[0,0,1,0,1,0]。

网络结构

Adrian Rosebrock 使用的网络结构是smallvggnet，结构如下:

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_1 (Conv2D)            (None, 96, 96, 32)        896       
_________________________________________________________________
activation_1 (Activation)    (None, 96, 96, 32)        0         
_________________________________________________________________
batch_normalization_1 (Batch (None, 96, 96, 32)        128       
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 32, 32, 32)        0         
_________________________________________________________________
dropout_1 (Dropout)          (None, 32, 32, 32)        0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 32, 32, 64)        18496     
_________________________________________________________________
activation_2 (Activation)    (None, 32, 32, 64)        0         
_________________________________________________________________
batch_normalization_2 (Batch (None, 32, 32, 64)        256       
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 32, 32, 64)        36928     
_________________________________________________________________
activation_3 (Activation)    (None, 32, 32, 64)        0         
_________________________________________________________________
batch_normalization_3 (Batch (None, 32, 32, 64)        256       
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 16, 16, 64)        0         
_________________________________________________________________
dropout_2 (Dropout)          (None, 16, 16, 64)        0         
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 16, 16, 128)       73856     
_________________________________________________________________
activation_4 (Activation)    (None, 16, 16, 128)       0         
_________________________________________________________________
batch_normalization_4 (Batch (None, 16, 16, 128)       512       
_________________________________________________________________
conv2d_5 (Conv2D)            (None, 16, 16, 128)       147584    
_________________________________________________________________
activation_5 (Activation)    (None, 16, 16, 128)       0         
_________________________________________________________________
batch_normalization_5 (Batch (None, 16, 16, 128)       512       
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 8, 8, 128)         0         
_________________________________________________________________
dropout_3 (Dropout)          (None, 8, 8, 128)         0         
_________________________________________________________________
flatten_1 (Flatten)          (None, 8192)              0         
_________________________________________________________________
dense_1 (Dense)              (None, 1024)              8389632   
_________________________________________________________________
activation_6 (Activation)    (None, 1024)              0         
_________________________________________________________________
batch_normalization_6 (Batch (None, 1024)              4096      
_________________________________________________________________
dropout_4 (Dropout)          (None, 1024)              0         
_________________________________________________________________
dense_2 (Dense)              (None, 6)                 6150      
_________________________________________________________________
activation_7 (Activation)    (None, 6)                 0         
=================================================================
Total params: 8,679,302
Trainable params: 8,676,422
Non-trainable params: 2,880
_________________________________________________________________

实际上Adrian Rosebrock使用的是传统的CNN结构，最后几层是全连接层，参数很多。整个模型训练下来有100M。但是实际上现在轻量网络都是将全连接层改为pooling，减少参数。下面给出一个我改进的网络结构，增加两层卷积层和GlobalAveraePooling2D层，训练完只有14M，accuray和原来的网络差不多，甚至更好：

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_1 (Conv2D)            (None, 96, 96, 32)        896       
_________________________________________________________________
activation_1 (Activation)    (None, 96, 96, 32)        0         
_________________________________________________________________
batch_normalization_1 (Batch (None, 96, 96, 32)        128       
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 32, 32, 32)        0         
_________________________________________________________________
dropout_1 (Dropout)          (None, 32, 32, 32)        0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 32, 32, 64)        18496     
_________________________________________________________________
activation_2 (Activation)    (None, 32, 32, 64)        0         
_________________________________________________________________
batch_normalization_2 (Batch (None, 32, 32, 64)        256       
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 32, 32, 64)        36928     
_________________________________________________________________
activation_3 (Activation)    (None, 32, 32, 64)        0         
_________________________________________________________________
batch_normalization_3 (Batch (None, 32, 32, 64)        256       
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 16, 16, 64)        0         
_________________________________________________________________
dropout_2 (Dropout)          (None, 16, 16, 64)        0         
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 16, 16, 128)       73856     
_________________________________________________________________
activation_4 (Activation)    (None, 16, 16, 128)       0         
_________________________________________________________________
batch_normalization_4 (Batch (None, 16, 16, 128)       512       
_________________________________________________________________
conv2d_5 (Conv2D)            (None, 16, 16, 128)       147584    
_________________________________________________________________
activation_5 (Activation)    (None, 16, 16, 128)       0         
_________________________________________________________________
batch_normalization_5 (Batch (None, 16, 16, 128)       512       
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 8, 8, 128)         0         
_________________________________________________________________
dropout_3 (Dropout)          (None, 8, 8, 128)         0         
_________________________________________________________________
conv2d_6 (Conv2D)            (None, 8, 8, 256)         295168    
_________________________________________________________________
activation_6 (Activation)    (None, 8, 8, 256)         0         
_________________________________________________________________
batch_normalization_6 (Batch (None, 8, 8, 256)         1024      
_________________________________________________________________
conv2d_7 (Conv2D)            (None, 8, 8, 256)         590080    
_________________________________________________________________
activation_7 (Activation)    (None, 8, 8, 256)         0         
_________________________________________________________________
batch_normalization_7 (Batch (None, 8, 8, 256)         1024      
_________________________________________________________________
max_pooling2d_4 (MaxPooling2 (None, 4, 4, 256)         0         
_________________________________________________________________
dropout_4 (Dropout)          (None, 4, 4, 256)         0         
_________________________________________________________________
global_average_pooling2d_1 ( (None, 256)               0         
_________________________________________________________________
activation_8 (Activation)    (None, 256)               0         
_________________________________________________________________
batch_normalization_8 (Batch (None, 256)               1024      
_________________________________________________________________
dropout_5 (Dropout)          (None, 256)               0         
_________________________________________________________________
dense_1 (Dense)              (None, 6)                 1542      
_________________________________________________________________
activation_9 (Activation)    (None, 6)                 0         
=================================================================
Total params: 1,169,286
Trainable params: 1,166,918
Non-trainable params: 2,368
_________________________________________________________________

损失函数

确定了网络结构之后就需要确定针对我们的问题选用什么样的loss function了。这也是多类别分类(multi-class classification)和 多标签(multi-label classification)的差别之处。
先看看多分类问题中用的softmax函数假设是网络最后的输出:

import math

def softmax(z):
    z_exp = [math.exp(i) for i in z]
    sum_z_exp = sum(z_exp)
    return [i / sum_z_exp for i in z_exp]

经过softmax 层 之后得到的是一个多项式概率分布，所有的节点概率和为1, 这种情况下每个类别的输出是不独立的。
假设网络最后的输出为:z = [-1.0, 5.0, -0.5, 5.0, -0.5]

In [4]: z = [-1.0, 5.0, -0.5, 4.7, -0.5]                                         
In [5]: softmax(z)                                                               
Out[5]: 
[0.0014152405960574873,
 0.5709488061694115,
 0.002333337273878307,
 0.4229692786867745,
 0.002333337273878307]

则我们的预测值为概率只为0.57的第二类。

而sigmoid层为:

def sigmoid(z):
    return [1 / (1 + math.exp(-n)) for n in z]

In [7]: z = [-1.0, 5.0, -0.5, 4.7, -0.5]                                         
In [8]: sigmoid(z)                                                               
Out[8]: 
[0.2689414213699951,
 0.9933071490757153,
 0.3775406687981454,
 0.990986701347152,
 0.3775406687981454]

在sigmoid函数下，假设我们的判断阈值为0.5，则我们的预测值应该是第2,4类。在这中情况下每个输出节点对网络预测都是独立的。这也是我们在多标签分类要解决的问题，我们希望网络的输出是独立的伯努利分布，每个节点对loss函数的贡献也是独立的，每个label的出现也是对立的，比如red, blue可能同时出现，可能同时不出现，也可能出现一个，但是最后的预测值最主要还是取决于你的训练样本，如果 你的训练样本没有红色和蓝色同时出现的，那么你的预测值也很大程度上不会有。

多标签编码(multi-hots-encoding)

可以使用scikit-learn.preprocessing的MultiLabelBinarizer来对多标签类进行类别编码, 具体用法如下:

In [9]: from sklearn.preprocessing import MultiLabelBinarizer                                 
In [10]: labels = [['blue', 'shirt'],['red', 'jeans'],['blue', 'jeans']]                      
In [11]: mlb = MultiLabelBinarizer()                                                          
In [12]: labels = mlb.fit_transform(labels)                                                   
In [13]: labels                                                                               
Out[13]: 
array([[1, 0, 0, 1],
       [0, 1, 1, 0],
       [1, 1, 0, 0]])
In [14]: print(mlb.classes_)                                                                  
['blue' 'jeans' 'red' 'shirt']

可以看到MLB是按照每个属性的字母序对当前的label进行编码的，出现编码为1， 不出现为0。
编码完成之后使用ImageDataGenerator 和图片同步输入到网络里进行训练就可以。

训练结果图:

Screen Shot 2018-12-13 at 11.48.50 AM.png

所有的代码在这: https://github.com/ItchyHiker/multi-label-classification-Keras

Reference

1. https://github.com/keras-team/keras/issues/741
2. https://www.depends-on-the-definition.com/guide-to-multi-label-classification-with-neural-networks/
3. https://www.pyimagesearch.com/2018/05/07/multi-label-classification-with-keras/
4. https://stackoverflow.com/questions/44164749/how-does-keras-handle-multilabel-classification

前两篇Keras实战文章:
Keras 实战 II: VGG16图片分类:https://www.jianshu.com/p/5a7df18498d4
Keras 实战 I: 卷积神经网络图片分类:
https://www.jianshu.com/p/dcb10f9a5b05