import csv
from array import array
from numpy import *
import scipy.io
- train.csv是训练样本集,大小42001x785,第一行是文字描述,所以实际的样本数据大小是42000x785,其中第一列的每一个数字是它对应行的label,可以将第一列单独取出来,得到42000x1的向量trainLabel,剩下的就是42000x784的特征向量集trainData,所以从train.csv可以获取两个矩阵trainLabel、trainData。
下面给出代码,另外关于如何从csv文件中读取数据,参阅:csv模块的使用
def loadTrainData():
l=[]
with open('train.csv') as file:
lines=csv.reader(file)
for line in lines:
l.append(line) #42001*785
l.remove(l[0])
l=array(l)
label=l[:,0]
data=l[:,1:]
return nomalizing(toInt(data)),toInt(label)
- 这里还有两个函数需要说明一下,toInt()函数,是将字符串转换为整数,因为从csv文件读取出来的,是字符串类型的,比如‘253’,而我们接下来运算需要的是整数类型的,因此要转换,int(‘253’)=253。toInt()函数如下:
#toInt()函数,是将字符串转换为整数
def toInt(array):
array = mat(array)
m,n = shape(array)
newArray=zeros((m,n))
for i in range(m):
for j in range(n):
newArray[i,j]=int(array[i,j])
return newArray
- nomalizing()函数做的工作是归一化,因为train.csv里面提供的表示图像的数据是0~255的,为了简化运算,我们可以将其转化为二值图像,因此将所有非0的数字,即1~255都归一化为1。nomalizing()函数如下:
def nomalizing(array):
m,n=shape(array)
for i in range(m):
for j in range(n):
if array[i,j]!=0:
array[i,j]=1
return array
- test.csv里的数据大小是28001x784,第一行是文字描述,因此实际的测试数据样本是28000x784,与train.csv不同,没有label,28000x784即28000个测试样本,我们要做的工作就是为这28000个测试样本找出正确的label。所以从test.csv我们可以得到测试样本集testData,代码如下:
def loadTestData():
l=[]
with open('test.csv') as file:
lines=csv.reader(file)
for line in lines:
l.append(line)
#28001*784
l.remove(l[0])
data = array(l)
return nomalizing(toInt(data))
def classify(inX, dataSet, labels, k):
inX = mat(inX)
dataSet = mat(dataSet)
labels = mat(labels)
dataSetSize = dataSet.shape[0]
diffMat = tile(inX, (dataSetSize,1)) - dataSet
sqDiffMat = array(diffMat)**2
sqDistances = sqDiffMat.sum(axis=1)
distances = sqDistances**0.5
sortedDistIndicies = distances.argsort()
classCount={}
for i in range(k):
voteIlabel = labels[0,sortedDistIndicies[i]]
classCount[voteIlabel] = classCount.get(voteIlabel,0) + 1
sortedClassCount = sorted(classCount.items(), key=lambda d:d[1], reverse=True)
return sortedClassCount[0][0]
def saveResult(result):
with open('result.csv','w',newline='') as myFile: #加newline=''
myWriter = csv.writer(myFile)
for i in result:
tmp=[]
tmp.append(i)
myWriter.writerow(tmp)
def handwritingClassTest():
trainData,trainLabel=loadTrainData()
testData=loadTestData()
m,n=shape(testData)
resultList=[]
for i in range(m):
classifierResult = classify(testData[i], trainData, trainLabel, 5)
resultList.append(classifierResult)
saveResult(resultList)
handwritingClassTest()
去除result文件中的空行
def delblankline(infile, outfile):
""" Delete blanklines of infile """
infp = open(infile, "r")
outfp = open(outfile, "w")
lines = infp.readlines()
for li in lines:
if li.split():
outfp.writelines(li)
infp.close()
outfp.close()
#调用示例
if __name__ == "__main__":
delblankline("result.csv","ok.csv")
根据Kaggle上的sample_submission.csv
文件的格式修改我们得到的预测值文件,并上传到Kaggle上,最终准确率为96.399%: