一.k-近邻算法概述

简单地说，k-近邻算法采用测量不同特征值之间的距离方法进行分类。
image.png
image.png

1.准备：使用Python导入数据

  1. from numpy import *
  2. import operator
  4. def createDataSet():
  5.     group = array([[1.0,1.1],[1.0,1.0],[0,0],[0,0.1]])
  6.     labels = ['A','A','B','B']
  7.     return group,labels
  1. Microsoft Windows [版本 10.0.18362.476]
  2. (c) 2019 Microsoft Corporation。保留所有权利。
  4. C:\Users\Joish>C:\Users\Joish\AppData\Local\Programs\Python\Python37-32\python.exe
  5. Python 3.7.4 (tags/v3.7.4:e09359112e, Jul  8 2019, 19:29:22) [MSC v.1916 32 bit (Intel)] on win32
  6. Type "help", "copyright", "credits" or "license" for more information.
  7. >>> import kNN
  8. >>> group,labels = kNN.createDataSet()
  9. >>> group
  10. array([[1. , 1.1],
  11.        [1. , 1. ],
  12.        [0. , 0. ],
  13.        [0. , 0.1]])
  14. >>> labels
  15. ['A', 'A', 'B', 'B']
  16. >>>

2.实施kNN算法

对未知类别属性的数据集中的每个点依次执行以下操作：
(1) 计算已知类别数据集中的点与当前点之间的距离；
(2) 按照距离递增次序排序；
(3) 选取与当前点距离最小的k个点；
(4) 确定前k个点所在类别的出现频率；
(5) 返回前k个点出现频率最高的类别作为当前点的预测分类。 

  1. from numpy import *
  2. import operator
  4. def createDataSet():
  5.     group = array([[1.0,1.1],[1.0,1.0],[0,0],[0,0.1]])
  6.     labels = ['A','A','B','B']
  7.     return group,labels
  8. def classify0(inX, dataSet, labels, k):
  9.     dataSetSize = dataSet.shape[0]
  10.     diffMat = tile(inX, (dataSetSize,1)) - dataSet
  11.     sqDiffMat = diffMat**2
  12.     sqDistances = sqDiffMat.sum(axis=1)
  13.     distances = sqDistances**0.5
  14.     sortedDisIndicies = distances.argsort()
  15.     classCount={}
  16.     for i in range(k):
  17.         voteIlabel = labels[sortedDistIndicies[i]]
  18.         classCount[voteIlabel] = classCount.get(voteIlabel,0) + 1
  19.     sortedClassCount = sorted(classCount.items(),
  20.      key=operator.itemgetter(1), reverse=True)
  21.     return sortedClassCount[0][0]

tile（）函数：

https://blog.csdn.net/qq_38669138/article/details/79085700

二.示例：使用k-近邻算法改进约会网站的配对效果

  1. def file2matrix(filename):
  2.     fr = open(filename)
  3.     arrayOLines = fr.readlines()
  4.     numberOfLines = len(arrayOLines)
  5.     returnMat = zeros((numberOfLines, 3))
  6.     classLabelVector = []
  7.     index = 0
  8.     for line in arrayOLines:
  9.         line = line.strip()
  10.         listFromLine = line.split('\t')
  11.         returnMat[index, :] = listFromLine[0:3]
  12.         classLabelVector.append(basestring(listFromLine[-1]))
  13.         index += 1
  14.     return returnMat, classLabelVector
  1. >>> reload kNN
  2. >>> datingDataMat, datingLabels = kNN.file2matrix('datingTestSet2.txt')
  3. >>> import matplotlib
  4. >>> import matplotlib.pyplot as plt
  5. >>> fig = plt.figure()
  6. >>> ax = fig.add_subplot(111)
  7. >>> from numpy import * 
  8. >>> ax.scatter(datingDataMat[:,1], datingDataMat[:,2], 15.0*array(datingLabels), 15.0*array(datingLabels))
  9. >>> plt.show()
  1. def autoNorm(dataSet):
  2.     minVals = dataSet.min(0)
  3.     maxVals = dataSet.max(0)
  4.     ranges = maxVals - minVals
  5.     normDataSet = np.zeros(np.shape(dataSet))
  6.     m = dataSet.shape[0]
  7.     normDataSet = dataSet - np.tile(minVals, (m, 1))
  8.     normDataSet = normDataSet/np.tile(ranges, (m, 1))
  9.     return normDataSet, ranges, minVals
  1. def datingClassTest():
  2.     hoRatio = 0.50
  3.     datingDataMat, datingLabels = file2matrix(datingTestSet.txt')      
  4.     normMat, ranges, minVals = autoNorm(datingDataMat)
  5.     m = normMat.shape[0]
  6.     numTestVecs = int(m * hoRatio)
  7.     errorCount = 0.0
  8.     for i in range(numTestVecs):
  9.         classifierResult = classify0(normMat[i, :], normMat[numTestVecs:m, :],
  10.                                      datingLabels[numTestVecs:m], 3)
  11.         print("the classifier came back with: %d, the real answer is: %d" % (
  12.             classifierResult, datingLabels[i]))
  13.         if classifierResult != datingLabels[i]:
  14.             errorCount += 1.0
  15.     print("the total error rate is: %f" % (errorCount / float(numTestVecs)))
  16.     print(errorCount)
  1. def classifyPerson():
  2.     resultList = ['not at all', 'in small doses', 'in large doses']
  3.     percentTats = float(input("percentage of time spent playing video games ?"))
  4.     ffMiles = float(input("frequent filer miles earned per year?"))
  5.     iceCream = float(input("liters of ice cream consumed per year?"))
  6.     datingDataMat, datingLables = file2matrix('G:/python/machinelearninginaction/Ch02/datingTestSet2.txt')
  7.     normMat ,ranges, minVals = autoNorm(datingDataMat)
  8.     inArr = np.array([ffMiles, percentTats, iceCream])
  9.     classifierResult = classify0((inArr-minVals)/ranges, normMat, datingLables, 3)
  10.     print ("You will probably like this person: ", resultList[classifierResult - 1])

注：遇到的坑实在太多，运行起来出了很多问题，至今无法一一解决，还需要再思考一下。