模式识别Machinelearning

思维导图包括模型评估与选择、线性模型、决策树、神经网络、特征选择chose features related to task、降维与量度学习、类clustering non supervised、贝叶斯分类器等内容。

模式识别Machinelearning

• 绪论
  • 基本术语 示例instance=样本sample Dataset 属性Attribute=特征feature性质 （属性)空间e.g.(色泽，根蒂，敲声) 特征向量 Generalization Ability泛化能力适用于新样本的能力 假设空间hypothes is seeing if which and how the feature is to define a good watermelon
  • 归纳偏好inductivebias脱离问题计算算法性能是不可取的。It is not advisable to calculate algorithm performance out of the problem.
    • NFL 没有完美的算法。No algorithm fit in all problem
  • getdata——preprocessing——train——test——improve
• 模型评估与选择
  • 经验误差training error generalization error——accuracy与过拟合overfitting and way to Mitigate overfitting
  • 评估方法：留出法；（important）交叉验证验证法；自助法situation
  • 性能度量：AccuracydicePrecisionRecallFalseOmissionRateTrueNegativeRatejaccard
  • *比较检验
  • 用某种实验评估方法测得学习器的某个性能指标，对结果进行比较，以此来选择模型。A model is selected by measuring a certain performance index of the learner by an experimental evaluation method, comparing the results to chose final model.
  • 偏差与方差
• 线性模型
  • 线性回归Linear regression set label to access real label y
    • ？正则化 regularization
  • 线性分类Linear classification according to label do classification
    • log linear-regression access to lny
  • 线性判别分析 Linear Disriminant Analysis 将样本投影到一条直线上，同类尽可能接近，异类尽可能远离。Project the sample onto a straight line, as close as possible to the same kind and as far away as possible from the heterogeneous.
    • how to find this straight line
    • 最大特点 N-dimensional data is reduced to N-1 dimension.
  • 多分类学习
  • 类别不平衡问题class-imbalance to solve it use rescaling
  • 以均方差度量性能
• 决策树
  • 基本流程
    • • to find highest purity attribute base on its indexs 信息熵information entropy smaller 信息增益information gain biger prefer Attribute with lots of options 增益率gain ratio max prefer Attribute with less option 基尼指数Gini index smaller
  • 剪枝处理 pruning is the method decision tree use to mitigate overfitting
    • 预剪枝 pruning while getting more generalizatation ability ；easy to cause unterfitting
    • 后剪枝 pruning after getting more spend
  • 连续值处理 use 二分法 bi-partition
  • *多变量决策树
• 神经网络
  • 神经元
    • 感知机Preceptron与多层网络
  • 误差逆传播算法 BP
  • 神经网络缓解过拟合策略
• 特征选择 chose features related to task
  • 作用 减轻维度灾难；降低学习难度mitigating dimensional disasters; Reduce learning difficulty
  • 流程 子集搜索与评价
  • 一般方法
    • 过滤式选择
    • 包裹式选择
    • 嵌入式选择 chose features while learning
• 降维与量度学习
  • find k近邻学习 Nearest Neighbor，if classification vote most win，if regression take its means ；Eorr≤2Best Error
  • At high latitude, the data distribution is sparse and the distance calculation is difficult, which is solved by dimensionality reduction algorithm.高纬度下，数据分布稀疏，距离计算困难，用降维算法解决。
  • *距离度量学习
• 聚类 clustering non-supervised
  • 性能度量
  • 原型聚类
    • k均值算法 k-means set original vectors calculate the distance between another vectors and them， divide other vectors into the same cluster depend on their distance；there should be a formula to calculate the neu origina vectors
  • 密度聚类 e.g. DBSCAN
  • 层次聚类
• 贝叶斯分类器
  • 贝叶斯决策论 基于概率和期望损失得到机器学习模型精度的理论上限。Based on probability and expected loss, the theoretical upper limit of the accuracy of the machine learning model is obtained.
  • generate model贝叶斯定理
  • 朴素贝叶斯分类器 Laplacian correction Smoothed probability values
• 支持向量机 KKT
  • 间隔与支持向量 最终模型仅与支持向量有关
  • 升维找划分超平面 核函数 为简化升维后向量的内积计算，使核函数=内积 决定SVM性能
  • 软间隔 soft margin
  • 正则化 regularization ≈罚函数法penalty method