1.1 为什么需要ML策略？

问题引入：如何提高机器学习的水平？可能有很多种想法（如，收集更多数据、收集更丰富多样的训练数据、增加梯度下降法迭代次数、增加网络规模、减小网络规模、使用随机失活（dropout）、添加L2正则化项、改变网络架构（激活函数、隐层单元数等）），应该选择哪些，舍弃哪些？
image.png

1.2 正交化

正交化的一个实例：老式电视机的调节按钮（上下、左右、形变等），每个按钮的功能明确而相对对立，即所谓“正交”（orthogonal）
image.png
机器学习中的假设链：
成本函数调优顺序：训练集 -> 开发集（验证集）-> 测试集 -> 真实世界
image.png
Orthogonalization（正交化）
Orthogonalization or orthogonality is a system design property that assures that modifying an instruction or a component of an algorithm will not create or propagate side effects to other components of the system. It becomes easier to verify the algorithms independently from one another, it reduces testing and development time.
When a supervised learning system is design, these are the 4 assumptions that needs to be true and orthogonal.
1. Fit training set well in cost function
- If it doesn’t fit well, the use of a bigger neural network or switching to a better optimization algorithm might help.
2. Fit development set well on cost function
- If it doesn’t fit well, regularization or using bigger training set might help.
3. Fit test set well on cost function
- If it doesn’t fit well, the use of a bigger development set might help
4. Performs well in real world
- If it doesn’t perform well, the development test set is not set correctly or the cost function is not evaluating the right thing.

1.3 单一数字评估指标

查准率（Precision）和查全率（Recall）难以同时兼顾，可综合考虑，用F1值来表示（F1=2PR/(P+R)）
image.png

另一个示例：使用平均值评价不同算法
image.png
查准率、查全率和F1值
image.png
image.png

1.4 优化指标和满足指标

评价指标可分为两类：优化指标（optimizing metrices）和满足指标（satisfying metrices）。优化指标需要重点关注，满足指标相当于约束条件而不得不考虑。
image.png
Satisficing and optimizing metric
There are different metrics to evaluate the performance of a classifier, they are called evaluation matrices. They can be categorized as satisficing and optimizing matrices. It is important to note that these evaluation matrices must be evaluated on a training set, a development set or on the test set.
Example: Cat vs Non-cat
image.png
In this case, accuracy and running time are the evaluation matrices. Accuracy is the optimizing metric, because you want the classifier to correctly detect a cat image as accurately as possible. The running time which is set to be under 100 ms in this example, is the satisficing metric which mean that the metric has to meet expectation set.
The general rule is:
image.png