Supervised Machine Learning Framework

by allenlu2007

本文參考台大林軒田教授的 machine learning foundation.




Is supervised learning possible?



Some points:

* 有training examples => 是 supervised learning

* Both y1, y2, …, yN and x1, x2, …, xN 都是 IID.  之間沒有 correlation.  因此無法 cover samples 之間有 correlation 的情況。如 reinforce learning 的最後 cost 是所有 decision 合在一起的結果。從另一個角度,如果把所有的 (x1, x2, …, xn) 視為更大 space 的一個新的 X1, 也許可以 …

* Probablisitic model: assuming IID.   -> cannot cover eninforcement learning or samples has correlation.


How about unsupervised learning?


Why H and A are separated!


Hypothesis (H) –> Training/Learning samples,  corresponding error -> Ein/Etrain or loss function

error type:  0/1 error, square error/loss, cross entropy loss, hinge loss, L1 loss, etc.


Algorithm -> minimize the training/learning error


Error weighting:  false positive or false negative


Model selection -> cross-validation samples, corresponding error -> Ecv (沒有 loss function)?

Or error weight selection (use precision/recall or F score) -> cross-validation samples

Both Model selection and error weight selection uses cross-validation samples, Do NOT use test samples to avoid contaminate test samples.


最後有  test samples,  Etest