Loss Function in Machine Learning

by allenlu2007

 

Loss function <->  Bregman divergence <-> exponential family

Square loss:  Gaussian distribution

cross entropy loss: Poisson distribution; Bernoulli distribution; multinomial distribution

Logic loss: Bernoulli distribution

Saito-?  loss (exponential loss?): exponential distribution

Exponential loss (random forest)?

Hinge loss

 

Loss function : divergence

–>  Optimization

 

Unsupervised clustering (K-mean, GMM, etc.)

Supervised Classification (similar to clustering), parameter estimation

ICA or PCA!!  Bregman Projection 

 

 

Advertisements