Probabilistic Graphical Models (PGM) Lecture

by allenlu2007

今天把 Stanford CS professor Daphne Koller 在 coursera 的 Probabilistic Graphic Models (PGM)看了 week1,有幾點感想。

1. Koller 和 Ng 是 coursera 的 cofounders.  Koller 是以色列裔美國人。雖然 Koller 和 Ng 都有 AI 領域的專長。我比較偏好 Ng 的對於理論部份熟悉的深度以及經驗的分享。Koller 比較快節奏,有些部份如 DBN 和 HMM 的關係就沒有很清楚。However, starting from week2, I like Koller’s overall orangization and rich material.

2. Koller no doubt 是 graph 的 master, 但常常沒有給一個明確目標 why to do this 或解釋可能的應用 (如 variable elimination lecture), 很容易讓聽者陷在 tedious 的公式推導中,有點像是 textbook 的 replay.  後來發現她的確寫了一本 PGM textbook and on-line readable.


1. 分類以及 logic 清楚: 依順序為 representation, inference, learning 讓學習者有大的 picture. 一直到看完 week2 lectures 才覺得 Koller 的教學像蓋房子,一塊塊的往上疊。如果沒有耐心聽完前面,很容易 got lost.

2. Belief propagation with PGM 可以用來解釋 turbo code and LDPC code performance close to Shannon channel capacity limit.  其中 decoder1 propagate information from prior to post to decoder2.  Decoder1 bit (prior to decoder2) –> MAP of decoder2    

3. EM algorithm 實用上要注意的點如 parameter 收歛性 and over fit,local minimum, initial value.



剛好看到 prof. Sam Roweis 在台北的 tutorial lecture.  我覺得非常清楚可以和 Koller 的 talk 互補。在聽過 Koller 之前如果先聽過 Roweis 應該會對大的 picture 更有概念。不過在 Koller 之後聽也非常有幫助。

Roweis 四個 lectures (MLSS in Taipei) 是我聽過最好的 PGM lectures.

Lecture1: introduction of PGM; conditional independent == factorization


Lecture3: latent variables, clustering

Lecture4: inference, belief propagation, junction tree: 非常棒,除非你已經精通 PGM, 應該會有豁然開朗 的感覺。  p(x|e) 就是用 query node as root (x, single variable), evident node (e, multiple variables) as leaves, trim out other leave nodes (sum to 1 for non-evident leaves).  Then use message passing to from leaf to root and root to leaf.   For multiple queries (again with single variable), do the messaging passing for every node back and forth (x2) obtains the entire tree CDP.

我 check Roweis 的 background, 1999 Caltech PhD, 待過 MIT and University of Toronto, then NYU professor.  I am so surprised to know he passed away in 2010.  What a pity to such a brilliant person!