by allenlu2007


cnn then clustering on feature!

apply to audio! emotion is a attribute

本計畫為“智慧型影音內容分析、創作及推薦”群體計畫之子計畫六。 近年來,具有多媒體功能之行動裝置快速普及於日常生活中,這些多媒體 應用衍生了許多重要相關研究議題,如拍攝影像之編修與影像內容分析 等。針對這些需求,本計畫希望能夠透過非監督式學習及半監督式學習的 技術來輔助發展視訊資料的內容分析技術,其中將分別從波譜式分群技術 (Spectral Clusteirng)、機率密度估測技術(Probability Density Estimation)以 及深度學習技術(Deep Learning)等三個不同的面向來討論這項議題。 在本計畫中,我們預計以三年的時間、針對三項主要議題來逐步進 行研究討論。第一年度將著重於從波譜式分群技術的角度來討論如何進 一步改善目前我們所提出的階層式影像晰分技術 (Hierarchical Image Matting),此外也將進一步討論視訊資料的晰分技術(Video Matting);第二 年度將著重於從機率密度估測技術的角度來討論視訊資料的拆解及分 析,其中我們計畫採用最新的貝氏循序切割演算法(Bayesian Sequential Partitioning);第三年則是將基於深度學習技術來自動地從視訊資料中學 習出有助於視訊內容分析的重要特徵,並據此進一步改善前兩年之研究成 果。此外,我們也將進一步討論機率密度估測技術與深度學習技術之間的 關聯性,希望能據此開發出有助於建構深度類神經網路之新式學習機制。 

This project is a sub-project of the joint-project “Intelligent Audio-Visual Content Analysis, Authoring, and Recommendation”. In recent years, mobile devices with various kinds of multi-media functionalities are prevalent in our daily life. Those new multi-media applications have been inspiring plentiful research topics, like the retouching/editing of image/video contents. In this project, we aim to discuss the development of video analysis techniques based on unsupervised learning tools or semi-supervised learning tools. In this project, we focus on three major aspects in unsupervised learning and semi-supervised learning: spectral clustering, probability density estimation, and deep learning. This is a three-year project. In the first year, we will focus on the improvement of the previously proposed hierarchical image matting technique from the aspect of spectral clustering. We will also discuss the extension of image matting to the matting of video data. In the second year, we will focus on the decomposition and analysis of visual data from the aspect of density estimation. Here, the newly proposed Bayesian Sequential Partitioning Algorithm will be adopted for density estimation. In the third year, we plan to automatically extract valuable features from visual data based on the state-of-the-art deep learning techniques. We will also apply these extracted features for the improvement of the techniques developed in the first two years. Besides, we will discuss the connections between density estimation and deep learning, hoping to develop a density-estimation-based mechanism for efficient learning of deep neural networks.