ÄãºÃ£¬»¶Ó­À´µ½¾­¹ÜÖ®¼Ò [µÇ¼] [×¢²á]

ÉèΪÊ×Ò³ | ¾­¹ÜÖ®¼ÒÊ×Ò³ | Êղر¾Õ¾

±´Ò¶Ë¹·ÖÎö·½·¨Ñо¿_µç×ÓÐÅÏ¢¹¤³Ì±ÏÒµÂÛÎÄ·¶ÎÄ

·¢²¼Ê±¼ä£º2015-03-16 À´Ô´£ºÈË´ó¾­¼ÃÂÛ̳
±´Ò¶Ë¹·ÖÎö·½·¨Ñо¿_µç×ÓÐÅÏ¢¹¤³Ì±ÏÒµÂÛÎÄ·¶ÎÄ Õª Òª»úÆ÷ѧϰ×÷ΪһÃÅÈ˹¤ÖÇÄܵĿÆѧ×Ô20ÊÀ¼Í50Äê´ú±»Ìá³öÒÔÀ´£¬¾­¹ýÈËÃǵIJ»¶ÏÑо¿£¬ÒÑÐγÉÁËÒ»Ì׿ÆѧϵͳµÄÀíÂÛ¡£»úÆ÷ѧϰÖÐÒ»¸öºÜÖØÒªµÄ²½ÖèÊÇÌØÕ÷µÄÑ¡ÔñÓëÌáÈ¡£¬Ô­Ê¼ÌØÕ÷µÄÊýÁ¿¿ÉÄܴܺ󣬻òÕß˵Ñù±¾´¦ÔÚÒ»¸ö¸ßά¿Õ¼äÖУ¬ÎÒÃÇÐèÒªÕÒµ½Ò»¸öºÏÀíµÄ·½·¨£¬½µµÍÌØÕ÷ÊýÁ¿µÄͬʱ£¬¾¡Á¿¼õÉÙÔ­ÌØÕ÷Öаüº¬ÐÅÏ¢µÄËðʧ£¬Òò×Ó·ÖÎö·¨¾ÍÊÇÕâÑùÒ»ÖÖ½µÎ¬µÄ·½·¨¡£È»¶øÓÉÓÚÒò×Ó·ÖÎöÄ£ÐÍÖдæÔÚ²»¿É¹Û²âµÄÒþ±äÁ¿£¬ÆÕͨµÄ¼«´óËÆÈ»·¨ºÜÄѵõ½Æä²ÎÊýµÄ¹À¼Æ¡£±´Ò¶Ë¹ÀíÂÛÌṩÁËÒ»ÖÖ¼ÆËã¸÷±äÁ¿ºóÑé¸ÅÂʵķ½·¨£¬ÕâÖÖ·½·¨»ùÓÚ¼ÙÉèµÄÏÈÑé¸ÅÂʺ͹۲⵽µÄÊý¾Ý£¬¿ÉÒԵõ½Ä£Ð͸÷±äÁ¿µÄºóÑé¸ÅÂÊ¡£±¾ÎĵŤ×÷ÕýÊÇÔÚÕâÖÖÑо¿±³¾°ÏÂÕ¹¿ªµÄ¡£ÔÚÇ°È˹¤×÷µÄ»ù´¡ÉÏ£¬±¾ÎÄ×ÅÖØÑо¿ÁËÈçºÎÔËÓñä·Ö±´Ò¶Ë¹Ëã·¨ÍƵ¼³öÒò×Ó·ÖÎö·¨µÄ±´Ò¶Ë¹ºóÑé·Ö²¼¹«Ê½¡£¹éÄÉÆðÀ´£¬±¾ÎĵÄÖ÷ÒªÄÚÈÝ°üÀ¨ÒÔÏÂËĸö·½Ã棺*¼òÒª½éÉܱ´Ò¶Ë¹»úÆ÷ѧϰµÄ»ù´¡ÖªÊ¶£¬°üÀ¨±´Ò¶Ë¹¶¨Àí£¬±´Ò¶Ë¹¹À¼ÆºÍ¼¸ÖÖÏÈÑé·Ö²¼¡£*¼òÒª½éÉÜÒò×Ó·ÖÎöÄ£ÐÍ£¬·ÖÎöÆ併άµÄ»úÀí¡£*Ϊ¹À¼ÆÒò×Ó·ÖÎöÄ£ÐÍÖеIJÎÊý£¬ÒýÈëEMËã·¨ºÍ±ä·Ö±´Ò¶Ë¹Ëã·¨£¬ÒÔ½â¾öÄ£ÐÍÖдæÔÚÒþ±äÁ¿µÄÎÊÌâ¡£*ÍƵ¼Òò×Ó·ÖÎö·¨µÄ±´Ò¶Ë¹ºóÑé·Ö²¼¹«Ê½£¬²¢ÓÃMatlab±à³ÌʵÏÖ£¬Í¨¹ýºÏ³ÉÊý¾Ý¼ìÑéÀíÂÛµÄÕýÈ·ÐÔ×îºó£¬ÎÒÃǶÔÈ«ÎŤ×÷½øÐÐÁË×ܽᣬ²¢Ö¸³ö½ñºóÐèÒª½øÒ»²½Ñо¿µÄһЩÎÊÌâ¡£¹Ø¼ü´Ê£ºÒò×Ó·ÖÎö·¨ ±´Ò¶Ë¹ÀíÂÛ ºóÑé·Ö²¼ EMËã·¨ ±ä·Ö±´Ò¶Ë¹Ëã·¨ ABSTRACTAs a kind of artificial intelligence science, machine learning was proposed in the 1950¡¯s and has formed a scientific and systematic theory. A very important step in machine learning is feature extraction and selection. The number of original features may be huge, or we can say that the sample is in a high dimensional space. So we need to find a reasonable approach which can not only reduce the number of observed variables, but also minimize the loss of the information contained in the original features. Factor analysis is such a dimension reduction method. However, because of the existence of unobserved hidden variables in the factor analysis model, the estimation of parameters using maximum likelihood solution becomes intractable. The Bayesian theory provides a solution to compute the posterior probabilistic of variables. Based on the assumption of prior probabilistic and the observed data, it can find the posterior probabilistic of all variables in the model. Based on previous work, this paper focuses on the derivation of the Bayesian posterior distribution of the parameters in factor analysis model via the Variational Bayesian algorithm. The main content of this paper is summarized as follows:Íê³É ʵÏÖimplementation* Briefly introduce the basic knowledge of Bayesian machine learning, including the Bayesian method, Bayesian inference and the choice of prior.* Briefly introduce the factor analysis model.* Introduce EM algorithm and Variational Bayesian algorithm for the estimation of parameters, in order to solve the problem of hidden variable.* Derive the Bayesian posterior distribution of the parameters, and code the algorithm with Matlab, which is validated by experiments using synthetic data.Finally, we conclude the paper with a summary and advance some suggestions for further research in factor analysis.Keywords: Factor analysis Bayesian theory Posterior probabilistic EM algorithm Variational Bayesian algorithm Ä¿ ¼µÚÒ»Õ Ð÷ÂÛ 2µÚ¶þÕ ±´Ò¶Ë¹ÀíÂÛ»ù´¡ÖªÊ¶ 22.1 ±´Ò¶Ë¹¹«Ê½ 22.2 ±´Ò¶Ë¹ÍÆ¶Ï 22.3 ÏÈÑé·Ö²¼µÄÑ¡Ôñ 22.3.1 ¿Í¹ÛÏÈÑé·Ö²¼ 22.3.2 Ö÷¹ÛÏÈÑé·Ö²¼ 22.3.3 ·Ö²ãÏÈÑé·Ö²¼ 22.4 С½á 2µÚÈýÕ Òò×Ó·ÖÎö·¨ 23.1 ÒýÑÔ 23.2 Òò×Ó·ÖÎö·¨ 23.1.1 Òò×Ó·ÖÎöÄ£ÐÍ 23.1.2 Òò×Ó·ÖÎöÄ£Ð͵ÄÐÔÖÊ 23.3 Òò×Ó·ÖÎöÓëÖ÷³É·Ö·ÖÎöµÄ±È½Ï 23.4 С½á 2µÚËÄÕ ±ä·Ö±´Ò¶Ë¹ÀíÂÛ 24.1 EMËã·¨ 24.1.1 EMËã·¨»ù±¾ÀíÂÛ 24.1.2 ¶ÔEMËã·¨µÄÀí½â 24.1.3 EMËã·¨ÊÕÁ²ÐÔ·ÖÎö 24.2 ±ä·Ö±´Ò¶Ë¹ 24.2.1 VBEMËã·¨ 24.2.2 ºóÑé·Ö²¼µÄÇó½â 24.3 С½á 2µÚÎåÕ ±ä·Ö±´Ò¶Ë¹Òò×Ó·ÖÎö 25.1 Ä£ÐͼÙÉè 25.2 ²ÎÊýÇó½â 25.3 ʵÑé·ÖÎö 25.4 С½á 2µÚÁùÕ ½áÊøÓï 2Ö л 2²Î¿¼ÎÄÏ× 2 µÚÒ»Õ Ð÷ÂÛÉú»îÖУ¬ÎÒÃÇÿʱÿ¿Ì¶¼ÔÚ¶ÔÖÜΧµÄÊÂÎï½øÐÐ×ÅÈÏÖªÓëʶ±ð£¬È»¶ø¶ÔÓÚÈËÄÔÖÐÕâÖÖÈÏÖªÓëʶ±ðµÄ»úÀí£¬ÈËÃÇÉÐδµÃµ½×¼È·µÄ½âÊÍ¡£»úÆ÷ѧϰÊÇÑо¿ÈçºÎʹÓüÆËã»úÄ£ÄâÈËÀàÈÏ֪ѧϰ¹ý³ÌµÄÒ»ÃÅ¿Æѧ£¬Ëü¶ÔÓÚÀí½âÈËÀà´óÄÔµÄѧϰ¹ý³ÌÓкܴó°ïÖú¡£¼ÆËã»úÔÚ¶ÔÖÜΧµÄÊÂÎï½øÐÐÈÏÖªµÄʱºò£¬ÊǶÔÏÖʵÖеÄÊÂÎィÁ¢ÊýѧģÐÍ£¬½«¹Û²âµ½µÄÐÅÏ¢½øÐÐÈ¡ÑùºÍÁ¿»¯£¬ÒÔÏòÁ¿µÄÐÎʽ´æ´¢ÔÚ¼ÆËã»úÖС£¼ÆËã»ú´ÓÒ»¸öÎïÌåÖÐÌáÈ¡µÄÐÅÏ¢¿ÉÒÔÊǶàÖÖ¶àÑùµÄ£¬±ÈÈ磬¶ÔÓÚÒ»¸öÆ»¹û£¬¿ÉÒÔÌáÈ¡ËüµÄÑÕÉ«ºÍÐÎ×´£¬È»¶ø¶ÔÓÚÒ»¸ö¸´ÔÓµÄÊÂÎ¿ÉÒÔÌáÈ¡µÄÌØÕ÷ÊýÁ¿¿ÉÄÜÊǾ޴óµÄ£¬ËüÃÇÔÚ¼ÆËã»úÖÐÒÔ¸ßάµÄ¾ØÕóÐÎʽ´¢´æ£¬´¦ÀíÕâÑùµÄ¸ßά¾ØÕóÊÇÁîÈËÍ·ÌÛµÄÒ»¼þÊ¡£ÐÒÔ˵ÄÊÇ£¬ÎÒÃÇ·¢ÏÖÕâЩÊýÁ¿ÅÓ´óµÄÌØÕ÷Ö®¼ä´æÔÚ×ÅijЩÏà¹ØÐÔ£¬Òò¶øͨ¹ý¶ÔÕâЩÐÅÏ¢½øÐÐÒ»¶¨µÄÊýѧ´¦Àí£¬¿ÉÒÔ¼ò»¯ËùÐèÒªµÄÌØÕ÷ÊýÁ¿£¬ÊµÏÖËùνµÄ½µÎ¬¡£Òò×Ó·ÖÎö·¨¾ÍÊÇÕâÑùµÄ½µÎ¬·½·¨¡£Òò×Ó·ÖÎö·¨ÊǶàԪͳ¼ÆѧÖеÄÒ»ÖÖ½µÎ¬·½·¨£¬Ëüͨ¹ýÑо¿¹Û²â±äÁ¿Ïà¹ØÕó»òЭ²îÕóµÄÄÚ²¿ÒÀÀµ¹Øϵ£¬½«¶à¸ö±äÁ¿×ÛºÏΪÉÙÊý¼¸¸öÒò×Ó£¬ÒÔÔÙÏÖÔ­±äÁ¿ÓëÒò×ÓÖ®¼äµÄÏà¹Ø¹Øϵ¡£Òò×Ó·ÖÎöµÄÖ÷ÒªÓ¦ÓÃÓÐÁ½¸ö·½Ã棺һÊÇÑ°Çó»ù±¾½á¹¹£¬¼ò»¯¹Û²âϵͳ£¬½«¾ßÓдí×Û¸´ÔÓ¹ØϵµÄ±äÁ¿×ÛºÏΪÉÙÊý¼¸¸ö²»¿É¹Û²âµ«Ï໥¶ÀÁ¢µÄËæ»ú±äÁ¿£¬¼´Òò×Ó£¬ÔÙÏÖÒò×ÓÓëÔ­±äÁ¿Ö®¼äµÄÄÚÔÚÁªÏµ£»¶þÊÇÕë¶Ô·ÖÀàÎÊÌ⣬½«Òò×Ó·ÖÎö·¨ÓÃÓÚÌØÕ÷ÌáÈ¡£¬Ê¹ÓýµÎ¬µÄÌØÕ÷ʵÏÖ¶ÔÔ­±äÁ¿µÄ·ÖÀࡣͨ³£Çé¿öÏ£¬ÎÒÃÇÊÖÍ·Ö»ÓÐÒ»×é¹Û²âÑù±¾£¬Òò¶øÊ×ÏÈÒª×öµÄ¾ÍÊÇÀûÓÃÑù±¾¶ÔÄ£ÐÍÖеIJÎÊý½øÐйÀ¼Æ¡£È»¶øÓÉÓÚÒò×Ó·ÖÎöÄ£ÐÍÖдæÔÚÎÞ·¨¹Û²âµÄÒþ±äÁ¿£¬Ôì³ÉÔÚ¶ÔÆäÄ£ÐͽøÐвÎÊý¹À¼Æʱ£¬Ó¦ÓÃ×î´óËÆÈ»¹À¼Æ·¨»áµ¼Ö¼ÆËã¹ýÓÚ¸´ÔÓ£¬ÕâʱÎÒÃÇÏëµ½ÔËÓÃÆÚÍû×î´ó£¨EM£©Ëã·¨½øÐвÎÊý¹À¼Æ¡£EMËã·¨ÊÇÒ»ÖÖ¸ßЧµÄµü´úËã·¨£¬Í¨¹ý²»¶ÏÓÅ»¯ËÆÈ»º¯ÊýµÄϽçÀ´½üËÆ×î´ó»¯ËÆÈ»º¯Êý£¬½ø¶øÇóµÃ²ÎÊýµÄ¹À¼ÆÖµ¡£Æä×î´óµÄÌصãÊÇÄܹ»´¦Àíº¬ÓÐÒþ±äÁ¿µÄÄ£ÐÍ¡£ÔÚÇó½âµÄ¹ý³ÌÖл¹Óõ½ÁËÒ»¸öÖØÒªµÄÀíÂÛ£¬±´Ò¶Ë¹ÀíÂÛ¡£±´Ò¶Ë¹ÀíÂÛÊÇÓÉÍÐÂí˹•±´Ò¶Ë¹Ìá³öµÄÄæ¸ÅÂÊÕâÒ»¸ÅÄî·¢Õ¹¶øÀ´µÄÒ»ÖÖÆÕ±éµÄÍÆÀí·½·¨£¬ÔÚ±´Ò¶Ë¹Ä£ÐÍÖУ¬²ÎÊý²»ÔÙÊÇÒ»¸öδ֪µÄ³£Êý£¬¶øÊǾßÓÐijһ·Ö²¼ÐÎʽµÄËæ»ú±äÁ¿¡£Ëüͨ¹ý¼ÙÉè²ÎÊýµÄÏÈÑé·Ö²¼£¬ÔÙ½áºÏÒÑÓеĹ۲âÊý¾Ý£¬¼ÆËã³ö²ÎÊýµÄºóÑé·Ö²¼¡£ÓúóÑé·Ö²¼µÄÆÚÍû×÷Ϊ²ÎÊýµÄ¹À¼ÆÖµ¡£È»¶øÔÚһЩ¸´ÔÓµÄÄ£ÐÍÖУ¬²ÎÊýµÄºóÑé·Ö²¼¾ßÓм«Æ临ÔÓµÄÐÎʽ£¬ÆÚÍûµÄ¼ÆËãͨ³£°üº¬¸´Ôӵĸßά»ý·Ö¡£ÕâʱÎÒÃÇÐèÒªÒýÈëÒ»ÖÖ¸ßЧµÄÓÅ»¯Ëã·¨£¬±ä·Ö±´Ò¶Ë¹£¨VB£©Ëã·¨¡£±ä·Ö±´Ò¶Ë¹Ë㷨ͨ¹ýÓÅ»¯ËÆÈ»º¯ÊýµÄ»ý·ÖÏÒÔÇóµÃ²ÎÊýºóÑé·Ö²¼µÄ¹À¼ÆÐÎʽ¡£±¾ÎÄÔÚ¼òÒª½éÉÜÁ˱´Ò¶Ë¹»úÆ÷ѧϰÀíÂ۵Ļù´¡ÉÏ£¬ÖصãÍƵ¼ÁËÒò×Ó·ÖÎö·¨µÄºóÑé·Ö²¼ÐÎʽ£¬¾ßÌåÄÚÈÝ°²ÅÅÈçÏ£ºµÚ¶þÕ½éÉÜÁ˱´Ò¶Ë¹ÍƶϵĻù´¡ÖªÊ¶£¬°üÀ¨±´Ò¶Ë¹¹«Ê½£¬±´Ò¶Ë¹¹À¼Æ£¬ÏÈÑé·Ö²¼µÄÑ¡ÔñÎÊÌâ¡£µÚÈýÕ½éÉÜÁËÒò×Ó·ÖÎö·¨¡£µÚËÄÕ½éÉÜÁËÆÚÍû×î´ó£¨EM£©Ëã·¨ºÍ±ä·Ö±´Ò¶Ë¹£¨VB£©Ëã·¨¡£µÚÎåÕÂÍƵ¼ÁËÒò×Ó·ÖÎöÄ£ÐÍÖвÎÊýºÍÒþ±äÁ¿µÄ±´Ò¶Ë¹ºóÑé·Ö²¼ÐÎʽ¡£µÚÁùÕ¶ÔÈ«ÎĵŤ×÷½øÐÐÁË×ܽᡣ µÚ¶þÕ ±´Ò¶Ë¹ÀíÂÛ»ù´¡ÖªÊ¶2.1 ±´Ò¶Ë¹¹«Ê½±´Ò¶Ë¹¹«Ê½Ô´ÓÚ±´Ò¶Ë¹ÔÚËûÉúǰΪ½â¾öÒ»¸öÄæ¸ÅÂÊÎÊÌâ¶øдµÄÎÄÕ£¬ÄÇʱµÄÈËÃÇÒѾ­Äܹ»¼ÆËãÕýÏò¸ÅÂÊ£¬ÄÇôʲôÊÇÄæ¸ÅÂÊÄØ£¿ÕâÔÚÉú»îÖÐÆäʵºÜ³£¼û¡£±ÈÈçÒ»ËùѧУÀïÃæÓÐ60%µÄÄÐÉú£¬40%µÄÅ®Éú¡£ÄÐÉú×ÜÊÇ´©³¤¿ã£¬Å®ÉúÔòÊÇÒ»°ë´©³¤¿ãÒ»°ë´©È¹×Ó¡£ÓÐÁËÕâЩÐÅÏ¢Ö®ºóÎÒÃǾͿÉÒÔÈÝÒ׵ؼÆËã¡°Ëæ»úÑ¡È¡Ò»¸öѧÉú£¬Ëû£¨Ëý£©´©³¤¿ãµÄ¸ÅÂʺʹ©È¹×ӵĸÅÂÊ¡±£¬Õâ¸ö¾ÍÊÇÇ°Ãæ˵µÄ¡°ÕýÏò¸ÅÂÊ¡±µÄ¼ÆË㡣Ȼ¶ø£¬¼ÙÉèÄã×ßÔÚУ԰ÖУ¬Ó­Ãæ×ßÀ´Ò»¸ö´©³¤¿ãµÄѧÉú£¨ºÜ²»ÐÒÄãÊǸ߶ȽüÊÓ£¬ÄãÖ»¿´µÃ¼ûËû£¨Ëý£©´©µÄÊÇ·ñÊdz¤¿ã£¬¶øÎÞ·¨È·¶¨Ëû£¨Ëý£©µÄÐԱ𣩣¬ÄãÄܹ»ÍƶϳöËû£¨Ëý£©ÊÇÅ®ÉúµÄ¸ÅÂÊÊǶà´óÂð£¿Äæ¸ÅÂÊÎÊÌâÓɴ˲úÉú¡£½ÓÏÂÀ´¾ÍÀ´¼ÆËãÒ»ÏÂÕâ¸ö¸ÅÂÊ¡£¼ÙÉèѧУÀïÃæÈ˵Ä×ÜÊýÊÇ ¡£60%ÊÇÄÐÉú£¬¶øÇÒËûÃǶ¼´©³¤¿ã£¬ÓÚÊÇÎÒÃǵõ½ÁË´©³¤¿ãµÄÄÐÉúµÄÈËÊý£º £¬£¨ÆäÖÐ ÊÇÄÐÉúµÄ¸ÅÂÊ£¬¼´60%£¬ ÊÇÄÐÉú´©³¤¿ãµÄ¸ÅÂÊ£¬±»³ÆΪÌõ¼þ¸ÅÂÊ£¬ÕâÀïÊÇ100% £¬ÒòΪËùÓÐÄÐÉú¶¼´©³¤¿ã£©¡£40%µÄÅ®ÉúÀïÃæÓÖÓÐÒ»°ëÊÇ´©³¤¿ãµÄ£¬ÓÚÊÇÎÒÃÇÓֵõ½ÁË´©³¤¿ãµÄÅ®ÉúµÄÈËÊý£º £¬ ÊÇÅ®ÉúµÄ¸ÅÂÊ£¬ ÊÇÅ®Éú´©³¤¿ãµÄ¸ÅÂÊ£¬¼´50%¡£ÏÖÔھͿÉÒÔÀ´¼ÆËãÅöµ½Ò»¸ö´©³¤¿ãµÄѧÉúÊÇÅ®ÉúµÄ¸ÅÂÊÁË£¬¼´ £¨2-1£©×¢Òâµ½£¬ £¬ÎÒÃdzÆÆäΪÍ걸ʼþ×顣ʽ£¨2-1£©¼´³£ËµµÄ±´Ò¶Ë¹¶¨Àí£¬Óù淶»¯µÄÊýѧÓïÑÔÃèÊöΪ£ºÉèʼþ ¹¹³ÉһϵÁл¥²»ÏàÈݵÄÍ걸ʼþ×飬Ôò¶ÔÈÎÒâʼþ ÓÐ £¨2-2£©Ê½£¨2-2£©±»³ÆΪ±´Ò¶Ë¹¹«Ê½£¬ÆäÖÐ £¬ ±»³ÆΪÏÈÑé¸ÅÂÊ£¬ ±»³ÆΪºóÑé¸ÅÂÊ¡£±´Ò¶Ë¹¹«Ê½·´Ó³ÁËÏÈÑé¸ÅÂÊÏòºóÑé¸ÅÂʵÄת»¯¡£ÎªÁËÒýÈ뱴Ҷ˹ͳ¼ÆÄ£ÐÍ£¬½«Ê½£¨2-2£©¸ÄдΪËæ»ú±äÁ¿µÄÐÎʽ£º¼Ù¶¨¹Û²âµÄÑù±¾ £¬ÊÇÀ´×Ô´ø²ÎÊýµÄ×ÜÌå £¬²ÎÊý ¿ÉÒÔÊÇÏòÁ¿£¬×ÜÌåµÄ¸ÅÂÊÃܶÈÊÇ £¬±´Ò¶Ë¹Ñ§ÅÉ°Ñ ¿´³ÉÊÇÓë Ò»ÑùµÄ¾ßÓÐijÖÖ¸ÅÂÊ·Ö²¼µÄËæ»ú±äÁ¿£¬Òò´Ë£¬Ó¦°Ñ¾­µäͳ¼ÆÖÐµÄ ¿´³ÉÊÇÌõ¼þ¸ÅÂÊÃÜ¶È £¬¼´ÒÑÖª²ÎÊý ʱ×ÜÌå µÄÃܶȡ£ÕâÑù£¬Ö»ÒªÖªµÀ µÄÏÈÑé¸ÅÂÊ £¬¾Í¿ÉÒÔͨ¹ý³éÈ¡Ñù±¾¹Û²âÖµÀ´µÃµ½¶Ô µÄеÄÈÏʶ£¬Çó³öºóÑé¸ÅÂÊ £¬¼´ £¨2-3£©ÔÚ±´Ò¶Ë¹Í³¼ÆÄ£ÐÍÖУ¬¶ÔÓÚ²ÎÊý µÄÒ»ÇÐÍƶ϶¼ÊÇ´ÓÆäºóÑé·Ö²¼³ö·¢µÄ¡£2.2 ±´Ò¶Ë¹ÍƶϼÙÉè¹Û²â±äÁ¿ ·þ´Ó¸ÅÂÊÃܶÈΪ µÄ·Ö²¼ÐÎʽ£¬ Ϊ´ý¹À²ÎÊý¡£ÏÖÔÚÓÐ ×é¹Û²âÑù±¾ £¬ËüÃÇÖ®¼äÏ໥¶ÀÁ¢£¬Ôò µÄÁªºÏ¸ÅÂÊÃܶÈΪ £¨2-4£©ÕâÒ»¸ÅÂÊËæ µÄÈ¡Öµ¶ø±ä»¯£¬ËùÒÔËüÊÇ µÄº¯Êý¡£ ±»³ÆΪÑù±¾µÄËÆÈ»º¯Êý¡£×î´óËÆÈ»¹À¼Æ·¨ÊÇÒ»ÖÖÔÚÒÑ֪ģÐ͵ÄһЩÑù±¾Êý¾ÝµÄÇé¿öÏÂÇó²ÎÊý×î¿ÉÄÜÈ¡ÖµµÄ·½·¨¡£Í¨¹ý¶ÔËÆÈ»º¯ÊýÇó¹ØÓÚ µÄÆ«µ¼Êý£¬¿ÉÒԵõ½ µÄ¹À¼ÆÖµ ¡£ÓÖÒòΪ Óë ÔÚͬһ ´¦È¡µ½¼«Öµ£¬ËùÒÔͨ³£½«ËÆÈ»º¯Êýд³É¶ÔÊýÐÎʽ ¡£±´Ò¶Ë¹ÀíÂ۵Ĺ۵ãÊÇÕâÑùµÄ²ÎÊý ·þ´Óijһ¸ÅÂÊÃܶȺ¯Êý £¬ÓÃÀ´±íʾÔÚÈ¡µÃÊý¾Ý֮ǰ£¬¶Ô È¡ÖµµÄ²Â²â£¬¼´Í¨³£Ëù˵µÄÏÈÑé·Ö²¼¡£ÀûÓñ´Ò¶Ë¹¶¨Àí£¬½«Êý¾ÝºÍ²ÎÊýµÄ·Ö²¼ÁªºÏÆðÀ´£¬ÔòÓÐ £¨2-5£©Ê½£¨2-5£©ÖеķÖĸÏî±»³ÆΪ¹éÒ»»¯³£Êý£¬¸Ã³£Êý¾­³£±»ºöÂÔ£¬ÒòΪÎÒÃǹØÐĵÄÊDz»Í¬²ÎÊýÖ®¼äµÄ±È½Ï£¬ËùÒÔ £¨2-6£©¿ÉÒÔ¿´µ½£¬ºóÑé·Ö²¼Æäʵ¾ÍÊÇÓÉËÆÈ»º¯ÊýºÍÏÈÑé·Ö²¼µÄ³Ë»ý¾ö¶¨µÄ¡£Õâ¸öºóÑé·Ö²¼ ÊǽøÐвÎÊý µÄµã¹À¼ÆµÄ³ö·¢µã¡£±´Ò¶Ë¹µã¹À¼ÆÓÐÒÔÏÂÈýÖÖ·½·¨[7]£º1£®×î´óºóÑé¹À¼ÆʹºóÑé·Ö²¼ ´ïµ½×î´óÖµµÄµã ±»³ÆΪ µÄ×î´óºóÑé¹À¼Æ¡£¼´ £¨2-7£©2. ºóÑé¾ùÖµ¹À¼ÆºóÑé·Ö²¼ µÄ¾ùÖµ ±»³ÆΪ µÄºóÑé¾ùÖµ¹À¼Æ£¬¼´ £¨2-8£©3. ºóÑéÖÐλÊý¹À¼ÆºóÑé·Ö²¼ µÄÖÐλÊý ±»³ÆΪ µÄºóÑéÖÐλÊý¹À¼Æ¡£ÏÈÑé·Ö²¼¸ÅÀ¨ÁËʵÑéÇ°¶Ô µÄÈÏʶ£¬¶øÔڵõ½Ñù±¾¹Û²âÖµ ºó£¬ÈÏʶÆðÁ˱仯£¬Õâ·´Ó³ÔÚ±´Ò¶Ë¹¹«Ê½ÖУ¬ºóÑé·Ö²¼×ÛºÏÁË µÄÏÈÑéÐÅÏ¢ÓëÑù±¾¹Û²âÖµÌṩµÄÓÐ¹Ø µÄÐÅÏ¢£¬ÊDZ´Ò¶Ë¹Í³¼ÆÍƶϵĻù´¡£¬ËùÒÔ±´Ò¶Ë¹ÍƶϵÄÔ­ÔòÊǶԲÎÊý Ëù×öÈκÎÍƶϱØÐë»ùÓÚÇÒÖ»ÄÜ»ùÓÚ µÄºóÑé·Ö²¼¡£2.3 ÏÈÑé·Ö²¼µÄÑ¡ÔñÓÉÉÏÒ»½Ú¿ÉÖª£¬²ÎÊý µÄºóÑé·Ö²¼ÓÉÁ½²¿·Ö×é³É£ºËÆÈ»º¯ÊýºÍÏÈÑé·Ö²¼¡£ÆäÖÐËÆÈ»º¯ÊýÊÇÄ£Ð͸ø¶¨µÄ£¬¶øÏÈÑé·Ö²¼ÔòÊÇÈËΪÉ趨µÄ£¬ÕâÀïÈ˵ÄÒâÔ¸ÆðÖ÷µ¼×÷Ó㬿ɼû£¬¸ø³öÒ»¸öºÏÀíµÄÏÈÑé·Ö²¼£¬¶Ô²ÎÊýµÄÇó½âÊ®·ÖÖØÒª¡£ÏÂÃæ½éÉܼ¸ÖÖ³£ÓõÄÏÈÑé·Ö²¼£º2.3.1 ¿Í¹ÛÏÈÑé·Ö²¼£¨Objective Prior£©¿Í¹ÛÏÈÑé·Ö²¼ÊÇÖ¸ÔÚÎÞÏÈÑé֪ʶ¿ÉÓõÄÇé¿öÏ£¬¶Ô²ÎÊýµÄÏÈÑé·Ö²¼×ö³öµÄÒ»ÖÖ¼ÙÉè¡£ÓÉÓÚÎÒÃÇ¶Ô ÊÇÎÞÖªµÄ£¬ËùÒÔÈÏΪÔÚÆäÈ¡Öµ·¶Î§ÄÚµÄÿһµã¶¼ÊǵȸÅÂʵģ¬·þ´ÓÒ»ÖÖ¡°¾ùÔÈ·Ö²¼¡±¡£¼´¼ÙÉè £¨2-9£©µ± ΪÎÞ½çÇøÓòʱ£¬ ²»ÊÇͨ³£ÒâÒåϵĸÅÂÊ·Ö²¼£¬Îª´ËÐèÒªÒý½ø¹ãÒåÏÈÑé·Ö²¼µÄ¸ÅÄî¡£³ÆÂú×ãÏÂÃæÁ½Ê½µÄ·Ö²¼Îª¹ãÒåÏÈÑé·Ö²¼[7]£º£¨1£© £¨2-10£©£¨2£© £¨2-11£©ÐèҪעÒ⣬°´Ê½£¨2-10£©µÄ¶¨Ò壬 ²¢²»ÊÇͨ³£ÒâÒåϵĸÅÂÊ·Ö²¼£¬µ«ÓÉÓÚʽ£¨2-11£©µÄ³ÉÁ¢£¬Òò´ËÀàËÆÓÚʽ£¨2-5£©ËùÈ·¶¨µÄºóÑé·Ö²¼ ÊÇ´æÔڵġ£µ± Âú×ãʽ£¨2-9£©ÇÒΪ¹ãÒåÏÈÑé·Ö²¼Ê±£¬ ³ÆΪ¹ãÒå¾ùÔÈ·Ö²¼¡£ÓÉÓÚ £¨2-12£©´Ó¶ø£¬µ± Ϊ¹ãÒå¾ùÔÈ·Ö²¼Ê±£¬ÓÐ £¨2-13£©¼´ËÆÈ»º¯ÊýÊǺóÑé·Ö²¼µÄºËÐÄ¡£Ê½£¨2-13£©¿ÉÒÔ¿´×öÊDZ´Ò¶Ë¹¼ÙÉèϵĺóÑé·Ö²¼ÐÎʽ¡£Í¨³£Ò²³ÆÂú×㱴Ҷ˹¼ÙÉèµÄÏÈÑé·Ö²¼Îª¡°ÎÞÐÅÏ¢ÏÈÑé·Ö²¼¡±£¨non-informative prior£©¡£2.3.2 Ö÷¹ÛÏÈÑé·Ö²¼£¨Subjective Prior£©Ö÷¹ÛÏÈÑé·Ö²¼Êǽ«ÏÈÑéÐÅÏ¢¾¡¿ÉÄܶàµÄѹËõ½øÏÈÑé·Ö²¼ÖС£ÏÈÑéÐÅÏ¢¿ÉÄÜÊÇÓÉÒÔÍùµÄ¾­ÑéºÍרҵ֪ʶ»ñµÃµÄ¡£ÏÂÃæ½éÉÜÒ»ÖÖÖØÒªµÄÖ÷¹ÛÏÈÑé·Ö²¼¡ª¡ª¹²éîÏÈÑé·Ö²¼£¨Conjugate prior£©¡£Ò»¸öÏÈÑé·Ö²¼Èç¹ûÊǹ²éîÏÈÑé·Ö²¼£¬ÔòËüµÄÓÉËÆÈ»º¯Êý ºÍÏÈÑé·Ö²¼ Ïà³ËµÃµ½µÄºóÑé·Ö²¼ ºÍÏÈÑé·Ö²¼¾ßÓÐͬÑùµÄÐÎʽ¡£±ÈÈ磬±äÁ¿ ·þ´Ó¶þÏî·Ö²¼ £¬ µÄÏÈÑé·Ö²¼Ñ¡Îª ·Ö²¼£¬¼´ £¨2-14£©ÓÖ²ÎÊýµÄËÆÈ»º¯ÊýΪ £¨2-15£©ËùÒÔ£¬ µÄºóÑé·Ö²¼Îª £¨2-16£©ÉÏʽÓÒ¶ËÊÇ ·Ö²¼µÄºË£¬¹Ê £¨2-17£©¿É¼û µÄÏÈÑé·Ö²¼ÓëºóÑé·Ö²¼¾ßÓÐÏàͬµÄ·Ö²¼ÐÎʽ£¬Ö»ÊǺóÑé·Ö²¼¶Ô²ÎÊý½øÐÐÁ˸üС£¹²éîÏÈÑé·Ö²¼ÒªÇó ÌṩµÄÐÅÏ¢ÓëÑù±¾·Ö²¼ ÌṩµÄÐÅÏ¢×ÛºÏÒԺ󣬲»¸Ä±ä µÄ·Ö²¼¹æÂÉ¡£ÕâʵÖÊÉÏÊÇÈÏΪÔÚÍÆ¶Ï µÄ·Ö²¼Ê±ÓÉÏÈÑé·Ö²¼ÌṩµÄÐÅÏ¢ÊÇÖ÷ÒªµÄ¡£¹²éîÏÈÑé·Ö²¼Ö»´æÔÚÓÚÖ¸Êý×åÄ£ÐÍÖУ¬³£¼ûµÄ¸ß˹·Ö²¼£¬¶þÏî·Ö²¼£¬Ù¤Âí·Ö²¼µÈ¶¼ÊÇÖ¸Êý×åÄ£ÐÍ¡£2.3.3 ·Ö²ãÏÈÑé·Ö²¼£¨Hierarchical Prior£©ÔÚ½øÐб´Ò¶Ë¹ÍƶϵÄʱºò£¬Ê×ÏȼÙÉèÒ»¸ö²ÎÊýµÄÏÈÑé·Ö²¼£¬ÀýÈ磬¼ÙÉè ·þ´ÓÙ¤Âí·Ö²¼£¬¼´ £¬ÕâʱÓÖÒýÈëÁËÁ½¸öеIJÎÊý ºÍ ¡£Èç¹û½«ÕâÁ½¸ö²ÎÊýÒ²¿´×÷Ëæ»ú±äÁ¿£¬¸øËüÃǼÓÉÏijÖÖ·Ö²¼ÐÎʽ£¬ ºÍ £¬ÕâÑù¾Í¿ÉÒÔͨ¹ý ºÍ À´¿ØÖÆ µÄ·Ö²¼£¬Õâ¾ÍÊÇÒ»¸ö·Ö²ãÏÈÑé·Ö²¼µÄÄ£ÐÍ¡£ÆäÖвÎÊý ºÍ ±»³ÆΪ³¬²ÎÊý£¨Hyperparameter£©¡£ÀàËƵģ¬»¹¿ÉÒÔÒýÈëеIJÎÊý À´¿ØÖÆ ºÍ £¬Õâ¸ö¹ý³Ì¿ÉÒÔ²»¶ÏµØÖظ´£¬Ö±µ½Ä³Ò»¸ö²ÎÊýµÄÏÈÑé·Ö²¼²»ÔÙÒÀÀµÆäËûµÄ²ÎÊýΪֹ¡£2.4 С½áÕâÒ»½Ú½éÉÜÁ˱´Ò¶Ë¹Í³¼ÆÍƶϵÄһЩ»ù´¡ÖªÊ¶£¬°üÀ¨±´Ò¶Ë¹¶¨Àí£¬²ÎÊýµÄ¹À¼ÆºÍ¼¸ÖÖ³£ÓÃÏÈÑé·Ö²¼µÄÑ¡È¡¡£±´Ò¶Ë¹¶¨ÀíÊDZ´Ò¶Ë¹Í³¼ÆÍƶϵĺËÐÄ£¬¶ø²ÎÊýµÄÏÈÑé·Ö²¼µÄÕýÈ·Ñ¡ÔñÊDzÎÊý¹À¼ÆºÏÀíÐԵĹؼü¡£ µÚÈýÕ Òò×Ó·ÖÎö·¨3.1 ÒýÑÔÔÚ¸÷¸öÁìÓòµÄ¿ÆѧÑо¿ÖУ¬ÍùÍùÐèÒª¶Ô·´Ó³ÊÂÎïµÄ¶à¸ö±äÁ¿½øÐдóÁ¿µÄ¹Û²â£¬ÊÕ¼¯´óÁ¿Êý¾ÝÒÔ±ã½øÐзÖÎöÑ°ÕÒ¹æÂÉ¡£¶à±äÁ¿´óÑù±¾ÎÞÒÉ»áΪ¿ÆѧÑо¿Ìṩ·á¸»µÄÐÅÏ¢£¬µ«Ò²ÔÚÒ»¶¨³Ì¶ÈÉÏÔö¼ÓÁËÊý¾Ý²É¼¯µÄ¹¤×÷Á¿£¬¸üÖØÒªµÄÊÇÔÚ´ó¶àÊýÇé¿öÏ£¬Ðí¶à±äÁ¿Ö®¼ä¿ÉÄÜ´æÔÚÏà¹ØÐÔ¶øÔö¼ÓÁËÎÊÌâ·ÖÎöµÄ¸´ÔÓÐÔ£¬Í¬Ê±¶Ô·ÖÎö´øÀ´²»±ã¡£Èç¹û·Ö±ð·ÖÎöÿ¸öÖ¸±ê£¬·ÖÎöÓÖ¿ÉÄÜÊǹÂÁ¢µÄ£¬¶ø²»ÊÇ×ۺϵġ£Ã¤Ä¿¼õÉÙÖ¸±ê»áËðʧºÜ¶àÐÅÏ¢£¬ÈÝÒײúÉú´íÎóµÄ½áÂÛ¡£Òò´ËÐèÒªÕÒµ½Ò»¸öºÏÀíµÄ·½·¨£¬¼õÉÙ·ÖÎöÖ¸±êµÄͬʱ£¬¾¡Á¿¼õÉÙÔ­Ö¸±ê°üº¬ÐÅÏ¢µÄËðʧ£¬¶ÔËùÊÕ¼¯µÄ×ÊÁÏ×÷È«ÃæµÄ·ÖÎö¡£ÓÉÓÚ¸÷±äÁ¿¼ä´æÔÚÒ»¶¨µÄÏà¹Ø¹Øϵ£¬Òò´ËÓпÉÄÜÓýÏÉÙµÄ×ÛºÏÖ¸±ê·Ö±ð×ۺϴæÔÚÓÚ¸÷±äÁ¿Öеĸ÷ÀàÐÅÏ¢¡£Òò×Ó·ÖÎö¾ÍÊÇÕâÑùÒ»ÖÖ½µÎ¬µÄ·½·¨¡£3.2 Òò×Ó·ÖÎö·¨Òò×Ó·ÖÎö£¨Factor Analysis, FA£©ÊǶàԪͳ¼Æ·ÖÎöÖеÄÒ»ÖÖÖØÒª·½·¨, ×îÔçÓÉÓ¢¹úÐÄÀíѧ¼ÒC.E.˹Ƥ¶ûÂüÌá³ö£¬ÆäÖ÷ҪĿµÄÊÇÓÃÀ´ÃèÊöÒþ²ØÔÚÒ»×é¹Û²âµ½µÄ±äÁ¿ÖеÄһЩ¸ü»ù±¾µÄ£¬µ«ÓÖÎÞ·¨Ö±½Ó²âÁ¿µ½µÄÒþÐÔ±äÁ¿£¨Hidden Variable£©¡£Òò×Ó·ÖÎöÀûÓýµÎ¬µÄ˼Ï룬´ÓÑо¿Ô­Ê¼±äÁ¿Ïà¹Ø¾ØÕóÄÚ²¿½á¹¹³ö·¢, °ÑһЩ´í×Û¸´ÔӵıäÁ¿¹é½áΪÉÙÊý¼¸¸ö×ÛºÏÒò×Ó¡£Æä»ù±¾Ë¼Â·ÊǸù¾ÝÏà¹ØÐÔ´óС½«±äÁ¿·Ö×飬ʹµÃͬ×éÄڵıäÁ¿Ö®¼äµÄÏà¹ØÐԽϸߣ¬²»Í¬×éÄڵıäÁ¿¼äµÄÏà¹ØÐԽϵ͡£Ã¿×é±äÁ¿´ú±íÒ»¸ö»ù±¾½á¹¹£¬ÓÃÒ»¸ö²»¿É¹Û²âµÄ×ۺϱäÁ¿±íʾ£¬Õâ¸ö»ù±¾½á¹¹³ÆΪ¹«¹²Òò×Ó¡£¶ÔÓÚËùÑо¿µÄÎÊÌâ¾Í¿ÉÒÔÓÃ×îÉÙ¸öÊýµÄ²»¿É¹Û²âµÄ¹«¹²Òò×ÓµÄÏßÐÔº¯ÊýÓëÌØÊâÒò×ÓÖ®ºÍÀ´ÃèÊö¹Û²âµ½µÄÿ¸ö·ÖÁ¿¡£±ÈÈ磬ij¹«Ë¾ÀÏ°å¶ÔӦƸÕß½øÐÐÃæÊÔ£¬²¢¸ø³öËûÃÇÔÚ15¸ö·½ÃæËùµÃµÄ·ÖÊý£¬Õâ15¸ö·½ÃæÊÇ£ºÉêÇëÊéµÄÐÎʽ£¨ £©£¬Íâò£¨ £©£¬×¨ÒµÄÜÁ¦£¨ £©£¬ÌÖÈËϲ»¶£¨ £©£¬×ÔÐÅÐÄ£¨ £©£¬¾«Ã÷£¨ £©£¬³Ïʵ£¨ £©£¬ÍÆÏúÄÜÁ¦£¨ £©£¬¾­Ñ飨 £©£¬»ý¼«ÐÔ£¨ £©£¬±§¸º£¨ £©£¬Àí½âÄÜÁ¦£¨ £©£¬Ç±Á¦£¨ £©£¬½»¼ÊÄÜÁ¦£¨ £©£¬ÊÊÓ¦ÐÔ£¨ £©¡£Í¨¹ýÒò×Ó·ÖÎö£¬Õâ15¸ö·½Ãæ¿ÉÒÔ¹é½áΪӦƸÕßµÄÍâ¶ÄÜÁ¦£¨ £©¡¢¾­Ñ飨 £©¡¢ÌÖÈËϲ»¶µÄ³Ì¶È£¨ £©¡¢×¨ÒµÄÜÁ¦£¨ £©ºÍÍâò£¨ £©ÕâÎå¸öÒò×Ó¡£ËäÈ»Òò×Ó·ÖÎö×îÔçÓÉÐÄÀíѧ¼ÒÌá³ö£¬µ«ÒòΪÆä¾ßÓнµÎ¬µÄÌØÐÔ£¬ÏÖÔÚÒѾ­¹ã·ºÓ¦ÓÃÓÚÈËÁ³Ê¶±ð¡¢ ÓïÒôʶ±ð¡¢ Web Îı¾ÌØÕ÷ÌáÈ¡¡¢ Éç»áµ÷²é¡¢ ÐÄÀí·ÖÎöºÍ½ÌÓýÆÀ¹ÀµÈÖî¶àÁìÓò¡£3.1.1 Òò×Ó·ÖÎöÄ£ÐÍ[8]Éè ÊÇ Î¬¿É¹Û²âµÄËæ»ú±äÁ¿£¬Æä¾ùֵΪ £¬Ð­·½²îΪ £» ÊÇ Î¬²»¿É¹Û²âµÄËæ»ú±äÁ¿£¬Æä¾ùֵΪ £¬Ð­·½²îΪ £»Í¨³£ÓÐ ¡£ ÊÇÓë »¥²»Ïà¹ØµÄ ά²»¿É¹Û²âµÄËæ»ú±äÁ¿£¬ÇÒÓоùֵΪ £¬Ð­·½²îΪ £»ÔòÒò×Ó·ÖÎöµÄÒ»°ãÄ£ÐÍΪ £¨3-1£©½«ÉÏʽд³É¾ØÕóÐÎʽΪ £¨3-2£©ÆäÖÐ ±»³ÆΪ µÄ¹«¹²Òò×Ó£¬ ±»³ÆΪ µÄÌØÊâÒò×Ó£»Ä£ÐÍÖеľØÕó ÊÇ´ý¹À¼ÆµÄϵÊý¾ØÕ󣬱»³ÆΪÒò×ÓÔغÉÕó¡£ÕâÀïÓÐÁ½¸öÖØÒªµÄ¼ÙÉ裺£¨1£©ÌØÊâÒò×ÓÖ®¼ä»¥²»Ïà¹Ø£¬ÇÒÓÐ £»£¨2£©ÌØÊâÒò×Ӻ͹«¹²Òò×ÓÖ®¼ä»¥²»Ïà¹Ø£¬¼´ ¡£3.1.2 Òò×Ó·ÖÎöÄ£Ð͵ÄÐÔÖÊ[8]1. µÄЭ·½²î µÄ·Ö½â £¨3-3£©2. Òò×ÓÔغÉÕó µÄͳ¼ÆÒâÒ壨1£© µÄÔªËØ £¨3-4£©¿É¼û ÖÐÔªËØ ¿Ì»­Á˱äÁ¿ Óë Ö®¼äµÄÏà¹ØÐÔ£¬³ÆΪ ÔÚ ÉϵÄÒò×ÓÔغɡ£ Ô½´ó£¬ËµÃ÷ ¶Ô µÄÓ°ÏìÔ½´ó¡££¨2£© µÄÐÐÔªËصÄƽ·½ºÍ ÒòΪ µÄ·½²îΪ £¨3-5£©¿ÉÒÔ¿´µ½£¬ µÄ·½²îÓÉÁ½²¿·Ö×é³É£º ÊÇÈ«²¿¹«¹²Òò×Ó¶Ô±äÁ¿ µÄ×Ü·½²îËù×öµÄ¹±Ï×£¬Ëü·´Ó³Á˹«¹²Òò×Ó¶Ô µÄÓ°Ï죬³ÆΪ¹²ÐÔ·½²î£» ÊÇÌØÊâÒò×Ó ¶Ô µÄ·½²î¹±Ï×£¬³ÆΪÌØÊâ·½²î¡££¨3£© µÄÁÐÔªËصÄƽ·½ºÍ ÒòΪ £¨3-6£©ÆäÖÐ £¬ Êǹ«¹²Òò×Ó ¶Ô µÄ×Ü·½²î¹±Ï×£¬Ëü·´Ó³ÁË ¶Ô µÄÓ°Ï죬ÊǺâÁ¿¹«¹²Òò×Ó ÖØÒªÐÔµÄÒ»¸ö³ß¶È¡£ÏÔÈ»£¬ Ô½´ó£¬±íÃ÷ ¶Ô µÄ¹±Ï×Ô½´ó¡££¨4£©Òò×ÓÐýתÒò×ÓÔغÉÕó ÊDz»Î¨Ò»µÄ£¬Éè ΪÈÎÒâ Õý½»¾ØÕó£¬Áî £¬ £¬ÔòÓÐ £¨3-7£© £¨3-8£© £¨3-9£©ËùÒÔÓÐ £¨3-10£©Òò´Ë¿ÉÒÔͨ¹ý¸øÒò×ÓÔغÉÕóÓÒ³ËÒ»¸öÕý½»¾ØÕó £¬À´ÐýתÒò×Ó£¬Ê¹ÐµÄÒò×ÓÓиüºÃµÄÒâÒå¡£3.3 Òò×Ó·ÖÎöÓëÖ÷³É·Ö·ÖÎöµÄ±È½ÏÖ÷³É·Ö·ÖÎöÊÇ£¨Principal Component Analysis£¬PCA£©Ò»ÖÖÓëÒò×Ó·ÖÎöÀàËƵĽµÎ¬·½·¨£¬Æä»ù±¾ÀíÂÛÊÇͨ¹ý¶Ôԭʼ¹Û²â±äÁ¿½øÐÐÏßÐÔ×éºÏ´Ó¶øµÃµ½Ö÷·ÖÁ¿¡£Éè ÊÇ Î¬Ëæ»ú±äÁ¿£¬¾ùֵΪ £¬Ð­·½²îΪ ¡£¶Ô ½øÐÐÏßÐԱ任ÓÐ £¨3-11£©½«ÉÏʽд³É¾ØÕóÐÎʽΪ £¨3-12£©ÎÒÃÇÏ£ÍûÑ°ÕÒÒ»×éеıäÁ¿ £¬Õâ×éеıäÁ¿ÒªÇó³ä·ÖµØ·´Ó³Ô­±äÁ¿µÄÐÅÏ¢£¬¶øÇÒÏ໥¶ÀÁ¢¡£ÎÒÃÇÖªµÀ£¬µ±Ò»¸ö±äÁ¿Ö»È¡Ò»¸öÊý¾Ýʱ£¬Õâ¸ö±äÁ¿£¨Êý¾Ý£©ÌṩµÄÐÅÏ¢Á¿ÊǷdz£ÓÐÏ޵ģ¬µ±Õâ¸ö±äÁ¿È¡Ò»ÏµÁв»Í¬Êý¾Ýʱ£¬¿ÉÒÔ´ÓÖжÁ³ö×î´óÖµ¡¢×îСֵ¡¢Æ½¾ùÊýµÈÐÅÏ¢¡£±äÁ¿µÄ±äÒìÐÔÔ½´ó£¬ËµÃ÷Ëü¶Ô¸÷ÖÖ³¡¾°µÄ¡°±éÀúÐÔ¡±Ô½Ç¿£¬ÌṩµÄÐÅÏ¢¾Í¸ü¼Ó³ä·Ö£¬ÐÅÏ¢Á¿¾ÍÔ½´ó¡£Ö÷³É·Ö·ÖÎöÖеÄÐÅÏ¢£¬¾ÍÊÇÖ¸±êµÄ±äÒìÐÔ£¬Óñê×¼²î»ò·½²î±íʾËü¡£´ÓÏßÐδúÊýµÄ½Ç¶ÈÀ´¿´£¬PCAµÄÄ¿±ê¾ÍÊÇʹÓÃÁíÒ»×é»ùÈ¥ÖØÐÂÃèÊöµÃµ½µÄÊý¾Ý¿Õ¼ä¡£¶øеĻùÒªÄܾ¡Á¿½ÒʾԭÓеÄÊý¾Ý¼äµÄ¹Øϵ¡£ËüÊÇÒ»¸öÏßÐԱ任¡£Õâ¸ö±ä»»°ÑÊý¾Ý±ä»»µ½Ò»¸öеÄ×ø±êϵͳÖУ¬Ê¹µÃÈκÎÊý¾ÝͶӰµÄµÚÒ»´ó·½²îÔÚµÚÒ»¸ö×ø±êÖᣨ³ÆΪµÚÒ»Ö÷³É·Ö£©ÉÏ£¬µÚ¶þ´ó·½²îÔÚµÚ¶þ¸ö×ø±êÖᣨµÚ¶þÖ÷³É·Ö£©ÉÏ£¬ÒÀ´ÎÀàÍÆ£¬±£³ÖÊý¾Ý¼¯µÄ¶Ô·½²î¹±Ï××î´óµÄÌØÕ÷£¬ÕâÊÇͨ¹ý±£ÁôµÍ½×Ö÷³É·Ö£¬ºöÂԸ߽×Ö÷³É·Ö×öµ½µÄ¡£ÕâÑùµÍ½×³É·ÖÍùÍùÄܹ»±£ÁôסÊý¾ÝµÄ×îÖØÒª·½Ãæ¡£PCAµÄÖصãÔÚÓÚ×î´óÏ޶ȵÄÌåÏÖÔ­Êý¾ÝËù°üº¬µÄÐÅÏ¢£¬¶øFAÄ£Ð͵ÄÖصãÔÚÓÚ½âÊÍԭʼ±äÁ¿Ö®¼äµÄÄÚÔÚ¹Øϵ¡£Ö÷³É·Ö·ÖÎöµÄÒ»¸öÏÔÖøÌصãÊÇûÓжԹ۲âÊý¾Ý½¨Á¢¸ÅÂÊÄ£ÐÍ£¬Ä£ÐÍÖеIJÎÊýÊǹ̶¨µÄ£¬ÐèÒª¼ÆËãÊý¾ÝµÄЭ·½²î¾ØÕ󣬵±¹Û²âÁ¿µÄάÊýºÜ¸ßʱ£¬¼ÆËãÁ¿»áºÜ´ó¡£Òò´ËBishopºÍTippingÔÚ1999ÄêÌá³öÁ˸ÅÂÊÖ÷³É·Ö·ÖÎöÄ£ÐÍ[5]£¨Probabilistic Principal Component Analysis£¬PPCA£©¡£Í¬ÑùµÄ£¬Éè ÊÇ Î¬Ëæ»ú±äÁ¿£¬¾ùֵΪ £¬Ð­·½²îΪ ¡£ ÊÇ Î¬²»¿É¹Û²âµÄÒþ±äÁ¿£¬Í¨³£·þ´Ó¸ß˹·Ö²¼ £¬ÔòÓÐ £¨3-13£©ÆäÖРΪԭʼ±äÁ¿ÔÚ×ø±ê¿Õ¼äµÄ¸÷¸ö·½ÏòÉÏËù¼ÓµÄÔëÉù£¬Í¬Ñù·þ´Ó¸ß˹·Ö²¼ ¡£ÓÚÊÇ¿ÉÒÔͨ¹ý×î´óËÆÈ»¹À¼ÆµÃµ½Ä£ÐÍÖеIJÎÊýºÍÒþ±äÁ¿¡£¿ÉÒÔ¿´µ½£¬¸ÅÂÊPCAÓëFAÄ£ÐÍÔÚÐÎʽÉÏÊ®·ÖÏàËÆ£¬Ö»ÊÇÁ½ÕßÔÚÔëÉù±äÁ¿ÉÏÏÞÖƲ»Í¬¡£¸ÅÂÊPCAÄ£ÐÍÒªÇóËù¼ÓÔëÉùÔÚ¸÷¸ö·½ÏòÉÏÒ»Ö£¬¶øFAÄ£ÐÍÔòÉáÈ¥ÁËÕâÒ»ÏÞÖÆ¡£Æä´Î£¬PCAÄ£ÐͺÍPPCAÄ£ÐͶ¼ÒªÇó¹Û²â±äÁ¿×Ó¿Õ¼äµÄ»ùÊÇÕý½»µÄ£¬ÓÉÓÚÁã¾ùÖµ¸ß˹±äÁ¿Õý½»Ôò¶ÀÁ¢£¬Òò´Ë£¬ÕâÁ½ÖÖÄ£ÐÍÏàÓ¦µÄÒþ±äÁ¿¸÷ÔªËØÊÇÏ໥¶ÀÁ¢µÄ¡£µ«ÊÂʵÉÏ£¬¶ÀÁ¢±äÁ¿²»Ò»¶¨¶¼ÊÇÕý½»µÄ£¬¶ÀÁ¢µÄÌõ¼þÒª±ÈÕý½»¸üÈõ£¬FAÄ£ÐͲ¢²»ÒªÇó¹Û²â±äÁ¿µÄ×Ó¿Õ¼äµÄ»ùÏ໥Õý½»£¬Í¨¹ýËÆÈ»º¯Êý×î´ó±£Ö¤Òþ±äÁ¿¸÷ÔªËصĶÀÁ¢Í³¼ÆÐÔ£¬Í¬Ê±»¹·Å¿í¶ÔÔëÉùµÄÏÞÖÆ£¬²¢²»ÒªÇóÔëÉù±äÁ¿¸÷ÔªËصķ½²î¶¼Ïàͬ¡£Òò´Ë¾ßÓиüÆÕ±éµÄÊÊÓÃÐÔ[4]¡£3.4 С½áÕâÒ»Õ½éÉÜÁËÒò×Ó·ÖÎö·¨µÄ»ù±¾ÖªÊ¶ºÍÖ÷ÒªÐÔÖÊ£¬²¢ÓëÖ÷³É·Ö·ÖÎö×÷Á˱Ƚϡ£Òª¶Ô¹Û²âÊý¾Ý½øÐÐÒò×Ó·ÖÎö£¬Ê×ÏȾÍÒª¶ÔÒò×Ó·ÖÎöÄ£ÐÍÖеIJÎÊý½øÐйÀ¼Æ£¬²ÎÊý°üÀ¨Òò×ÓÔغÉÕó ºÍÌØÊâÒò×ӵķ½²î ¡£ÔÚÔËÓü«´óËÆÈ»·¨¹À¼Æʱ£¬ÓÉÓÚÄ£ÐÍÖдæÔÚ²»¿É¹Û²âµÄÒþ±äÁ¿£¬ÒªÕÒµ½Ê¹¶ÔÊýËÆÈ»º¯Êý´ïµ½×î´óÖµµÄ²ÎÊýÊÇÊ®·ÖÀ§Äѵģ¬ÔÚÏÂÃæµÄÒ»ÕÂÖУ¬½«½éÉÜÒ»ÖÖ²ÎÊý¹À¼ÆµÄÌرðËã·¨£¬EMËã·¨¡£ËüÔÚ½â¾öÄ£ÐÍÖдæÔÚÒþ±äÁ¿µÄ²ÎÊý¹À¼ÆÕâÒ»·½ÃæÊ®·ÖÓÐЧ¡£µÚÁùÕ ½áÊøÓïÒò×Ó·ÖÎö·¨×÷ΪһÖÖÓÐЧµÄÊý¾Ý½µÎ¬µÄ·½·¨£¬¹ã·ºµÄÓ¦ÓÃÓÚÈ˹¤ÖÇÄÜ£¬»úÆ÷ѧϰºÍģʽʶ±ðÁìÓò¡£ÓÉÓÚÄ£ÐÍÖдæÔÚ²»¿É¹Û²âµÄÒþ±äÁ¿£¬Ê¹µÃ²ÎÊýµÄ¹À¼Æ±È½Ï¸´ÔÓ¡£±¾ÎÄ´Ó±´Ò¶Ë¹ÀíÂÛ³ö·¢£¬ÔËÓñä·Ö±´Ò¶Ë¹EMËã·¨ÍƵ¼³öÒò×Ó·ÖÎöÄ£ÐÍÖвÎÊýºÍÒþ±äÁ¿µÄºóÑé·Ö²¼ÐÎʽ£¬²¢ÔÚÒ»¶¨³Ì¶ÈÉϽâ¾öÁËÒþ±äÁ¿Î¬ÊýµÄ×Ô¶¯È·¶¨ÎÊÌâ¡£ÏÖ½«±¾ÎĵÄÖ÷Òª¹¤×÷¹éÄÉÈçÏ£º*½éÉÜÁ˱´Ò¶Ë¹ÀíÂ۵Ļù´¡ÖªÊ¶£¬°üÀ¨±´Ò¶Ë¹¶¨Àí£¬±´Ò¶Ë¹¹À¼ÆºÍÏÈÑé·Ö²¼µÄÑ¡Ôñ¡£*½éÉÜÁËÒò×Ó·ÖÎöÄ£ÐÍ£¬²¢ÓëÖ÷³É·Ö·ÖÎö×÷¼òÒª±È½Ï¡£*ÔËÓñä·Ö±´Ò¶Ë¹Ëã·¨ÍƵ¼³öÁËÒò×Ó·ÖÎö·¨µÄ±´Ò¶Ë¹ºóÑé·Ö²¼¹«Ê½£¬²¢ÓÃMatlab±à³ÌʵÏÖ¡£ ±¾ÎÄÉдæÔÚһЩÎÊÌâûÓнâ¾ö£¬±ÈÈçËã·¨Ö»Äܽâ¾öÒþ±äÁ¿Î¬Êý½ÏÉÙµÄÊý¾Ý£¬ÇÒ³ÌÐò¶ÔÓÚ³õʼÌõ¼þµÄÉèÖùýÓÚÃô¸ÐµÈ£¬ÕâЩ½«ÔÚ½ñºóµÄÑо¿¹¤×÷Öеõ½½øÒ»²½µÄ½â¾ö¡£ Ö л±ÏÒµÉè¼Æ×÷Ϊ´óѧËÄÄêµÄ×îºóÒ»¿Î£¬½Ì»áÁËÎÒÐí¶à¶«Î÷¡£ÔÚ×ö±ÏÉèµÄÕ⼸¸öÔÂÀÎÒѧµ½µÄ²»Ö»ÊÇרҵÉϵÄÀíÂÛ֪ʶ£¬¸üÖØÒªµÄÊǽ¥½¥Ñ§»áÈçºÎÈ¥×öÒ»ÏîÑо¿£¬ÈçºÎ¶ÀÁ¢½â¾öÓöµ½µÄÎÊÌ⣬ÈçºÎ²éÔÄ×ÊÁϵȵȣ¬Îª½«À´Ñо¿Éú½×¶ÎµÄѧϰ×÷ÁË×¼±¸¡£µ±È»£¬²»ÄÜÍü¼ÇµÄÊÇÄÇЩһֱÔÚÎÒÉí±ß°ïÖúÎÒµÄÈË£¬ÎÒµÄÀÏʦ¡¢Í¬Ñ§ºÍÅóÓÑ£¬ÔÚ´Ë£¬ÏòËûÃÇÖÂÒÔ×î³ÏÖ¿µÄлÒâ©uÊ×ÏÈ£¬ÎÒÒªÖÔÐĵظÐлÎÒµÄÖ¸µ¼ÀÏʦ¶ÅÀ¼¸±½ÌÊÚ¡£ÔÚÕû¸öµÄ±ÏÉè¹ý³ÌÖУ¬¶ÅÀÏʦ¶ÔÓÚÎÒÔÚ¹¤×÷ÉÏÓöµ½µÄÎÊÌ⣬ʼÖÕ¸øÓèÄÍÐÄϸÖµĽ²½âºÍ°ïÖú£¬¾¡Ö°¾¡Ôð£¬ÎÒÄܹ»Ë³ÀûµÄÍê³É±ÏÒµÉè¼ÆÀë²»¿ªËýµÄ°ïÖú¡£Í¬Ê±ÎÒ»¹Òª¸ÐлÓëÎÒͬÔÚÒ»¸ö±ÏÉèС×éµÄÀîÖ¾Åô£¬çÃÏþÓîºÍÑÖÀ¥Í¬Ñ§£¬ÔÚ±ÏÉè¹ý³ÌÖÐÎÒÃÇ»¥Ïàѧϰ£¬Ï໥¹ÄÀø£¬¹²Í¬½ø²½¡£»¹ÓУ¬¸Ðл´óѧËÄÄêÓëÎÒ³¯Ï¦Ïà´¦µÄÎÒµÄÉáÓÑÃÇ£¬Ð»Ð»ËûÃÇËÄÄêÀ´´ø¸øÎҵĿìÀÖ£¬ËûÃÇÊÇÎÒÒ»ÉúµÄÅóÓÑ¡£×îºó£¬ÉîÉîµØ¸ÐлÎҵļÒÈ˶ÔÎҵĹØÐĺÍÖ§³Ö£¬ÎÒ»á¸ü¼ÓŬÁ¦µÄ£¡ ²Î¿¼ÎÄÏ×[1] M. J. Beal, Variational Algorithms for Approximate Bayesian Inference, Phd. Thesis, University College London (UCL), May 2003; [2] C. M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006;[3] F. B. Nielsen, Variational Approach to Factor Analysis and Related Models, Master Thesis, Technical University of Denmark, May 2004; [4] ¶ÅÀ¼, À×´ï¸ß·Ö±æ¾àÀëÏñÄ¿±êʶ±ð·½·¨Ñо¿, ²©Ê¿Ñ§Î»ÂÛÎÄ, Î÷°²µç×ӿƼ¼´óѧ, 2007Äê4ÔÂ.[5] C. M. Bishop, M. E. Tipping, Probabilistic Principle Component Analysis, September 27 1999;[6] M. J. Beal, Z. Ghahramani, The Variational Bayesian EM Algorithm for Incomplete Data: with Application to Scoring Graphical Model Structures, University College London (UCL), 2003;[7] ·¶½ð³É, ÷³¤ÁÖ, Êý¾Ý·ÖÎö, ±±¾©: ¿Æѧ³ö°æÉç, 2002;[8] ¸ß»Ûè¯, ʵÓÃͳ¼Æ·½·¨ÓëSASϵͳ, ±±¾©: ±±¾©´óѧ³ö°æÉç, 2001,10;[9] ±ßÕØì÷, ÕÅѧ¹¤, ģʽʶ±ð (µÚ¶þ°æ). ±±¾©: Ç廪´óѧ³ö°æÉç, 2000;[10] Àî²ýÀû, ÉòÓñÀû, ÆÚÍû×î´óËã·¨¼°ÆäÓ¦ÓÃ, ¼ÆËã»ú¹¤³ÌÓëÓ¦ÓÃ, 2008, 44(29). 61-63;[11] J. A. Bilmes, A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models, International Computer Science Institute, 1998, 4;[12] Àî²ýÀû, Àî˾¶«, »ùÓÚEMËã·¨µÄÒò×Ó·ÖÎöÖÐÒþ±äÁ¿µÄÌõ¼þ¸ÅÂÊÃܶȺ¯Êý, ÊýѧµÄʵ¼ùÓëÈÏʶ, 2009Äê7ÔÂ, 39¾í(14ÆÚ). 132-135
¾­¹ÜÖ®¼Ò¡°Ñ§µÀ»á¡±Ð¡³ÌÐò
  • ɨÂë¼ÓÈë¡°¿¼ÑÐѧϰ±Ê¼ÇȺ¡±
ÍƼöÔĶÁ
¾­¼ÃѧÏà¹ØÎÄÕÂ
±êÇ©ÔÆ
¾­¹ÜÖ®¼Ò¾«²ÊÎÄÕÂÍƼö