Understanding Belief Propagation and its Generalizations Jonathan

32

Transcript of Understanding Belief Propagation and its Generalizations Jonathan

Page 1: Understanding Belief Propagation and its Generalizations Jonathan

������������ ���������������������������! � "�! #�����!�$ ����%�&�'( ������� �����)* #����+��

,.-0/2143&5216/87�9;:*<>=@?A=2?B1CED�FHG

I�JLK�MONQPSR�T>URWVX R�Y[Z4NQ\]T4^�_a` CEb JSIcK&daeVc_&T>\fT>\]RSgHY"_hNQikj lWPaY

mn?BoAoA?B1qpsr"9�t2u�<L<4pv1q/CED�FHG

I�JLK�MONQPSR�T>URWVX R�Y[Z4NQ\]T4^�_a` CEb JSIcK&daew NQ_W_WY�R�x0gHY"_hNQikj lWP�Y

:*16?Bu"my<>?Az&z{ l}|6PcP�i~P w�X PaY"�4�4�Q_hN { lW\�_Wx6lh_�R�xqT D x4^a\�x6_h_hNQ\�x6^

� |6_*��_WZ4NQ_�U8�Hx6\��S_�NQ��\���V�P w�� _�NQ�6�QR�i�_WYeLK&eaJ�� � _hNQ�6�QR�i�_WYE`6����N}R�_WiVLU�_h\]���}gHlh�Wj |L����\kj R�laj \�i

�y�[�4�0����2�

� _�_h�>�6i]R�\�x�\�x�R��Q�>�QP�NQ\]R�i w R���|4\]P�x�|4P&U¡ ¢\�x w _hNQ_Wx4lW_W£v�4NQPaZ4i�_WY"�¤R�NQ\]��_!\�x����}R��Q\����Q\]lWR�i�6|cV>��\�lW�W`qlWP�Y"�6�>�Q_hN¥�>\���\�Pax.`0_hN�NQP�N�¦�lhP�N�NQ_Wlh�Q\�x4^�lhPLT4\�x6^"�Q|6_hP�N�Vc`6R�xqT b ��`6R�x6T+�Q|6_*�4NQ\�x>¦lW\��6i�_W�OZ0_W|6\�xqT��Q|6_Z0_Wi�\�_ w �4NQP��qR�^aR��Q\�Pax¨§©M�ªO«OR�i�^aP�NQ\¬�Q|4YE`aU­|6\�l}|"\���R�x"_�®"lW\�_Wxc�OU�RWV[�QP��Pai��S_[\]x w _hNQ_hx6lW_[�4NQPaZ4i�_WY"��Z6R���_&T¨Pax¨�6R�����\�x6^�i�Pcl&R�i�Y"_W���QR�^a_W�Wj � _*��N�VE�QP�T>_W�S_Wi�Pa�¯R�6x4\�°6_&T¤R��6�4NQPaR�l}|~`�U­\��Q|"_��6R�Y"�6i�_W�W`ax6P��}R��Q\�Pax~`SR�x6T¤^�N}R��4|6\�l&R�iqY"PLT4_Wi���Z0P�N�NQP&U'_&T w NQPaYR�i�i~�Q|4_HNQ_Wi�_W��R�xc�¥T4\���lW\��4i]\�x4_W�Wj� _�_h�>�6i]R�\�x¨�Q|6_[lWi�Pa��_¤lWP�x6x4_Wlh�Q\�Pax¯Z0_h�¢U'_W_Wx��Q|6_#M�ª8R�i�^aP�NQ\¬�Q|6Y±R�xqT��Q|6_[M'_h�Q|6_#R��>¦�4NQP&�>\�Y�R��Q\�Pax�P w ���}R��Q\����Q\�l&R�i.�6|cV>��\�lW�Wjc��x��6R�N��Q\�lW�4ifR�N&`LU�_H��|4P&U²�Q|qR���M�ª�l&R�x�Pax4i¬V�lWPax>¦�S_hNQ^�_¥�QP�R³°6�>_&T#�0P�\�xL�O�Q|qR��'\���R�i���P#R*���}R��Q\�P�xqR�N�V¤�0Pa\�xc��P w �Q|4_�MO_h�Q|6_�R��4�4NQP&�>\]Y�R��Q\�Pax�QP��Q|4_ w NQ_W_#_hx6_hNQ^�Vcj � |6\��HNQ_W���4i¬�*|6_Wi��4�H_h�>�qR�\�x¨�Q|4_[���4lWlW_W����_h�*P w �Q|6_[M�ª8R�i�^aP�NQ\��Q|4YE`R�x6T#_WxqR�Z6i�_W��lWPax4x6_Wl��Q\]P�x6���QP³Z0_Y�RaT>_�U­\¬�Q|¤�aR�NQ\]R��Q\�Pax6R�i0R��6�4NQPaR�l}|6_W���QP*R��6�4NQP&�>\�Y�R��Q_\�x w _�NQ_Wx6lW_�j� |6_ElhPax6x4_Wlh�Q\�Pax´P w M�ªµU­\¬�Q|´�Q|6_+M'_h�Q|4_¨R��6�>NQP&�4\�Y�R��Q\]P�x´R�i���P����6^a^�_W���Q��R¯U�R&V��QPlWPax4����NQ�6lh�#x6_hU¶Y"_W���QR�^�_!�6R�����\�x6^vR�i�^�P�NQ\¬�Q|6Y"�[ZqR���_WT�P�x�\]Y"�>NQP��S_WY"_Wxc�Q�³�QP¯M'_h�Q|4_a· �R��4�4NQP&�>\]Y�R��Q\�Pax�\�xL��NQPLT>�6lW_WT�ZcV�¸*\�¹L�6l}|6\R�xqT�P��Q|4_hNQ�Wj � |6_!x6_hUº^�_Wx6_�N}R�i�\�»W_&T´Z0_Wi�\�_ w

Page 2: Understanding Belief Propagation and its Generalizations Jonathan

�4NQP��qR�^aR��Q\�Paxµ§½¼³M'ª'«"R�i�^�P�NQ\¬�Q|6Y"��R�NQ_���\�^�x6\¬°ql&R�xc�Qi�V�Y"P�NQ_ER�lWlW�>N}R��Q_��Q|qR�x´P�N}T4\�xqR�N�VM�ª w P�N*��PaY"_��4NQP�Z6i�_WY"�Wj � _[\�i�i��6����N}R��Q_#|6P&Uy�QP�lWP�x6����NQ�4lh��¼³M�ª8R�i]^�P�NQ\¬�Q|6Y"��U­\��Q|¯RT4_h�}R�\�i]_WT+_h�4R�Y"�4i]_�j

¾ ¿4À¶ÁEÂÄÃyÂ�ÀÆÅ� � À�Ç Èyà �yÉ´Ê ¿�Å �µËÍÌÏΠǵ Ë�Ð

� _�U­\�i�i>Z0_OT4_h��lhNQ\�Z6\�x6^�R�x6T�_h�>�4ifR�\�x6\�x6^¥R�x[R�i�^�P�NQ\¬�Q|6YE`�l&R�i�i]_WT¯ �Z0_Wi�\]_ w �4NQP��qR�^aR��Q\�Pax4£#§©M�ªO«�`&U­|6\�l}|\��@���6�4�0Pa��_&T*�QP¥��Pai��S_¤ �\]x w _hNQ_hx6lW_W£��4NQP�Z6i�_WY"�W`�R��@i]_WR�����R��4�4NQP&�>\�Y�R��Q_Wi¬Vcj&��x w _�NQ_Wx6lW_O�4NQP�Z6i�_WY"�2lhPaY"_�6�!\�xEY�R�xcV�T4\¬Ñ�_hNQ_hxL�­��lh\]_hxL�Q\¬°6l³°q_Wi]T4�W`4��P"\¬�­\���x4P���Q|qR��¥���4NQ�4NQ\���\�x6^[�Q|qR���R#^�PcPLTER�i�^�P�NQ\¬�Q|6YÆ�QP��Pai��S_"���4l�|Ä�4NQPaZ4i�_WY"�*|qR��[Z0_W_hxvNQ_W�0_&R��Q_&T4i¬V¯NQ_h¦�T4\���lWP��S_hNQ_WT.j~�¢x w R�lh�&`2Pax6_"lWR�x���|4P�UÒ�Q|6R������6l}|R��4�qR�NQ_Wxc�Qi�V�T4\¬Ñ�_hNQ_Wxc�³Y"_��Q|6PLT4�HR��³�Q|4_ w P�N�UR�N}T>¦½ZqR�l�¹cU�R�N}TÄR�i�^aP�NQ\¬�Q|4YE`0�Q|6_�ÓH\¬�Q_hNQZ6\�R�i�^aP�NQ\¬�Q|4YE`\¬�Q_hN}R��Q\��S_'T4_WlhPLT4\�x6^�R�i�^aP�NQ\¬�Q|6Y"� w P�N�¼³R�i]i]R�^a_hN�lWPLT>_W�;R�x6T³�Q�4NQZ0PclWPLT>_W�W`�ª@_&R�NQik· �;Z0_Wi�\�_ w �>NQPa�6R�^SR��Q\�PaxR�i�^�P�NQ\¬�Q|6Y w P�NEM�RWVc_h��\fR�x²x6_h��U�P�NQ¹>�W`O�Q|6_�¸�R�i]Y�R�x´°qi¬�Q_hN&`�R�x6T��Q|6_E��N}R�x6� w _�N�¦�Y�R���NQ\¬�ÔR��6�4NQPaR�l}|\�xy�6|cV>��\�lW�¯R�NQ_�R�i�i[���0_WlW\]R�i¤lWR���_W�vP w �Q|6_�M'ªÕR�i�^�P�NQ\¬�Q|6YÖT4\���lWP��S_hNQ_WT×\�xÒT4\¬Ñ�_hNQ_Wxc�¨��lW\�_Wxc�Q\¬°qllWP�Y"Y#�6x4\¬�Q\]_h�Wj� _�U­\�i]iO_WY"�6|6R���\�»W_��Q|qR���R�Y#�4i��Q\¬¦�T4\���lW\��4i]\�x6R�N�V¯R��4�4NQPaR�l}|vV>\�_Wi]T4��\�Y"�0P�N��}R�xc�*\�x4��\]^�|c�Q�*\]xc�QP��Q|6_M�ª8R�i�^�P�NQ\¬�Q|6YEj � _[�Q|6_�NQ_ w P�NQ_#Z0_W^�\]x¨U­\¬�Q|ÄR���Y�R�i�i;���>NQ�S_hVEP w  �\]x w _hNQ_hx6lW_W£!�4NQP�Z6i�_WY"� w NQPaY��Q|6_b �}`alhPaY"�6�>�Q_hN��L\���\]P�x~`����}R��Q\����Q\�l&R�i0�4|cV4��\�lW�W`aR�x6T#T>\]^�\¬�}R�iqlWP�Y"Y#�4x6\�l&R��Q\�P�x6�;i�\¬�Q_hN}R��Q�>NQ_W�W`aU­|4\]l}|#U­\�i�iR�i���P[^a\��S_³�4�R�l}|qR�x6lW_¥�QP[\�xc��NQPLT4�4lW_��Q|4_³T>\�Ñ�_�NQ_Wxc��¹L\�xqT4�OP w ^�N}R��4|6\�l&R�i~Y"PLT>_Wi��'�Q|qR��R�NQ_H�6��_&T"�QPT4_h��lhNQ\�Z0_#�Q|6_W��_#�4NQPaZ4i�_WY"�Wj � |6\��³°6NQ������_Wl��Q\]P�xÄ\���R!Z6R���\�l#NQ_W�L\�_hUÒP w U�_Wi�i¬¦½¹>x4P&U­xÄY�R��Q_�NQ\fR�ik`2Z4�4�U�_¥|6P��0_¥�Q|6R��O\¬�'U­\�i�i0Z0_��4��_ w �4iq\�x"_Wi��6lW\]T6R��Q\�x6^��Q|6_¥T4_W_h�"��\]Y"\�i]R�NQ\¬��V"Z0_h�¢U'_W_Wx��4NQP�Z6i�_WY"��\�x��Q|4_W��_T4\¬Ñ�_hNQ_hxL�;°6_Wi]T4�Wj � |6_�NQ_&R�T4_hN�\�xc�Q_hNQ_W���Q_&T#\�x[P��Q|6_�N�NQ_h�>\�_hU­��P w �Q|4\���Y�R��Q_hNQ\]R�i>Y�RWV�U�R�xc�;�QP³lWP�x6���4i��§k¸*��l}|6\���l�|6R�x4^4`6Ø4NQ_hVc`4R�x6T G Pc_Wi�\�^a_�N*I�JaJ>K&«¥R�xqT¯§©Ø4NQ_hV¨K&e�eaÙS«�j

Ú 9 Ú Û#Ü :!Ý*7~Þ Ü!ßàß ÝHr�m�á¤â�ã¨7�¢x[�Q|6_ b �2i]\¬�Q_hN}R��Q�4NQ_a`@ �M�RWVc_W��\]R�x�x4_h�¢U'P�NQ¹L��£�R�NQ_­�>NQPaZqR�Z6i¬V��Q|6_Y"Pa�����0P��6�6i]R�NO��V4�0_P w ^�N}R��6|6\�l&R�iY"PLT4_hi[§©ª@_&R�NQi#K&e�ÙaÙLä � _Wx6��_hxyK&eae�åS«�j � |4_hV²R�NQ_¨�6��_&TÔ\�xÔ_h�>�0_hN��!��V>���Q_WY"��\�x>�aPai��L\]x4^��4NQP�Z6i�_WYT4P�Y�R�\�x6�Ä���6l}|¶R���Y"_&T4\�l&R�i�T4\]R�^�x6Pa��\��W`[Y�R��Òi�_&R�NQx4\�x6^4`[i]R�x4^a�qR�^a_Ô�4xqT>_hNQ���}R�x6T4\�x6^>`�|6_W�>NQ\]���Q\�l��_&R�NQl�|.`SR�xqT���PHPax.j � _OU­\�i�i>�}R�¹S_R���R�x#_��6R�Y"�6i�_O�Q|4_'Y"_&T>\�l&R�i>T4\]R�^ax6P���\����>NQPaZ4i]_hYEj { �6�4�0Pa��_O�Q|6R��U�_³U�R�xc�­�QP#lWP�x6����NQ�4lh��R#Y�R�l}|6\�x6_�U­|6\�l�|�U­\�i]i2R��>�QPaY�R��Q\�l&R�i�i�V!^a\��S_�T>\]R�^ax4Pa��_W� w P�NH�6R��Q\�_Wxc�Q�Wj6Ø6P�N_&R�l�|E�6R��Q\�_Wxc�&`LU�_³U­\�i�i2|6R��a_H��PaY"_#§k�0Pa����\�Z6i¬V!\]x4lWPaY"�4i�_h�Q_�«'\�x w P�NQY�R��Q\�P�x~`4���6l}|ER��­|4\]�­��V>Y"�4�QP�Y"�R�x6T+�Q_W���HNQ_h���6i¬�Q�W`�R�x6T+U�_�U�P��6i]T�i]\�¹S_��QPEæAçWè&é�ê�Q|6_��4NQP�ZqR�Z6\�i�\���V+�Q|6R��HR�^�\��S_Wx¯T4\���_&R���_[P�N³��_h�HP wT4\���_&R���_W�\��l&R��4��\�x6^�|6\�����V>Y"�4�QP�Y"�Wj � _HR�i���P¤R������6Y"_­�Q|qR���U�_�¹>x4P&Uy§k�4NQ_h���6Y�R�Z4i¬V w NQP�Y¶_h�>�0_hN��RaT>�L\]lh_�«~�Q|6_O���}R��Q\����Q\�l&R�i6T4_W�0_Wx6T4_Wx4lW\�_W�;Z0_h��U�_W_Wx[T4\¬Ñ�_hNQ_hxL����V>Y"�>�QPaY"�W`W�Q_W���;NQ_h���6i¬�Q�W`�R�xqT[T4\���_&R���_W�WjØ6P�N³_h�4R�Y"�4i�_a`0i�_h�³�4�HlWP�x6��\]T4_�NH�Q|6_�°ql��Q\]P�xqR�i*  b ��\]R�£!_h�4R�Y"�6i�_�P w G R��>NQ\¬�Q»W_Wx¯R�x6T { �6\�_W^�_Wi�|qR�i¬�Q_hN§ G R��4NQ\¬�Q»W_WxER�xqT { �4\]_h^a_Wi�|qR�i¬�Q_hN[KWeaÙ�ÙS«�`q��|4P&U­xE\�x+°q^��4NQ_#K�j�¢xvU�P�N}T>�W`2�Q|4_�ëL�6R�i�\¬�}R��Q\��S_"���}R��Q\]���Q\�l&R�iT4_h�0_WxqT>_Wx6lW\�_W����|4P�U­x�\�xÄ�Q|4\�����Y�R�i�i�M�RWVc_W��\]R�x�x6_h��U�P�NQ¹l&R�xEZ0_*T4_h��lhNQ\�Z0_&TER�� w Pai�i�P�U­�WìKaj b NQ_Wlh_Wxc�¥��NQ\����QP b ��\]R+§ b «'\]x4lhNQ_&R���_W�¥�Q|4_*l�|6R�x4lW_W�P w �Q�4Z0_hNQlW�4i]P���\���§ � «�jILj { Y"P�¹>\�x4^+§ { «'\��¥R[NQ\���¹ w R�l��QP�N w P�N¥Z0P��Q|+i��6x4^¤l&R�x4lW_hN�§ G «�R�xqT+Z>NQPax4l�|4\��Q\���§©M«�jd>j � |6_��4NQ_W��_hx6lW_�P w _W\¬�Q|6_�N¤§ D «­�Q�6Z0_�NQlW�6i�Pa��\��³P�N³i��6x4^"l&R�x4lW_hN*lWR�x�Z0_[T4_h�Q_Wl��Q_&TEZcV�R�x¨í¥¦½N}RWVNQ_W���6i¬��§kí*«�`>Z4�4��Q|6_Hí�¦©N}RWV+R�i�Pax6_*lWR�x6x4P���T4\����Q\�x6^��6\���|EZ0_h��U�_W_hx+�Q|6_WYEj

�4jHîHV>���6x4Pc_&R�§kî�«�§k��|4P�N��Qx6_h���HP w Z4NQ_WR��Q|0«­Y�RWVEZ0_�l&R��4��_&T¯ZcV+Z4NQPax4l�|4\¬�Q\]�[§©M«�`�P�NH_W\¬�Q|6_hN¤§ D «�Q�6Z0_hNQlh�6i�Pa��\��¥P�N¥i��6x6^[l&R�x4lW_hN&j

Page 3: Understanding Belief Propagation and its Generalizations Jonathan

t­?Bïqð2u�< Ú � |4_³°qlh�Q\�Pax6R�i¥  b ��\fR�£�M�RWVc_W��\]R�x�x4_h�¢U'P�NQ¹0j

D R�l}|vx4PLT4_[NQ_h�4NQ_W��_Wxc�Q�*R���R�NQ\]R�Z4i]_[�Q|qR��Hl&R�x¯Z0_#\�xvR!T>\���lhNQ_h�Q_#xL�4Y#Z0_hN�P w �0Pa����\�Z6i�_¤���}R��Q_W�Wj � _UNQ\¬�Q_Hñ0ò w P�N��Q|4_���R�NQ\]R�Z6i�_�NQ_W�>NQ_W��_Wxc�Q\�x6^��Q|6_HT4\¬Ñ�_hNQ_Wxc���0P�����\�Z6i�_H���}R��Q_W�P w x6PLT>_Hó�jL�¢x�RaT4T4\¬�Q\�Pax��QP�Q|6_Oëc�qR�i]\¬�}R��Q\]�a_'T>_W�0_WxqT>_Wx6lh\]_h�@T4_h��lhNQ\�Z0_&T*ZLVH�Q|6_�M�RWVc_W��\]R�x�x6_��¢U�P�NQ¹³^�N}R��6|.`&�Q|6_hNQ_OR�NQ_'ëL�6R�xc�Q\¬�Q\��S_���}R��Q\����Q\�l&R�i�NQ_hifR��Q\�Pax6��|4\��6�³�Q|qR��HU'_#R�����\]^�xv�QP�_&R�l�|¯R�N�NQP&Uµ\�x¨�Q|6_�^�N}R��6|.j b ����PclW\]R��Q_WT¨U­\¬�Q|¯_&R�l}|R�N�NQP&Un\��+R�lWP�xqT>\��Q\�P�xqR�i��4NQP�ZqR�Z4\�i]\¬��V.ì w P�N+_h�4R�Y"�6i�_a`�U�_�UNQ\¬�Q_�ô2§kñ�õ@ö ñ�÷q« w P�N��Q|6_�lhPaxqT>\¬�Q\]P�xqR�i�4NQP�ZqR�Z6\�i�\���V"P w R*�qR��Q\�_Wxc�|6R&�>\�x4^�i��4x6^�l&R�x4lW_hN^�\��S_Wx��Q|qR��'|6_�T4Pc_W�'P�N­T>Pc_W��x6P�����Y"P�¹S_ajLØ6P�N��Q|6\��i�\�x6¹0`2U'_��QRWV¯�Q|qR��[�Q|4_ { x6PLT>_�\����Q|6_� ��qR�NQ_hxL�Q£�P w �Q|6_ G x4PLT4_�Z0_Wl&R��6��_�ñ õ \���lWPax6T4\¬�Q\�Pax6R�i�i¬VT4_h�0_WxqT>_Wxc�'P�x�ñ ÷ j { P�Y"_­x6PLT4_h�'i�\�¹S_¥�Q|4_�î8x4PLT4_¥Y"\]^�|c�'|6R��a_�Y"P�NQ_­�Q|6R�x�Pax4_��6R�NQ_Wxc�&`c\�x"U­|6\�l}|l&R���_#U�_#T4_h°6x6_��Q|6_W\¬N*lWP�xqT>\��Q\�P�xqR�i��>NQPaZqR�Z6\�i�\¬�Q\]_h�*\]x��Q_hNQY"�HP w R�i�i;�Q|6_W\¬N*�6R�NQ_Wxc�Q�Wä0�Q|L�6�¥U�_[UNQ\¬�Q_ô@§kñ�ø�ö ñ0ùú�ñ0ûO« w P�N­�Q|6_*lWP�xqT4\¬�Q\�Pax6R�i.�4NQP�ZqR�Z4\�i]\¬��V+P w |qR&�L\�x6^"TLV4���4x6Pc_&RLjü P��Q_[�Q|6R��H�Q|4_#M'R&VS_W��\]R�xÄx4_h�¢U'P�NQ¹�T>_h°qx4_W�³R�xv\�x6T4_W�0_Wx6T4_Wx4lhVE����NQ�4lh�Q�4NQ_�ì0�Q|6_��4NQPaZ6R�Z4\]i�\¬�¢V��Q|6R��R#x4PLT4_*\��­\]x+Pax6_HP w \¬�Q�­���}R��Q_W�¥T4_h�0_WxqT>�­T4\¬NQ_Wlh�Qi¬V!Pax6i¬V!Pax��Q|6_*���}R��Q_h�¥P w \¬�Q�¥�6R�NQ_Wxc�Q�Wj6Ø6P�N­x4PLT4_W�i�\�¹S_ b P�N { �Q|6R��OT4P³x4P���|qR&�S_­R�xcV[�6R�NQ_Wxc�Q�W`�U�_\�xc��NQPLT4�6lh_¥�>NQPaZ6R�Z6\�i�\¬�Q\�_W�Oi�\�¹S_�ô@§kñ ÷ «2�Q|qR��OR�NQ_¥x4P��lWP�xqT4\¬�Q\�Pax4_&T+P�x�R�xcV!P��Q|6_�N�x4PLT4_W�Wj6�¢x!^a_Wx4_hN}R�ik`�R¤M'R&VS_W��\]R�x�x6_��¢U�P�NQ¹v§©R�xqT!�Q|4_³P��Q|4_hN¥^�N}R��6|6\�l&R�iY"PLT4_hi]���Q|qR��OU'_­lWPax4��\fT>_hN�«�\��OY"Pa�����6��_ w �6iq\ w \¬�'\��O���qR�NQ��_�`SU­|6\�l}|"Y"_&R�x4���Q|6R��OY"Pa���OP w �Q|6_x4PLT4_W�T4P#x6P��­|6R&�S_*R¤T>\�NQ_hlh�¥���}R��Q\����Q\�l&R�i;T>_W�0_Wx6T4_Wx4lW_aj�¢xHPa�4N2_h�4R�Y"�4i]_�`��Q|4_�P��S_�N�¦¢R�i�iSP�N� f�¢Pa\�xc�Q£��>NQPaZqR�Z6\�i�\¬�¢V�ô2§�ý�ñ.þ�«�ÿ�ô@§kñ��Oú�ñ�÷.ú�ñ���ú�ñ0õ@ú�ñ�û­ú�ñ0ù�ú�ñ���úQñ0ø«�Q|qR����Q|6_¥�qR��Q\�_Wxc�|qR�����PaY"_�lWPaY[Z6\�xqR��Q\]P�x�P w ��V>Y"�>�QPaY"�W`a�Q_h����¦½NQ_h���6i¬�Q�W`4R�xqT�T4\���_&R���_W�W`>\��@���6���'�Q|6_�4NQPLT>�6lh�­P w R�i�i��Q|6_*�4NQP�ZqR�Z6\�i�\��Q\�_W�¥P w �Q|6_³�6R�NQ_Wxc�¥x6PLT>_W�¥R�xqT!R�i�i~�Q|4_³lWP�xqT4\¬�Q\�Pax6R�i~�4NQPaZ6R�Z4\]i�\¬�Q\�_W�Wì

ô@§�ý�ñ.þ�«���ô2§kñ � «Bô@§kñ ÷ «Bô@§kñ � ö ñ � «Bô@§kñ õ ö ñ ÷ «Bô@§kñ û ö ñ ÷ «Bô2§kñ ù ö ñ õ úQñ � «Bô@§kñ ø ö ñ û ú�ñ ù «Bô2§kñ � ö ñ ù «²§�K�«C P�NQ_�^a_hx6_hN}R�i]i¬Vc`qR"M�RWVc_W��\]R�x¨x6_��¢U�P�NQ¹!\��HR"T4\¬NQ_Wlh�Q_&T+R�lhV>lWi�\�l[^�N}R��4|�P w�� N}R�xqT>PaYº��R�NQ\]R�Z4i�_W�Hñ0ò�Q|qR��¥T4_h°6x6_W�¥R��¢Pa\�xc�¥�4NQPaZ6R�Z4\]i�\¬�¢V w �6x6l��Q\]P�x

ô@§kñ�ú�ñ���ú� � � �ú}ñ��¥«�� ��ò��� ô2§kñ0ò�ö �����>§kñ0ò½«�« §½Ia«

U­|6_�NQ_������>§kñ0ò©«�T>_Wx6P��Q_W���Q|4_[���}R��Q_W�³P w �Q|6_��6R�NQ_Wxc�Q�HP w x4PLT4_�ó�`.R�xqTE\ w x6PLT>_[ó�|6R���x6P��qR�NQ_hxL�Q�W`U�_*�}R�¹S_ô@§kñ ò ö �����4§kñ ò «�«���ô@§kñ ò «�j0M�V´ ¢T4\¬NQ_Wlh�Q_&T�R�lhV>lWi�\�l�^�N}R��4|~` £¤U�_�Y"_&R�x+�Q|qR��¥�Q|4_�R�N�NQP&U­�³T>P

Page 4: Understanding Belief Propagation and its Generalizations Jonathan

x6P��¥i]PcP���R�NQP��6x6T+\�x�R#lhV>lWi�_��S\���\��­���Q\]i�i~�0Pa����\�Z6i�_*�Q|qR��¥�Q|4_³i�\�x6¹L� w P�NQY�R#i�PcP��EU­|6_Wx!Pax6_�\�^ax4P�NQ_W��Q|6_³R�N�NQP&U­�Wj �4N�^aPaR�i@U­\�i�i;Z0_*�QP�lhPaY"�6�>�Q_*lW_hN��}R�\]x"!$#�ê&%�æAç�#('4�>NQPaZ6R�Z6\�i�\¬�Q\�_W�Wj.Ø6P�NH_h�4R�Y"�4i]_�`6U�_�Y"\�^a|c�¥UR�xc��QPÔlWP�Y"�6�>�Q_Ä�Q|6_��>NQPaZqR�Z6\�i�\¬�¢V �Q|qR��vRÔ�6R��Q\�_Wxc�¯|qR��¯RÔlW_�N��}R�\�xµT4\���_&R���_aj�MOVn ¢\�x w _hNQ_Wx4lW_a` £²U�_��\�Y"�6i¬V�Y"_&R�x¯�Q|4_#lWP�Y"�6�>�}R��Q\�Pax¨P w �Q|6_W��_#Y�R�NQ^a\�x6R�i;�4NQP�ZqR�Z6\�i�\��Q\�_W�Wj C R��Q|4_WY�R��Q\�l&R�i�i�Vc`.Y�R�NQ^a\�xqR�i�4NQP�ZqR�Z6\�i�\��Q\�_W��R�NQ_¥T4_h°6x6_&T[\]x¤�Q_�NQY"�OP w ���4Y"��P��a_hN�R�i�i6�Q|6_­�0Pa����\�Z4i]_¥���}R��Q_h�'P w R�i]i4�Q|6_­P��Q|4_hN'x4PLT4_W�\�xv�Q|4_#��V>���Q_WYEj~Ø6P�N�_��6R�Y"�6i�_#\ w U'_#UR�xc�*�Q|6_¤Y�R�NQ^a\�xqR�i��>NQPaZ6R�Z6\�i�\¬�¢V¯P w �Q|6_#i]R����[x4PLT4_³ô2§kñ � «�`U�_*\�xE^�_Wx6_�N}R�i.x6_W_&T��QP¤lWPaY"�4�4�Q_

ô2§kñ���«��*),+.-/),+102 � � 3)+5476 - ô@§kñWú�ñ���ú�ñ�8�ú9 � � �ú}ñ��¥« §©dS«

� _­U­\�i�i�NQ_ w _hN��QP�Y�R�NQ^�\�xqR�i0�>NQPaZ6R�Z6\�i�\¬�Q\�_W���Q|qR��R�NQ_�lWPaY"�4�4�Q_&T"R��6�>NQP��>\�Y�R��Q_hi�V"R��¤ �Z0_Wi�\]_ w �W` £"R�x6TT4_hx6P��Q_³�Q|4_³Z0_Wi�\�_ w R��­x4PLT4_*ó�ZcV;:�§kñ0ò©«�j� w U�_�|qR&�S_���PaY"_¥\]x w P�NQY�R��Q\�Pax+R�Z0Pa�>���PaY"_�P w �Q|6_�x6PLT4_h�³§k_aj ^4jLU�_�¹Lx6P&UÔ�Q|qR����Q|6_¥�qR��Q\�_Wxc�­T4Pc_W�x6P��¥��Y"Pa¹S_³\�x!Pa�4N�  b ��\]R�£"_h�4R�Y"�6i�_�«�`>�Q|6_Wx�U�_³U­\�i�i2Z0_�R�Z4i�_³�QP#°6���Q|6_HlWP�N�NQ_W���0P�xqT>\]x4^"�aR�NQ\]R�Z6i�_R�x6T�U'_�U­\�i]i>x6P��;|qR&�S_O�QPH���6Y P��S_hN;�Q|qR��;x6PLT>_aj � _'U­\�i�i>l&R�i�i>���6l}|#R¥x6PLT4_�R�xÄ �PaZ4��_hNQ�aR�Z6i�_W£�x6PLT4_�`\�x�lWPaxc��N}R����'�QP��Q|4_�P��Q|6_hN�x6PLT4_h�OU­|6\�l}|�R�NQ_+ �|6\]T6T>_Wx6£�x4PLT4_W�Wj>Ø6P�N­R�i�i0P��4N�^�N}R��4|6\�l&R�i�Y"PLT>_Wi��W`SU�_U­\�i�i@T4_hx6P��Q_*P�Z6��_hNQ��R�Z4i�_*x6PLT4_h�ZLV�°6i]i�_&TL¦½\]x!lW\¬NQlWi�_W�W`�R�xqT!|4\]T6T4_hx+x6PLT>_W�­ZcV�_WY"�4��V�lW\¬NQlWi�_W�WjØ6P�N*��Y�R�i�i�M�RWVc_W��\]R�xvx4_h�¢U'P�NQ¹L�W`0U�_[l&R�x¯_&R���\�i¬V¯T4P�Y�R�NQ^�\�xqR�i�\�»&R��Q\]P�xÄ���4Y"�³T4\¬NQ_Wl��Qi�Vc`qZ6�4�H�6x w P�N�¦�Q�6x6R��Q_Wi¬Vc`.�Q|6_�xL�6Y[Z0_hN�P w �Q_hNQY"��\�xÄ�Q|6_����4Y"��U­\�i�i�^�NQP�U¶_h�>�0Pax4_Wxc�Q\fR�i�i�VvU­\¬�Q|��Q|6_�xL�6Y#Z0_�N�P w|6\]T4T4_Wx�x4PLT4_W�[\]x��Q|6_�x4_h�¢U'P�NQ¹0j � |6_!�L\�N��Q�4_�P w �Q|6_!M�ªÒR�i�^aP�NQ\��Q|4Y \��#�Q|6R���U�_+lWR�x��6��_�\¬�¤�QPlWP�Y"�6�4�Q_­Y�R�NQ^a\�xqR�i0�4NQPaZ6R�Z4\]i�\¬�Q\�_W�W`4R��i�_&R����­R��4�4NQP&�>\�Y�R��Q_Wi¬Vc`S\�x!R³�Q\]Y"_­�Q|qR���^�NQP�U­�P�x6i¬V"i�\�x6_&R�NQi¬VU­\¬�Q|��Q|6_�xL�6Y#Z0_�N�P w x6PLT4_h�H\�x��Q|6_���V>���Q_WYEj0Ø6P�NH�Q|6R��¥NQ_&R���P�x~`.M�ªÔl&R�x¨Z0_[�4��_&TE\�x¯�>N}R�lh�Q\�lW_[R��R�x´ ¢\�x w _hNQ_Wx6lh_�_hx6^a\�x4_W£"R�l��Q\]x4^"Pax��Q|6_*���}R��Q\����Q\�l&R�i;T6R��}R#_Wx6lhPLT4_&T�\]xER#i]R�NQ^a_�M�RWVc_h��\fR�x�x6_h��U�P�NQ¹�jM'_ w P�NQ_*U�_³�Q�>NQx!�QP"M'ª;`4|6P&U'_W�S_hN&`>U�_�T>_W��lhNQ\�Z0_*��PaY"_*P��Q|6_hN¥\�x w _hNQ_Wx6lh_*�4NQPaZ4i�_WY"�Wj

Ú 9�< = Ü Þhâm�Þ&7~Ý?> Ü â�ã+áA@ â Ü!ßCB áD>�t­ÞhÝFE B 7M�ªÆ|qR���R�i���P´NQ_Wlh_Wxc�Qi�V8Z0_h^a�6x²�QP�Z0_Ä�6��_WT R��¨R�x� ¢_Wx4^a\�x6_W£ w P�N¨i]P&U¦½i�_W�S_Wi�lWP�Y"�6�>�Q_hNE�L\���\�Pax�4NQP�Z6i�_WY"��§©Ø4NQ_W_WY�R�x~`�ª;R���»h�QP�N&`#R�x6T X R�NQY"\�l�|6R�_Wi!I�J�JaJa«�j[���6Y�R�x6�¯P w �Q_hx��}R�¹a_ w P�N�^�N}R�xc�Q_&T�Q|6_!��Pai��4�Q\�P�xÔP w R��6�6R�NQ_Wxc�Qi¬V���\�Y"�6i�_+lWP�Y"�6�>�Q_hN"�L\���\]P�x´�>NQPaZ6i�_WY"�#i]\�¹S_!�Q|6_+��_h^aY"_Wxc�}R��Q\�P�xÔR�x6TNQ_WlWP�^ax4\��Q\�P�x!P w P�Z>�¢_Wlh�Q�W`>P�N��Q|4_¥T4_h�Q_Wl��Q\]P�x+R�xqT¤\�xc�Q_hNQ�4NQ_��}R��Q\�Pax�P w Y"P��Q\�Pax.j � _¥��Pai��S_­�Q|6_h��_¥�}R���¹>���P�R��>�QPaY�R��Q\]lWR�i�i¬V+�Q|qR��¥\���l&R�xEZ0_����4NQ�4NQ\���\�x6^"|4P&UyT>\�®¤lW�6i¬�¥\¬��\��­�QP¤�Q_&R�l}|¨R#lWP�Y"�6�>�Q_hN­�QP¤��Pai��S_�Q|6_��QR�Y"_H�}R���¹L�W`4^�\��S_Wx����6���¥R���_hNQ\�_W�­P w �¢U�P�¦�T4\�Y"_Wx6��\�P�xqR�i.R�N�N}R&V>�P w �6\¬�>_Wi���R�i��6_W�Wj x4_HPaZ4���}R�lWi�_�QP��4NQPa^�NQ_W���\�x!lhPaY"�6�>�Q_hNO�>\���\�Pax!|qR���Z0_W_Wx"�Q|6_�T4\¬®"lh�6i¬�¢V�P w °6xqT4\�x4^��Q|4_WP�NQ_��Q\]lWR�i�i¬V!��P�i�\fT�Y"PLT4_Wi���Q|qR��¥R�NQ_�lWP�Y"�6�4�}R��Q\�PaxqR�i�i�V���N}R�lh�}R�Z4i]_�jª;R�\¬N�U­\���_ C R�NQ¹SP��vN}R�xqT4P�Y±°q_Wi]T4��§ C+F Ø'· �}«��4NQP��L\]T4_�R�����N}R�lh�Q\��S_��Q|6_WP�NQ_h�Q\�l&R�i�Y"PLT>_Wi�� w P�N#��PaY"_lWP�Y"�6�4�Q_�N;�L\]��\�P�x"�4NQPaZ4i�_WY"�Wj��¢x#���4l�|[�4NQP�Z6i�_WY"�W`�U�_x6P�NQY�R�i�i¬V�UR�xc���QP³\�x w _hNOR�NQ_W�>NQ_W��_Wxc�}R��Q\�Pax#P wU­|qR��Q_W�S_hN�\]�# �NQ_&R�i]i¬V�Pa�>�'�Q|4_hNQ_W£ w NQPaY��Q|6_�T6R��}R*�Q|qR��'U'_*R�NQ_�^a\��S_Wx.`cU­|6\�l�|�\���6i¬�Q\�Y�R��Q_hi�V"R���U�P�¦T4\�Y"_Wx4��\�PaxqR�i;R�N�N}RWV�P w x>�4Y#Z0_hNQ�NQ_W�>NQ_W��_Wxc�Q\�x6^��6\¬�>_Wi@\�xc�Q_Wx4��\��Q\�_W�Wj � P��}R�¹S_�R"lWPax4lhNQ_h�Q_�_h�4R�Y"�6i�_a`���6�4�0Pa��_­�Q|qR��'U�_­UR�xc�'�QP�\�x w _�N'�Q|6_�T4\����}R�x6lW_¥P w �Q|6_­PaZL��_hlh�Q��\�x�R���lh_Wx6_ w NQPaYÒ�Q|6_¥�L\�_hU�_hN&j � |qR��\��W`0\�Y�R�^�\]x4_*�Q|qR��¥U�_[R�NQ_�^�\]�a_Wx¨REK&JaJ�J�ZcV¯K&J�JaJ"^�N}RWVL¦½��l&R�i�_#\�Y�R�^a_a`qR�xqT+U'_[�0P���_�Pa�4NH�4NQP�Z6i�_WYR��³��N�V>\�x6^��QP�\]x w _hN�T>\����}R�x6lh_¤�aR�i��6_W��GLòlWP�N�NQ_W���0Pax6T4\�x6^��QP!\�xc�Q_Wx4��\���V¨�aR�i��6_W��HWò�`�U­|4_hNQ_#óN}R�x4^a_W�P��S_hN��Q|6_�Y"\�i�i]\�P�x��0P�����\�Z6i�_!�4\¬�4_hi'�0Pa��\¬�Q\�Pax4�Wj N&`;\�x6���Q_&R�T�P w T4\����}R�x6lW_a`2U�_�Y"\]^�|c�#Z0_�\�x w _hN�NQ\�x6^��PaY"_#P��Q|6_�N�ëL�6R�xc�Q\¬�¢V¯R�Z0Pa�4�*�Q|4_¤��lW_Wx4_��c���4l�|ÄR���|6\�^�|4¦©NQ_W��Pai��>�Q\]P�x�T4_��}R�\�i����Q|6R���R�NQ_¤Y"\�����\]x4^E\�x�Q|6_H\]Y�R�^a_a`4P�N¥�Q|6_HPa�4�Q\�l&R�iJI6P�U \�xER[��_hNQ\�_W��P w \�Y�R�^�_W�W`q_h�Ql�j

Page 5: Understanding Belief Propagation and its Generalizations Jonathan

�¢x¨^�_Wx6_hN}R�ik`0U�_¤R������6Y"_��Q|qR��HU�_[PaZ4��_hNQ�S_���PaY"_[ëL�6R�xc�Q\¬�Q\�_W�*R�Z0P��4���Q|6_�\]Y�R�^a_LKSò�`�R�x6TE�Q|qR���U�_UR�xc�[�QP�\�x w _hN¤��P�Y"_�P��Q|6_�N¤ëc�qR�xc�Q\��Q\�_W�[R�Z0P��4���Q|6_��6x6T4_hNQi¬V>\�x6^E��lh_Wx6_�ñ0ò�j � |4_�\�x6T4\�lW_W��ó³lWP��6i]TNQ_W�>NQ_W��_Wxc�*��\�x6^�i]_#�6\¬�>_Wi;�0Pa��\¬�Q\�Pax4�W`~P�N³�Q|6_�V�Y"\�^a|c�HNQ_W�>NQ_W��_Wxc�³�Q|4_#�0Pa��\¬�Q\�Pax¯P w R���Y�R�i�i��6R��Ql}|¨P w�6\¬�>_Wi��Wj � _ w �4N��Q|6_�N¥R������6Y"_��Q|qR���Q|4_hNQ_³\��­��PaY"_³���}R��Q\����Q\]lWR�i@T>_W�0_Wx6T4_Wx4lhV�Z0_h�¢U'_W_Wx!ñ ò R�xqTMK ò R��_&R�l�|��0P���\��Q\�P�x�ó�`2U­|4\�l�|�U�_"UNQ\¬�Q_!R��¤R¤�¢Pa\�xc�#�>NQPaZqR�Z6\�i�\¬�¢V"N ò §kñ ò úOK ò «�j � |6_ w �4x6lh�Q\�PaxPN ò §kñ ò úQK ò «\��HP w �Q_Wx¯l&R�i�i�_&TE�Q|6_¯ ¢_h�>\]T>_Wx6lW_h£ w P�NHñ0ò�j�Ø@\�xqR�i�i¬Vc` w P�N³�4���QP��0Pa����\�Z6i¬V�Z0_[R�Z4i�_��QP�\�x w _hN*R�xLVL�Q|4\]x4^R�Z0P��4���Q|4_H��lW_hx6_a`c�Q|4_hNQ_H|6R��'�QP�Z0_H��P�Y"_H����NQ�6l��Q�4NQ_��QP��Q|6_¥ñ0ò�jL�¢x�^a_hx6_hN}R�i©`>\ w U'_³T>P[x4P��­R������6Y"_���6l}|¨����NQ�6lh�Q�>NQ_a`�lWP�Y"�6�4�Q_�N³�L\���\]P�xv�>NQPaZ4i]_hY"�*R�NQ_[\]x4|6_hNQ_hxL�Qi¬V�|4Pa�0_Wi�_W����i¬V¯\�i�i�¦½�0P���_&T.j � _�_Wx4lWPLT4_�Q|6_+R������4Y"_&T�����NQ�4lh�Q�4NQ_!P w �Q|4_+��lW_hx6_EZcV��QRWV>\]x4^v�Q|6R��¤�Q|4_+x6PLT>_W�"ó�R�NQ_�R�N�N}R�x4^a_&T�\]x´R¯��U�P�¦T4\�Y"_Wx4��\�PaxqR�i0^�NQ\]T.`4R�xqT"��lW_Wx4_³�aR�NQ\]R�Z6i�_W�'ñ ò ��|6P��6i]T.`L\�x6��P w R�NHR���0Pa����\�Z6i�_a`>Z0_! ¢lWP�Y"�qR��Q\�Z4i]_h£�U­\¬�Q|x6_WR�NQZcVÄ��lh_Wx6_"��R�NQ\]R�Z4i�_W�#ñ�RS`2R���NQ_W�4NQ_W��_hxL�Q_WTÄZcV�R+lhPaY"�qR��Q\�Z6\�i�\���V w �6x6l��Q\]P�x2S�ò R>§kñ0ò�ú�ñ�R�«�`.U­|6_hNQ_S�ò R"P�x6i¬VElhPax6x4_Wlh�Q�³x4_&R�NQZcV+�0Pa��\¬�Q\�Pax6�Wj � _��Q|4_WxE�}R�¹a_��Q|4_�P��S_�N}R�i�iq��P�\]xc�H�4NQPaZ6R�Z4\]i�\¬�¢VEP w R���lh_Wx6_ñ0ò�R�x6TER�x!\�Y�R�^a_TKaò��QP#Z0_

ô@§�ý&ñ.þcú�ý1K6þ�«�� KU � V ò R�W S ò R §kñ ò ú�ñ R « � ò N ò §kñ ò úQK ò « §k�c«

U­|6_�NQ_ U \���R­x6P�NQY�R�i�\�»&R��Q\�Pax[lWPax4���}R�xc��R�x6T��Q|4_'�>NQPLT4�6l��;P��S_hN§kó�Xc«2\��;P��S_hN§ w P�N�_h�4R�Y"�4i�_�«.x4_&R�NQ_W���x6_h\]^�|LZ0P�NQ�­Pax+�Q|4_³�Qëc�qR�NQ_*ifR����Q\�lW_ajb ^�N}R��6|6\�l&R�i­T4_W�4\�lh�Q\�Pax�P w �Q|6\��#Y"PLT>_Wi�\��#��|4P&U­x�\�x�°q^��4NQ_+ILj � |4_!°qi�i�_&TL¦�\�x�lW\¬NQlWi�_W�¤NQ_h�4NQ_W��_Wxc�

t­?Bï0ð2u�</<$b �Qëc�qR�NQ_*i]R����Q\�lW_��qR�\�N�U­\���_ C R�NQ¹aP�� F R�xqT>PaYºØ@\�_Wi]T�j

�Q|6_*P�Z6��_hNQ�a_&T+\�Y�R�^�_�x4PLT4_W�YK ò `4U­|6\�i�_*�Q|6_*_WY"�>�¢V�lW\¬NQlWi�_W�­NQ_W�4NQ_W��_hxL�¥�Q|6_� �|6\]T6T>_Wx6£#��lW_Wx6_*x4PLT4_W�ñ0ò�j � |6_ C R�NQ¹SP��"N}R�x6T4PaY °q_Wi]T�\]��QR�\]T��QP#Z0_! ¢�6R�\¬N�U­\���_W£"Z0_Wl&R��6��_H�Q|4_HlWP�Y"�qR��Q\�Z4\]i�\¬�¢V w �6x4lh�Q\�Pax6�Pax4i¬V�T4_W�0_hxqTÄP�x��6R�\¬NQ�¤P w ��\¬�Q_W�#ó*R�x6TZX4j��¢x�lWP�xL��N}R����#�QP¯M�RWVc_W��\]R�x�x4_h�¢U'P�NQ¹L�W`;�Q|6\���^�N}R��6|6\�l&R�iY"PLT4_hiq\���6xqT>\¬NQ_Wlh�Q_&T�j � |6_hNQ_�\���x6P�x6P��Q\]P�x~`>R���\���6���qR�i�i�V¤\]Y"�4i�\]lh\���\]x!R�M�RWVc_W��\]R�x�x6_h��U�P�NQ¹�`c�Q|6R���Q|6_���R�NQ\]R�Z4i�_¤R���Pax4_[x4PLT4_�ñ0òO\��*R"l&R��6�QR�i* ¢�qR�NQ_Wxc�Q£�P w \¬�Q�Hx6_h\]^�|LZ0P�N³ñ[Rc`0��P�U�_��6��_��4xqT4\¬NQ_Wl��Q_&TlWP�Y"�qR��Q\�Z4\]i�\¬�¢V w �6x4lh�Q\�Pax4��S ò R §kñ ò ú�ñ R «*\]x4���Q_&RaT�P w lWP�xqT4\¬�Q\�Pax6R�i�>NQPaZqR�Z6\�i�\¬�¢V w �6x6l��Q\]P�x6��ô@§kñ ò ö ñ R «�jü _h�S_hN��Q|6_hi]_h���W`4P��4N¥R�^a_Wx6T6R#\�xET>Pa\�x6^[\�x w _�NQ_Wx6lW_�U­\]i�i~Z0_H�S_hN�V���\�Y"\�i]R�N&ì>U�_³U­\�i�i.U�R�xc��QP#lhPaY"�6�>�Q_�Q|6_OZ0_Wi�\�_ w ��:a§kñqò�«�` w P�NOR�i�iL�0Pa��\¬�Q\�Pax4��óQ`���PHR��@�QPHZ0_�R�Z6i�_'�QP¥\�x w _hN���PaY"_h�Q|4\�x6^³R�Z0Pa�>�@�Q|6_O�6xqT>_hNQi¬V>\]x4^�6x4¹Lx6P&U­x���lW_Wx6_�j x4lW_ER�^SR�\]x´RÄT>\�NQ_hlh�"lWP�Y"�6�4�}R��Q\�Pax�P w Y�R�NQ^a\�xqR�i­�4NQPaZ6R�Z4\]i�\¬�Q\�_W�¤U�P��6i]T��}R�¹S__h�>�0Pax4_Wxc�Q\]R�iO�Q\�Y"_�§©R�xqTÄ�Q|6_"xL�4Y#Z0_hN�P w x4PLT4_W��\�����V>�6\�l&R�i�i¬V��a_hN�VÄi]R�NQ^a_�«���P¨U'_�x6_W_&TÄR w R����Q_hNR�i�^�P�NQ\¬�Q|6YÆi�\�¹S_�M�ª@j

Page 6: Understanding Belief Propagation and its Generalizations Jonathan

Ú 9�\ rL]+Ý^=�á¤r�r¤7 Ü�ßCB Þ&7~Þ ßZ_ >Òá B ÝFE7����\��;U�P�N��Q|#�}R�¹L\�x6^³RH��Y�R�i�i4T4_h�QP��4N��QPH��|4P�U��Q|qR��;�Q|6_�6R�\¬N�U­\���_ C+F Ø�T4_h��lhNQ\�Z0_&T#ZcV�_&ëc�qR��Q\�P�xE§k�c«l&R�x!_&R���\�i�V�Z0_¥Z4NQP��6^a|c�\�xc�QP�R w P�NQY¶NQ_hlWPa^�x6\�»&R�Z4i�_H�QP[�4|cV4��\�lW\����Q�R����Q|6_� �ª@P����Q�Y"PLT4_Wi�£�§©M�R��L�Q_hNK&e�ÙSI�«�j G _��!�4��T4_h°6x6_+�Q|4_Ô ¢\�xc�Q_hN}R�lh�Q\�Pax6£a`cò R>§kñ0ò�ú�ñ�R�«#Z0_h��U�_W_Wx��Q|6_E��R�NQ\]R�Z4i�_W��R���x6_W\�^�|>Z0P�NQ\�x6^x6PLT>_W�+ZcVb`Lò R>§kñqò�ú�ñ�R�«Z�¡i�xYS�ò Rc§kñ0ò�ú�ñ�R�«!R�xqT´�Q|6_8 ½°q_Wi]T4£PcqòQ§kñ0ò©«�R��!_&R�l}|8x6PLT4_¨ZcVbc6òQ§kñqò�«Z�i�xdN�§kñqò¢úQKSò�«�j@§©M'_hl&R��4��_[U�_[T4P�\�x w _hNQ_Wx4lW_ w P�N�R"^�\]�a_Wx���_h�³P w Kaò�`qU�_�l&R�x¨\�x w R�lh�³lWP�x6��\]T4_�NH�Q|6_AKSòR��³°4�>_&T��aR�NQ\fR�Z6i�_W��R�x6T¯���4Z6���4Y"_��Q|4_WYÕ\�xc�QP!P��4N�T>_h°qx4\¬�Q\]P�xÄP w cqò�§kñ0ò½«�j «�� w U�_¤x4P&UÒT4_h°6x6_��Q|6_ª@P����Q�­Y"PLT>_Wi­ ¢_Wx4_hNQ^�V>£�R�� e §�ý&ñ.þ�«��gfZ) V

ò R�W `Lò R>§kñ0ò�ú�ñ[R�«�f") ò cqòQ§kñ0ò½« §iha«R�x6T+R��6�0_&R�i~�QP"M'P�i¬�Q»WY�R�x4x~· �­ifRWU w NQP�YÆ���}R��Q\����Q\�l&R�i~Y"_Wl�|6R�x4\]lh�Wì

ô2§�ý�ñ.þ�«�� KUkjml ù V�n +1oqp W,r � §©åS«

U�_���_W_��Q|qR��[Pa�>N¤�6R�\¬N�U­\���_ C+F Ø lWP�N�NQ_W���0Pax6T4�¤_h�4R�lh�Qi¬V��QP¯R¯ª@P����Q�#Y"PLT>_Wi�R��"R��Q_WY"�0_hN}R��Q�>NQ_s _&ëc�qR�i­�QP�Kaj � |6_Ex4P�NQY�R�i]\�»&R��Q\�PaxÔlWP�x6���}R�xL� U \���¹Lx4P�U­x�\�x��Q|6_+�6|cV>��\]lh�¤i�\��Q_�N}R��Q�4NQ_¨R��"�Q|6_ ¢�6R�N��Q\¬�Q\�Pax w �4x6lh�Q\�P�x~j £v� w �Q|6_�xL�6Y[Z0_hN�P w ���}R��Q_W�¤R��[_&R�l}|�x6PLT4_�\��#_��6R�lh�Qi¬V��¢U'P4`@�Q|4_�Y"PLT4_WiO\]�l&R�i]i�_&T��Q|6_Ô ��¢��\�x6^�Y"PLT>_Wikj £v�¢x��Q|6\���l&R���_a`O�6|cV>��\�lW\����Q����PaY"_h�Q\�Y"_W�"�>NQ_ w _hN��QPvl�|6R�x6^�_��aR�NQ\]R�Z6i�_W�w NQP�Ynñ ò �QPMt ò U­|4\�l�|ElWR�xE�}R�¹a_�P�x+�Q|6_���R�i��6_h��P w K�P�NAf*Ka`0R�xqT+�QP w �4N��Q|6_�N¥NQ_h����NQ\]l����Q|4_WY"��_Wi��S_W��QPu` ò R \�xc�Q_hN}R�lh�Q\�P�x6��U­|4\�l�|¨|qR&�S_�R"��V>Y"Y"_h��NQ\�l w P�NQYn�Q|6R���l&R�x�Z0_�UNQ\¬���Q_Wx�\�x¨�Q_�NQY"�¥P w R� ����6\�x^ai]R�����£�_hx6_hNQ^�V w �6x4lh�Q\�Pax�§ C _W»WR�N}T.`qª;R�NQ\���\k`0R�xqT!Ó�\¬N}R���P�NQPEKWeaÙ(va«e §�ýwt�þ�«��gfx) V

ò R�W `Lò R(t&òqtORdf") ò cqòqt&ò §iva«�¢x"�Q|4_�lWP�xc�Q_h�L��P w �Q|6_��¢��\�x6^�Y"PLT>_Wik`c�Q|6_¥\]x w _hNQ_hx6lW_��4NQPaZ4i�_WY¶P w lWP�Y"�6�4�Q\�x4^*Z0_Wi�\]_ w �3:a§kñqò�«�l&R�x�Z0_Y�R��4�0_&T�Paxc�QP#�Q|6_H�6|cV>��\]lh�¥�>NQPaZ4i]_hYºP w lWP�Y"�6�>�Q\]x4^#i�Pcl&R�iH ¢Y�R�^ax4_h�Q\�»&R��Q\�Pax4��£y ò@ÿz:a§it&òJ�×K�«�f{:�§it�òJ�gf*K�«� §©ÙS«

Ú 9�| r Ü!ß+ß ÝHâ _ â Ü =T]¨7 Ü�ßCB t ÜM} r#á¤â _ â Ü =T]¨7b x4P��Q|4_hN*�4NQP�Z6i�_WY w P�N*U­|4\]l}|v�Q|4_"M�ª R�i�^aP�NQ\¬�Q|6Y±|qR��*^a\��S_Wxv_h�>lW_Wi�i�_Wxc�³NQ_W���4i��Q��\��³�Q|6_[\¬�Q_hN}R��Q\��S_T4_hlWPLT4\�x6^�P w _hN�NQP�N�¦½lWP�N�NQ_Wlh�Q\�x6^�lWPLT4_h�H§©Ø4NQ_hV#R�x6T C R�l}¹aRWV�K&e�eaÙ>ä C l D i�\�_WlW_a` C R�lW¸�RWVc`LR�x6T X |6_hx6^K&e�eaÙa«�j�M�ª \��¤�Q|6_ET>_WlWPLT>\]x4^�R�i]^�P�NQ\¬�Q|6Y �4��_&T��QP�T4_hlWPLT4_E��P�Y"_+P w �Q|6_!Z0_W�����4N}R�lh�Q\�l&R�i¥lWPLT4_h�W`\�x6lWi��6T4\�x6^Ä�Q�>NQZ0PclWPLT4_h�¯§©MO_hN�NQPa�.`¼Hi]R&�L\]_h�4��`�R�xqT � |4\¬�Q\]Y�R&����|4\�Y�R�K&e�eadS«�R�x6TÔ¼³R�i]i]R�^a_hN+lWPLT4_W�§½¼³R�i�i]R�^�_hN�K&eaå�dS«�jHî³_WlWPLT>\]x4^´_hN�NQP�N�¦½lWP�N�NQ_Wl��Q\]x4^ÔlWPLT>_W�¨\]�¨\�x w R�l��ÄR�x×_Wi�_W^aR�xc�¯_h�4R�Y"�4i]_vP w R�x ¢\�x w _hNQ_Wx4lW_W£��4NQPaZ4i�_WY��S�Q|4_¨NQ_hlW_W\��S_hN+P w �Q|6_¯lWPLT4_&T²Y"_W���QR�^a_¯�Q|qR��+|qR��+Z0_W_Wx8lWP�N�NQ�6�>�Q_&TÔZcV²Rx6P�\���Vvl}|qR�x6x4_Wi�\]�³��N�V>\�x6^��QPvæAçWè&é�ê¥�Q|6_#Y"_h���QR�^a_¤�Q|6R��*U�R��*\�x6\¬�Q\]R�i�i¬Vv��N}R�x6��Y"\¬���Q_&T�jO§©Ø4P�N�^a_hx6_hN}R�iZqR�l�¹cNQP��6xqT�Pax!_hN�NQP�N�¦½lWP�N�NQ_Wlh�Q\�x6^"lWPLT>\�x6^4`6��_W_"§½¼³R�i�i]R�^�_hN#KWeaå�ÙS«�j «¼³R�i�i]R�^�_hN"lWPLT>_W�[R�xqT¯�Q�4NQZ0PclWPLT>_W�[lWR�x�Z0_ w P�NQY#�6i]R��Q_&T�R���~�#�ê}æ,���w�����6éQ���2�O�1�cé��"§ C R�l�¹�RWV�KWeaeaea«�j�¢xvR!Z4i�Pcl�¹��6R�NQ\¬�¢VL¦½l�|4_Wl}¹�lWPLT4_a`�U'_#U­\�i]i��¢V>�4\]lWR�i�i¬V¯��N�V��QP���_Wx6T{��\�x w P�NQY�R��Q\�PaxÄZ6\¬�Q�³\�x�R�Z6i�Pcl�¹P w�� Z6\¬�Q�Wj � U­\�i�i¥Z0_+^�NQ_WR��Q_hN"�Q|6R�xa����PÄR��¤�QP¯�4NQP��L\fT>_!NQ_&T>�6x6T6R�x4lhV��Q|6R��"l&R�x�Z0_+�6��_&T��QPNQ_WlWP��S_�N w NQPaY¶_hN�NQP�NQ�\�xqT>�6lW_WT�ZcV¤�Q|4_³x6P�\]��V�l}|qR�x6x4_Wikj G _h��4�^a\��S_³R���Y�R�i�i�_h�4R�Y"�4i]_�ì>R�x � �²åL`�"�Æd¨Z4\�xqR�N�VÄlWPLT4_�`;��|6P&U­x�ZcV�\¬�Q�v  � R�x4x6_hN#^�N}R��6|6£¨\�x�°q^��4NQ_�d>j bº� R�x6x4_hN[^�N}R��4|8§ � R�x4x6_hN

Page 7: Understanding Belief Propagation and its Generalizations Jonathan

t­?Bï0ð2u�</\$b×� R�x6x4_hN¥^�N}R��4| w P�N�R#��Y�R�i�i � �²åL`��$�²d#�6R�NQ\¬�¢V�l}|6_Wl}¹�lWPLT4_�jK&e�Ù>K&«­\��­R#�4\]l��QP�NQ\]R�i.NQ_W�4NQ_W��_hxL�}R��Q\�Pax+P w �Q|6_H�qR�NQ\¬��V>¦½l}|6_Wl}¹"lWP�x6����N}R�\�xc�Q�­Pax��Q|6_³lWPLT>_hU�P�N}T4��§ki�_W^SR�ilWP�x4°q^��4N}R��Q\]P�x6�­P w Z4\¬�Q�}«'P w R�x!_hN�NQP�N�¦½lWP�N�NQ_Wlh�Q\�x6^"lhPLT4_aj D R�l�|E�Qëc�6R�NQ_HNQ_W�>NQ_W��_Wxc�Q�¥R#�qR�NQ\¬�¢V�l�|4_Wl}¹�`R�x6T�_WR�l}|�lW\¬NQlWi�_�lWP�x6x4_Wlh�Q_&T��QPÄR¯�Qëc�qR�NQ_�NQ_W�4NQ_h��_Wxc�Q��R¯Z4\��#�Q|qR��¤�qR�N��Q\�lW\��6R��Q_W��\�x��Q|6R��"�6R�NQ\¬�¢Vl}|6_Wl}¹0j��¢x�Pa�>N"_h�4R�Y"�4i]_�lWPLT>_a`��Q|6_!°4NQ���"�qR�NQ\¬�¢V�l}|6_Wl}¹ w P�NQlW_W�¤�Q|4_+���6YàP w Z6\¬�Q�k��K�`7�¤IL`OR�x6T�[�+�QP+Z0_¤_W�S_Wx.`�U­|6\�i�_���_hlWPax6Tv�qR�NQ\¬�¢V¯l�|4_Wl}¹ w P�NQlW_h���Q|4_¤���6Y±P w Z6\¬�Q�L��Ka`��#dL`@R�xqT2��h��QPEZ0__W�S_hx~`0R�xqT+�Q|4_��Q|4\�N}TE�6R�NQ\¬�¢V+l�|4_Wl}¹ w P�NQlW_W�H�Q|4_����6YÆP w Z6\¬�Q���¤IL`J�#dL`�R�x6TZ�#å¤�QP�Z0_�_h�S_Wx~j � |6_Pax4i¬V"_W\�^a|c�OlWPLT4_hU'P�N}T>�'�Q|qR����QR��Q\�� w V"�Q|6_W��_�Q|4NQ_h_��6R�NQ\¬�¢VL¦½l�|4_Wl}¹[lWP�x6����N}R�\]xc�Q�R�NQ_��(�m�m�m�m�>`��m���1���(�a`���1���1���a`����m�m�m�9�4`��1�(���m�1�4`��1���m�9���a`3�m�9�m���m�a`@R�x6Ta�m�m�9�m�m�4j ü P��Q_[�Q|qR�� w P�N��Q|6\���lWPLT4_�`~�Q|4_#°6NQ����Q|4NQ_h_�Z4\��Q�'R�NQ_­�Q|6_� ¢\�x w P�NQY�R��Q\]P�x6£�Z6\¬�Q�W`c��P�\ w VcP���R�NQ_¥��N}R�x6��Y"\¬���Q\�x6^*Y"_h���QR�^a_h�W`>R�xqT¤VcP��"\�xL�Q_hxqT.`w P�N¥_h�4R�Y"�6i�_a`c�QP���N}R�x4��Y"\��'�Q|6_����1�¤Y"_W���QR�^�_a`LVcPa��U­\�i�i2�6��_¥�Q|6P���_³Z6\¬�Q�­R���Q|4_�°6NQ����Q|>NQ_W_HZ4\¬�Q�P wVcP��4N*lWPLT>_hU�P�N}T.`2R�x6T¨�Q|4_Wx��Q|6_[NQ_hY�R�\�x6\�x6^!Z6\¬�Q�*R�NQ_¤�6x4\fëc�4_Wi¬V¯T4_h�Q_hNQY"\�x4_&T�§ w P�N*�Q|6R��*lWR���_#VcP��U�P��6i]T!��N}R�x6��Y"\¬��Q|6_T���1���1����lWPLT>_hU�P�N}T.j «� |6_"NQ_Wlh\]�4\�_Wxc�#P w R¨lhPLT4_&TÄY"_h���QR�^a_�`�R w �Q_hN¤\¬�[|qR���Z0_W_WxÄlWP�N�NQ�4�4�Q_&TÄZcVÄREx6P�\]��VÄl�|6R�x4x6_Wik`@Y�RWV°qx6T¯�Q|6R����Q|6_¤NQ_Wlh_W\��S_&T�Y"_W���QR�^a_�\���x4P��#R+lWPLT>_hU�P�N}T�R w �Q_hN#R�i�ikj;Ø4P�N[_h�4R�Y"�6i�_a`~i�_h���6�����6�4�0Pa��_�Q|qR��*�Q|6_$���1���1����Y"_W���QR�^a_¤U�R��*��N}R�x4��Y"\����Q_WT.`.Z6�>��Z0_Wl&R��4��_"P w �Q|4_¤x6P�\���VÄl}|qR�x6x6_hi©`.P�x6_"P w �Q|6_Z6\¬�Q�¥UR��FIq\��6�0_&T�R�xqT+�Q|4_����(�m�1���v �U�P�N}T4£�UR���NQ_WlW_h\]�a_&T.j � |4\]�¥U�P�N}T¯T>Pc_W�³x4P���lWP�N�NQ_W���0P�xqT��QPR�xcV¯lWPLT>_hU�P�N}T.`2R�xqT¯\¬��\]�*�Q|4_H��P�Z�P w �Q|4_"T4_hlWPLT4\�x6^+R�i�^aP�NQ\¬�Q|6Y±�QPE\�x w _hN*U­|6\�l}|�lWPLT4_�U�P�N}T¯UR��R�l��Q�qR�i�i¬VE��_hxL�&jqØ6P�N�Pa�>N¥lWPLT4_a`6U�_�l&R�x!NQ_&RaT>\]i¬V���_h_��Q|6R��­�Q|6_����1���1����lWPLT>_hU�P�N}T¨\��­�Q|6_*P�x6i¬VEP�x6_�Q|qR��³\��³P�x6i¬V¨R���\]x4^ai�_#Z4\¬��¦iI6\��vRWU�R&V w NQPaY��Q|6_�NQ_WlW_W\��S_WT¨U'P�N}T.`���P�\ w �Q|4_�Z6\¬��¦&Iq\��¨�4NQPaZ6R�Z4\]i�\¬�¢VE\]���Y�R�i�ik`>\¬�\���NQ_&R���P�xqR�Z4i�_H�QP¤T>_WlWPLT4_¥�QP[�Q|6R��lWPLT>_hU�P�N}T.j>�Hx w P�N��Q�6xqR��Q_Wi¬Vc` w P�N­Y�R�xcV¤lWPLT>_W�W` � R�x6T�ÄU­\�i�iZ0_!i]R�NQ^�_a`���P��Q|qR��[�Q|6_�xL�6Y#Z0_�N[P w lWPLT4_hU'P�N}T>�"\]�[_h�>�0Pax4_Wxc�Q\fR�i�i�V�|>�4^a_!R�x6TÄU�_+lWR�x6x4P����\�Y"�6i¬V¨NQ_W��P�N����QPE_��6R�Y"\�x6\�x6^+R�i�iO�Q|4_¤lWPLT4_�U�P�N}T>�#R�xqT¨°qxqT>\�x6^!�Q|4_"lWi�Pa��_W���[Pax4_aj b �*\��[ZcV¨x4P�UZ0_WlWP�Y"\�x6^�R w R�Y"\]i�\]R�NHNQ_ w N}R�\]x.`0U�_�U­\�i]i;Z0_[R�Z4i�_��QP¤�Q�>NQxE�QP��Q|4_�M�ª²R�i�^aP�NQ\¬�Q|6YE`�Z0_Wl&R��6��_�\¬��U­\�i�iU�P�NQ¹!\�x+R[�Q\�Y"_H�Q|qR���P�x6i¬V�^�NQP&U­�¥i]\�x4_&R�NQi¬V+U­\¬�Q| � j� |6_�T4_WlWPLT>\�x6^Ô�>NQPaZ6i�_WY l&R�x Z0_�^a\��S_Wx×R´�4NQP�ZqR�Z6\�i�\]���Q\�l w P�NQY#�4i]R��Q\�Pax.j � _�R������4Y"_v�Q|qR���U�_|qR&�S_¯NQ_WlW_W\��S_WT×R���_WëL�4_Wx6lh_ÄP w$� Z6\¬�Q��K ò `�R�x6T U'_�R�NQ_Ä��N�V>\�x6^��QP�°6xqTÔ�Q|6_ � Z6\¬�Q��P w �Q|6_��N}R�x4��Y"\¬���Q_&T�lhPLT4_hU'P�N}TÔñ0ò¢j G _h���4�W` w P�N��Q|6_+�6�4NQ�0P���_EP w ��\�Y"�4i]\�lW\¬��Vc`�R������6Y"_E�Q|6R��"�Q|4_Ex6P�\]��Vl}|qR�x4x6_Wi'\]�v ¢Y"_WY"P�N�V>i]_h���W` £��Q|6R��[\]�#_&R�l}|�Z6\¬�¤óH\]�AIq\��6�0_WT�\]x6T4_W�0_hxqT4_hxL�Qi¬VvP w _W�S_hN�VÄP��Q|4_hN¤Z4\��&j� |qR��#Y"_WR�x6�*�Q|6R���U�_�l&R�x�R�����PclW\]R��Q_ERElWP�xqT4\¬�Q\�Pax6R�iO�4NQPaZ6R�Z4\]i�\¬�¢V+ô@§kñ0ò�ö KSò½«³U­\¬�Q|�_&R�l}|�NQ_WlW_h\]�a_&TZ6\¬�&jqØ6P�N�_h�4R�Y"�4i]_�`6\ w �Q|6_*°6NQ���¥Z4\���\��¥NQ_hlW_W\��S_&T�R��³R$�4`qR�xqT+U'_�¹>x4P&U �Q|6R��¥�Q|6_*Z6\¬�Q��R�NQ_�Iq\��6�0_WT

Page 8: Understanding Belief Propagation and its Generalizations Jonathan

U­\¬�Q|�R*�4NQPaZ6R�Z4\]i�\¬�¢V$�~`a�Q|4_Wx#�Q|6_­�>NQPaZqR�Z6\�i�\¬�¢V#�Q|6R����Q|6_�°4NQ���'Z4\���UR��O��N}R�x6��Y"\¬���Q_&T¤R���RT�*\���Kf;�~`U­|6\�i�_��Q|6_��4NQP�ZqR�Z4\�i]\¬��VE�Q|qR��H\¬�¥UR�����N}R�x6��Y"\¬���Q_&T+R��³R���\��T�~j { P�ô@§kñ��×J6ö K�F� JS«3��§�KYf��0«�`U­|6\�i�_Hô2§kñF�¶Kcö K���µJS«Y���~j � |6_[P��a_hN}R�i�i;�4NQP�ZqR�Z6\�i�\���V�P w _&R�l}|¨lWPLT4_�U�P�N}T¯\��³�4NQP��0P�N��Q\�Pax6R�i;�QP�Q|6_¥�4NQPLT>�6lh�'P w �Q|6_W��_¥Pax4_h¦½x6PLT>_��>NQPaZqR�Z6\�i�\¬�Q\]_h�Wì�� �ò��� ô2§kñ ò ö K ò «�jLM'�4�O�QP[Y�R�¹S_­���4NQ_­�Q|qR��'U�_�Pax4i�VlWP�x6��\]T4_hN@lWPaY[Z6\�xqR��Q\]P�x6�@P w ñ ò �Q|qR���R�NQ_OlWPLT4_hU'P�N}T>�W`�U'_OUNQ\¬�Q_�R¥R;��Pa\�xc�;�>NQPaZqR�Z6\�i�\¬�¢V w �6x6l��Q\]P�x��Q|6R��lWP�Y#Z6\�x4_W�H�Q|6_h��_#lWP�xqT4\¬�Q\�Pax6R�i��4NQP�ZqR�Z4\�i]\¬�Q\�_W�³U­\¬�Q|v�Q|4_#�6R�NQ\¬�¢V¨l}|6_Wl}¹ElWP�x6����N}R�\]xc�Q�Wj2Ø6P�N³_h�4R�Y"�6i�_a`w P�N¥Pa�4N � �²å>`��$�²d#lWPLT>_*^a\��S_Wx�R�Z0P��S_a`LU�_*UNQ\¬�Q_³�Q|6_³P��S_�N}R�i�i4�¢Pa\�xc���>NQPaZ6R�Z6\�i�\¬�¢V+T4\�����NQ\�Z6�>�Q\]P�xR��

ô@§�ý&ñ.þcú}ý5K6þ�«�� KU S�q���c§kñWú�ñ���ú�ñ��W«�S�i8O�c§kñJWú�ñ�8�ú�ñ��W«�S���8Q�c§kñ���ú�ñ�8�ú�ñ��&« �� ò��� ô@§�Kaò�ö ñqò�«� §©eS«

�¢x��Q|6_R�Z0P��S__&ëc�qR��Q\]P�x~` w �4x6lh�Q\�P�x6��i�\]¹a_3S q��� §kñ ú�ñ � ú�ñ � «;R�NQ_�6R�NQ\¬�¢V�l}|6_Wl}¹ w �6x4lh�Q\�Pax4�Wj � |4_hV�|qR&�S_�Q|6_��aR�i��6_#K�\ w �Q|6_����6Y¶P w �Q|6_h\�NR�NQ^a�4Y"_Wxc�Q��\��_W�S_Wx.`4R�xqT�|qR&�S_¥�Q|6_��aR�i]�4_³J�\ w �Q|6_H���4Y¶P w �Q|6_W\¬NR�NQ^��6Y"_Wxc�Q�\��­PLT6T�j�¢x#^a_Wx4_hN}R�ik` w P�N'R³�qR�NQ\¬��V#l}|6_Wl}¹�lWPLT4_U­\¬�Q|#��N}R�x4��Y"\¬���Q_&T#Z6\¬�Q��ñ0ò~R�xqT�NQ_WlW_W\��S_&T[Z6\¬�Q�7Kaò�`cR�x6T � f$��qR�NQ\¬�¢V�l�|4_Wl}¹>�W`>U�_*lWR�x+UNQ\¬�Q_³�Q|6_O��Pa\�xc���4NQPaZ6R�Z4\]i�\¬�¢V!T>\]����NQ\�Z4�4�Q\�Pax�R��

ô2§�ý�ñ.þSú�ý5K6þ�«�� KU � l���  �� S   §�ý�ñ.þ   « �� ò��� ô@§�K ò ö ñ ò « §�K&JS«

U­|6_�NQ_¥U'_¥|6R&�S_­�6��_&T[�Q|4_­x6P��}R��Q\�PaxkS   §�ý&ñ.þ   «;�QP�T>_Wx6P��Q_­�Q|4_Y�L�Q|#�qR�NQ\¬��V¤l}|6_Wl}¹ w �6x4lh�Q\�Pax$S   R�x6T\¬�Q�¥R�NQ^a�4Y"_Wxc�Q��ý�ñ.þ   jb T4_WlWPLT>\�x6^[R�i�^aP�NQ\��Q|4Y w P�N­�qR�NQ\¬��V�l�|4_Wl�¹#lWPLT4_W��U­|4\�l�|�Y"\�x6\�Y"\�»W_W�O�Q|4_HxL�6Y[Z0_hNOP w Z4\¬�Q�'�Q|qR��­R�NQ_T4_hlWPLT4_&T[\]x4lWP�N�NQ_hlh�Qi¬V#\��;�QP³lWP�Y"�6�4�Q_��Q|6_�Y�R�NQ^a\�xqR�i6�>NQPaZqR�Z6\�i�\¬�Q\]_h�~ô@§kñqò©« w P�NO_W�S_�N�V[Z4\¬��óQ`aR�x6T��Q|4_Wx�QP+�Q|>NQ_W��|6P�ifTv_&R�l}|�Z6\¬���QP�\¬�Q��Y"Pa�����4NQP�ZqR�Z4i�_"�aR�i]�4_aj b ���6���6R�ik`@R�T>\¬NQ_Wlh��lWPaY"�4�4�}R��Q\]P�xÄP w �Q|6_Y�R�NQ^�\�xqR�i��>NQPaZqR�Z6\�i�\¬�Q\]_h��U­\�i�i��}R�¹S_#R�x¯_h�>�0Pax6_hxL�Q\]R�i]i¬V¯|L�6^�_[�Q\�Y"_a`���P!U'_#NQ_W��P�N��*�QP!�Q|6_[_h®"lW\�_Wxc�M�ªµR�i]^�P�NQ\¬�Q|6YEj � |6_!M�ªµR�i�^aP�NQ\¬�Q|6YE`@U­|4\]i�_!x6P��[_h�4R�l�� w P�N�¼³R�i]i]R�^a_hN�lWPLT>_W�#P�N#�Q�>NQZ0PclWPLT4_W�W`@\]��S_hN�VE_hÑ�_Wlh�Q\��S_�\�xv�>N}R�lh�Q\�lW_a`.R�xqT�|qR��³Z0_W_Wx¨�4��_&T¨�QP�R�l�|4\]_h�S_#x4_&R�N�¦ { |6R�x4x6Pax�i�\�Y"\¬�*�0_hN w P�NQY�R�x4lW_§ C R�l}¹�R&V¨KWeaeaea«�j � R�¹L\]x4^[�Q|4_³Y"Pa���­�>NQPaZqR�Z6i�_*��R�i��6_³P w _&R�l�|!Z6\¬�­\�xqT4_h�0_WxqT>_Wxc�Qi�V�Y"\�^�|L�R�lh�Q�6R�i�i¬Vx6P��+V>\�_Wi]T R���R�i�\]T8lhPLT4_hU'P�N}T.`¥��P�Y"P�NQ_v��Pa�6|4\����Q\]lWR��Q_&T R�i�^aP�NQ\¬�Q|6Y"�E|6R��a_vZ0_W_hx×T>_W�L\]��_WT²�Q|6R����NQ_&R��lWPaY"�4�4�Q_&T[Z0_Wi�\]_ w �­R��ONQ_Wi�\fR�Z6\�i�\¬�¢V�Y"_&R����>NQ_W�O�Q|6R��'Y�R&V[�Q|6_Wx�Z0_ w _&T"\�xc�QP#R�x6P��Q|6_hN�T4_WlWPLT>\�x6^R�i�^�P�NQ\¬�Q|6Y�§©Ø6P�����P�NQ\�_hN³I�JaJLK�«�jb è�#m���i��ê;%�êQ#�~���§k¸*��l�|4\���l�|6R�x6^>`�Ø>NQ_hVc`OR�xqT G Pc_Wi�\�^a_hNEI�J�J>K&«�\���RÄ^a_hx6_hN}R�i]\�»&R��Q\�Pax8P w R � R�x4x6_hN^�N}R��6|.`�U­|4Pa��_E^�N}R��4|�i�PcPa¹L�"_W����_hxL�Q\]R�i]i¬V�\]T4_Wxc�Q\�l&R�ik`�Z6�>�¤x4P�UÆ_WR�l}|��Qëc�qR�NQ_El&R�x�NQ_W�>NQ_W��_Wxc�Z#�ç��w �4x6lh�Q\�P�x´P w \¬�Q����R�NQ\]R�Z6i�_W��§A�Q|6_!x6PLT>_W�"\¬��\]��R����}R�l}|6_&T��QPc«�j D R�l}| w �4x6lh�Q\�P�x´l&R�x´|qR&�S_!�aR�N�V>\]x4^xL�6Y[Z0_hNQ�HP w �aR�NQ\]R�Z6i�_W�W`~\�x6lWi��6T4\�x6^#�¢�6����P�x6_aj � _[�}R�¹S_#�Q|4_H��P�\�xL���>NQPaZ6R�Z6\�i�\¬�¢VvT4\�����NQ\�Z6�>�Q\�Pax�P w Rw R�lh�QP�N�^�N}R��6|EP w�� ��R�NQ\]R�Z4i]_h�¥U­\¬�Q|Z¡ w �4x6lh�Q\�Pax4��QP¤Z0_

ô@§�ý&ñ.þ�«�� KU£¢�  �� S   §�ý�ñ.þ   « §�KaK�«

ü P��Q_��Q|6R��³U'_#���4Z6���6Y"_�R�xLVÔ ¢PaZ4��_hNQ�S_&T>£!x4PLT4_W�FKSò�\�xc�QP w �6x4lh�Q\�Pax6���Q|qR����Q|6_hVE^a_Wx4_hN}R��Q_[P w �Q|6_ ¢|4\]T6T4_hx6£¤x4PLT4_W�ñ0ò�j

Ú 9�¤ } á ß @!ÝHâ³r�Þ ßZ_¥_ â Ü =T]EÞ }[Ü E¦>Òá B ÝFE7���¥\��¥�S_hN�V!_&R���V!�QP"lWPaxL�S_hN���R�NQZ6\¬��N}R�N�V!�qR�\�N�U­\���_ C+F Ø'· �¥P�NHM�RWVc_W��\]R�x�x4_h�¢U'P�NQ¹L��\�xc�QP"_&ëc�6\���R�i�_Wxc�

Page 9: Understanding Belief Propagation and its Generalizations Jonathan

t¥?fï0ð2u�<Z| X P�xL�S_hN��Q\�x6^"R[�qR�\¬N�U­\���_ C+F Ø�\�xc�QP¤R w R�lh�QP�NH^�N}R��4|~j

w R�lh�QP�N­^�N}R��4|6�Wj>�¢x"°q^��4NQ_��[U�_���|6P&U R���Y�R�i�i��qR�\¬N�U­\���_ C+F Ø�R�x6T��Q|4_H_&ëc�4\]��R�i�_Wxc� w R�l��QP�N­^�N}R��6|.jü P��Q_��Q|qR��¨\]x4���Q_&RaT P w �4��\]x4^  ¢P�Z6��_hNQ��R�Z4i�_W£²x6PLT>_W�W`�U'_�\�xL��NQPLT>�6lW_Ä_&ëc�6\��aR�i�_Wxc� w R�l��QP�Nv^�N}R��4|w �4x6lh�Q\�P�x6��P w R���\]x4^ai�_¯�aR�NQ\fR�Z6i�_aj � |4_ w R�l��QP�NE^�N}R��4| w �6x4lh�Q\�Pax4�uS   §�ý�ñ.þ   «¤U­\]i�i*Z0_¨_&ëc�6\���R�i�_Wxc��QP#_h\��Q|4_hN­�Q|6_H��U�P[x6PLT4_ w �6x4lh�Q\�Pax4�dS�ò R>§kñqò¢úQñ�R�«'\ w �Q|4_hV�i�\�x6¹"�¢U'PÄ ¢|4\]T6T4_hx6£#x4PLT4_W�W`4P�N­�Q|4_³��\�x6^ai�_x6PLT>_ w �4x6lh�Q\�Pax4�dN ò §kñ ò úOK ò «'\ w �Q|4_hV�R�NQ_�R����}R�l}|6_&T��QP"R#��\�x6^�i]_¨ �|6\]T6T>_Wx6£[x6PLT4_�j�¢x�°q^��4NQ_xh¯U'_¯^�\��S_¨RÄ��Y�R�i�iHM�RWVc_W��\]R�x²x4_h�¢U'P�NQ¹�R�x6T��Q|6_�_&ëc�4\]��R�i�_Wxc� w R�lh�QP�N+^�N}R��6|.j ü P��Q\�lW_�Q|qR����Q|6_�NQ_´\���R�i¬URWV>��R w R�lh�QP�N�^�N}R��6| w �6x4lh�Q\�Pax i]\�x4¹>\�x4^yR8x6PLT4_ÔR�xqTÒR�i]i�P w \¬�Q���qR�NQ_hxL�Q�W`Z0_Wl&R��6��_*�Q|qR��H\��­�Q|6_ w P�NQYÆP w �Q|6_��¢Pa\�xc�³�4NQP�ZqR�Z6\�i�\���VET>\�����NQ\]Z4�4�Q\�Pax�\�x¨R"M�RWVc_h��\fR�x¨x6_h��U�P�NQ¹0j � |6_w R�lh�QP�N'^�N}R��6| w �4x6lh�Q\�P�x6��S   §�ý�ñ.þ   «;T4\¬NQ_Wl��Qi�V#lWP�N�NQ_h���0PaxqT¤�QP³�Q|4_�M'R&VS_W��\]R�x�x6_h��U�P�NQ¹[�>NQPaZqR�Z6\�i�\¬�Q\]_h�ô@§kñ ò ö �����>§kñ ò «�«�jØ@\�xqR�i]i¬Vc`>\�x�°6^a�>NQ_³å>`LU'_H^a\��S_��Q|6_ w R�lh�QP�N­^�N}R��6|�NQ_W�4NQ_W��_hxL�}R��Q\�Pax+P w �Q|4_ � �²å>`��$�²d�lWPLT4_H�Q|6R��U�_�T4\���lW�6����_WT#R�Z0P��S_�j � |6_'NQ_Wlh_W\��S_&T�Z6\¬�Q��KSò0R�NQ_'NQ_W�4NQ_h��_Wxc�Q_&T�\�x��Q_hNQY"�2P w �Q|6_ w �6x4lh�Q\�Pax4�2�Q|qR��;�Q|6_hV\�xqT>�6lW_*P�x+�Q|6_*lWP�N�NQ_W���0Pax6T4\�x6^"Z4\¬�Q�¥ñ ò j � |6_�NQ_[R�NQ_�R�i���P w R�lh�QP�N�^�N}R��6| w �6x6l��Q\]P�x6�¥lWP�N�NQ_h���0PaxqT>\�x6^�QP#�Q|4_³P�NQ\�^a\�x6R�i@�6R�NQ\¬�¢V�l}|6_Wl}¹ w �6x4lh�Q\�Pax4�WjM'_Wi�\�_ w �4NQPa�6R�^aR��Q\�PaxvR�i�^aP�NQ\¬�Q|6Y"�H���0_hlW\¬°ql[�QP�M�RWVc_h��\fR�x¨x6_h��U�P�NQ¹L�W`��6R�\¬N�U­\���_ C+F Ø'· �W`.R�xqT w R�lh�QP�N^�N}R��6|4�¥|6R&�S_³R�i]i~Z0_W_Wx+T4_W�S_hi]P��0_&T.`4R�xqT��Q|4_hV�R�NQ_�R�i�i~Y�R��Q|6_WY�R��Q\]lWR�i�i¬V!_WëL�4\��aR�i]_hxL�&`>Z4�4�\¬�U�P��6i]TlW_hN��}R�\�x6i¬VylWP�x w �6��\�x6^ w NQP�Y �Q|4_��0_&T6R�^aP�^a\�l&R�i"�0P�\�xL�vP w �L\�_hU �QP²�4NQ_W��_Wxc�vR�i�i#�Q|4_W��_�T4\¬Ñ�_hNQ_Wxc��S_hNQ��\�P�x6�Wj��¢x w R�lh�&`.R��HU�_[|qR&�S_­���6���H��_W_Wx.`.R�NQZ6\¬��N}R�N�V¯M�RWVc_W��\]R�x¯x6_h��U�P�NQ¹>�HR�x6T¨�6R�\¬N�U­\���_ C+F Ø'· �l&R�xÄZ0_¤_&R���\�i�VÄlWPaxL�S_�N��Q_&T¯\�xc�QP+_WëL�4\��aR�i]_hxL� w R�lh�QP�N�^�N}R��4|6�W`2R�x6TÄR��³U'_"��|qR�i]i���_W_"\�x�R!Y"PaY"_Wxc�&`Pax4_­l&R�x�R�i]��P�lWP�xL�S_hN��'R�NQZ6\¬��N}R�N�V w R�lh�QP�N'^�N}R��6|6��\�xc�QP*_&ëc�6\��aR�i�_Wxc�'M'R&VS_W��\]R�x�x6_h��U�P�NQ¹>��P�NO�qR�\¬N�U­\]��_C+F Ø'· �Wj � _[l&R�x¯�Q|L�4�³l�|4PcPa��_��QP!U�P�NQ¹EU­\¬�Q|ÄM�RWVc_h��\fR�xÄx4_h�¢U'P�NQ¹L�W`��qR�\�N�U­\���_ C+F Ø'· �W`.P�N w R�lh�QP�N^�N}R��6|4�­U­\¬�Q|4Pa�4�¥i�Pa��\�x4^�R�xcV�^a_Wx4_hN}R�i�\¬�¢Vcj� _;U­\�i�iSl}|6PcPa��_��QP w PclW�4�2P�x*�qR�\¬N�U­\���_ C+F Ø'· �Wj �4N�NQ_&R���Pax w P�N2�Q|4\]�.l}|6Pa\�lW_�\��2�4�4NQ_Wi¬V��0_&T6R�^aPa^�\�l&R�i �R�i¬�Q|4Pa�6^�|�R�i�iq�Q|6_�T4\¬Ñ�_hNQ_Wxc��M�ª�R�i�^aP�NQ\¬�Q|6Y"�R�NQ_�Y�R��Q|6_WY�R��Q\]lWR�i�i¬V"_&ëc�6\���R�i�_Wxc�&`a�Q|4_��S_�NQ��\]P�x�P w M�ªw P�N­�qR�\�N�U­\���_ C+F Ø'· �¥\����P�Y"_hU­|qR����\�Y"�6i�_hN­\�x w P�NQY Z0_hl&R��4��_³\¬�­|qR��P�x6i¬V�Pax4_³¹L\�xqT"P w Y"_h���QR�^a_�`U­|6\�i�_³�Q|6_HM�ª´R�i�^aP�NQ\¬�Q|4Y"� w P�N�Q|4_³P��Q|4_hN­^�N}R��6|6\�l&R�i~Y"PLT>_Wi��­R�NQ_³x4P�NQY�R�i]i¬V!T4_h��lhNQ\�Z0_&T!�4��\]x4^[��U�P

Page 10: Understanding Belief Propagation and its Generalizations Jonathan

t­?Bï0ð~u�<x¤ X P�x>�a_hN��Q\�x6^"R¤M'R&VS_W��\]R�x ü _��¢U�P�NQ¹�\]xc�QP"R w R�l��QP�N�^�N}R��4|~j

¹L\�xqT4��P w Y"_W���QR�^a_W�Wj � |4_+x6_h_&T w P�N¤�¢U'P¯¹L\�xqT>�#P w Y"_h���QR�^a_h�"R�NQ\���_W�"_W\¬�Q|4_hN¤Z0_WlWR��6��_�P w �Q|6_!R�N�¦NQP&U­��\]xÄM�RWVc_W��\]R�xvx6_h��U�P�NQ¹>�W`~P�N*�Q|4_#�¢U'P+¹L\�xqT>�³P w x4PLT4_W�*\�x w R�l��QP�N�^�N}R��6|6�Wj.��x�R�xcV¨l&R���_a`.�Q|6_M�ªµR�i]^�P�NQ\¬�Q|6Yà�Q|qR��[Pax4_+T>_hNQ\��S_W�[ZLVÄ�6��\�x6^��Q|4_!�6R�\¬N�U­\���_ C+F Ø w P�NQY#�4ifR��Q\�Pax�Pax�R�lWPaxL�S_�N��Q_&TM�RWVc_W��\]R�x�x4_h�¢U'P�NQ¹[P�N w R�l��QP�N�^�N}R��4|"\]�O�4NQ_Wlh\]��_hi�V¤Y�R��Q|6_hY�R��Q\�l&R�i�i¬V[_&ëc�6\��aR�i�_Wxc�'R��'_h�S_hN�V¤\¬�Q_hN}R��Q\�Pax�QP[�Q|4_³M�ª�R�i�^�P�NQ\¬�Q|6YºR��\¬�­\]�x4P�NQY�R�i]i¬V�T>_W��lhNQ\�Z0_&T w P�N­�Q|4_HP��Q|4_hN^�N}R��6|4\�l&R�i.Y"PLT4_Wi��W`L��P#U'_�U­\�i�iR�l��Q�qR�i�i¬V!i]P���_�x6P��Q|4\�x6^"R��¥R�i�i2ZcV��6��\�x6^[�Q|6\��­T4_W��l�NQ\]�>�Q\�Pax~j� |6_�NQ_&R�T4_hN³U­|4P�\]��\�Y"�qR��Q\�_Wxc�¥�QP!i�_&R�NQx¯R�Z0Pa�>�*M�ª²l&R�x¨x6P&Uy��¹L\��¯R�|6_WRaTE�QP"�Q|6_�x6_h�L����_Wlh�Q\�Pax.`Z6�>� w P�N��Q|6_¤�4�4NQ�0Pa��_h�*P w lWP�Y"�6i�_h�Q_Wx4_W���³U�_#_h�>�6i]R�\�xÄ|4P&Uµ�QP+lWP�x>�a_hN���R�NQZ4\���N}R�N�V w R�l��QP�N�^�N}R��6|6�\�xc�QP¯_WëL�4\��aR�i]_hxL�#M�RWVc_W��\]R�x�x4_h�¢U'P�NQ¹L�¤R�xqT��6R�\¬N�U­\���_ C+F Ø'· �Wj � P¯lhPaxL�S_hN��#R w R�l��QP�N"^�N}R��6|�\�xc�QPR�xv_&ëc�6\���R�i�_Wxc��M�RWVc_W��\]R�x�x6_h��U�P�NQ¹�`.lWP�xL�S_hN���_W�S_�N�V w �6x4lh�Q\�Paxvx6PLT4_#\]xc�QP�R�x× �PaZ6��_�NQ�aR�Z6i�_W£Ex4PLT4_�Q|qR��#\��[PaZ6��_�NQ�S_&TÄ�QP�Z0_�\�x�\��Q��°6NQ���[���}R��Q_aj � |6_Wx�Y�R�¹S_¤�Q|6_��4NQP�ZqR�Z6\�i�\���V�P w �Q|4_"°6NQ���[���}R��Q_�P w�Q|qR��HP�Z6��_hNQ��R�Z4i�_[x4PLT4_a`0^�\��S_Wx�\¬�Q�H�6R�NQ_Wxc�Q�W`0_&ëc�4\]��R�i�_Wxc�­�QP"�Q|6_ w R�lh�QP�N³^�N}R��4| w �4x6lh�Q\�Pax��Q|6R��¥UR��_Wi�\�Y"\�xqR��Q_WTv§k��_W_*°q^��4NQ_�vcj «� _"��|4P&U¶�Q|6_"lhPaxL�S_hNQ��\�Pax�P w R w R�l��QP�N¤^�N}R��4|�\]xc�QP¨RE�6R�\¬N�U­\���_ C+F Ø ^�N}R��4|6\�l&R�i�i¬V�\�xÄ°6^a�4NQ_�ÙLjD R�l}| w �6x6l��Q\]P�x�\�x��Q|6_ w R�l��QP�N�^�N}R��6|�\��"lWPaxL�S_hN��Q_WT�\�xc�QPvRv|6\]T4T4_Wx�x6PLT>_+\�x�R¯�6R�\¬N�U­\���_ C+F ØU­\¬�Q|¨R�xEP�Z6��_hNQ��R�Z4i�_�x4PLT4_*|6R�x6^�\�x6^"P�ÑvP w \¬�&jqØ6P�N�_h�4R�Y"�4i]_�`6R w �6x4lh�Q\�Pax�S   §kñqò¢úQñ�Rcú�ñ � «P w �Q|>NQ_W_x6PLT>_W��\]x¤R w R�l��QP�N�^�N}R��4|¤U­\]i�iqZ0_�NQ_h�4NQ_W��_Wxc�Q_&T¤ZcV[R³x6_�U´|4\fT4T4_Wx#x6PLT4_ñ   R�x6T"R³x6_hU�PaZ4��_hNQ��R�Z6i�_x6PLT>_uK   j � |6_!x6PLT>_+ñ   l&R�x�Z0_+\�x�R��"Y�R�xLVÄ���}R��Q_W��R��¤�Q|6_!�4NQPLT4�4lh�¤P w R�i]i�Q|4_+��R�NQ\]R�Z4i]_h�"P w�Q|6_�lWP�N�NQ_W���0Pax6T4\�x6^ w �4x6lh�Q\�P�xaS   j�Ø6P�N#_h�4R�Y"�4i]_�`@\ w �Q|4_ w R�lh�QP�N"^�N}R��6| w �4x6lh�Q\�P�x��Q|6R��[U�_�U�_hNQ_lWP�x>�a_hN��Q\�x6^�UR��3S   §kñ0ò�ú�ñ�RSú�ñ � «�`>R�xqT"ñ0ò�`>ñ�RS`>R�x6T�ñ � U'_hNQ_³_&R�l�|�Z6\�xqR�N�V�x6PLT>_W�'�}R�¹L\�x6^[Pax��aR�i��6_W��EP�N;�a`.�Q|6_Wx¯�Q|6_"x4_hU x6PLT4_"ñ   lWPa�4i]Tv�}R�¹S_"P�x�_W\�^a|c����R�i��6_W�*N}R�x6^a\�x4^ w NQPaY§�m�(�+�QP��m�m�aj � |6_ ¢_h�>\]T>_Wx6lW_h£�N;§kñ   úQK   «~\��;��_h�;�QP�lhP�N�NQ_W���0P�xqT��QP¥�Q|6_ w R�l��QP�N�^�N}R��6| w �6x4lh�Q\�PaxLSH§�ý�ñ�þ�«�j�Ø6P�N@_h�4R�Y"�6i�_a`\ w SH§�ý&ñ.þ�«@\��'R³�6R�NQ\¬�¢V[l}|6_Wl}¹ w �6x4lh�Q\�Pax~`��Q|qR�x$N;§kñ   úQK   «@U­\�i�ia�¢�6����Z0_­_&ëc�qR�iL�QP¤K\ w ñ   lhP�N�NQ_W���0P�xqT4�

Page 11: Understanding Belief Propagation and its Generalizations Jonathan

t­?Bï0ð2u�</¨$� |6_ w R�lh�QP�NH^�N}R��6|!NQ_W�4NQ_W��_hxL�Q\�x4^#�Q|6_'�¢Pa\�xc�¥�4NQPaZ6R�Z4\]i�\¬�¢V w �4x6lh�Q\�P�xEP w R[��Y�R�i�i~�6R�NQ\¬�¢Vl}|6_Wl}¹�lWPLT4_aj

�QP[R����}R��Q_�U­\¬�Q|!R�x�_W�S_Wx�xL�6Y[Z0_hN�P w Pax4_W�R�x6T�J�P��Q|6_hN�U­\���_aj4Ø@\�xqR�i]i¬Vc`cU'_�x4_W_&T¤�QP[\�xc��NQPLT4�4lW_­�Q|6_x6P��}R��Q\�Pax�ñ   §kó�«��QP�T4_Wx4P��Q_³�Q|6_*���}R��Q_*�Q|qR��­�Q|4_*ó©�Q|�x4PLT4_*��|4Pa�6i]TEZ0_*\�xE�QP¤lWP�N�NQ_W���0Pax6T+U­\¬�Q|E�Q|6_��x6PLT>_*Z0_W\�x6^¤\�x+���}R��Q_*ñ   j � |6_*lhPaY"�qR��Q\�Z6\�i�\���V w �6x4lh�Q\�Pax4�dS   òQ§kñ   ú�ñqò�«�Z0_h��U�_W_hxER#x4_hU x6PLT4_Hñ  R�x6T+R�xEP�N}T>\]x6R�N�V!x6PLT>_³ñ ò ��|6Pa�4i]T!Z0_*��_h�­�QP¤P�x6_³\ w ñ   §kó�«7�Ôñ ò R�x6T!»W_�NQP"P��Q|6_�N�U­\]��_�j© Ð�ªE� À�Ç � õǬ«ÔÂ Ë ¿4Â�Á É Ã Î8ÉE� È �"ª ¿ Î ÀØ6P�N��Q|4_�NQ_WR���Pax4�H^�\]�a_Wx�\�x+�Q|6_��4NQ_hlW_&T4\�x4^"��_Wlh�Q\�Pax.`qU�_*U­\�i�i w PclW�4�HPax��6R�\¬N�U­\���_ C+F Ø'· �Wj � _�l&R�xlWP�x6��\]T4_hNH�Q|6_Ä �PaZ4��_hNQ�S_&T>£!x4PLT4_W��K ò �QP!Z0_�°6�>_&T�`qUNQ\¬�Q_DN ò §kñ ò «�R���R���|6P�N���¦½|qR�x6T w P�NTN ò §kñ ò úQK ò «�`R�x6T w PclW�6�¥Pax+�Q|4_'��P�\�xL�¥�>NQPaZqR�Z6\�i�\¬�¢V+T4\�����NQ\�Z6�4�Q\�P�x w P�N¥�Q|4_³�6x4¹Lx6P&U­x+��R�NQ\]R�Z4i�_W�­ñ0ò�ì

ô@§�ý&ñ.þ�«�� KU � V ò R�W S�ò Rc§kñ0ò�ú�ñ�R�« � ò N.ò�§kñ0ò½« §�K�Ia«

�¢x��Q|4_+M�ª R�i�^�P�NQ\¬�Q|6YE`;U'_+\�xc��NQPLT4�6lh_+��R�NQ\]R�Z4i]_h�¤���6l}|´R�� y ò R §kñ R «�`;U­|4\�l�|�l&R�x�\�xc�Q�6\¬�Q\��S_Wi¬V�Z0_�6x6T4_hNQ���QPcPLT�R��³R� �Y"_W���QR�^�_W£ w NQPaYnR¤|6\]T6T>_WxEx4PLT4_�ó'�QP¤�Q|6_�|4\fT4T4_Wx+x6PLT4_FX�R�Z0Pa�>��U­|6R������}R��Q_x6PLT>_kXÄ��|6P��6i]T�Z0_!\�xקk��_W_+°6^a�>NQ_�eS«�j � |6_+Y"_W���QR�^�_ y ò RL§kñ[R�«[U­\�i�i�Z0_+RÄ�S_Wl��QP�N"P w �Q|6_!�QR�Y"_T4\�Y"_Wx4��\�PaxqR�i�\���V�R��'ñ�Rc`aU­\��Q|�_&R�l�|�lWP�Y"�0Pax6_hxL�OZ0_W\�x6^��4NQP��0P�N��Q\�Pax6R�i0�QP�|6P&U²i�\�¹S_Wi¬V"x4PLT4_¥ó2�Q|6\�x6¹L�\¬�³\��H�Q|6R��³x4PLT4_�X�U­\�i�i�Z0_�\�x��Q|6_�lWP�N�NQ_W���0P�xqT>\]x4^!���}R��Q_aj��¢xE�Q|4_#M'ª R�i�^aP�NQ\¬�Q|4YE`0�Q|6_�Z0_Wi�\]_ w R��*Rx6PLT>_�ó'\��H�>NQPa�0P�N��Q\�P�xqR�i~�QP��Q|4_*�4NQPLT>�6lh��P w �Q|6_�i�Pcl&R�i@_W�L\]T4_Wx4lW_�R��¥�Q|qR���x6PLT>_�§&N ò §kñ ò «�«�`qR�xqT�R�i�i�Q|6_HY"_W���QR�^�_W��lhPaY"\�x6^[\]xc�QP¤x4PLT4_*ó�ì: ò §kñ ò «��*��N ò §kñ ò « �R1­[� V ò�W y R}ò §kñ ò « §�K&dS«

U­|6_�NQ_u��\��"R�x6P�NQY�R�i�\�»&R��Q\�P�x�lWP�x6���}R�xL�+§A�Q|6_�Z0_Wi�\�_ w �"Y[�6�������6Y��QP�K�«�R�xqT � §kó�«�T4_Wx4P��Q_W���Q|6_x6PLT>_W�­x6_W\�^�|>Z0P�NQ\�x6^¤ó¥§k��_W_*°q^��4NQ_#KWJ>j « � |4_*Y"_W���QR�^�_W��R�NQ_�T4_��Q_hNQY"\�x6_&T���_Wi w ¦½lWP�x6��\����Q_Wxc�Qi¬VEZcV��Q|6_Y"_W���QR�^a_*�6��T4R��Q_HNQ�4i�_W�Wìy ò RL§kñ�R�«7® )q+1o N�òQ§kñ0ò©«�S�ò RL§kñ0ò�ú�ñ�R�« �� ­m� V ò�W,¯iR y � ò §kñ0ò©«� §�KW�c«

Page 12: Understanding Belief Propagation and its Generalizations Jonathan

t­?Bï0ð2u�<{° X PaxL�S_hN��Q\�x6^¤R w R�lh�QP�N�^�N}R��6|+\�xc�QP"R¤M'R&VS_W��\]R�x�x6_��¢U�P�NQ¹0j

ü P��Q_��Q|6R��P�x��Q|4_¥NQ\�^a|c��¦½|qR�xqTL¦���\]T>_a`cU�_¥�}R�¹S_¥�Q|6_H�>NQPLT4�4lh��P��S_hNR�i�i�Y"_W���QR�^�_W�^aP�\]x4^#\�xc�QP#x4PLT4_Hó_h�>lW_W�>� w P�NO�Q|6_P�x6_­lWPaY"\�x4^ w NQPaYÒx4PLT4_7X4j � |6_­Y"_W���QR�^a_h¦½�4��T6R��Q_�NQ�4i�_¥\�����|6P&U­x�T4\]R�^�N}R�Y�R��Q\�l&R�i�i¬V\�x+°q^��4NQ_#K�Kaj� |4_hNQ_�T4P"�Q|6_W��_�NQ�6i�_W��lWPaY"_ w NQPaYM±��½�³\��Hx4P��¥�QPcP�|qR�N}TE�QP"��|4P�U×�Q|qR��¥�Q|6_h��_�NQ�4i�_W�H^�\]�a_[Z0_Wi�\�_ w ��Q|qR���R�NQ_*_h�4R�l���\ w �Q|4_³�qR�\�N�U­\���_ C+F Ø�\��� ¢��\�x6^�i¬V>¦½lWP�x6x4_Wlh�Q_&T>£4ä6�Q|6R��­\��W`q\ w �Q|4_hNQ_*R�NQ_�x4P¤i�PcPa�6�¥\�x�Q|6_"�6R�\¬N�U­\���_ C+F Ø'j � _¤U­\�i�i�x6P��[^�\��S_�R+�4NQPcP w |4_hNQ_a`~Z6�>�¥�¢�6���[RE��Y�R�i�i�_h�4R�Y"�4i]_¤�Q|6R���Y"\�^a|c�|6_hi]��lWP�xL�>\�x4lW_�Q|6_NQ_&RaT>_hN&j X Pax4��\]T4_hN��Q|4_­x6_h��U�P�NQ¹�U­\¬�Q| w Pa�>N|6\]T6T>_Wx"x6PLT>_W�'��|4P&U­x�\ w °q^��4NQ_�K�Icj� _�U­\]i�i~lWPaY"�4�4�Q_H�Q|4_³Z0_Wi�\�_ w R��¥x6PLT>_���K*�4��\�x6^#�Q|4_*Z0_Wi�\]_ w �4NQP��qR�^aR��Q\�PaxENQ�4i�_W�Wj � _³|qR&�S_:w�§kñJ}«��²��N�&§kñ}« y ���§kñ}« §�K5ha«�H��\�x4^#�Q|6_³Y"_h���QR�^a_�¦��4��T6R��Q_³NQ�6i�_W� w P�N y ���§kñ}«�`>U�_³°6xqT: §kñ «��²��N §kñ «�)&+10�S q� §kñ ú�ñ � «�N � §kñ � « y 8O� §kñ � « y �O� §kñ � « §�K&åS«

�H��\�x4^#�Q|6_³Y"_h���QR�^a_�¦��4��T6R��Q_³NQ�6i�_W� w P�N y 8O� §kñ � «�R�xqT y �O� §kñ � «'U�_³°6xqT:.&§kñ}«��b��N��§kñQ«�),+10�S�q�a§kñWúQñ��W«�N³��§kñ��W«�),+1´ZN�8S§kñ�8&«�S���8S§kñ���ú�ñ�8&«�)&+9µxN��a§kñ��&«�S����S§kñ���úQñ��&«8§�K5va«Ø@\�xqR�i]i¬Vc`0ZcV�NQ_WP�NQ^aR�x4\]»h\]x4^��Q|4_����6Y"�W`q\¬�H\]�¥_&R���VE�QP���_��Q|6R��¥�Q|6_�Z0_Wi�\�_ w R���x6PLT4_���K�\����Q|4_��QR�Y"_R���Q|4_*_h�4R�lh�­Y�R�NQ^a\�xqR�i~�>NQPaZqR�Z6\�i�\¬�¢V+R��¥x6PLT>_¤Kaì:w&§kñJ}«7�b� )+101¶ +1´1¶ +9µ ô@§�ý�ñ�þ�«���ô&§kñ}« §�K&ÙS«

Page 13: Understanding Belief Propagation and its Generalizations Jonathan

t¥?fï0ð2u�</· X P�xL�S_hN��Q\�x6^"R w R�l��QP�N�^�N}R��4|+\�xc�QP"R#�6R�\¬N�U­\���_ C+F Ø'j

����\���_&R���V��QPÄlWPaxL�L\�x6lW_EP�x6_W��_hi w `R�xqT��QPv�4NQP��S_a`��Q|6R���M�ª \]x w R�lh��^a\��S_W�"�Q|6_+_h�4R�lh��Y�R�NQ^a\�xqR�i�4NQP�ZqR�Z6\�i�\��Q\�_W� w P�N³R�i�i.�Q|4_*x6PLT4_h�\]xER�xcV���\]x4^ai¬VL¦½lWPax4x6_Wlh�Q_WT+^�N}R��6|~j�¢x8R��>N}R�lh�Q\�l&R�i*lWPaY"�4�4�}R��Q\�P�x~`P�x6_¯���}R�N��Q�!U­\¬�Q|²�Q|6_¨x4PLT4_W�+R��+�Q|4_v_WT4^a_¯P w �Q|4_v^�N}R��6|.`­R�x6TPax4i¬V�lhPaY"�6�>�Q_W��RvY"_h���QR�^a_EU­|4_Wx´P�x6_�|6R���R&�aR�\]i]R�Z6i�_¨R�i�i­�Q|6_EY"_W���QR�^a_W��x4_WlW_W���QR�N�Vcj � |L�6�W`O\�xPa�>N¨_h�4R�Y"�4i]_�`�P�x6_�U'Pa�6i]T ���}R�N��¨U­\¬�Q| y 8O��R�xqT y �O��`HR�xqT8�Q|6_Wx lWP�Y"�6�4�Q_ y ��&`�R�x6T �Q|4_Wx°qx6R�i�i¬V�lWPaY"�4�4�Q_�:.Wj4�¢x�^a_Wx4_hN}R�ik`q\¬�­\��­_&R���V��QP¤��_h_H�Q|qR��­_&R�l}|EY"_W���QR�^a_³x4_W_&T!P�x6i¬V�Z0_³lWP�Y"�6�4�Q_WTPax4lW_ w P�N���\�x6^�i�V¨lhPax6x4_Wlh�Q_&T¨^�N}R��6|4�Wj � |6R���Y"_&R�x6�H�Q|6R��³�Q|4_[U­|4Pai�_¤lWP�Y"�6�4�}R��Q\�Pax¨�}R�¹S_W�³R��Q\�Y"_�4NQP��0P�N��Q\�Pax6R�i¥�QPÄ�Q|6_ExL�4Y#Z0_hN"P w i�\�x6¹L��\]x��Q|6_E^�N}R��6|.`OU­|4\�l�|Ô\���T>N}R�Y�R��Q\�l&R�i]i¬V�i�_W�����Q|6R�x��Q|6__h�>�0Pax4_Wxc�Q\]R�i�i¬V[i]R�NQ^a_��Q\�Y"_'�Q|qR��;U�P��6i]T[Z0_'NQ_WëL�4\¬NQ_&T��QP�lWPaY"�4�4�Q_'Y�R�NQ^a\�xqR�i4�>NQPaZ6R�Z6\�i�\¬�Q\�_W��xqR�\��S_hi�Vcj� |6_"�0P�\�xL��P w �L\]_�U�\�i�i��6����N}R��Q_&T�ZcV¯�Q|4\���_��6R�Y"�6i�_¤���4^a^a_h���Q���Q|6R���Z0_Wi�\�_ w �4NQP��qR�^SR��Q\�P�x�\]�[R!U�R&VP w P�NQ^SR�x6\�»W\�x6^8�Q|6_µ �^ai�PaZ6R�i�£ lhPaY"�6�>�}R��Q\�Pax×P w Y�R�NQ^�\]x6R�i[Z0_Wi�\]_ w �Ä\�x×�Q_�NQY"�¯P w ��Y�R�i�i�_hNÄi�Pcl&R�ilWP�Y"�6�4�}R��Q\�Pax6�Wj � |6_�Y"_W���QR�^�_7IqP&UÄ\]�@R�¹L\�x��QP�Q|4_�I6P&U�P w RNQ\]�a_hN&`�R�x6T*_&R�l}|�Y"_W���QR�^�_O���4Y"Y�R�NQ\�»W_W�R�i�ic�Q|6_OlWPaY"�4�4�}R��Q\]P�x6�2�Q|6R��;PclhlW�4N�NQ_&T w �>N��Q|6_hN��4�6����NQ_&R�YyR�i]P�x6^¥�Q|6_'Z>N}R�x4l�|4_W�2�Q|qR�� w _h_&T[\�xc�QP¥�Q|6R��Y"_W���QR�^a_aj � |L�6��\�x#Pa�>N�_h�4R�Y"�6i�_a` y ��&§kñ}«2\���RH���6Y"Y�R�N�V�P w R�i�i>�Q|4_�lWPaY"�4�4�}R��Q\]P�x6�@�Q|6R���PclWlW�>NQ_&TR���x4PLT4_W�'IL` d>`cR�xqT[�4j � |4\��O\��;�Q|6_lWi]R�����\]lWR�i0�0Pa\�xc��P w �L\�_hU*`cR�xqT#\¬�O��_W_WY"�;�QP³���4^a^a_h���'�Q|qR���\���Y[�6���Z0_#�a_hN�V�\�Y"�0P�N��}R�xL�H�Q|qR��H�Q|6_hNQ_[Z0_#x6P�i�PcPa�4�*\]x¨�Q|6_[�qR�\�N�U­\���_ C+F Ø'` w P�N�P��Q|6_�N�U­\]��_¤�Q|4_#_Wxc�Q\¬NQ_R�NQ^��6Y"_Wxc� w P�N­�Q|6_³_��6R�lh�Qx6_h����P w M�ª�Z>NQ_&R�¹L�¥T4P&U­x.j{ P w P�N@�Q|4_��Q\�Y"_�Z0_W\�x6^>`&i�_h�;�6�~lWPaxc�Q\�xL�6_�Pa�>N;T>\]��lh�6����\�Pax�P w ��\�x6^ai¬VL¦½lWPax4x6_Wl��Q_&T"§ki]PcP��4¦ w NQ_h_�«~�qR�\¬N�U­\]��_C+F Ø'· �Wj � _*U­\]i�i@°6xqTE\¬�H�a_hN�VElWP�xL�S_Wx6\�_Wxc�¥�QP�\]xc��NQPLT>�6lW_*�Q|6_*��U�P�¦½x4PLT4_�Y�R�NQ^a\�x6R�i@�>NQPaZqR�Z6\�i�\¬�Q\]_h�ô ò R §kñ ò ú�ñ R «�` w P�NO��U�P³x4_W\�^a|LZ0P�NQ\�x6^*��\¬�Q_W��ó@R�xqTTX4`�U­|6\�l}|"R�NQ_­PaZ>�}R�\�x6_WT#ZcV#Y�R�NQ^a\�xqR�i]\�»W\�x4^³�Q|6_@��P�\�xL�

Page 14: Understanding Belief Propagation and its Generalizations Jonathan

t­?Bï0ð2u�</¸$b xE\�i�i��6����N}R��Q\]P�x�P w �Q|4_*Y"_W���QR�^�_W�­�qR�����_WTE\]xEM�ª@j y ò R §kñ R «�l&R�xEZ0_³�Q|6P��6^�|L�­P w R��¥R ¢Y"_h���QR�^a_h£ w NQP�YÆR[|6\]T4T4_Wx�x6PLT>_³ó��QP��Q|6_�|6\]T6T>_Wx!x4PLT4_YX"R�Z0Pa�4�U­|6R��­���}R��Q_³x6PLT>_YX¤��|6Pa�4i]T!Z0_\�x~j

�4NQP�ZqR�Z6\�i�\���V w �6x4lh�Q\�Pax+P��S_�N�_h�S_hN�V�x6PLT>_³_h�>lW_W�>�¥�Q|4_H�¢U'P¤x6PLT>_W�­ó'R�xqT�X4ìô0ò R>§kñ0ò¢úQñ�R�«�ÿ )¹5º ¹ o�» � V +1o�¶ +9» W ô@§�ý.¼Lþ�«� §�K&eS«

� _�UNQ\¬�Q_�R³Z0_Wi�\�_ w _&ëc�qR��Q\�P�x w P�N'�Q|6_���U�P�¦�x4PLT4_­Z0_Wi�\�_ w �7:Wò R>§kñ0ò�ú�ñ[R�«;R�x6R�i�Pa^�Pa�4��i�V"�QP*_WëL�6R��Q\�Pax+§�K&dS«w P�N­�Q|6_*Pax4_h¦½x6PLT>_³Z0_Wi�\�_ w �Wì:Wò RL§kñ0ò�ú�ñ[R�«��b��S�ò R>§kñqò�ú�ñ�R�«�N�ò�§kñ0ò½«�N�RL§kñ[R�« �� ­[� V ò�W,¯qR y � ò §kñ0ò½« �½ ­m� V ò�W,¯Qò y ½ R §kñ�R�« §½I�JS«

U­|6_�NQ_T�M¾ � §kó�«Q¿�X¤Y"_&R�x4�'�Q|qR���U�_HlWPax4��\fT>_hN�R�i�i�x6_W\�^�|>Z0P�NQ�Y��P w x4PLT4_³ó�_h�>lW_W�>� w P�N­x6PLT>_YX4j � _T4_h��lhNQ\�Z0_O�Q|6\��~_&ëc�qR��Q\�P�x#T>\fR�^�N}R�Y"Y�R��Q\�l&R�i�i¬V*\�x�°q^a�>NQ_¥KWd>j��½�@l&R�x[Z0_����4���Q\�°6_&T*Pax���\�x6^ai¬VL¦½lWPax4x6_Wl��Q_&T^�N}R��6|4��ZcV�U'P�NQ¹L\�x6^EP��4���Q|qR�� w P�N����4l�|v^�N}R��4|6�*�Q|6_[�¢U�P�¦½x6PLT4_"Z0_hi]\�_ w ��U­\�i�i�lWP�N�NQ_W���0P�xqTv�QP!�Q|6__h�4R�l��¥��U�P�¦�x4PLT4_³Y�R�NQ^a\�xqR�i2�>NQPaZqR�Z6\�i�\¬�Q\]_h�Wj���'\���R�i���P�U�P�N��Q|�x4P��Q\�x6^��Q|qR��'�Q|qR��'ZcV"lhPaY#Z4\�x6\�x6^*_&ëc�6R��Q\�Pax4�H§�K&da«'R�x6T�§½I�JS«;U­\¬�Q|��Q|4_�Y�R�NQ^a\�xqR�i�¦\�»&R��Q\�P�x�lWPax6T4\¬�Q\�Pax :hòQ§kñqò�«��*)�+9»x:hò Rc§kñ0ò�ú�ñ�R�« §½ILK�«

U�_#lWPax4��\]���Q_hxL�Qi¬VvT4_�NQ\]�a_[�Q|6_#Y"_W���QR�^�_¤�6��T4R��Q_[NQ�4i�_W�"§�KW�S«�j'§ { _W_[°6^a�4NQ_!KW� w P�N[R+T>\]R�^�N}R�Y"Y�R��Q\�l�S_hNQ��\�P�xEP w �Q|4\���T>_hNQ\��aR��Q\�Pax~j «� |6_¨M'ª¶R�i�^�P�NQ\¬�Q|6YE`'R���T4_�°qx6_WT�\�x��Q_�NQY"�"P w �Q|4_EZ0_Wi�\�_ w _&ëc�qR��Q\�Pax6�¨§�KWdS«"R�x6T §½I�Ja«�`�R�xqT��Q|6_Y"_W���QR�^a_h¦½�6��T4R��Q_�NQ�6i�_W�¤§�Kh�c«�`.T>Pc_W�³x6P��³Y�R�¹S_�NQ_ w _hNQ_Wx4lW_[�QP"�Q|4_��QP��0Pai�Pa^�VvP w �Q|4_[^�N}R��4|¨�Q|6R��*\¬�\��'NQ�6x4x6\�x6^�P�x~j � |L�6�W`S�Q|4_hNQ_H\���x4P��Q|4\]x4^*�QP[���QP����4� w NQP�Y¶\�Y"�6i�_WY"_hxL�Q\�x4^*\��'Pax!R�^�N}R��4|��Q|6R��'|qR��i�PcPa�4�Wj x6_#���}R�N��Q�*U­\¬�Q|���PaY"_"\�x6\¬�Q\]R�i���_h��P w Y"_W���QR�^�_W�!§kP�x6_"�4���qR�i�i¬V¯Z0_W^a\�x4��U­\¬�Q|�lhPaY"�6i�_h�Q_hi�V

Page 15: Understanding Belief Propagation and its Generalizations Jonathan

t­?Bï0ð2u�< Ú�À b T>\fR�^�N}R�Y"Y�R��Q\�l NQ_W�4NQ_h��_Wxc�}R��Q\�Pax P w �Q|6_ÆM�ªÏZ0_hi]\�_ w _&ëc�6R��Q\�PaxÁ: ò §kñ ò «Â���N ò §kñ ò «�� R5­m� V ò�W y R}ò §kñ ò «�j�6xLZ4\fR���_&T�Y"_W���QR�^a_W�}«�`�R�xqT���\�Y"�6i¬V�\¬�Q_hN}R��Q_W�#�Q|6_�Y"_W���QR�^a_h¦½�4��T6R��Q_�NQ�6i�_W�[�6xc�Q\�iO�Q|4_hVÔ§k�0P�����\]Z4i¬Vq«lWP�x>�a_hNQ^a_a`SR�x6T#�Q|6_Wx¤P�x6_­l&R�x¤NQ_&RaT¤P�Ñ!�Q|4_­R��6�>NQP&�4\�Y�R��Q_Z0_Wi�\]_ w � w NQPaYÒ�Q|6_­Z0_Wi�\�_ w _&ëc�qR��Q\�Pax6�WjcM'�4�Pax4_OU'Pa�6i]T*x6P��;x6_hlW_W���QR�NQ\�i¬V�_h�>�0_Wlh�~�Q|6_'R�i�^aP�NQ\��Q|4Y �QP­U�P�NQ¹³U�_Wi�ikj b �@ª@_WR�NQiSUR�NQx6_&T�`� ¢\ w U'_'\�^ax4P�NQ_�Q|6_�_h�>\����Q_Wx6lh_#P w i�PcPa�4��R�xqT��0_hNQY"\¬�¥�Q|6_[x6PLT4_h���QP�lWPaxc�Q\�xL�6_�lWPaY"Y[�6x6\�l&R��Q\�x6^¤U­\¬�Q|v_WR�l}|vP��Q|6_hNR��\ w �Q|4_Hx4_h�¢U'P�NQ¹¤U�_�NQ_*��\]x4^ai¬V�lWPax4x6_Wl��Q_&T.`>Y"_W���QR�^a_W�Y�RWV�lW\¬NQlW�4ifR��Q_³\�xqT>_h°qx4\��Q_hi�V�R�NQPa�4xqT��Q|4_W��_i�PcPa�4�W`2R�xqTE�Q|6_��4NQPclh_W���*Y�RWVEx6P��³lWP�xL�S_hNQ^a_��QP�R����}R�Z6i�_#_&ëc�4\]i�\�Z4NQ\��4Y"£¯§©ª@_WR�NQi�K&e�ÙaÙa«�j x4_[l&R�x\�xqT>_W_&T¤°qx6T�_��6R�Y"�6i�_W�OP w ^�N}R��4|6\�l&R�i�Y"PLT>_Wi��'U­\¬�Q|!i�PcPa�4�W`LU­|6_hNQ_a` w P�N­lW_hN��}R�\�x!�6R�N}R�Y"_��Q_hN���R�i��6_h�W`�Q|6_¥M�ªÄR�i�^aP�NQ\��Q|4Y w R�\�i��'�QP*lWP�x>�a_hNQ^a_a`SP�NO�4NQ_&T>\]l��Q�OZ0_Wi�\�_ w �'�Q|qR��'R�NQ_�\�x6R�lWlW�>N}R��Q_�§ C �4NQ�4|cVc` � _W\����W`R�x6T � P�N}T6R�x�KWeaeaea«�j x#�Q|4_P��Q|6_�N'|6R�xqT�`a�Q|4_­M�ª�R�i�^�P�NQ\¬�Q|6Y |qR��OZ0_W_hx"���6lWlW_h��� w �6iqR��OR*T4_WlWPLT>\�x6^R�i�^�P�NQ\¬�Q|6Y w P�N³_hN�NQP�N�¦�lhP�N�NQ_Wlh�Q\�x4^�lhPLT4_W�HT4_h°6x6_&T+P�x � R�x4x6_hN�^�N}R��6|4�¥�Q|6R��¥|qR&�S_�i�PcPa�4�#§©Ø4NQ_hV+R�x6TC R�l}¹�R&V�K&e�eaÙa«�`;R�x6T�R�i���P w P�N���PaY"_"lWP�Y"�6�>�Q_hN*�L\���\]P�x��4NQP�Z6i�_WY"�³U­|6_�NQ_¤�Q|6_"�6x6T4_hNQi¬V>\�x6^ C+F Ø\ w�w �6i�i@P w i�PcPa�6�#§©Ø4NQ_W_WY�R�x~`6ª;R���»h�QP�N&`.R�xqT X R�NQY"\�l�|6R�_Wi;I�JaJ�JS«�j���x+�Q|6_�x6_��>�¥��_Wl��Q\]P�x~`qU'_[_��4�4i]R�\�xU­|cV��Q|6\��­���6lWlW_h���¥Y"\�^�|L�­x4P��¥Z0_³��P"���>NQ�4NQ\���\�x6^"R w �Q_hN�R�i�ikjà ÁEÃ×Â� Â�À�Â�ÃyÈy¿4 Ð

�¢x*�Q|4\]�;��_hlh�Q\�Pax~`�U'_OU­\�i�i4\�xc��NQPLT4�4lW_O�Q|4_'M'_��Q|6_�R��6�>NQP��>\�Y�R��Q\�P�x��QP­�Q|6_ w NQ_W_�_hx6_hNQ^�V�R�xqT*��|4P�U��Q|6R���Q|6_­°6�>_&T¤�0P�\�xL�Q�OP w �Q|6_�M�ª�R�i�^aP�NQ\¬�Q|6Y¶lhP�N�NQ_W���0P�xqT"�QP��Q|4_����}R��Q\�PaxqR�N�V��0Pa\�xc�Q�'P w �Q|4_HM'_��Q|6_ w NQ_W__Wx4_hNQ^�Vcj~Ø@\¬NQ����U�_¤UR�xc�*�QP+_h�>�6i]R�\]x.`~�QP!NQ_&RaT>_hNQ�*U­|6P+Y"\�^�|L��x4P���|6R��a_¤R+�4|LV>��\�lW�*Z6R�l}¹>^�NQPa�4xqT.`�Q|6_+lWPax4lW_W�4��P w R×  w NQ_W_¨_Wx6_�NQ^�Vcj £Ô§ { _h_�R�i���P²§,ÄO_&T4\]T4\]RÄI�JaJLK�« w P�N+Y"P�NQ_EZqR�l�¹L^�NQP��6x6T.j « b i�i¥P wPa�>N"^�N}R��6|4\]lWR�iY"PLT4_hi]�¤T4_h°6x6_!R"�¢Pa\�xc�"�4NQP�ZqR�Z6\�i�\���V w �4x6lh�Q\�Pax¯ô@§�ý&ñ.þ�«�j;� w U'_!|6R&�S_!��P�Y"_+P��Q|6_hNR��4�4NQP&�>\�Y�R��Q_��P�\]xc�H�4NQPaZ6R�Z4\]i�\¬�¢V w �4x6lh�Q\�PaxZ:�§�ý�ñ.þ�«�`qU�_�l&R�x¯T>_h°qx4_�R� ¢T4\����}R�x4lW_W£¯§k¹Lx6P&U­x¨R��¥�Q|6_¸*�6i�i�ZqR�l�¹c¦ G _h\]Z4i�_hN�T4\����}R�x4lW_�«'Z0_h��U�_W_Wx#ô@§�ý�ñ.þ�«�R�x6Tu:a§�ý&ñ.þ�«'ZcVÅ §�:�§�ý�ñ.þ�«Wö�ö ô2§�ý�ñ.þ�«7� ) n +.p :�§�ý�ñ.þ�«Li�x :�§�ý�ñ.þ�«ô@§�ý&ñ.þ�« §½IaIa«

� |6_­¸*�4i]i�Z6R�l}¹L¦ G \]_hZ6i�_hN'T4\����}R�x4lW_¥T4Pc_W�Ox6P��O|qR&�S_­R�i�i6�Q|6_­�4NQP��0_hN��Q\�_W��U�_¥x6P�NQY�R�i�i�V#R�����Pclh\fR��Q_�U­\¬�Q|T4\����}R�x6lW_W�Wì>\¬�\���x4P�����V>Y"Y"_h��NQ\�l¥R�x6T!T>Pc_W��x4P����QR��Q\�� w V��Q|6_¥��NQ\fR�x6^�i]_�\�x6_&ëc�qR�i�\���Vcj ü _W�a_hN��Q|6_Wi�_W���W`L\¬�

Page 16: Understanding Belief Propagation and its Generalizations Jonathan

t­?Bï0ð2u�< ÚqÚ b T>\]R�^�N}R�Y"Y�R��Q\�l�NQ_W�4NQ_W��_hxL�}R��Q\�Pax¨P w �Q|6_�M�ª²Y"_W���QR�^a_��6��T6R��Q_�NQ�4i�_W� y ò R §kñ R «3®Æ +1o N ò §kñ ò «�S ò R §kñ ò ú�ñ R «�� � ­[� V ò�W�¯qR y � ò §kñ ò «�j � |6_¯���6Y"Y�R��Q\]P�xÔ��V>Y#Z0P�iH\�xqT>\]lWR��Q_W�+�Q|6R���U�_vR�NQ_���6Y"Y"\�x4^[P��S_hN¥R�i�i~�Q|6_H���}R��Q_W�¥P w x4PLT4_*ó�j

\���6��_ w �6i.Z0_Wl&R��6��_³\¬�­\��¥R�i¬URWV>�­x6Pax>¦½x6_W^aR��Q\��S_³R�xqT�\��­\��»W_�NQP¤\ w R�x6T!P�x6i¬V�\ w �Q|6_¥�¢U�P#�4NQPaZ6R�Z4\]i�\¬�¢Vw �4x6lh�Q\�P�x6�Y:a§�ý&ñ.þ�«R�x6T[ô@§�ý&ñ.þ�«�R�NQ_*_&ëc�qR�iO§ X P��S_hN�R�xqT � |4PaY�R���KWeaeLK�«�j{ �}R��Q\����Q\]lWR�iH�4|cV4��\�lW\����Q��^a_Wx4_hN}R�i�i¬V²R������6Y"_+�Q|6R��!M'P�i��Q»hY�R�x6x.· �!i]RWU¶ô2§�ý�ñ.þ�«u� Ç j l ù V�n +.p W�r � \]���NQ�6_�j~��xÄP��4N�lWPaxc�Q_h�L�&`�U�_"lWR�x!�¢�6����lWP�x6��\]T4_hN�\¬�[R��}R��4�QPai�P�^�Vc`;T>_h°qx4\�x6^��Q|6_� ¢_hx6_hNQ^�V4£ e §�ý&ñ.þ�«�jØ6�>N��Q|6_hNQY"P�NQ_a` w NQPaYÒP��4N��0Pa\�xc��P w �L\�_hU*`a�Q|4_" ��Q_WY"�0_hN}R��Q�4NQ_W£ s \������4���'R³�6R�N}R�Y"_h�Q_hN;�Q|qR��OT4_h°6x6_W�R¤��l&R�i]_�P w �6x6\¬�Q� w P�N��Q|4_*_Wx6_hNQ^�Vc`qR�xqT w P�N���\�Y"�6i�\�lW\¬�¢Vc`>U�_�l&R�x+l�|4PcPa��_�Pa�>NH�4x6\¬�Q�­��P¤�Q|qR�� s �µK�j{ �4Z6���Q\¬�Q�4�Q\�x4^¤M'P�i��Q»hY�R�x6x.· �¥i]RWU \]xc�QP[Pa�4N�T>\]���}R�x6lW_*Y"_WR����4NQ_�`qR�xqT!��_����Q\]x4^ s �yK�`4U'_*°qxqT"�Q|qR��Å §�:�ý�ñ�þ>ö�ö ô@§�ý�ñ.þ�«�� ) n +.p7:a§�ý&ñ.þ�« e §�ý�ñ�þ�«³È ) n +.p7:a§�ý�ñ�þ�«ci]xY:�§�ý�ñ.þ�«JÈ�i�x U §½I�dS«

{ P[U�_���_W_*�Q|qR��¥�Q|4_*¸*�6i�i�ZqR�l�¹c¦ G _W\�Z4i]_�N³T>\]���}R�x6lW_*U­\�i�i2Z0_�»W_hNQP>`0R�xqT!�Q|4_hNQ_ w P�NQ_*�Q|6_�R��4�4NQP&�>\�Y�R��Q_�4NQP�ZqR�Z6\�i�\���V w �6x6l��Q\]P�x�:a§�ý&ñ.þ�«�U­\�i]i2_&ëc�qR�i.�QP¤�Q|6_*_h�4R�lh�H�>NQPaZ6R�Z6\�i�\¬�¢V w �6x4lh�Q\�Pax"ô@§�ý&ñ.þ�«�`>U­|6_hxE�Q|6_ëc�qR�xL�Q\¬��VÉ §�:�§�ý�ñ.þ�«�«�� ) n +.p :a§�ý�ñ�þ�« e §�ý�ñ.þ�«JÈ ) n +.p :a§�ý�ñ�þ�«>i�xY:a§�ý&ñ.þ�«��*Ê[§�:�ý&ñ.þ�«�f2˧�:&ý�ñ.þ�« §½I��c«

R�l}|6\�_W�a_W�*\¬�Q��Y"\�x4\]Y�R�i;�aR�i��6_¤P wYÌ ÿÍf!i�x U j Ì \���lWR�i�i�_&Tv�Q|6_Ä ¢�H_hi]Y"|4Pai�» w NQ_W_¤_Wx4_hNQ^�Vc` £+U­|4\]i�_�Q|6_�Y"P�NQ_�\�Y"�0P�N��}R�xc� w �6x6l��Q\]P�xqR�i É §�:a§�ý&ñ.þ�«�«¤�4x w P�N��Q�6x6R��Q_Wi¬V´T4Pc_W��x6P��!|6R&�S_�R�xqR�Y"_+�Q|qR���\]��6x4\��S_hNQ�QR�i�i¬V+R�^�NQ_h_&T��6�0Pax.ä4U'_*U­\]i�i@�4��_³�Q|6_*x6R�Y"_� �¼H\�Z4Z6� w NQ_W_�_Wx4_hNQ^�Vcj £ � |4_�°4NQ���­�Q_hNQYÆ\]x!�Q|6_¼H\�Z6Z4� w NQ_h_�_Wx4_hNQ^�V¤\]�Ol&R�i�i�_&T"�Q|6_� �R&�S_�N}R�^a_�_Wx6_�NQ^�V>£�Ê�`cU­|6\�i�_¥�Q|4_���_WlhPaxqT"\��O�Q|4_­x6_W^aR��Q\��S_HP w �Q|6_ ¢_hxL��NQP��cV4£DË'j

Page 17: Understanding Belief Propagation and its Generalizations Jonathan

t­?Bï0ð2u�< Ú <$b �6R�\¬N�U­\���_ C+F ØÄU­\¬�Q| w P��4N¥|6\]T4T4_Wx!x6PLT4_h�Wj

t­?Bï0ð2u�< Ú \$b T>\]R�^�N}R�Y"Y�R��Q\�lÆNQ_W�4NQ_W��_hxL�}R��Q\�Pax P w �Q|6_nM�ª �¢U'P�¦½x6PLT>_nZ0_Wi�\�_ w _&ëc�qR��Q\]P�x:Wò R>§kñqò¢úQñ�R�«��b��S�ò RL§kñ0ò�ú�ñ�R�«�N.ò�§kñ0ò©«�N�Rc§kñ�R�«�� � ­[� V ò�W�¯qR y � ò §kñqò�«�� ½ ­m� V ò�W,¯Qò y ½ R §kñ�R�«� \.9 Ú rL]+Ý^>yÝ Ü!ßMÎ t­ÞhÝ�E B t­â�ÝHÝ Ý ß Ý�â _ :� |cV�\]��\¬�E�4��_ w �4i��QP�T4_h��lhNQ\�Z0_ÄR���V>���Q_WY \�xÔ�Q_hNQY"��P w \¬�Q��¼H\�Z6Z4� w NQ_W_¯_Wx6_hNQ^�V É \�x6���Q_&R�TÔP wT4\¬NQ_Wl��Qi�V[\�x#�Q_hNQY"�;P w \¬�Q�.�¢Pa\�xc�O�>NQPaZ6R�Z6\�i�\¬�¢V#T4\�����NQ\�Z6�>�Q\]P�x�± x6_'NQ_WR���Pax"\��;�Q|qR���\���\���P w �Q_Wx"�0Pa����\�Z6i�_�QP�Y�R�¹S_��4NQP�^�NQ_W����ZcV�lhPax6����NQ�4lh�Q\�x6^�R�xqR�i¬VL�Q\�l&R�i�i¬V´��N}R�lh�}R�Z4i�_vR��6�>NQP��>\�Y�R��Q\�P�x6�"�QP É §,Ä'_&T>\]T4\]RI�J�J>K&«�j'Ø4P�N¤_��6R�Y"�6i�_a`;i�_h�¤�6�[NQ_��Q�4NQx��QP¨�qR�\¬N�U­\���_ C+F Ø'· �"R�x6T�NQ_h����NQ\]l��"Pa�>NQ��_Wi��S_W�#�QP¨lWP�x6��\]T4_hNR��4�4NQP&�>\�Y�R��Q_��¢Pa\�xc���4NQPaZ6R�Z4\]i�\¬�¢VvT>\]����NQ\�Z4�4�Q\�Pax4�A:�§�ý�ñ.þ�«��Q|qR���|qR&�S_¤R!�qR�N��Q\�lW�4i]R�NQi¬VÄ��\�Y"�4i]_ w P�NQYEì�Q|6_�V+R�NQ_ w R�lh�QP�NQ\�»W_&TEP��S_�N¥�Q|4_³��\¬�Q_W�Wì :�§�ý�ñ.þ�«�� �

ò:Wò�§kñ0ò½« §½I(ha«

U­|6_�NQ_­�Q|4_Y:Wò�§kñ0ò½«;R�NQ_¥���6ZL��_Wlh���QP³�Q|6_lhPax6����N}R�\�xL� Æ ò :hò�§kñ0ò©«7�×K�j ü P��Q_�Q|qR���U­\¬�Q|6\�x¤�Q|6\��� ¢Y"_&R�x4¦°q_hifT>£ÄR��6�>NQP��>\�Y�R��Q\�P�x~`��Q|6_�Pax4_h¦½x6PLT4_!Z0_Wi�\�_ w ��R�NQ_u:Wò�§kñ0ò½«#R�x6T��Q|4_!��U�PÄx4PLT4_!Z0_Wi�\]_ w ��R�NQ_#�¢�6����Q|6_��4NQPLT>�6lh�Q��P w �Q|6_�lWP�N�NQ_W���0P�xqT>\]x4^!P�x6_h¦½x4PLT4_#Z0_hi]\�_ w �Wì³: ò R §kñ ò ú�ñ R «Y�¦: ò §kñ ò «�: R §kñ R «�j��H��\�x6^��Q|6\��

Page 18: Understanding Belief Propagation and its Generalizations Jonathan

t­?Bï0ð2u�< Ú | � |6_��QPa�Ò_&ëc�qR��Q\]P�x�\���R T4\]R�^�N}R�Y"Y�R��Q\�l´�S_hNQ��\�P�x�P w �Q|4_´_&ëc�qR��Q\�Pax�: R §kñ R «²�Æ +1o : ò R §kñ ò ú�ñ R «�jqMOVElWR�x6lh_Wi�i]\�x4^��Q|4_��6\�_WlW_W�¥�Q|qR��HR�NQ_[lhPaY"Y"Pax!Pax�Z0P��Q|���\fT>_W�¥P w �Q|6_*_&ëc�6R��Q\�Pax.`U�_�T>_hNQ\��S_¤�Q|6_#Y"_W���QR�^�_h¦½�6��T6R��Q_¤NQ�6i�_W� y ò R>§kñ[R�«T® Æ + o N�òQ§kñqò�«�S�ò R>§kñqò¢ú�ñ�R�«�� � ­m� V ò�W,¯iR y � ò §kñqò�«�`��|6P&U­xEZ0_hi]P&U*j

R��4�4NQP&�>\�Y�R��Q_'�¢Pa\�xc���>NQPaZ6R�Z6\�i�\¬�¢V w �4x6lh�Q\�Pax.`6\¬�¥\���_WR���V��QP¤lWPaY"�4�4�Q_H�Q|4_*R��4�4NQP&�>\�Y�R��Q_�¼H\�Z6Z4� w NQ_W__Wx4_hNQ^�Vcj � |6_*_hx6_hNQ^�V+P w R#lWP�x4°q^��4N}R��Q\]P�x�P w R[�qR�\¬N�U­\���_ C+F Ø�\��e §�ý&ñ.þ�«��gfx) V

ò R�W i�xYS�ò RL§kñ0ò�ú�ñ�R�«�f�) ò i�xdN�ò�§kñ0ò½« §½I�åS«

��P#�Q|4_*Y"_&R�x>¦½°6_Wi]T!R&�S_�N}R�^a_*_hx6_hNQ^�V+\��Ê ¢�Ï §�ý5:hò�þ�«7�gf�) V ò R�W )+1o�¶ +9» :hò}§kñqò©«�:QR>§kñ[R�«ci�xdS�ò R>§kñqò�ú�ñ�R�«�f�) ò ) +1o :Wò�§kñ0ò½«>i�xYN�ò}§kñqò©« §½I(va«U­|6\�i�_*�Q|6_³Y"_WR�x4¦©°6_Wi]T�_Wxc��NQP��LV�\��Ë ¢LÏ §�ý1:Wò¢þ�«��gfx) ò ) + o;:hò}§kñ0ò©«Li�xd:hò�§kñ0ò½« §½I�ÙS«

Page 19: Understanding Belief Propagation and its Generalizations Jonathan

R�x6T+Y"_&R�x4¦©°q_Wi]T+¼H\]Z4Z6� w NQ_W_�_Wx4_hNQ^�V+\�� É ¢LÏ �gÊ ¢�Ï f�Ë ¢�Ï j ü P��Q_*�Q|qR��­U­|6\�i�_��Q|4_ w �4i]i;¼H\]Z4Z6�w NQ_h_�_Wx6_�NQ^�V�\���R w �4x6lh�Q\�P�x#P w �Q|6_ w �6i�i���Pa\�xc���4NQP�ZqR�Z6\�i�\���V[T4\�����NQ\�Z6�4�Q\�P�x~`��Q|6_OY"_&R�x>¦½°6_Wi]T w NQ_W__Wx4_hNQ^�V\��'P�x6i¬V"R w �4x6lh�Q\�P�x�P w �Q|6_­Pax4_h¦½x6PLT4_­Z0_Wi�\�_ w �Wj { \�x6lW_­U�_­¹Lx6P&U��Q|6_�¼H\�Z6Z6� w NQ_h_�_Wx4_hNQ^�V¤\]�OZ0Pa�4xqT4_WTZ0_Wi�P&U8ZcV"�Q|6_H��_Wi�Y"|6P�i]» w NQ_W_H_Wx6_hNQ^�Vc`L\¬�\���NQ_&R���P�xqR�Z4i�_³�QP#��_&R�NQl�| w P�N�Q|6R��lhPax4°6^a�>N}R��Q\�Pax!P w : ò · ��Q|qR��HY"\�x4\]Y"\�»W_h� É ¢LÏ j � |6\���\]�­�Q|6_����}R�xqT6R�N}TE�aR�NQ\]R��Q\�Pax6R�i6�¢�6���Q\¬°ql&R��Q\�Pax w P�NHY"_&R�xE°q_Wi]T!�Q|4_WP�N�Vcj§ { _h_�§ � P�N}T4R�x.`�¼H|6R�|>N}R�Y�R�x4\k` � R�R�¹L¹SP�ifRL`�R�xqT { R��4i*K&eae�ÙS«"R�x6T § � R�R�¹L¹SP�ifR�I�JaJ�JS« w P�N��Q�4�QP�NQ\]R�i\�xc��NQPLT4�6l��Q\]P�x6��QP¤��R�NQ\]R��Q\�P�xqR�i~Y"_h�Q|6PLT>�Wj «\.9�< rL]+Ý Û ÝHrA]EÝ�t­â�ÝHÝ�Ý ß Ý�â _ :� |6_����4���Q\¬°ql&R��Q\]P�x w P�N[�Q|6_�M'_h�Q|4_ w NQ_W_!_Wx6_hNQ^�V�\��¤T>\¬Ñ�_hNQ_Wxc� w NQPaY �Q|qR��#Z0_W|4\�xqTv�Q|4_�Y"_WR�x4¦©°6_Wi]T�Q|6_hP�N�Vc`6R�i¬�Q|6P��6^a|E�Q|4_hNQ_�R�NQ_���PaY"_*��\�Y"\�i]R�NQ\¬�Q\�_W�Wj � _³U'Pa�4ifTEi�\�¹S_*�QP"T>_hNQ\��S_�R�¼H\]Z4Z6� w NQ_h_[_Wx4_hNQ^�V�Q|qR��¥\]��R w �6x6l��Q\]P�xEP w Z0P��Q|+�Q|6_*P�x6_h¦½x4PLT4_*Z0_Wi�\�_ w �F: ò §kñ ò «R�x6T��Q|6_³��U�P�¦�x4PLT4_*Z0_Wi�\�_ w ��: ò R §kñ ò úQñ R «�j� |6_*Z0_hi]\�_ w �H��|4Pa�6i]T+P�Z0_hV��Q|6_*x6P�NQY�R�i�\�»&R��Q\�P�x�lWPax6T4\¬�Q\�Pax6� Æ +1o : ò §kñ ò «�� Æ +1o�¶ +9» : ò R §kñ ò ú�ñ R «7�yKR�x6T �Q|4_�Y�R�NQ^a\�x6R�i�\�»&R��Q\�PaxµlhPaxqT>\¬�Q\]P�x6�Z: ò §kñ ò «P� Æ +9» : ò R §kñ ò úQñ R «�j�M'_Wl&R��6��_�P w �Q|4_��qR�\¬N�U­\]��_�4NQP��0_hN��¢V�P w �6R�\¬N�U­\���_ C+F Ø'· �W`'�Q|6_¨P�x6_h¦½x6PLT>_¨R�x6T��¢U'P�¦½x6PLT>_¯Z0_hi]\�_ w �ER�NQ_ÄR�lh�Q�qR�i]i¬V´���4®"lW\�_Wxc��QP�T4_h�Q_hNQY"\�x4_¨�Q|4_ÄR&�S_�N}R�^a_v_Wx6_�NQ^�Vcj­�¢x²P��Q|6_hN+U�P�N}T>�W` w P�N¯R�xcVÔ�6R�\¬N�U­\���_ C+F Ø�R�xqT w P�N¨R�xLVR��4�4NQP&�>\�Y�R��Q_���P�\�xL���4NQP�ZqR�Z6\�i�\���V w �6x6l��Q\]P�x����6l}|+�Q|qR����Q|4_*Pax4_h¦½x6PLT4_�Y�R�NQ^a\�xqR�i2�4NQP�ZqR�Z6\�i�\��Q\�_W�HR�NQ_:Wò�§kñ0ò½«;R�x6T[�Q|6_���U�P�¦�x4PLT4_­Y�R�NQ^�\�xqR�i4�4NQP�ZqR�Z4\�i]\¬�Q\�_W�'R�NQ_Y:Wò RL§kñqò�ú�ñ�R�«�`��Q|6_­R&�S_hN}R�^a_­_Wx4_hNQ^�V�U­\�i]i6|qR&�S_�Q|6_ w P�NQY Ê*�gf�) V

ò R�W :hò RL§kñ0ò�ú�ñ�R�«Li�xYS�ò RL§kñ0ò�ú�ñ�R�«�f�) ò :WòQ§kñ0ò©«>i�xdN�òQ§kñqò�« §½I�eS«

� |6_ÄR&�S_hN}R�^�_Ä_Wx4_hNQ^�V´U­|6_Wx lWPaY"�4�4�Q_&T´U­\¬�Q|8�Q|6_¯_h�4R�lh�+Y�R�NQ^�\]x6R�i*�>NQPaZqR�Z6\�i�\¬�Q\]_h�"ô ò §kñ ò «!R�x6Tô ò R §kñ ò ú�ñ R «[U­\�i�i³R�i���P�|qR&�S_+�Q|4\]� w P�NQYE`O��P�\ w �Q|6_+Pax6_�¦�x4PLT4_�R�xqT��¢U�P�¦½x6PLT4_�Z0_hi]\�_ w ��R�NQ_�_h�4R�l��&`�Q|6_³R��a_hN}R�^�_�_hx6_hNQ^�V!^�\��S_WxEZcV�_&ëc�qR��Q\�Pax�§½I�eS«'U­\�i]i~Z0_*_h�4R�lh�&j� |6_*_hxL��NQP��cV+\��­x6P�����P"_WR���V+�QP#PaZ4�}R�\�x~`0R�xqT!U'_*Y#�4���­x6P�NQY�R�i�i�V!��_h���Qi�_ w P�N³R�x�R��4�4NQP&�>\]Y�R��Q\�Pax~j� _¥lWP��6i]T�lhPaY"�6�>�Q_�Q|6_¥_Wxc��NQPa�cV¤_h�4R�lh�Qi¬V"\ w U'_HlWP��6i]T�_��4�4i�\]lh\��Qi¬V"_��4�>NQ_W���'�Q|4_��¢Pa\�xc�­T4\�����NQ\�Z6�>�Q\]P�x:a§�ý&ñ.þ�«�\]x��Q_hNQY"��P w �Q|4_HPax4_h¦½x6PLT>_³R�x6T¤�¢U�P�¦½x6PLT4_HZ0_Wi�\]_ w �Wj4� w Pa�4N�^�N}R��4|!U'_hNQ_H��\�x6^�i¬V>¦½lWP�x6x4_Wlh�Q_&T�`U�_lWR�x¤\�x w R�lh�'T4P¥�Q|qR��&ja��x[�Q|qR���l&R���_�`aU�_¹Lx4P�U��Q|6R��;�Q|6_lhP�N�NQ_Wlh���¢Pa\�xc�O�>NQPaZ6R�Z6\�i�\¬�¢V#T4\�����NQ\�Z6�>�Q\]P�xl&R�xEZ0_HUNQ\¬���Q_WxE\�x+�Q|4_ w P�NQY :a§�ý&ñ.þ�«�� � V ò R�W :Wò R>§kñqò�ú�ñ�R�«� ò :hò�§kñ0ò½«�Ð o l §©daJS«

U­|6_�NQ_uÑ ò \��#�Q|4_!xL�6Y[Z0_hN�P w x6PLT>_W�¤x6_h\]^�|LZ0P�NQ\�x6^¯x6PLT>_+󤧩ª@_&R�NQi³K&e�ÙaÙS«�jO�H��\�x4^¨�Q|4\�� w P�NQYE`@U�_PaZ>�}R�\�x+�Q|4_*M'_h�Q|4_�R��6�>NQP&�4\�Y�R��Q\]P�x!�QP#�Q|4_³_Wxc��NQPa�cVË û³Ò�Ó�Ô1Ò �¦f ) V

ò R�W )+1o�¶ +9» : ò R §kñ ò ú�ñ R «Li]x3: ò R §kñ ò ú�ñ R «JÈ ) ò §�Ñ ò f�K&« )�+1o : ò §kñ ò «>i�x3: ò §kñ ò « §©d>K�«

Ø6P�N*R"��\�x6^ai¬VL¦½lWPax4x6_Wl��Q_&TE^�N}R��6|��Q|6_Wx.`q�Q|6_�M'_��Q|6_[R��6�>NQP&�4\�Y�R��Q\]P�x�P w Z0P��Q|��Q|4_�_hx6_hNQ^�V¨R�x6T+�Q|6__Wxc��NQPa�cV[U­\]i�i0|6R��a_­�Q|4_­lWP�N�NQ_Wl�� w �4x6lh�Q\�P�xqR�i�T>_W�0_Wx6T4_Wx4lW_Pax"�Q|6_­Z0_Wi�\�_ w �W`6R�xqT#�Q|4_¥��R�i��4_W�'P w �Q|4Pa��_Z0_Wi�\�_ w �³�Q|6R��HY"\]x4\�Y"\]»h_��Q|4_#M'_��Q|6_ w NQ_W_¤_hx6_hNQ^�V É ûJÒ�Ó,Ô5Ò �Õʲf"Ë û³Ò�Ó�Ô1Ò U­\�i�i;lWP�N�NQ_h���0PaxqT��QP��Q|6__h�4R�l��#Y�R�NQ^a\�xqR�i'�4NQP�ZqR�Z6\�i�\��Q\�_W�Wj�Ø4P�N[^�N}R��4|6��U­\¬�Q|�i]PcP��6�W`@�Q|4_�M'_h�Q|6_�_Wxc��NQP��LVÄR�x6T w NQ_h_!_Wx4_hNQ^�VU­\�i�i'Pax4i¬VvZ0_¤R��6�>NQP&�4\�Y�R��Q\]P�x6�W`2R�i�Z0_W\¬��Pax4_W�*�Q|qR���R�NQ_"�6���6R�i�i¬VÄëc�4\��Q_#^aPcPLT.j2�¢xvlWPaxc��N}R������QP��Q|6_Y"_&R�x4¦©°q_Wi]T w NQ_h_*_Wx6_hNQ^�Vc`L�Q|6_³MO_h�Q|6_ w NQ_W_*_Wx4_hNQ^�V�\��*ç��w��^�_Wx6_�N}R�i�i¬VER�x+�6�4�0_hNZ0P��6xqT�Pax��Q|6_¥��NQ�6_w NQ_h_*_Wx6_hNQ^�Vcj

Page 20: Understanding Belief Propagation and its Generalizations Jonathan

� |6_�R&�S_�N}R�^a_�_Wx4_hNQ^�V+l&R�x+Z0_³UNQ\����Q_hx�\]xER w P�NQYÆ�Q|6R��¥\�����\�Y"\�i]R�N¥�QP#�Q|qR��¥P w �Q|6_�MO_h�Q|6_�_Wxc��NQPa�cV\ w U�_H\]xc��NQPLT>�6lW_H�Q|4_³i�Pcl&R�i~_Wx6_�NQ^a\�_W� e ò�§kñ0ò½«��gf+i�xdN�ò�§kñ0ò½«�R�xqT e ò Rc§kñ0ò�ú�ñ�R�«��gf+i]x3S�ò R>§kñqò�ú�ñ�R�«fi�xdN.ò�§kñ0ò½«�f¯i]xdN�Rc§kñ�R�«�j4�H��\�x4^#�Q|6_³Y�R�NQ^a\�xqR�i�\]»WR��Q\�Pax�lWP�xqT>\��Q\�P�x6�¥Pax!�Q|4_*Z0_Wi�\�_ w �W`4U�_*PaZ>�}R�\�xʲ�*) V

ò R�W )+1oQ¶ +9» :Wò RL§kñqò�ú�ñ�R�« e ò RL§kñ0ò�ú�ñ[R�«³ÈÖ) ò §�ÑWò³f�K�«�) +1o :Wò�§kñ0ò½« e òQ§kñqò©« §©dSIa«

U­|6\�l}|�\]�'_h�4R�lh�Qi¬V¤�Q|4_¥�QR�Y"_ w P�NQY¶R��'�Q|4_�MO_h�Q|6_HR��6�>NQP&�4\�Y�R��Q\]P�x¤�QP*�Q|6_¥_Wxc��NQPa�cV¤_h�>lW_h�4�'�Q|6R���U�_|qR&�S_�NQ_W�6i]R�lh_&T!�Q|6_*i�x3:H�Q_�NQY"��U­\��Q|+i]PclWR�i2_hx6_hNQ^�V e �Q_hNQY"�Wj\.9�\ ÝT×DØ+Þ�@ Ü E'Ý ßZ} Ý á"t Û = r¤á rA]EÝ Û ÝHrA]EÝ Ü =T=*â"áLÙ!Þ�> Ü r�Þ&á ß�¢x��Q|6_³�>NQ_W�L\�Pa�6�­��_Wl��Q\]P�x~`4U�_*�QRWU8�Q|qR��­�Q|4_*M'_h�Q|4_ w NQ_W_*_Wx4_hNQ^�VÉ û³Ò�Ó�Ô1Ò §�:Wò�§kñ0ò©«�úQ:hò RL§kñ0ò�ú�ñ�R�«�«Ú� ) V

ò R�W )+1o�¶ +9» :hò Rc§kñ0ò�ú�ñ�R�«.§ e ò RL§kñ0ò�ú�ñ[R�«³È�i�xY:Wò Rc§kñ0ò�ú�ñ�R�«�«f )ò§�Ñ ò f�K�« ) +1ou: ò §kñ ò «0§ e�ò §kñ ò «³È�i]xY: ò §kñ ò «�« §©dadS«

\��_&ëc�qR�iq�QP��Q|4_H_h�4R�l��¥¼H\�Z6Z6� w NQ_W_³_hx6_hNQ^�V w P�N­�6R�\¬N�U­\���_ C+F Ø'· ��U­|4_Wx��Q|6_W\¬N^�N}R��6|�|qR��x6P�i�PcPa�6���P��Q|qR��'�Q|6_�M'_h�Q|6_ w NQ_W_�_Wx6_�NQ^�V�\���Y"\�x6\�Y�R�i w P�N'�Q|6_HlhP�N�NQ_Wlh��Y�R�NQ^a\�x6R�i��Wj � _�R�i���P#¹Lx6P&UÔ�Q|6R���M�ª^a\��S_h�lWP�N�NQ_Wl���Y�R�NQ^a\�xqR�i���U­|6_Wx��Q|6_³^�N}R��6|!|qR��x6P#i�PcP��6�*§�§�K&da«R�xqT¨§½I�Ja«�j �¢x+P��Q|6_hNU'P�N}T>�W`4U­|4_Wx�Q|6_�NQ_³R�NQ_�x6P�i�PcPa�4�W`4�Q|4_HM�ª�Z0_Wi�\�_ w �­R�NQ_H�Q|4_�^�i]P�ZqR�i.Y"\�x4\]Y�R³P w �Q|4_HM'_��Q|6_ w NQ_W_³_Wx4_hNQ^�VcjS���'�Q�>NQx6�Pa�>�­�Q|qR��­U'_�lWR�xE�QRWV!Y"P�NQ_aì#M�hé��Y��è�Û}é�' æ½éAè��Y%�æ,Ü�é��A#DÝ7Þ2ß�àcéQ�A~���æBç���æAçx#�ç���%�êQ#�~���æ è�#�ç��u��ç�' �æ è����6é��;#�ê�éL'��1�O#('����i#.�kæ&��ç�#�ê��A~���æAç�������è����6éTÝHé����4éOè�ê�é}é[é�ç�é�ê�%w�[á� P¥��_W_��Q|6\��W`&U�_�x6_W_WT*�QP¥R�T6T G R�^�N}R�x4^a_�Y[�6i¬�Q\��6i�\�_hNQ�~�QP É û³Ò�Ó�Ô1Ò \�x�P�N}T>_hN@�QP w P�NQYyR G R�^�N}R�x6^a\]R�xDâìã ò R>§kñ�R�«;\]�OR³Y#�4i¬�Q\]�4i�\]_�N��Q|6R���_Wx w P�NQlh_W�O�Q|4_Y�R�NQ^a\�x6R�i�\�»&R��Q\�Pax�lWP�x6����N}R�\]xc�7:Wò�§kñ0ò½«�� Æ R :Wò R>§kñqò¢úQñ�R�«�`U­|6\�i�_3ä>ò R�R�xqTAä>ò2R�NQ_Y[�6i¬�Q\��6i�\�_hNQ���Q|qR���_Wx w P�NQlW_�Q|6_x6P�NQY�R�i�\�»&R��Q\�P�x6�'P w :Wò RHR�x6T�:Wò�j � |6_­_&ëc�qR��Q\]P�xå õå.æ o�» V + o ¶ + » W � J[^a\��S_W�Wì

i�xY: ò R §kñ ò ú�ñ R «��gf e�ò R §kñ ò ú�ñ R «JÈ ã ò R §kñ R «JÈ ã R�ò §kñ ò «³È�ä ò R f�K §©d��c«

� |6_*_WëL�6R��Q\�Pax å õå.æ o V + o W �²J#^a\��S_W�Wì§�Ñ&ò³f�K�«h§ki�xY:Wò�§kñ0ò½«³È²K&«��Ò§�K3f/Ñ&ò½« e ò�§kñ0ò½«³È )R1­[� V ò�W ã R}òQ§kñqò�«³È2ä>ò §©dmha«

T4\¬Ñ�_hNQ_hxL�Q\]R��Q\]x4^��Q|4_ G R�^�N}R�x4^a\]R�x¤U­\¬�Q|�NQ_W���0_Wlh�;�QPH�Q|4_ G R�^�N}R�x6^�_¥Y[�6i¬�Q\��6i�\�_hNQ�@^�\]�a_W�;�Q|6_�Y�R�NQ^a\�xqR�i�¦\�»&R��Q\�P�x�lWPax4����N}R�\�xc�Q�Wjü P&U*`>���6�4�0Pa��_H�Q|6R���U'_³|qR&�S_HR���_��­P w Y"_h���QR�^a_h��R�xqT�Z0_Wi�\�_ w ��Q|qR��­R�NQ_³R�°6�>_&TL¦½�0Pa\�xc��P w M�ª@j � _T4_�°qx6_ ã ò Rc§kñ�R�«ZcV ã ò RL§kñ�R�«��Ôi�x �� ­m� V R�W,¯}ò y � R §kñ�R�« §©daåS«

R�x6T#�6��\�x4^³�Q|6_­M�ª�_&ëc�6R��Q\�Pax4�¥§�KWdS«�R�x6TE§½I�JS«;\¬�'\���_&R���V[�QP³��|6P&U��Q|qR�� ã ò RHR�x6T[�Q|6_­Z0_Wi�\�_ w �'�QR��Q\�� w V�Q|6_*���}R��Q\�PaxqR�NQ\¬�¢V�lWP�xqT>\��Q\�P�x6��§©d��4` d(ha«�j { \]Y"\�i]R�NQi¬Vc`6^a\��S_WxEZ0_hi]\�_ w �³R�xqT G R�^�N}R�x6^a_�Y#�4i¬�Q\]�4i�\]_�NQ�­�Q|6R��

Page 21: Understanding Belief Propagation and its Generalizations Jonathan

�QR��Q\�� w Vv�Q|4_¤���}R��Q\�Pax6R�NQ\¬�¢Vv_&ëc�qR��Q\�Pax6��§©d��4` dmh�«³U'_"�6��_"d�å!�QP�T>_h°qx4_#Y"_W���QR�^a_W��R�x6Tv\¬��\���_&R���V¯�QP��|6P&U8�Q|qR��­�Q|6_W��_³Y"_h���QR�^a_h�HR�xqT!Z0_hi]\�_ w ��Y[�6���­�QR��Q\]� w V!�Q|4_�M�ª�°4�>_&T>¦½�0Pa\�xc�_&ëc�6R��Q\�Pax4�Wj� _­l&R�x��6��_�Q|6_ w R�lh���Q|6R��'M�RWVc_W��\]R�x�x6_��¢U�P�NQ¹L�W`c_hN�NQP�N�¦½lWP�N�NQ_Wlh�Q\�x6^�lWPLT>_W�W`>R�x6T w R�l��QP�N^�N}R��6|4�'l&R�xZ0_lWP�x>�a_hN��Q_&T#\�xc�QP³�qR�\¬N�U­\]��_ C+F Ø'· ���QP�T4_�°qx6_ w NQ_W_­_Wx4_hNQ^a\�_W� w P�N'RHU­\]T4_�N}R�x6^a_­P w T4\¬Ñ�_hNQ_Wxc��¹L\]x6T4�P w Y"PLT4_hi]�Wjq�¢x�_W�S_�N�VEl&R���_a`0U�_[|qR&�S_�R�x¨\]T4_Wxc�Q\�l&R�i;���QP�N�V�ìq�Q|6_[M�ª²R�i]^�P�NQ\¬�Q|6Y w P�N*_&R�l}|¯T4\¬Ñ�_hNQ_Wxc�Y"PLT4_hilWP�N�NQ_W���0P�xqT>�#�QP¯�Q|4_+���}R��Q\]P�xqR�N�V��0P�\�xL�#P w RÄM'_h�Q|4_ w NQ_W_+_Wx4_hNQ^�V w P�N"�Q|6R��"Y"PLT>_Wikj � |6_NQ_&R�T4_hN¥\�xc�Q_hNQ_W���Q_&T+\�x!Y"P�NQ_*T>_h�}R�\�i�����|4Pa�4ifT!lWPax4���6i¬��§,Ä'_&T>\]T4\]R>`4Ø4NQ_W_hY�R�x~`4R�xqT � _W\����HI�JaJLK&RS«�j� |6_EM�ª�R�i�^aP�NQ\¬�Q|4Y w P�N�^�N}R��4|6\�l&R�i�Y"PLT>_Wi��#U­\¬�Q|´i�PcPa�4�"\]�"x4P���^��qR�N}R�xc�Q_W_&T��QPÄlWP�x>�a_hNQ^a_a`�Z4�4�Z0_Wl&R��6��_E�Q|4_�M�ªµ°6�>_&T��0P�\�xL�Q�"lhP�N�NQ_W���0P�xqT��QPÄM'_h�Q|6_ w NQ_W_�_Wx4_hNQ^�V�Y"\�x6\�Y�R>`�P�x6_ElWR�xÔ��\�Y"�6i¬Vl}|6PcPa��_!�QPÄY"\�x6\�Y"\�»W_��Q|6_�M'_��Q|6_ w NQ_W_E_Wx4_hNQ^�V�T>\�NQ_hlh�Qi¬Vcj { �6l}| w NQ_W_E_hx6_hNQ^�V�Y"\]x4\�Y"\]»WR��Q\�Pax4�"R�NQ_��i�P&U�_hN�Q|6R�x"�Q|6_HM�ª�R�i�^aP�NQ\��Q|4YE`LZ6�>�'�Q|6_�V�R�NQ_HR��i�_&R����­^a�qR�N}R�xc�Q_W_&T"�QP�lWPaxL�S_hNQ^�_aj�§ � _hi]i�\�x6^[R�x6T� _W|´I�J�J>K�ä�ÄO�6\�i�i�_�I�JaJ>K&« x��Q|6_+P��Q|6_hN#|qR�xqT.`�_hY"�6\¬NQ\�l&R�i_h�>�4i]P�N}R��Q\�Pax�P w �Q|6\��¤\]T4_WR¯\�xqT>\�l&R��Q_W��Q|qR��*U­|6_WxÄM�ª w R�\�i����QP+lWPaxL�S_hNQ^�_a`~\���\���R+lWi��4_¤�Q|qR��*�Q|4_#NQ_W���4i��Q� w NQPaY±Y"\�x6\�Y"\�»W\�x6^��Q|6_"M'_��Q|6_R��4�4NQP&�>\�Y�R��Q\�Pax�U­\]i�i2R�i���P"Z0_�ëc�6\¬�Q_³\�xqR�lWlW�4N}R��Q_S§ � _hi]i�\�x6^�R�xqT � _W|¨I�JaJ>K&«�j

ç è ¿�è^é¶Å Ê ¿ �yÉ�É Ã Îaê ¿ Ì �"ª ¿ Î À Ð몲Îìª´Ê Â ÁEÃyÂÄÂÂ�À�Â�ÃyȦí

ª�|LV>��\�lW\����Q�W`[Z0_W^a\�x6x4\�x6^×U­\¬�Q|¶¸*\�¹L�6l}|6\�§k¸*\�¹L�6l}|6\EKWemhcKa䤸*\�¹L�6l}|6\EKWeae��c«�`¤|6R&�S_²T4_h�S_Wi�Pa�0_&T RY"_h�Q|4PLT.`���PaY"_h�Q\�Y"_W�.¹Lx6P&U­x�R��.�Q|6_� ¢lWi��4���Q_hN@��R�NQ\]R��Q\�P�xqR�iaY"_h�Q|6PLT�` £P w T4_hNQ\��L\�x6^¥R��4�4NQP&�>\]Y�R��Q\�Pax6�W`�Q|qR��­\�Y"�4NQP��S_³P�x�R�x6T!^a_hx6_hN}R�i]\�»W_*�Q|4_�M'_h�Q|6_�R��6�>NQP��>\�Y�R��Q\�P�x!�QP#�Q|4_�¼H\�Z6Z6� w NQ_W_*_Wx6_�NQ^�Vcjq¼H\]�a_Wx�Q|6_�lWi�Pa��_ONQ_Wi]R��Q\�Pax4��|6\��#Z0_h��U�_W_hx��Q|4_'M'_��Q|6_�R��6�>NQP��>\�Y�R��Q\�P�x[R�xqT³�Q|6_'M�ª¨R�i�^aP�NQ\¬�Q|6YE`�\¬�;\��@xqR��Q�4N}R�i�QP*R���¹[U­|6_��Q|6_hN��Q|6_�NQ__h�>\]���O^a_hx6_hN}R�i]\�»W_WT"M�ª�R�i�^�P�NQ\¬�Q|6Y"��U­|6P���_­°4�>_&T[�0P�\]xc�Q��lWP�N�NQ_h���0PaxqT#�QP��Q|6_���}R��Q\�P�xqR�N�V��0P�\�xL�Q�2P w �Q|6_W��_O\]Y"�>NQP��S_&T*R��4�4NQP&�>\�Y�R��Q\�Pax4�Wj b �2U'_'��|6R�i�ic��_W_'\�x��Q|6_�x6_h�L�~��_Wlh�Q\�Pax.`��Q|6_R�x4��U�_hN�\��¥VS_W�Wj6��x w R�lh�&`S�¢�6����R��¥P�N}T>\]x6R�N�V+M�ª´l&R�x�Z0_�T4_h°6x6_&T�U­\¬�Q|6Pa�>��NQ_ w _hNQ_Wx4lW_*�QP¤�Q|6_�M'_��Q|6_R��4�4NQP&�>\�Y�R��Q\�Pax.`�U'_E��|qR�i�i�Z0_+R�Z6i�_E�QPÄT4_h°6x6_+^�_Wx6_�N}R�i�\�»W_&TÔM�ª R�i�^�P�NQ\¬�Q|6Y"�"U­\¬�Q|6P��4�"NQ_ w _hN�NQ\�x6^T4\¬NQ_Wl��Qi�Vµ�QP8�Q|6_�¸*\�¹L�6l}|6\"R��6�4NQP&�>\�Y�R��Q\�Pax4�v�QP8U­|6\�l}| �Q|4_hVµlWP�N�NQ_W���0Pax6T.j � |L�6�W`*�Q|4_�NQ_&R�T4_hN�4NQ\�Y�R�NQ\]i¬V´\]xc�Q_hNQ_h���Q_&TÔ\�x \�Y"�4i]_hY"_Wxc�Q\]x4^�\�Y"�4NQP��S_&T�M�ªºR�i�^�P�NQ\¬�Q|6Y"�!lWPa�4ifT²Z0_W^�\�x ZcV�NQ_&RaT>\�x6^�Q|6_[x6_h�L�H��_Wlh�Q\�Paxv\]Y"Y"_WT4\]R��Q_Wi¬Vc`qR�xqT¨Pax4i¬V�NQ_h�Q�4NQx¨�QP��Q|4\��³��_Wlh�Q\�Paxvi]R��Q_hN*�QP�Z0_h���Q_hN*�4xqT>_hNQ���}R�x6T�Q|6_³R��6�>NQP&�4\�Y�R��Q\]P�x+Y�R�T4_³ZcV��Q|6_³^�_Wx6_�N}R�i�\�»W_&T�M�ª�R�i�^aP�NQ\��Q|4Y"�Wj�¢xÄR!^�_Wx6_hN}R�i[ �¸*\�¹>�4l�|4\�£�R��4�4NQP&�>\]Y�R��Q\�Pax~`��Q|4_ w NQ_W_"_hx6_hNQ^�Vv\���R��6�4NQP&�>\�Y�R��Q_&TvR���R+���4Y P w �Q|6_i�Pcl&R�i w NQ_W_*_Wx4_hNQ^a\�_W�¥P w R[��_h�¥P w NQ_W^�\]P�x6�¥P w x4PLT4_W�Wj � |6_� �lWi��6���Q_hN��aR�NQ\]R��Q\�Pax6R�i2Y"_��Q|6PLT4£[�4NQP��L\]T4_W�Pax4_"URWVÄ�QP�l}|6PcP���_��Q|6R�����_h�[P w NQ_W^a\�P�x6�Wj x4_�Z0_h^a\�x6��U­\¬�Q|�REZqR���\�l���_h�[P w lWi��6���Q_�NQ�#P w x4PLT4_W�U­|6\�l}|�\�x6lWi��6T4_"_W�S_�N�Vv\�xc�Q_hN}R�lh�Q\�Pax�R�x6TÄx6PLT>_"\�xÄ�Q|6_"�6R�\¬N�U­\���_ C+F Ø'`;R�xqTv�Q|4_WxÄ���4Z4��N}R�lh�Q�*�Q|6_w NQ_h_*_Wx6_hNQ^�\�_W��P w P��S_hN�¦½lWPa�4xc�Q_&T�\�xc�Q_hNQ��_Wl��Q\]P�xENQ_W^a\�P�x6�W`0R�xqT��Q|6_Wx+RaT6T>�­ZqR�l}¹"�Q|6_ w NQ_W_*_Wx6_�NQ^a\�_W�¥P w�Q|6_HP��a_hN�¦½lWPa�4xL�Q_WT!\�xc�Q_hNQ��_Wlh�Q\�Pax4�­P w \�xc�Q_hNQ��_Wlh�Q\�Pax4�W`0R�xqT!��P#Pax~jØ@\�^a�>NQ_�K5h��|6P&U­�;R�x�_h�4R�Y"�6i�_�P w �Q|6_OM'_h�Q|4_'R��6�4NQP&�>\�Y�R��Q\�Pax�R�xqT�R¸*\]¹L�4l�|4\cR��4�4NQP&�>\]Y�R��Q\�Pax w P�N�R��Y�R�i�iS�6R�\¬N�U­\���_ C+F Ø'j&�¢x*�Q|6\��.°q^a�>NQ_�U'_'|6R��a_�P�Y"\¬���Q_&TH�Q|6_�PaZ4��_hNQ�S_&T*x6PLT>_W�~�Q|6R��@R�NQ_'�4xqT>_hNQ���QPcPLT�QP�Z0_ElhPax6x4_Wlh�Q_&T��QPÄ_&R�l�|´|6\]T6T>_Wx�x4PLT4_aj � |6_EM'_h�Q|6_¨R��6�>NQP��>\�Y�R��Q\�P�x´l&R�x´Z0_ElWP�x6��\]T4_�NQ_&TÔR��R��qR�N��Q\�lW�6i]R�N!¸*\�¹L�6l}|6\¥R��4�4NQP&�>\]Y�R��Q\�Pax~`�U­|6_�NQ_¨U'_�l�|4PcPa��_¨�Q|6_EZ6R���\�l¨lWi��6���Q_hNQ��P w �Q|4_�lWi��6���Q_hN��R�NQ\]R��Q\�Pax6R�i­Y"_h�Q|4PLT��QP¨Z0_��Q|6_���_h�"P w R�i�i�qR�\¬NQ�"P w |6\]T6T>_Wx�x6PLT4_h�Wj � _!T4_h°6x6_��Q|6_+i�Pcl&R�i w NQ_W__Wx4_hNQ^�V!\�x>�aPai��L\]x4^¤R#��\�x4^ai�_�x6PLT4_HóOZcVÉ ò�§�:WòQ§kñ0ò©«�«��*) + o;:hòQ§kñqò�«h§ki�xY:Wò�§kñ0ò½«³È e ò�§kñ0ò½«�« §©dmva«

Page 22: Understanding Belief Propagation and its Generalizations Jonathan

t­?Bï0ð2u�< Ú ¤$� |4_³ZqR���\]l*lhi]�4���Q_hNQ�\�x!�Q|4_³M'_h�Q|4_*R��4�4NQP&�>\�Y�R��Q\�Paxv§ki]_ w ��«�R�x6T!R�¸*\�¹>�4l�|4\�R��4�4NQP&�>\�¦Y�R��Q\�P�xħANQ\�^a|c��« w P�NHR[��Y�R�i�i~��\¬�L¦½x6PLT>_*�qR�\¬N�U­\���_ C+F Ø'j

R�x6T��Q|4_*i]PclWR�i w NQ_W_*_Wx4_hNQ^�V!\�x>�aPai��L\]x4^[��U�P¤x4PLT4_W�­ZcVÉ ò R §�: ò R §kñ ò ú�ñ R «�«�� )+ o ¶ + » : ò R §kñ ò ú�ñ R «h§ki�xY: ò R §kñ ò ú�ñ R «³È e¥ò R §kñ ò ú�ñ R «�« §©daÙS«

�¢x��Q|6_³M'_h�Q|6_³R��6�>NQP&�4\�Y�R��Q\]P�x w P�N­�Q|6\��_h�4R�Y"�4i�_H�qR�\¬N�U­\]��_ C+F Ø'`L�Q|6_³Z6R���\�l*lWi��6���Q_hNQ�­R�NQ_�R�i�i��Q|6_�qR�\¬NQ��P w lWP�x6x6_hlh�Q_&T"x6PLT>_W�W`>R�x6T#�Q|6_¥\]xc�Q_hNQ��_hlh�Q\�Pax�NQ_W^a\�Pax4�R�NQ_�R�i�iq�Q|4_���\�x6^�i�_Hx6PLT>_W�Wj ü P��Q\�lW_¥�Q|6R��U­|6_hx!U�_*���}R�N��¥ZLV����4Y"Y"\�x6^�P��a_hN¥R�i�i.i]PclWR�i w NQ_W_*_Wx4_hNQ^a\�_W�¥P w �6R�\¬NQ�­P w x4PLT4_W�W`>U�_HP��a_hNQlWPa�4xc��Q|6_i�Pcl&R�i w NQ_W_#_Wx4_hNQ^�V¨P w x4PLT4_W�[Ka`.d>`��>`.R�x6T¯å�Pax4lW_a`.R�xqTE�Q|6_�i�Pcl&R�i w NQ_h_¤_Wx6_�NQ^a\�_W�³P w x6PLT>_W�*I�R�x6Th���U­\�lW_aj � R�¹L\�x6^!x6PLT>_"I!R���R�x¯_h�4R�Y"�4i]_�`0U�_¤lWP��6xc�*\¬�*\�x¨�Q|6_ w NQ_W_¤_Wx4_hNQ^a\�_W��P w �Q|6_[�qR�\�NQ�$î¬K&I5ï©`î I�d5ï½`>R�xqTCî I(h1ï©`6R�xqT#U'_���|4Pa�6i]T¤|6R��a_¥P�x6i¬V"lWP��6xc�Q_&T¤\¬�OPax6lh_a`L��P*U�_¥x6_W_&T#�QP����4Z4��N}R�l�� É �a§�:9��§kñ��W«�«�¢U­\�lW_�j { �6Z>��N}R�lh�Q\�x6^�_&R�l�|¯P w �Q|6_#��\�x4^ai�_h¦½x6PLT>_#i�Pcl&R�i w NQ_h_¤_Wx6_�NQ^a\�_W��R��4�4NQP��4NQ\]R��Q_Wi¬Vc`�U�_�°qx6T��Q|6R��w P�N­�Q|6\��­_h�4R�Y"�6i�_a`É ûJÒ�Ó,Ô5Ò � É q� È É ��8 È É �O� È É ��� È É i� È É �Q� È É 8Q�f É f É 8 f É � f É � f�I É � f�I É � §©daeS«

U­|6_�NQ_¤U�_"|qR&�S_"���4�6�>NQ_W����_&T¯�Q|4_"_h�>�6i�\�lW\¬�[T4_W�0_Wx6T4_Wx4lW_¤P w _&R�l}|�i�Pcl&R�i w NQ_W_"_hx6_hNQ^�VÄP�x�\¬�Q��i�Pcl&R�iZ0_Wi�\�_ w �Wj ü P��Q\]lh_��Q|6R���Q|4\]�­_&ëc�6R��Q\�Pax!\]�����4���¥R#�qR�N��Q\�lW�6i]R�N�l&R���_*P w _&ëc�qR��Q\�Paxħ©d�dS«�jM'_h���Q_�Nĸ*\�¹L�6l}|6\[R��4�4NQP&�>\]Y�R��Q\�Pax6�¯R�NQ_´T>_hNQ\��S_&T×ZcV����4���Ä_h�L�Q_Wx6T4\�x6^��Q|6\��v¹L\�xqT P w i�Pa^�\]l�j�Ø6P�N_h�4R�Y"�4i�_a`�\ w U'_ElWPax4��\]T4_hN¤�Q|4_+lWi��4���Q_hN�P w¥w P��4N�x6PLT>_W�Cî¬K�I��[h1ïH\�x�°q^a�>NQ_vK1hL`�U�_ElWR�x�T4_h°6x6_�\��Q�i�Pcl&R�i w NQ_W_�_Wx4_hNQ^�V��QP¤Z0_É q���O� §�: q���O� §kñ ú�ñ � ú�ñ � ú�ñ � «�«

Page 23: Understanding Belief Propagation and its Generalizations Jonathan

� )+.-9¶ +105¶ +9µ5¶ +1ð�: q���O� §kñ ú�ñ � ú�ñ � úQñ � «h§ki�x3: q���O� §kñ ú�ñ � ú�ñ � ú�ñ � «³È e�q���O� §kñ ú�ñ � ú�ñ � úQñ � «�§k�SJS«U­|6_�NQ_³�Q|6_³i�Pcl&R�i@_Wx6_�NQ^�V e q���O�S§kñhú�ñ���ú�ñ���ú�ñ��W«�\��e q���O�a§kñWúQñ���ú�ñ���ú�ñ��h«�� f i�xYS�q�a§kñWú�ñ��&«�fvi�xYS�i�S§kñWú�ñ��W«�fvi�x3S��Q�c§kñ���ú�ñ��W«�fvi]x3S��O�a§kñ���úQñ��W«f i�xdN�&§kñ}«�fÄi�xYN³�a§kñ��W«�f¯i�xFN��S§kñ��W«�fvi�xdNJ��§kñ��W«� §k�4K�«

� |6\���T>_h°qx4\��Q\�P�x�P w �Q|4_'i�Pcl&R�iL_Wx4_hNQ^�V�\���R­^a_Wx4_hN}R�i�\�»&R��Q\]P�x¤P w Pa�>N;�4NQ_W�L\�Pa�4�@x6P��Q\�Pax~ì&U'_2���4���;\�x6lWi��6T4_R�i�iq�Q|4_¥lhPaY"�qR��Q\�Z6\�i�\���V¤Y�R���NQ\]lh_W�'R�xqT"_W�L\]T4_Wx4lW_�Q_hNQY"���Q|qR��O\�x�I6�6_Wx4lW_­Pax6i¬V#x6PLT4_h�'\�x"Pa�>N�lWi��6���Q_�N&j�¢x+Pa�4N�_h�4R�Y"�4i�_*P w R¤¸*\�¹>�4l�|4\2R��6�>NQP��>\�Y�R��Q\�P�xE\�xE°q^a�>NQ_"K5hL`6U�_*�}R�¹S_�P��4N�ZqR���\�l�lWi��6���Q_�NQ���QP"Z0_�Q|6_ w Pa�4N�¦½x4PLT4_�lWi��6���Q_hNQ�Dî¬K&I��[h1ïOR�xqT2î I�dmh�å5ï©`2R�xqTEU�_����6Z>��N}R�lh�HP�ѯ�Q|6_�\�xL�Q_�NQ��_Wlh�Q\�Pax�NQ_W^�\]P�xaî Iwh5ï©`U­|6\�l}|+U­\�i�i2Z0_³P��S_hN�¦½lWP��6xc�Q_&T�j � |4_*¸*\�¹>�4l�|4\ w NQ_W_*_Wx4_hNQ^�V!\�x!�Q|4\]�­l&R���_*U­\]i�i~Z0_ÉTñ ò ��ò.ó Ô ò� É q���O� È É ��8O��� f É �Q� §k�cIa«

U­|6_�NQ_�U'_[|6R&�S_�R�^aR�\�x¨���6�6�>NQ_W����_&T+�Q|4_�_��4�4i�\]lh\��HT4_h�0_WxqT>_Wx6lW_*P w �Q|6_�i�Pcl&R�i w NQ_W_�_Wx6_hNQ^�\�_W�HP�xE�Q|6_i�Pcl&R�i~Z0_Wi�\�_ w �Wj

t­?Bï0ð2u�< Ú ¨ Ø6P��4N�¦½x6PLT>_�Z6R���\�l³lWi��4���Q_hNQ�\�x+R[R�¸*\�¹L�6l}|6\�R��6�4NQP&�>\�Y�R��Q\�Pax w P�N¥R[��Y�R�i�i0x6\�x6_h¦½x4PLT4_�qR�\¬N�U­\]��_ C+F Ø'j

Ø@\�^a�>NQ_�KWå���|4P�U­�+R�Y"P�NQ_¨^a_Wx4_hNQ\�lÄ��\¬�Q�qR��Q\�Pax~j�§ b ^SR�\]xÔU'_vP�Y"\¬���Q_&T´�Q|6_¯PaZ4��_hNQ�S_&TÔx4PLT4_W� w P�NlWi]R�NQ\¬��Vcj « � |4_Wx+U�_��6��_*�Q|4_*\]x6T4\�l&R��Q_&T�ëc�qR�N��Q_��Q��P w x4PLT4_W��R���Pa�>NHZ6R���\�l�lWi��6���Q_hNQ��\]xE�Q|4_*lWi��6���Q_hN��R�NQ\]R��Q\�Pax6R�i~Y"_h�Q|6PLT�`>U'_³°qx6TÉ ñ ò ��ò5ó Ô ò � É q���O�7È É ��8O����È É �O�Qô�õ�È É ���QõQöf É �Q�3f É �O�Yf É ���df É ��õ�È É � §k�SdS«

ü P��Q\�lW_��Q|6R��'�Q|4_H�6R�\¬NQ��P w x6PLT>_W��î I(h1ï©`�î �mh5ï©`�î h�å5ï©`qR�xqT�î h�Ù.ï._&R�l�|!R��4�0_&R�N�\]x"��U�P�ZqR���\�l�lWi��6���Q_hNQ�W`>��P�Q|qR��­U�_�Y#�4�������4Z4��N}R�lh�¥P�ѯ�Q|6_W\¬N¥i�Pcl&R�i w NQ_W_�_Wx6_hNQ^�\�_W��P�x6lW_�j�M'�4�¥P�x6lW_*U�_�T>P¤�Q|qR��­U'_���_h_��Q|6R��x6PLT>_Ah¤R��4�0_&R�NQ��\]x w Pa�>NHZ6R���\�l�lWi��6���Q_hNQ�HR�x6T!�Q|6_Wx!UR��¥���6Z>��N}R�lh�Q_WT+\�x�_&R�l}|�P w �Q|4_*\]xc�Q_hNQ��_hlh�Q\�PaxNQ_W^�\]P�x6�W`6��P#�Q|qR���\¬�Q�­i�Pcl&R�i w NQ_W_�_Wx4_hNQ^�V�Y[�6���­Z0_*R�T6T4_WT+R�^aR�\�x~j

Page 24: Understanding Belief Propagation and its Generalizations Jonathan

Ø6P�NR*^a_Wx4_hN}R�i0�6R�\¬N�U­\���_ C+F Ø'`SU�_�T4_h°6x6_¥R*¸*\�¹>�4l�|4\qR��6�4NQP&�>\�Y�R��Q\�Pax�ZcV"R*��_h�'P w NQ_W^a\�P�x6�3÷8R�x6TR!��_���P w �O�wøLç��kæAç�%�ç�ø�!$Û}é�ê���ù�ú w P�N�_&R�l}|vNQ_W^�\�Pax2�u¾P÷�j � |6_¤lWP��6xc�Q\�x6^�xL�6Y#Z0_�NQ�HY#�4���*�QR��Q\�� w V�Q|6_lhPaxqT>\¬�Q\]P�x#�Q|qR���_&R�l}|¤\�xc�Q_hN}R�lh�Q\�P�x"R�x6T#x6PLT>_\]���6i¬�Q\�Y�R��Q_hi�V�lWP��6xc�Q_&T�Pax6lh_­U­|4_Wx#P�x6_­\�x6lWi��6T4_W��Q|6_HlWPaxc��NQ\�Z6�>�Q\]P�x6�­P w _h�S_hN�V"NQ_W^�\]P�x~jqØ6P�NQY�R�i]i¬Vc`>�Q|6\��Y"_&R�x6���Q|qR��­U�_³NQ_&ëc�6\¬NQ_H�Q|6R��)ú9­[û ù�ú�î ó�¾C�5ï�� )ú9­[û ù�ú7î ó�úiXD¾C�5ï��×K §k�a�c«

w P�N¤R�i�i�x6PLT>_W�[ó�R�x6T��qR�\�NQ��P w x4PLT4_W��ó�úiX�\]x��Q|4_"�qR�\�N�U­\���_ C+F Ø'j2�H_hNQ_�î ó�¾C�5ï'\��[R�x�\�xqT>\]lWR��QP�Nw �4x6lh�Q\�P�xv_&ëc�6R�i@�QP�Pax6_#\ w x6PLT>_#ó­\��³\�x¯NQ_h^a\�Pax/�!R�xqT¨_&ëc�qR�i@�QP!»W_�NQP+P��Q|4_hN�U­\���_aj.�¢x¨�Q|6_�lWi��6���Q_hN��R�NQ\]R��Q\�Pax6R�i~Y"_h�Q|6PLT�`>�Q|4\��¥\��­lWP�xqT4\¬�Q\�Pax+l&R�xEZ0_H^a�qR�N}R�xc�Q_W_&T��QP¤Z0_³�QR��Q\���\¬°q_WTEZLV!T>_h°qx4\]x4^ù ú �×KYf )ü ­ ü ò9ý Ò ú V ú�W ù ü §k�[ha«U­|6_�NQ_At1þaô j �4§��a«O\���Q|6_*��_h�¥P w R�i�i2���4�0_hN�¦©NQ_W^a\�Pax4�­P w �LjG _��#ý�ñ.þ ú Z0_"R!���}R��Q_"P w R�i]i��Q|6_#x6PLT4_h�*\]x¯NQ_W^a\�P�x2�>`~R�xqT/: ú §�ý�ñ.þ ú «�Z0_#�Q|4_¤Z0_Wi�\�_ w \�xv�Q|4_¤���}R��Q_ý�ñ�þ ú j4î³_h°qx4_H�Q|6_H_Wx6_hNQ^�V+P w R[NQ_W^�\�Pax�ZcVe úa§�ý&ñ.þ.ú�«�ÿ¦f+i�x � V

ò R�W S�ò R>§kñ0ò¢ú�ñ[R�«�fÄi�x � ò N�ò�§kñ0ò½« §k�SåS«

U­|6_�NQ_+�Q|6_!�4NQPLT4�4lh�Q�"R�NQ_EP��S_hN�R�i]i¥x6PLT>_W�¤P�N��qR�\¬NQ�"P w x6PLT>_W�¤�Q|qR���R�NQ_+lWPaxc�}R�\�x4_&T�_Wxc�Q\¬NQ_Wi¬V�\�xNQ_W^�\]P�x��>j � |4_Wx+�Q|6_H¸*\]¹L�4l�|4\ w NQ_W_*_Wx4_hNQ^�V!\��

ÉTñ �*)ú9­mû ù9ú�ÿ�),+��x:�úa§�ý�ñ.þ5ú�« e ú�§�ý�ñ.þ5ú�«JÈa)�+��x:�úS§�ý�ñ.þ5ú�«Li�xY:9úS§�ý&ñ.þ.ú�« � §k�[va« �w lWP��4NQ��_a`~�Q|6_"Z0_Wi�\�_ w �L: ú §�ý�ñ.þ ú «H\]xvNQ_W^a\�Pax��!Y#�4�������6YÕ�QP�P�x6_�R�x6TvZ0_"lWP�x6��\����Q_Wxc��U­\��Q|v�Q|6_Z0_Wi�\�_ w �­�Q|qR��¥\]xc�Q_hNQ��_hlh�U­\¬�Q|C�>j�¢x�^a_hx6_hN}R�i©`a\�x6lhNQ_&R���\�x6^H�Q|4_���\�»W_P w �Q|4_�ZqR���\�llWi��6���Q_hNQ�;\�Y"�4NQP��S_h�@�Q|6_�R��6�>NQP��>\�Y�R��Q\�P�x#Pax4_�PaZ>�}R�\�x6�ZcV¨Y"\�x6\�Y"\�»W\�x6^��Q|6_#¸*\�¹L�6l}|6\ w NQ_W_"_Wx4_hNQ^�Vcj b �³U�_¤�QRWU*`2�Q|4_¤M'_��Q|6_ w NQ_W_"_Wx6_�NQ^�V w P�N[R!�qR�\¬N�U­\]��_C+F ØyR�i¬NQ_&R�T>V�|qR��"R�x�R&�S_hN}R�^�_E_Wx4_hNQ^�V��Q_hNQY �Q|qR��"\��"_h�4R�lh�&`OR�x6T��Q|6\��"lWPaxc�Q\�xL�6_W�[�QPvZ0_���NQ�6_P w R�i�i@\�Y"�4NQP��S_&T�¸*\�¹>�4l�|4\2R��6�>NQP��>\�Y�R��Q\�P�x6�Wj � |6_�\�Y"�4NQP��S_WY"_Wxc�­R�NQ\���_W��\�xE�Q|4_³��NQ_&R��QY"_Wxc�­P w �Q|6__Wxc��NQPa�cVc`4U­|6\�l�|¨Z0_WlhPaY"_W�¥\�x6lhNQ_&R���\�x6^ai¬V¯R�lWlW�4N}R��Q_¤R����Q|6_�ZqR���\�l#lWi��6���Q_�NQ�HZ0_WlWP�Y"_�ifR�NQ^a_hN&j.��xE�Q|6_i�\�Y"\��U­|4_hNQ_*R#Z6R���\�l*lWi��6���Q_�N­lWP��S_hNQ�¥R�i�i��Q|6_³x4PLT4_W�\�x!�Q|4_³��V>���Q_WYE`c�Q|6_H¸*\]¹L�4l�|4\�R��6�>NQP&�4\�Y�R��Q\]P�xw P�N­�Q|6_*_Wxc��NQPa�cV�Z0_WlhPaY"_W�­_h�4R�l��&j

� ÈyÂ�À�ÂÄà �µË ¿��"¿4ÀÆÈ «ÔÂ Ë ¿>Â�Á É Ã Î²ÉE� È �"ª ¿ Î À�¢x¨P�N}T4\�xqR�N�V¯M�ª@`2R�i�i�Y"_h���QR�^a_h��R�NQ_¤R�i¬URWV>� w NQPaYÕR!��\�x6^�i]_#x6PLT4_��QP+R�x6P��Q|6_hN*��\�x4^ai�_¤x6PLT>_aj.���H\]�xqR��Q�4N}R�i2�QP�_h�>�0_Wlh�¥�Q|qR���Y"_W���QR�^a_W� w NQPaYn^�NQP��6�6��P w x6PLT>_W���QP�P��Q|6_hN�^�NQP��6�6��P w x6PLT>_W�HlhPa�6i]T�Z0_Y"P�NQ_'\]x w P�NQY�R��Q\��S_a`aR�x6T[�Q|L�6�;i�_&RaT��QP³Z0_h���Q_hN�\�x w _hNQ_Wx6lh_aj � |6R���\����Q|4_�ZqR���\�l¥\�xc�Q�6\¬�Q\��S_�\]T>_&R³Z0_W|6\�x6T^a_hx6_hN}R�i]\�»W_WT+Z0_Wi�\�_ w �4NQP��qR�^SR��Q\�P�x�§½¼³M�ªO«�j � |4_³Y�R��Q|4_WY�R��Q\�l&R�iS���4���Q\�°6l&R��Q\�P�xEP w ¼³M'ª�R�i]^�P�NQ\¬�Q|6Y"�\��2�Q|qR��&`&\ w U'_�T>_h°qx4_OY"_W���QR�^a_W�@R�xqT*Y"_W���QR�^a_h¦½�4��T6R��Q_�NQ�6i�_W�@R��6�>NQPa�>NQ\fR��Q_Wi¬Vc`&U�_Ol&R�x���|6P&U��Q|6R��~�Q|6_°6�>_&T[�0Pa\�xc�Q��P w R�¼³M�ª�R�i�^�P�NQ\¬�Q|6Y�R�NQ_­_&ëc�6\���R�i�_Wxc���QP³�Q|6_���}R��Q\]P�xqR�N�V"�0Pa\�xc�Q��P w R³lWP�N�NQ_h���0PaxqT>\�x6^

Page 25: Understanding Belief Propagation and its Generalizations Jonathan

¸*\�¹L�6l}|6\�R��4�4NQP&�>\�Y�R��Q\�Pax¨�QP!�Q|4_ w NQ_W_#_Wx6_hNQ^�Vcj.�¢x¨�Q|6\��*�qR��0_hN&`�U�_#U­\�i�i'x4P��*^�\]�a_#�Q|6_¤�>NQPclW_&T>�4NQ_�QP�lWP�x6����NQ�6l��#R¨¼³M'ªµR�i�^aP�NQ\��Q|4Yà\�x��Q|4_"^a_Wx4_hN}R�i�lWR���_a`2P�N��Q|6_��4NQPcP w P w �Q|6_"_&ëc�4\]��R�i�_Wx4lW_"�QP�R¸*\�¹L�6l}|6\�R��4�4NQP&�>\�Y�R��Q\�Pax.j � _[NQ_ w _hN��Q|6_[\]xc�Q_hNQ_h���Q_&T¯NQ_&R�T4_hN*�QPv§,Ä'_WT4\]T4\]R>`~Ø4NQ_h_WY�R�x.`.R�xqT � _h\]���I�J�J>KWRS«�j��¢x4���Q_&RaT�`@�QP¨\]i�i��6����N}R��Q_!�Q|4_!\]T4_WR��#\�xL�SP�i]�a_&T.`@U'_�U­\�i�i�U�P�NQ¹����Q_W�4¦½ZcVL¦½���Q_W�Ä�Q|4NQP��6^�|��Q|6_ ¢lWR�x6P�x6\�l&R�i]£�Y"_h�Q|4PLT��QP#lWPax4����NQ�6lh��R"¼³M�ª�R�i�^aP�NQ\¬�Q|6Y w P�N³R�x+_h�4R�Y"�4i�_³�qR�\�N�U­\���_ C+F Ø'j{ �0_hlW\¬°ql&R�i]i¬Vc`�i�_h���6�vNQ_h�Q�4NQxÒ�QP °6^a�4NQ_ K&å R�xqTÒ��N�Vµ�QP lhPax6����NQ�4lh��Ry¼³M'ªàR�i]^�P�NQ\¬�Q|6Y �Q|6R��lWP�N�NQ_W���0Pax6T4�O�QP³�Q|4_¸*\�¹>�4l�|4\qR��6�4NQP&�>\�Y�R��Q\�Pax¤T4_W��lhNQ\�Z0_&T¤\�x¤�Q|4_ifR�������_Wl��Q\]P�x~j � |4_­ZqR���\�l¥lWi��6���Q_hNQ�|qR�T w P��4N"x4PLT4_W�[_&R�l}|~ì3î¬K&I��[h1ï©`�î I�d(h�å5ï½`dî �[hwv�Ù5ï½`OR�xqTbî h�åaÙ�e.ï©j � |6_�°6NQ���#���Q_W��\�x�lhPax6����NQ�4lh�Q\�x6^¯R¼³M�ªyR�i�^aP�NQ\¬�Q|4Yà\��*�QPE°qx6TÄR�i�i��Q|6_"\�xc�Q_hNQ��_Wlh�Q\�Pax�NQ_h^a\�Pax4�[P w �Q|4_¤ZqR���\�l�lhi]�4���Q_hNQ�W`�R�xqT�R�i]i��Q|6_W\¬N\�xc�Q_hNQ��_Wlh�Q\�Pax¨NQ_W^�\]P�x6�W`2R�x6T¨��P�Pax.j � _�°qx6T��Q|4_[\�xc�Q_hNQ��_Wl��Q\]P�xvNQ_W^�\�Pax6�$î Iwh5ï©`�î �mh5ï©`�î h�å.ï©`2R�x6T"î h�Ù.ï©`R�x6T��Q|4_*��\]x4^ai�_³NQ_W^�\]P�xE�Q|qR��­\]��R�x!\�xL�Q_�NQ��_Wlh�Q\�Pax�P w \�xL�Q_�NQ��_Wlh�Q\�Pax4�Wì�î h5ï©j

t­?Bï0ð2u�< Ú °$� |6_¤|4\�_hN}R�NQl}|cVvP w NQ_W^a\�P�x6��\�x�Pa�>N�x6\�x6_h¦½x4PLT4_¤_h�4R�Y"�6i�_aj F _h^a\�Pax4�[R�NQ_"lWPax4x6_Wl��Q_&T�QP#�Q|4_W\¬NHT>\¬NQ_Wlh�­���6Z>¦©NQ_W^a\�Pax4�­Z0_Wx6_&R��Q|+�Q|6_hYEj

� |6_�x6_h�L�O���Q_W��\]x�lWP�x6����NQ�4lh�Q\�x6^[R#¼³M�ª�R�i]^�P�NQ\¬�Q|6Y¶\]�O�QP�P�NQ^SR�x6\�»W_*R�i]i0�Q|4_¥NQ_W^�\�Pax6�\�xc�QP�R�ê�é&%�æ&��ç%�êQ#�~��LìOR�|6\�_hN}R�NQl}|cV�P w NQ_W^�\�Pax6�¤R�xqT��Q|4_W\¬N� �T>\¬NQ_Wlh�Q£¯���6Z4¦©NQ_W^�\�Pax6�Wj b T>\¬NQ_Wlh�¤���4Z4¦©NQ_W^�\]P�xat!P w RNQ_W^�\]P�x{�"\���R����6Z4¦©NQ_W^�\�Pax¯P w ÷µ�Q|qR��*\��*x6P��*R�i���P�R����6Z>¦©NQ_W^a\�Pax¯P w R�x4P��Q|6_�N*���6Z>¦½NQ_h^a\�Pax¯P w �>j���x°q^��4NQ_�K5vc`.U'_#^�\]�a_��Q|4_[|4\]_�N}R�NQl}|LVEP w NQ_h^a\�Pax4� w P�N*Pa�>N*_h�4R�Y"�4i]_�j ü P��Q_��Q|6R��HNQ_h^a\�Paxaî h1ï�\��³x6P��*RT4\¬NQ_Wl������4Z4¦©NQ_W^�\]P�xEP w NQ_W^a\�Pax�î¬K&I��[h1ï©`0Z0_Wl&R��6��_*\¬�¥\]�¥R�i]��P�R[���6Z4¦©NQ_W^�\�PaxEP w NQ_W^a\�P�x�î Iwh5ï©j� |6_�x6_��>�[���Q_W��\��[�QP¨lWP�x6����NQ�4lh�¤Y"_W���QR�^a_W�[lWPax4x6_Wl��Q\]x4^¯R�i�i�NQ_W^a\�P�x6�D�+�QP¯R�i]iO�Q|6_W\¬N"T>\¬NQ_Wlh�¤���4Z4¦NQ_W^�\]P�x6�DtSj2�¢x�P��Q|6_hN�U�P�N}T4�W`@U'_+R�����PclW\]R��Q_ER�Y"_W���QR�^a_"U­\¬�Q|�_&R�l}|�i�\]x4_!\�x�°6^a�4NQ_�K1vLj X P�x6��\]T4_hNw P�N�_h�4R�Y"�6i�_��Q|6_+Y"_h���QR�^a_�lhPax6x4_Wlh�Q\�x6^¯NQ_W^a\�P�xgî¬K�I��[h5ï��QPÄNQ_W^�\]P�xgî I(h1ï©j � _!l&R�x�lWPax4��\]T4_hN��Q|6\���QP�Z0_¯R�Y"_W���QR�^�_ w NQPaYÍx6PLT>_W�¯K¯R�xqTÔ���QP�x4PLT4_W�EI�R�xqT²hcj � _¨U­\�i�i*T>_Wx6P��Q_¨�Q|4\��!Y"_W���QR�^a_ZcV y i���A�Q�S§kñ���ú�ñ��W«�j;��x�^a_Wx4_hN}R�ik`�U�_!l&R�x�lWP�x6��\]T4_hN"R¯Y"_W���QR�^a_!lWPax4x6_Wlh�Q\�x4^vR�NQ_W^�\]P�xa�¨R�xqT�R���6Z>¦©NQ_W^a\�Pax�t³�QP#Z0_�R#Y"_W���QR�^�_ w NQP�Y �Q|6Pa��_*x4PLT4_W�­\�xC���Q|qR��HR�NQ_*x6P��­\]x�t³�QP[�Q|6_³x4PLT4_W�­\�xZtaj

Page 26: Understanding Belief Propagation and its Generalizations Jonathan

� |6_�x6_��>�~���Q_W��\��@�S_hN�VH\�Y"�0P�N��}R�xc�&j � _�lWPax4����NQ�6lh�2Z0_Wi�\]_ w _&ëc�6R��Q\�Pax4� w P�N;_W�S_�N�VHNQ_W^�\]P�xL�L`�R�lWlWP�N}T>\�x6^�QP"�Q|6_*NQ�6i�_��Q|qR��H�Q|6_�Z0_Wi�\�_ w :9úS§�ý&ñ.þ.ú�«\��³�4NQP��0P�N��Q\�Pax6R�i@�QP"�Q|6_��4NQPLT>�6lh��P w _W�S_�N�VElWP�Y"�qR��Q\�Z4\]i�\¬�¢VY�R���NQ\¬��R�xqT�_W�L\]T4_Wx4lW_!�Q_hNQYàlWPaxc�}R�\]x4_&T�lWPaY"�4i]_��Q_Wi¬V�\�x��Q|4_!NQ_W^�\]P�x~`OR�x6T�_h�S_hN�V�Y"_W���QR�^a_+�Q|6R��^aPc_h�³\�xL�QP�x4PLT4_W��\]x��Q|4_�NQ_W^�\�Pax w NQPaYnx6PLT>_W�HP��4�Q��\]T4_��Q|6_�NQ_W^a\�P�x~j.Ø6P�N³_h�4R�Y"�4i�_a` w P�N³�Q|4_�NQ_W^�\�PaxlWP�x6��\����Q\�x6^³���4���¥P w x4PLT4_AhL`>�Q|6_*Z0_hi]\�_ w _&ëc�qR��Q\�P�xE\��: � �²��î N � ï[î y ����� y ���A� y ����� y õ���� ïm §k�SÙS«

U­|6_�NQ_a`J�¨\��*R�x6P�NQY�R�i�\�»&R��Q\�P�xÄlWP�x6���}R�xc�&`.R�xqT�U�_#|6R&�S_#���4�6�4NQ_h����_&T¨R�i�i;�Q|6_#P�ZL�>\�P��6� w �6x6l��Q\]P�xqR�iT4_h�0_WxqT>_Wx6lW_h�OP�x��Q|4_����}R��Q_W�­ý&ñ.þcj � |4\��'_&ëc�qR��Q\�Pax�\��'\�i�i]�4����N}R��Q_&T�\�x"°q^a�>NQ_*K&Ù�Pax�Z0P��Q|¤�Q|4_¥NQ_W^�\�Pax^�N}R��6|!R�x6T"�Q|4_�P�NQ\�^a\�xqR�i.�6R�\¬N�U­\���_ C+F Ø'j � R�¹L\�x6^*�Q|6_­NQ_W^�\]P�x{î �[h1ï~R��R�x�_��6R�Y"�6i�_¥P w R���U�P�¦½x4PLT4_

t­?Bï0ð2u�< Ú ·%� |4_!Z0_hi]\�_ w _&ëc�qR��Q\�Pax":1�;�Í��î N³��ï[î y ����� y ����� y ���A� y õ�����ï w P�N#�Q|4_�NQ_W^�\�Pax²î h5ï©`\�i�i]�4����N}R��Q_&TEZ0P��Q|+P�x+�Q|6_HNQ_h^a\�PaxE^�N}R��4|�§ki�_ w ��«R�x6T+Pax��Q|6_*P�NQ\]^�\�xqR�i~�qR�\�N�U­\���_ C+F Ø8§ANQ\�^�|L��«�j

NQ_W^�\]P�x~`6\��Q�­Z0_Wi�\�_ w _&ëc�6R��Q\�Pax!\]�Wì:��O�Y�²��î N��9N³��S��O��ï[î y q���T�O� y ô�õ��T�O� y ����� y ����� y õ�����ïcú §k�SeS«U­|6\�l}|�U�_�\�i�i]�4����N}R��Q_�\�xÄ°q^��4NQ_�K&eLj � R�¹L\�x6^E�Q|4_¤NQ_W^a\�P�xbî�K&I��mh5ï¥R��#R�x�_��6R�Y"�6i�_"P w R w Pa�4N�¦½x4PLT4_NQ_W^�\]P�x~`6\��Q�­Z0_Wi�\�_ w _&ëc�6R��Q\�Pax!\]�Wì:wq���O�d�b��î N�ON³��N��1N³��S�q��S�i�9S��Q��S��O��ï[î y 8Q�����Q� y ô�õ��T�O� y ����� y õ�����ïSú §ih�JS«U­|6\�l}|!\��\�i�i��6����N}R��Q_WT!\�x�°6^a�>NQ_*I�JLj � |6_�NQ_h^a\�PaxEZ0_hi]\�_ w _&ëc�qR��Q\]P�x6�R�NQ_³xqR��Q�4N}R�i0^a_Wx4_hN}R�i�\�»&R��Q\�P�x6�¥P w�Q|6_HM'_h�Q|4_³Z0_Wi�\�_ w _&ëc�qR��Q\�Pax6�W`>Z4�4�P�x6_³Y"\�^a|c����Q\�i�i�U�Pax6T4_hN¥R�Z0Pa�>���Q|6_W\¬N;�¢�6���Q\¬°ql&R��Q\�Pax~j b i¬�Q|4Pa�6^�|U�_¨R�NQ_¨PaY"\¬���Q\�x6^v�Q|4_��4NQPcP w |6_hNQ_a`��Q|6_��0P�\�xL��\����Q|qR��"�Q|6\���xqR��Q�>N}R�i¥URWV��QPÄlWPax4����NQ�6lh��Z0_Wi�\�_ w_&ëc�qR��Q\�Pax6�¥\��H�>NQ_WlW\���_Wi¬VEU­|qR���VSPa��x6_h_&T w P�NH�Q|4_#¼³M'ª´°6�>_&T+�0P�\]xc�Q�­�QP�Z0_����}R��Q\�PaxqR�N�V��0Pa\�xc�Q�¥P w�Q|6_HlWP�N�NQ_W���0P�xqT>\]x4^"¸*\�¹>�4l�|4\ w NQ_W_*_Wx4_hNQ^�Vcj

Page 27: Understanding Belief Propagation and its Generalizations Jonathan

t­?Bï0ð2u�< Ú ¸$� |6_�Z0_Wi�\�_ w _&ëc�qR��Q\]P�x�: �O� � ��î N � N � S �O� ï�î y q���T�O� y ô�õ��T�O� y ����� y ����� y õ��A� ï�` w P�N�Q|6_"NQ_W^�\�Pax²î �[h5ï©`�\�i�i��6����N}R��Q_WT�Z0P��Q|�Pax��Q|4_"NQ_W^a\�Pax�^�N}R��6|²§ki�_ w ��«[R�xqTÄP�x��Q|4_"P�NQ\�^a\�x6R�i�qR�\¬N�U­\]��_C+F Ø §ANQ\�^a|c��«�j ü P��Q_*�Q|qR���U'_�\]x4lWi��qT>_[R�i�i;Y"_W���QR�^�_W���Q|6R���\]Y"�4\�x6^a_*�4�0PaxE�Q|4_�NQ_h^a\�Pax"î �[h1ï�P�N�\��Q����6Z>NQ_W^a\�P�x2î h5ï©j� |6_Ox6_h�L�&`�R�xqT*°qxqR�ik`����Q_h�[\�x�lWPax4����NQ�6lh�Q\�x6^HRH¼³M'ªvR�i]^�P�NQ\¬�Q|6Yµ\��@�QP¥_Wx w P�NQlW_O�Q|6_'Y�R�NQ^a\�xqR�i�\]»WR��Q\�PaxlWP�xqT4\¬�Q\�Pax�NQ_Wi]R��Q\�x4^#_&R�l�|��qR�\¬NP w NQ_W^a\�P�x6���Q|qR��R�NQ_³lWPax4x6_Wl��Q_&T"\]x"�Q|4_�|4\]_�N}R�NQl}|LV���|4P&U­x�\�x�°q^��4NQ_K5vcj~Ø6P�N�_h�4R�Y"�6i�_[�Q|4_#Y�R�NQ^�\�xqR�i�\�»&R��Q\]P�x�lWP�xqT4\¬�Q\�Pax¯lWP�x6x6_hlh�Q\�x6^!�Q|4_[NQ_W^�\�Paxzî h5ï�U­\¬�Q|v�Q|4_[NQ_W^�\�Paxî �[h1ï­\]�A: � §kñ � «L� Æ +9µ : �O� §kñ � ú�ñ � «�j;� w U�_�lWPaY[Z6\�x6_¤�Q|6R���U­\��Q|�P��4N#�4NQ_W�L\�Pa�4�[Z0_Wi�\�_ w _&ëc�qR��Q\�Pax6�§k�SÙa«R�xqT¯§k�Sea«�`6U�_³°6xqT�`4ZcV�l&R�x6lW_Wi�i�\�x6^"lWP�Y"Y"Pax��Q_hNQY"�W`L�Q|6_³Y"_h���QR�^a_��4��T6R��Q_�NQ�6i�_y ���A��§kñ��W«�® �d) � 0 N³�S§kñ��&«�S��O�a§kñ���ú�ñ��W« y q���T�O�a§kñ���ú�ñ��&« y ô�õ����Q�a§kñ���ú�ñ��W« §ihLK�«� |6_vlWPai�i�_Wlh�Q\�P�xyP w R�i�i*�Q|6_vZ0_Wi�\]_ w _&ëc�qR��Q\�Pax6�ER�xqT²Y"_W���QR�^a_��4��T6R��Q_¯NQ�4i]_h�¨T4_h°6x6_W�!Pa�4N¨¼³M�ªR�i�^�P�NQ\¬�Q|6YEj b ¼³M'ª×R�i�^�P�NQ\¬�Q|6Y±NQ�6x4�³\�xv�Q|4_¤�QR�Y"_#U�RWVÄR��*�Q|6_¤M'ª×R�i�^�P�NQ\¬�Q|6YEj x6_[x6P�NQY�R�i�i�V\�x6\¬�Q\]R�i�\�»W_W��R�i�i��Q|6_HY"_h���QR�^a_h�­�QP��Q|6_W\¬N­�4x>Z4\]R���_&T����}R��Q_W�W`6R�x6T��Q|4_Wx+\¬�Q_hN}R��Q_W���Q|6_HY"_W���QR�^�_³�6��T4R��Q_NQ�6i�_W�#�6xc�Q\�i'�Q|6_hV´§k|6P��0_ w �4i]i¬V6«�lhPaxL�S_hNQ^�_aj lWl&R���\]P�xqR�i]i¬Vc`�\¬�¤\��¤|4_Wi�� w �4i w NQP�Yà�Q|4_!�0P�\]xc�#P w �>\�_hUP w lWPaxL�S_�NQ^a_Wx4lW_!�QP¯Pax4i�V�Y"P��S_��6R�N���¦©URWV�R��¤_&R�l}|�\¬�Q_hN}R��Q\�Pax��QP&U�R�N}T4�¤�Q|4_!x4_hUº��R�i��6_h�¤P w �Q|6_Y"_W���QR�^a_W�Wj � |4_WxElWP�xL�S_hNQ^a_hx6lW_�P w �Q|6_*Y"_h���QR�^a_h�³\��HR�l�|4\]_h�S_&T.`6�Q|6_�T4_W��\¬NQ_&TEZ0_hi]\�_ w �³l&R�x�Z0_�NQ_&R�TP�Ñ w NQPaY �Q|6_*Z0_hi]\�_ w _&ëc�qR��Q\�P�x6�Wj�HP&U�U�_hi]i[T4P²¼³M�ªÕR�i]^�P�NQ\¬�Q|6Y"�¨U�P�NQ¹�±�Ø6P�N¯R´i�Pax4^a_hNÄR�x4��U�_hN¨U­\¬�Q|µT4_��}R�\�i��W`H�Q|6_vNQ_&RaT>_hN¯\]�NQ_ w _hN�NQ_&Tv�QP�§,ÄO_&T>\fT>\]R>`~Ø4NQ_W_WY�R�x~`~R�x6T � _W\����#I�JaJLKWZ0«�`~U­|6_hNQ_¤U'_�T4_W��l�NQ\]Z0_"_��4�0_�NQ\]Y"_hxL�Q��U­|6_hNQ_¼³M�ª R�i�^aP�NQ\¬�Q|6Y"�*l&R�xv��\�^ax4\¬°ql&R�xL�Qi¬V¨P��4��¦½�0_hN w P�NQY P�N}T>\�xqR�N�V¯M'ª P�x¯��U�P�¦�T>\]Y"_hx6��\�Pax6R�i;�qR�\¬N�U­\]��_C+F Ø'· �W`SR�x6T w P�NT4_hlWPLT4\�x6^H_hN�NQP�N�¦½lWP�N�NQ_Wlh�Q\�x6^�lWPLT>_W�Wj � |6_��|4P�N��'R�x4��U�_hNO\����Q|6R��'¼³M�ª�R�i]^�P�NQ\¬�Q|6Y"�x6_WR�NQi¬VvR�i¬URWV>�[\�Y"�>NQP��a_a`.R���i�_&R�������i�\�^a|c�Qi¬Vc`2P��S_�N*�Q|6_¤�0_hN w P�NQY�R�x6lW_"P w P�N}T>\]x6R�N�VÄM�ª@`@R�xqT¨�Q|6_hV

Page 28: Understanding Belief Propagation and its Generalizations Jonathan

t­?Bï0ð2u�< < À� |6_¨Z0_Wi�\�_ w _&ëc�qR��Q\�Paxb: q���O� �Á��î N N � N � N � S q� S i� S �Q� S �O� ï[î y 8Q�����Q� y ô�õ��T�O� y ����� y õ��A� ï©` w P�N�Q|6_³NQ_h^a\�Pax"î¬K�I��[h5ï©`�\�i�i��6����N}R��Q_WTEZ0P��Q|�Pax!�Q|6_³NQ_W^�\]P�x¨^�N}R��4|�§ki�_ w ��«­R�x6T+Pax+�Q|4_�P�NQ\�^a\�xqR�i@�qR�\¬N�U­\]��_C+F Ø8§ANQ\�^�|L��«�j

l&R�x+��\�^ax4\�°6l&R�xc�Qi¬V�Pa�>��¦��0_�N w P�NQY¶P�N}T4\�x6R�N�V!M�ª�\ w �Q|6_�^�N}R��4|6\�l&R�i~Y"PLT>_Wi��6xqT>_hNlWP�x6��\]T4_�N}R��Q\�Pax!|qR����|6P�N��¥i]PcP��6�Wjb � w P�N��Q|6_­lWP�Y"�6i�_h�>\¬�¢V[P w ¼³M�ª@`S�Q|4_�Z6RaT¤x6_�U­�'\��O�Q|6R��O\��O^�NQP&U­��_��4�0P�x6_Wxc�Q\]R�i�i¬V¤U­\¬�Q|"�Q|6_­��\�»W_¥P w�Q|6_�ZqR���\�l'lWi��6���Q_�NQ�@�Q|qR��@R�NQ_Ol�|4Pa��_Wx.j � |6_'^�PcPLT�x4_hU­�@\��2�Q|6R��;\ w �Q|6_�ZqR���\�lOlWi��6���Q_hNQ�;_Wx4lWPaY"�6R����.�Q|6_��|6P�N��Q_W����i�PcPa�6��\�x[�Q|4_�^�N}R��6|4\]lWR�i>Y"PLT4_Wik`�Pax4_��6���6R�i�i¬V#_Wi�\�Y"\�xqR��Q_W�;x6_&R�NQi¬V#R�i�i>�Q|4_�_hN�NQP�NOR�����PclW\]R��Q_&TU­\¬�Q|!�Q|4_*M�ª�R�i�^aP�NQ\��Q|4YEjqØ4P�N­Y�R�xcV�^�N}R��6|4\�l&R�i.Y"PLT4_Wi��W`.§k_aj ^4j4�Qëc�qR�NQ_³i]R����Q\�lW_*�qR�\¬N�U­\]��_ C+F Ø'· �}«�`�6��\�x4^H���6l}|�ZqR���\�l�lhi]�4���Q_hNQ��R�l��Q�qR�i�i¬V�\�x>�aPai��S_W�@P�x6i¬V�Y"\�x6\�Y�R�i�i¬V*Y"P�NQ_'lWP�Y"�6�4�}R��Q\�Pax��Q|qR�x�P�N}T>\�xqR�N�VM�ª@j¤.9 Ú _!Û = } E�ØE7~r�ÝHâ�Þ ßZ_ Ü!ßCB ,JØ ßZ} r[ÞWá ß r[â�Ý�Ý*7b ���}R�xqT4R�N}T#R�i�^aP�NQ\¬�Q|4Y w P�N�_h�4R�lh��\]x w _hNQ_hx6lW_\�x#^�N}R��4|6\�l&R�i>Y"PLT4_hi]�;\��;�Q|6_���øLç����kæ&��çk�kê�é}é;R�i]^�P�NQ\¬�Q|6Y§ X P&U�_Wi�i­K&e�eaÙS«�`.U­|6\�l}|¯U'_¤Z4NQ\�_�I4V�\�i�i]�4����N}R��Q_"\�x¨°q^a�>NQ_"IcK&R�¦�T�j � |4_¤^�N}R��6|¯\��³°6NQ���D�kê}æ&#�ç[%wø�'�#.��éO�§k\kj _aj2U'_!R�T6TÄ_WT4^a_h�[��PE�Q|6R���_W�S_hN�V¯lhV>lWi�_"P w i�_Wx4^��Q|� dE|6R���REl}|6P�N}T6«�j2��x�Pa�>N�_h�4R�Y"�6i�_a`.�Q|6\��^a\��S_h���Q|6_�^�N}R��6|�\�xE°6^a�4NQ_#ILKhZ~j � _³�Q|6_WxE°6xqT!�Q|4_�Y�R��>\�Y�R�i.lWi�\fëc�4_W�HP w �Q|4_���6x4lh�Q\�Pax���NQ_W_�R�x6TlWP�x6x6_hlh�³�Q|6_h��_¤\�xÄR���NQ_W_+§A°6^a�4NQ_¤ILKhl�«�j � |4_#�0P��Q_Wxc�Q\fR�i��*Paxv�Q|4_#���6R�x6x4\�x6^���NQ_W_"R�NQ_[��_h����P!�Q|6R���Q|6_*��NQ_W_�T>_h°qx4_W�HR�x�_&ëc�6\���R�i�_Wxc� C+F Ø��QP¤�Q|6_�P�NQ\�^�\]x6R�i;�>NQPaZ6i�_WYEj x6lW_³U'_[|6R&�S_�R�x¨_&ëc�6\���R�i�_Wxc���NQ_W_a`4U�_�l&R�x���\�Y"�4i�V�NQ�6xEM�ªÔPaxE�Q|4_­�¢�6x4lh�Q\�PaxE��NQ_W_*�QP"PaZ>�}R�\�x�Y�R�NQ^�\�xqR�i���PaxE�Q|6_�lWi�\]ëc�6_W��§©R�x6TR�x¨R�T6T>\��Q\�P�xqR�i2Y�R�NQ^a\�x6R�i�\�»&R��Q\�Pax¨P�x�_&R�l}|�lWi�\]ëL�4_³U­\�i]i;^�\��S_��6�­Y�R�NQ^�\]x6R�i��HP�x�\�xqT4\��L\]T4�6R�i.x6PLT4_h�}«�j�HP&U'_W�S_hN&`6��\]x4lW_��Q|6_*x6PLT>_W�H\�xE�Q|4_����6x4lh�Q\�PaxE��NQ_W_�lWP�N�NQ_W���0Pax6T+�QP"P��a_hNQi]R��6�4\�x6^���_h�Q�¥P w �aR�NQ\]R�Z6i�_W�\�x��Q|6_�P�NQ\�^a\�x6R�i­^�N}R��6|~`�P�x6_�U�P��6i]T�i]\�¹S_!R¯M�ªµR�i�^aP�NQ\¬�Q|6Yà�Q|6R��[�}R�¹S_W�#RaT>�aR�xL�}R�^a_�P w �Q|6\�� w R�lh�

Page 29: Understanding Belief Propagation and its Generalizations Jonathan

x1

x2

x3

x4

x5

x1

x2

x3

x4

x5

R Z

x1,x

2,x

3x

2,x

3,x

5x

2,x

4,x

5

c1 c

2c

3

x1,x

2,x

3x

2,x

3,x

5x

2,x

4,x

5

x2,x

3x

2,x

5

c1 c

2c

3

s1 s

2

l T

x1

x2,x

3 x4,x

5

z1 z

2z

3

_t­?Bï0ð2u�<z< Ú 1~9 R�x��4xqT>\�NQ_hlh�Q_&TÄ^�N}R��6|�U­\¬�Q|�i�PcPa�4�Wj�� 9 �Q|6_"�QR�Y"_"^�N}R��6|�R w �Q_hN���NQ\fR�x6^��6i]R��Q\�Pax.ì_&T>^a_W�¥|qR&�S_*Z0_W_hx�RaT4T4_&T+��P¤�Q|6R��¥_W�S_�N�V+lhV>lWi�_�P w i�_Wx4^��Q|� ´d¤|qR��HR#l}|6P�N}T�� 9 �Q|6_��¢�6x6l��Q\]P�xE��NQ_W_\���R+���qR�x6x6\�x4^+��NQ_W_¤P w �Q|6_¤Y�R��4\�Y�R�i;lWi�\fëc�4_W��P w �Q|6_[��NQ\fR�x6^��6i]R��Q_&Tv^�N}R��4|~j =O9 \�x w _hNQ_Wx6lh_�\�x¯�Q|6_���4x6lh�Q\�P�x[��NQ_W_�lWR�x#Z0_'�0_hN w P�NQY"_&T[ZcV*NQ�6x4x6\�x6^�M�ª¨Pax#RH^�N}R��6|[�Q|6R��;\�x4lWi��qT4_h���Q|4_'lWi�\]ëc�6_W��R�xqT��Q|6_��_W�6R�N}R��QP�NQ�Wj <�9 R�x¨R�i¬�Q_hNQxqR��Q\]�a_[URWVE�QP�lWP�xL�S_hN��¥�Q|6_�P�NQ\�^a\�x6R�i�^�N}R��6|¨\�xL�QP�R¤��NQ_W_�\����Q|6_�Y"_h�Q|4PLTP w lWi��6���Q_hNQ\�x6^>j

Page 30: Understanding Belief Propagation and its Generalizations Jonathan

�QP+NQ_&T>�6lW_¤�Q|4_¤��\�»W_�P w �Q|4_"Y"_W���QR�^�_W�Wj@��xv�Q|4_ { |6R w _�N�¦ { |6_Wx4P&VÄR�i�^aP�NQ\¬�Q|4Y�§ { |qR w _hN[R�xqT { |6_WxcVcP&VK&e�eaJa«2Y"_W���QR�^�_W�;Z0_h�¢U'_W_Wx�lWi�\]ëL�4_W�;R�NQ_ w �4x6lh�Q\�Pax4�@P w �Q|6_����}R��Q_�P w �Q|4_d�héq~�#�ê�#w�i��ê��@P�N�\�xc�Q_hNQ_W��_hlh�Q\�Pax6�Z0_h��U�_W_Wx"lhi]\]ëc�6_h�Wj � |6\���R�i]^�P�NQ\¬�Q|6Y�\��'_&ëc�6\���R�i�_Wxc���QP*�QP�NQ�4x6x4\]x4^*M�ª�Pax�R³��NQ_W_­\�x"U­|6\�l}|¤U�_¥|qR&�S_RaT4T4_&TE_��>��N}R¤x4PLT4_W�­�QP�T4_hx6P��Q_*�Q|4_���_h�qR�N}R��QP�NQ�[§A°q^a�>NQ_[ILK&T6«�j � |6\��HR�i�^aP�NQ\��Q|4Yn\���¹>x4P&U­x�R��¥�Q|6_^a_hx6_hN}R�i]\�»W_WTET4\�����NQ\�Z6�>�Q\]�a_Hi]RWU8\]x��Q|6_�\]x w P�NQY�R��Q\�Pax��Q|6_WP�N�V�i]\¬�Q_hN}R��Q�4NQ_¤§ b ��\.R�x6T C l D i�\�_WlW_�I�JaJaJa«�jb NQ_Wi]R��Q_&TEY"_h�Q|6PLT!\]��ª@_&R�NQi©· ��Y"_h�Q|6PLT�P w lWi��6���Q_hNQ\�x4^6§©ª@_&R�NQi'K&e�ÙaÙS«­\�i]i��4����N}R��Q_&T�\�x+°q^��4NQ_[ILKh_ajq��x�Q|6\���Y"_��Q|6PLT¤U�_ w P�NQY¶lWi��6���Q_hNQ��P w x4PLT4_W� w NQPaY¶�Q|4_�P�NQ\�^a\�xqR�i~^�N}R��6|!R�x6T"lWPax4����NQ�6lh�­R�x6_hUÔ^�N}R��4|w NQP�Y �Q|6_¤lhi]�4���Q_hNQ�����6l}|v�Q|6R��*�Q|6_#x6_hUÒ^�N}R��6|vNQ_W�4NQ_W��_hxL�Q�³�Q|4_¤�QR�Y"_#�4NQPaZ6R�Z4\]i�\¬�¢VvT>\]����NQ\�Z4�4�Q\�Pax.jM�ª�\]��Q|4_Wx!NQ�4x+Pax!�Q|6_³lWi��4���Q_hN¥^�N}R��4|~jM'P��Q|E�Q|6_���4x6lh�Q\�PaxE��NQ_h_[R�xqT+�Q|4_�lhi]�4���Q_hNQ\�x6^�R�i�^�P�NQ\¬�Q|6Y"�¥�qR����HY"_W���QR�^a_W�¥�Q|qR��HR�NQ_ w �6x6l��Q\]P�x6��P wlWi��6���Q_�NQ�¥P w x6PLT>_W�Wj6��P&U×R�NQ_³�Q|6_W��_�R�i�^aP�NQ\¬�Q|4Y"�­NQ_hifR��Q_&T!�QP�¼³M'ª�±� |6_*��|4P�N���R�x4��U�_hN�\���Q|qR��¥Z0P��Q|!�Q|6_��¢�6x4lh�Q\�Pax+��NQ_W_�R�xqT��Q|6_*lWi��6���Q_�NQ\]x4^"R�i�^�P�NQ\¬�Q|6Y"��R�NQ_*���0_Wlh\fR�il&R���_W�*P w ¸*\]¹L�4l�|4\;R��4�4NQP&�>\]Y�R��Q\�Pax6�Wj.Ø4P�N*_h�4R�Y"�6i�_a`0�Q|6_­���4x6lh�Q\�P�x¯��NQ_W_#R�i�^aP�NQ\¬�Q|6Y±\�x¯°6^a�>NQ_¤IcK&TR��4�4NQP&�>\�Y�R��Q_W���Q|4_�¼H\�Z6Z6� w NQ_W_�_Wx6_hNQ^�V��6��\�x6^>ìÉTñ ò ��ò5ó Ô ò� É q��8 È É ��8O� È É ���O� f É ��8 f É �Q� §ihaIa«U­|6\�i�_*�Q|6_³lhi]�4���Q_hNQ\�x6^¤Y"_��Q|6PLT!\�x�°q^a�>NQ_�ILKh_�R��6�>NQP&�4\�Y�R��Q_W��Q|6_�¼H\�Z6Z4� w NQ_W_*_Wx4_hNQ^�V!�6��\�x4^4ìÉ ñ ò ��ò.ó Ô ò � É q��8�È É ��8Q�O�Yf É ��8 §ih�dS«M'P��Q| _WëL�6R��Q\�Pax4��h�I�R�xqTÔ_&ëc�6R��Q\�Pax h�d�R�NQ_vl&R���_W�+U­|4_hNQ_¯�Q|4_¨¸*\]¹L�4l�|4\³R��4�4NQP&�>\�Y�R��Q\�Pax²\��+x4Pi�Pax4^a_hN¤R�x�R��4�4NQP&�>\�Y�R��Q\�Pax.ì@\¬�¤\��[R�x�_h�4R�lh�[_h�>�4NQ_h����\]P�x~j;�½��l&R�x�Z0_���|6P&U­x��Q|qR��[U­|4_Wx6_h�S_hN��Q|6_NQ_W^�\]P�x�^�N}R��6|�§k_aj ^4j4°q^��4NQ_¤K5v�«­lhPaxc�}R�\�x6�¥Pax4i¬V+�¢U'P¤i�_W�S_Wi���§k\kj _�jqNQ_W^a\�P�x6��R�xqT��Q|6_W\¬N�\]xc�Q_hNQ��_hlh�Q\�Pax6�}«R�x6T��Q|4_ONQ_W^�\]P�x¤^�N}R��6|[|qR���x4PHlhV>lWi�_W�W`��Q|6_O¸*\]¹L�4l�|4\LR��6�>NQP&�4\�Y�R��Q\]P�x#\��;_h�4R�l��­§,Ä'_&T>\]T4\]R>`�Ø4NQ_W_hY�R�x~`R�x6T � _W\����³I�JaJ>KWR>ä b ��\~R�x6T C l D i�\]_hlW_[I�JaJLK�«�j� |4\]i�_#�Q|4_H���4x6lh�Q\�P�xv��NQ_W_�R�xqT¨lWi��6���Q_hNQ\�x6^+Y"_h�Q|6PLT>�³l&R�xvV4\�_Wi]T¯_h�4R�lh��¸*\�¹L�6l}|6\;_h�>�6R�x6��\�P�x6�W` w P�NY�R�xcV��>N}R�lh�Q\�l&R�iH�>NQPaZ6i�_WY"�[�Q|6_hV�NQ_&ëc�4\�NQ_!NQ_W^a\�Pax´��\]»h_W�"�Q|qR��"R�NQ_�_Wx4P�NQY"Pa�4�Wj � |6_E^�PSR�i¥P w �Q|6_^a_hx6_hN}R�i'¸*\�¹L�6l}|6\ w N}R�Y"_hU�P�NQ¹Ä\��*�QP+°qx6Tv_h�>�qR�x6��\�Pax4���Q|6R��[R�NQ_"P w NQ_&R���P�xqR�Z6i�_�R�lWlW�>N}R�lhVÄZ6�>��P ww R�Ni�_W���lWP�Y"�6i�_h�>\¬�¢V#�Q|6R�x"�Q|6_����4x6lh�Q\�P�x���NQ_h_ajL�¢x�^a_hx6_hN}R�i©`LZcV"l}|6PcP���\�x6^[R�x!R��6�>NQPa�>NQ\fR��Q_H¸*\�¹L�6l}|6\R��4�4NQP&�>\�Y�R��Q\�Pax+R�xqT!lWP�N�NQ_W���0P�xqT>\]x4^"^a_Wx4_hN}R�i�\�»W_&T�M'ª´R�i�^�P�NQ\¬�Q|6YE`6Pax6_*lWR�x�R�T����4���­�Q|6_³��N}RaT>_h¦½P�ÑZ0_h��U�_W_Wx�R�lWlW�4N}R�lhV�R�x6T�lWPaY"�4i]_��4\¬��Vcj b �¤R¨�4N}R�l��Q\]lWR�iY�R����Q_�N&`;|6P&U �QP¨l}|6PcPa��_��Q|6_� ¢P��4�Q\�Y�R�i�£¸*\�¹L�6l}|6\@R��6�>NQP��>\�Y�R��Q\�P�x¯\������Q\]i�i�Y"P�NQ_�R�xvR�N��³�Q|qR�x¨R���lh\]_hx6lW_aj � _[P�x6i¬V�P�Ñ�_hNH�Q|4_#R�T4�L\�lW_��Q|6R��Pax4_���|6P��6i]T���N�V��QP�\�x6���>NQ_E�Q|qR��"�Q|4_���|6P�N��Q_W����i]PcP��6��\�x´�Q|6_E^�N}R��4|²R�NQ_¨_Wxc�Q\¬NQ_Wi¬V�\]x4lWi��qT>_&T´\�x¸*\�¹L�6l}|6\0NQ_W^a\�P�x6�W`q��P[�Q|qR��­�Q|4_hV!R�NQ_*|qR�xqT>i]_WT!_h�4R�l��Qi�Vcj

� Ð é ÌsÌ � Ã2í� |6_H���6lWlW_h���¥P w M�ª�R�xqT+¼³M�ª�R�i�^aP�NQ\��Q|4Y"�­\]�­_h�>lW\¬�Q\�x6^>`6Z0_hl&R��4��_³\¬�­Y"_&R�x4���Q|qR��­Y�R�xcV�T4\¬Ñ�_hNQ_Wxc�¹L\�xqT4�~P w �4NQP�Z6i�_WY"�~�Q|6R��2��_W_WY"_&T*��P�T4\¬®"lW�4i��~�QP�|6R�x6T4i�_a`�\�xL�SPai��L\�x6^­^�N}R��6|6�~U­\��Q|�Y�R�xcV�x6PLT>_W�;R�x6Ti�PcPa�4�W`2l&R�x�R�lh�Q�qR�i]i¬V¯Z0_#|6R�x6T4i�_&T¨�6��\�x6^!_h®"lW\�_Wxc��R�xqT¨��V4���Q_hY�R��Q\�l&R�i�i¬V¯lWP�N�NQ_hlh�}R�Z4i]_�R�i�^aP�NQ\¬�Q|4Y"�Wj� |6_h��_'R�i]^�P�NQ\¬�Q|6Y"�@R�NQ_�Y#�6l}| w R����Q_hN@�Q|6R�x C P�xL�Q_ X R�NQi�PHR��6�>NQPSR�l�|4_W�W`�R�x6T³�Q|6_�R��4�4NQP&�>\]Y�R��Q\�Pax6�.�QP�Q|6_ w NQ_W__Wx4_hNQ^�V��Q|qR����Q|4_hV[R�NQ_­_hÑ�_Wl��Q\]�a_Wi¬V#\�Y"�6i�_WY"_hxL�Q\�x4^HR�NQ_¥Y"P�NQ_���Pa�4|6\����Q\�l&R��Q_WT"R�xqT#R�lWlW�>N}R��Q_�Q|qR�x´ �Y"_&R�x>¦½°6_Wi]T4£#R��6�>NQP&�4\�Y�R��Q\]P�x6�Wj� _*_hxqT+U­\¬�Q|¨R¤���0_WlW�4ifR��Q\�Pax~ìq�0_hNQ|qR��6�¥\���\��Hx4P��¥_Wxc�Q\¬NQ_Wi¬V�R¤lWP�\]x4lW\]T4_Wx4lW_��Q|qR��¥Y"_W���QR�^a_h¦½�6R�����\�x6^\�xE�Q|6_�M�ª²R�i]^�P�NQ\¬�Q|6Y�NQ_W��_WY[Z6i�_W�­�Q|6_�Y"_W���QR�^�_h¦½�qR�����\]x4^"�Q|qR��H^aPc_W��Pax�Z0_h��U�_W_hx¨�Q|4_�x6_W�>NQPax6�¥\�xPa�>N¥Z4N}R�\�x4�Wj

Page 31: Understanding Belief Propagation and its Generalizations Jonathan

Ã������ � ��� � � �b �¢\k` { j C j�R�xqT C l D i�\]_hlW_a` F j � jL§½I�JaJaJa« � |6_�¼H_Wx6_hN}R�i�\]»h_&T[î³\�����NQ\�Z6�>�Q\��S_ G R&U*j������ �"!6ê�#�ç���#m���kæ&��ç����ç#�}ç&è���ê�!D#w�kæ&��ç$!J�4éO��ê��*�aå>ì dSIwh1�cd��Sdb �¢\k` { j C j2R�x6T C l D i�_W\�lW_a` F j � §½I�J�J>K&« � |4_�¼H_Wx4_hN}R�i�\�»W_&T�î³\�����NQ\�Z6�>�Q\��S_ G RWU�R�x6TvØ>NQ_W_ D x4_hNQ^�VC \�x6\�Y"\�»&R��Q\�P�x~j � P�Z0_��4�6Z4i]\���|4_&T"\]x"�Q|4_�Þ�êQ�1�}éQéQ��æAç�%w����èF���6é&%('.���*)¥ç4ç�ø�#('+)F',']é�ê��i��ç�,���ç&è&é�ê��é�ç��}é���ç-,��w!�!Lø>ç4æ&�O#w�kæ&��ç+./,���ç��kêQ�w' .�#�ç��0,��.!�~�ø��kæAç[%[á

M�R��>�Q_�N&` F j � j2§�KWeaÙSI�«�j1��àm#m���&' �32��(' Ü�éQ�54/�1�cé�' �³æBç�2��i#w�kæ,�O�kæ&�O#('14ÄéQ����#�ç6æ����}j b l&R�T4_WY"\�l*ª;NQ_W���WjM'_hN�NQP��~` X j�` b j�¼Hi]R&�>\�_W�>��`&R�x6T*ª@j � |6\¬�Q\�Y�R��¢��|6\�Y�R*§�K&e�eadS«�j ü _&R�N { |6R�x4x6PaxHi]\�Y"\¬�~_hN�NQP�N�¦½lWP�N�NQ_Wlh�Q\�x6^lWPLT>\]x4^HR�xqT�T4_WlhPLT4\�x6^>ì � �4NQZ0P�¦�lhPLT4_W�Wj���x�Þ�êQ�1�}éQéQ��æAç[%w�768'�'8%9����� �:�}ç���é�ê}ç�#.�kæ&��ç�#w';,���çWè&é�êQé�ç��}é��ç$,��w!L!�øLç6æ&�O#w�kæ���ç��Q`�¼H_hx6_W��R>` { U­\¬�Q»W_�NQifR�xqT�`q�6�.j�K&Jaå��.�6K&Jmv�J>j

X P��S_hN&` � j C j>R�xqT � j b j � |6P�Y�R��³§�KWeae>K&«�j+�Y']é�!"é�ç�������è<�}çWè9��ê�!$#w�kæ&��ç-!J�6éQ��ê���j � P�|6x � \�i�_hV�R�x6T{ P�x6�Wj

X P&U�_Wi�ik` F j2§�K&e�eaÙS«�j���xc��NQPLT4�6l��Q\]P�xE�QP"�¢x w _hNQ_Wx4lW_ w P�N³M�RWVc_W��\]R�x ü _��¢U�P�NQ¹L�Wj0�¢x C j � P�N}T4R�x�§ D T�j «�`= éQ#�ê}ç6æAç[%�æAç->�ê�#�~��Læ���#w'14/�1�cé�' �Qj C � � ª�NQ_h���WjØ6P�����P�NQ\�_hN&` C j§½I�JaJ>K&«�j����Q_hN}R��Q\��S_ F _Wi�\]R�Z4i�\���V�M�R���_&T�î³_WlWPLT>\�x6^¨P w G P&U�î³_Wx4��\���V�ª;R�NQ\���V X |4_Wl�¹X PLT4_h�Wj � |6_­ª;NQPclW_W_WT4\�x6^��OP w �Q|6_� DOD�D ��xL�Q_�NQxqR��Q\�P�xqR�i { V4Y"�0P���\��6YyP�x#�¢x w P�NQY�R��Q\�Pax � |4_WP�N�Vc`� R���|6\�x6^��QPaxEî X `6�.j0I�dad>j

Ø4NQ_W_hY�R�x~` � j � j�` D j X j.ª;R���»h�QP�N&`.R�xqT j � j X R�NQY"\]l}|qR�_Wi§½I�J�JaJa«�j G _&R�NQx4\�x6^�i]P&U¦½i�_W�S_Wi;�L\���\�Pax.j�}ç��&'�á@?�áA,��w!F~�ø���é�êCB.æ,�Qæ&��ç�D�E~§�K�«�`0I(h9�c�[vcjØ4NQ_hVc`4M�j � j2§�K&eae�ÙS«�j >�êQ#�~��Læ&�O#('14x�1�cé�' ��è9��ê94x#m���LæAç0é = éQ#�ê}ç6æAç[%;#�ç��GF³æ�%�æ,�i#('�,��w!L!�øLç6æ&�O#w�kæ���ç6jC � � ª;NQ_W���WjØ4NQ_hVc`qM�j � j�R�xqT�î#j � j X j C R�l}¹aRWV�§�KWeae�ÙS«�j b NQ_W�SP�i��4�Q\�Pax.ì.M'_Wi�\�_ w �>NQPa�6R�^SR��Q\�Pax¯\]x¨^�N}R��6|4��U­\¬�Q|lhV>lWi�_W�Wjc��x C j � P�N}T6R�x~` C j { jS¸*_WR�NQx6�W`cR�xqT { j b j { Pai�i]R�§ D T4�Wj «�`H)d�wÜwá0æAç3I�é�øLêQ#('+�}çWè9��ê�!$#w�kæ���çÞ�êQ�1�}é��O�}æBç�%J2��.�O��é�!��Q`qÓ�Pai��4Y"_#K&JLj C � � ª;NQ_W���Wj

¼³R�i�i]R�^�_hN&` F j0¼[j~§�K&e�åada«�j = �LK��q�cé�ç��Qæ,���A~�#�ê}æ,���M���6éQ���;�O�1�cé��Qj C � � ª;NQ_W���Wj¼³R�i�i]R�^�_hN&` F jc¼[j4§�K&e�åaÙS«�jM�}ç&è9��ê�!$#.�kæ&��çC!J�6éQ��ê��D#�ç��ON­é�' æ&#mÛ9']éO,��w!�!Lø>ç4æ&�O#w�kæ&��ç4j � Pa|6x � \]i�_hV#R�x6T{ P�x6�Wj

� R�R�¹L¹SPai]RL` � jO§½I�JaJ�JS«�j � �4�QP�NQ\fR�iOP�x���R�NQ\]R��Q\�Pax6R�i'R��6�>NQP&�4\�Y�R��Q\]P�xvY"_��Q|6PLT4�Wj~R&��R�\�i]R�Z6i�_�Pax4i�\]x4_�R��|c���Q�~ì P8P&UUU*j R�\kj Y"\��&j _&T>��P��0_WPa�4i]_�P��QPaY"Y"\QP��qR��0_�NQ�Wj |c�QY"ikj

� _Wx4��_Wx~`6Ø'j~§�K&eae�åS«�jR)¥çS�}ç��kêQ�1�wø����kæ&��ç{�i��ÝF#w�Sé��}æ�#�ç#I�é��TK7��êQ�.�Qj { �>NQ\]x4^a_hN&j� P�N}T6R�x.` C j¥�}j]`&U@j³¼H|6R�|>N}R�Y�R�x4\k` � j � R�R�¹L¹SPai]RL`³R�xqT G j { R��4i�§�K&eae�ÙS«�j b x \]xc��NQPLT>�6lh�Q\�Pax �QP��R�NQ\]R��Q\�Pax6R�i�Y"_h�Q|4PLT4� w P�NH^�N}R��6|6\�l&R�i�Y"PLT4_Wi��Wj0��x C j � P�N}T4R�x�§ D T�j «�` = éQ#�ê�ç4æAç�%!æBç�>�ê�#�~��Læ&�O#('4/�1�cé�' �Qj C � � ª�NQ_h���Wj

¸*\�¹L�6l}|6\k` F j~§�K&e(hLK�«�j�Þ��[�w��áRN¥é�ÜwáRV(6�`0e�ÙaÙ>j¸*\�¹L�6l}|6\k` F j�§�KWeae��S«�j { �0_WlW\]R�i.\������6_*\�x+|4Pax6P�N­P w F j>¸*\�¹>�4l�|4\kj>ª�NQP�^�N&j � |4_WP�N&j6ª�|LV>�Wj { �4�6�4i©j�`L�SP�ikjKaK1hL`~K&eae��4j

¸*��l}|6\���l�|6R�x4^4`�Ø'j F j�`;M¥j � j;Ø>NQ_hVc`;R�xqTÄ��j ¦ b j G Pc_hi]\�^�_hN�§½I�J�J>K&«�j�ØqR�l��QP�N¤^�N}R��4|6�[R�xqTÄ�Q|4_"���6Y¤¦�4NQPLT>�6lh�¥R�i�^aP�NQ\¬�Q|4YEj;����� �W!4êQ#�ç��9áR�}ç&è��[áA!J�6éQ��ê��SD�X�`6�aeaÙ1��hLKWe>j

G R��4NQ\¬�Q»W_Wx.` { j G j6R�xqT�î#j � j { �6\�_W^�_Wi�|qR�i��Q_�N[§�KWeaÙ�ÙS«�j G Pcl&R�i~lWPaY"�4�4�}R��Q\]P�x6�'U­\¬�Q|E�>NQPaZ6R�Z6\�i�\¬�Q\�_W��P�x^�N}R��6|4\]lWR�iq����NQ�4lh�Q�4NQ_W�OR�x6T#�Q|6_W\¬N'R��4�6i�\�l&R��Q\�Pax"�QP*_h�>�0_hN�����V>���Q_WY"�¥§AU­\¬�Q|�T4\���lW�6����\�P�x0«�j�?��wøLê}ç�#('��è����6éON3�w�(#('R2��i#w�kæ,�O�kæ&�O#(';2��1�hæ½é����Y.A20é�ê�æ½é��dÝ[Z(E�`~K1h(v1�LIaI��4j

C R�l}¹�R&Vc`�î#j � j X j4§�K&e�eaea«�j>¼HPcPLT¤_hN�NQP�N�¦½lWP�N�NQ_Wlh�Q\�x6^�lWPLT>_W��ZqR���_&T¤Pax¤�a_hN�V����qR�NQ��_­Y�R���NQ\]lh_W�WjH�������!6ê�#�ç���áR�}ç&è9�[á\!J�6éQ��ê��*D]Z�`qdae�e5�c�ad>K�j

Page 32: Understanding Belief Propagation and its Generalizations Jonathan

C l D i�\]_hlW_a` F j � j�`Lî#j � j X j C R�lW¸�RWVc`LR�x6T � j>Ø'j X |4_Wx6^�§�K&e�eaÙa«�j � �4NQZ0P�T4_WlhPLT4\�x6^�R��'R�x�\�x6���}R�x6lW_P w ª@_&R�NQi©· �7^ Z0_Wi�\�_ w �4NQP��qR�^aR��Q\�Pax.·4R�i�^aP�NQ\��Q|4YEj+����� �_?�á��çJ2�é�'�á`)¥êQéQ#w�¥æAça,��w!L!ká16�b~§½I�«�`qKW�aJ5�K5h�ILj

C _W»&R�N}T.` C j�`2¼[j2ª;R�NQ\���\©`2R�x6T C j b j�ÓH\¬N}R���P�NQP�§�K&e�Ùmv�«�j�2w~qæAçx%('�#.���A���6éQ��ê��x#�ç���Û}é��(��ç���j � P�NQi]T{ lh\]_hxL�Q\¬°6laj

C �4NQ�4|cVc`q¸¤j�`�Ä#j � _W\����W`.R�x6T C j � P�N}T4R�x�§�K&e�eaeS«�j G PcPa�cV�Z0_Wi�\�_ w �4NQP��qR�^aR��Q\�Pax w P�N*R��4�4NQP&�>\�Y�R��Q_\�x w _hNQ_Wx6lh_aì0R�xE_WY"�6\¬NQ\�l&R�i~���Q�6T>VcjL�¢xCÞê��1�5áOc.ç��}é�ê��i#�æAç����!æAçS)7�}j

ª@_&R�NQi©` � j!§�K&eaÙ�ÙS«�jMÞ�êQ�(Û�#(Ûhæ�' æ,�O�kæ&�ÔêQéQ#w����ç4æAç�%µæAç¶æAç���é�',' æ�%cé�ç��x�O�.�O��é�!��ed+ç�é��TK7��êQ�.�²��è{~�'�#wø��Qæ&Û9']éæAçWè&é�êQé�ç���é�j C P�NQ^aR�x�¸�R�� w Y�R�x6x~j

{ |6R w _hN�¼[j F j�R�xqT { |4_Wx6P&V�ª;j ª@j�j­§�K&e�eaJS«�jOª;NQPaZ6R�Z6\�i�\¬�¢V�ª;NQPa�6R�^SR��Q\�Pax~j<)¥ç6ç�#(' �;��èf4x#w���6é�!$#.�kæ&���#�ç��G)¥ê}æ,�kæ ß7��#w'1�}ç���é�'�' æ�%cé�ç��}é­ILì dSIwv1�cdmhaIcj� R�x6x6_�N&` F j C j;§�K&e�Ù>K&«�j b NQ_WlW�4NQ��\��S_[R��4�4NQPaR�l}|E�QP�i�P&UµlWP�Y"�6i�_h�>\¬�¢V!lWPLT4_h�WjR��� ���g!6êQ#�ç���á��}ç&è��[á!J�6éQ��ê��#�L!��ih+X&`�h�dad5��h��mvLj� _hi]i�\�x6^>` C j�R�xqT"Ä#j � j � _W|ק½I�JaJ>K&«�j­M'_Wi�\�_ w Pa�>�Q\�Y"\]»WR��Q\�Pax.ì b ���}R�Z6i�_¯R�i¬�Q_hNQx6R��Q\��S_E�QP�Z0_Wi�\�_ w�4NQP��qR�^SR��Q\�P�x~jÄO_&T4\]T4\]RL` � j { j~§½I�JaJLK�«�j b x!\]T>\]P���V>x6lhN}R��Q\]l��¢Pa�>NQx6_hV�Z0_hVcP�xqT�Y"_&R�x�°q_Wi]T��Q|6_WP�N�VcjL�¢x+î[j { RaR�T+R�x6TC j �4�0_hN*§ D T4�Wj «�`;)Y�wÜ.#�ç���é&4ÄéQ#�çSjOæ½é�'��k4Äé������1�w�ml]!J�6éQ��ê��u#�ç��LÞê�#m���kæ&�}é�j C � � ª;NQ_W���WjÄO_&T4\]T4\]RL` � j { j�` � j � jÒØ4NQ_W_WY�R�x~`ÒR�xqT Ä#j � _W\�����§½I�JaJLK&RS«�j�M'_��Q|6_ w NQ_W_ _Wx6_hNQ^�\�_W�W`¸*\�¹L�6l}|6\vR��6�>NQP&�4\�Y�R��Q\]P�x6�W`ER�xqT Z0_hi]\�_ w �>NQPa�qR�^SR��Q\]P�x R�i�^aP�NQ\��Q|4Y"�Wj b ��R�\�i]R�Z4i�_ P�x6i�\�x6_�R��|c���Q�~ì P8P&UUU*j Y"_hNQikj lhPaY*P�NQ_h�0P�N��Q�mP ��F I�J�J>K�¦}KWå�P�\]x6T4_h��j |c�QY"ikjÄO_&T4\]T4\]RL` � j { j�` � j � j~Ø4NQ_W_hY�R�x~`0R�xqTCÄ[j � _W\����¤§½I�JaJ>KhZ0«�j X |qR�N}R�lh�Q_�NQ\]»h\]x4^EZ0_Wi�\�_ w �>NQPa�6R�^SR��Q\�PaxR�x6T²\��Q�+^a_Wx4_hN}R�i�\�»&R��Q\�P�x6�Wj b ��R�\�i]R�Z6i�_vPax6i�\�x6_ÄR��+|L���Q�.ì P8P�UUU*j Y"_hNQikj lWP�Y*P�NQ_W�0P�N��Q�mP ��F I�JaJLKh¦K5hLP�\�xqT>_h��j |L�QY"ikjÄO�6\�i�i]_�` b j G j­§½I�JaJLK�«�j b T>Pa�6Z4i�_h¦½i]PcP��´R�i�^aP�NQ\¬�Q|4Y �QPvY"\�x4\]Y"\�»W_��Q|6_+M'_h�Q|4_ER�x6T�¸*\�¹L�6l}|6\ w NQ_W__Wx4_hNQ^a\�_W�Wj6�Hx6�4�6Z4i]\���|4_&T.j