Retrieval and browsing of news video by human relationshipide/res/paper/J01... · The Institute of...

6
The Institute of Electronics, Information and Communication Engineers Technical Report of IEICE. 101-8430 2-1-2 Tel: (03)4212-2585 Fax: (03)3556-1916 {ide,satoh}@nii.ac.jp Retrieval and browsing of news video by human relationship Ichiro Ide Shin’ichi Satoh National Institute of Informatics 2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo 101-8430, Japan Tel: +81-3-4212-2585 Fax: +81-3-3556-1916 {ide,satoh}@nii.ac.jp Abstract Most conventional news video retrieval and browsing was based on simple keyword retrieval and browsing of the retrieved segment. We are considering a retrieval and browsing method that utilizes structured information derived from the target video contents. In this presentation, we will introduce a re- trieval and browsing method based on human relationship that is derived from cooccurence of human names in large news video corpus. key words news video, retrieval, browsing, human relationship, caption, human name

Transcript of Retrieval and browsing of news video by human relationshipide/res/paper/J01... · The Institute of...

Page 1: Retrieval and browsing of news video by human relationshipide/res/paper/J01... · The Institute of Electronics, Information and Communication Engineers Technical Report of IEICE.

���������� The Institute of Electronics,

Information and Communication Engineers

���Technical Report of IEICE.

����������������� !"

# $ % & ' ( ) %

*+,-./01

2 101-8430 3456789:;< 2-1-2Tel: (03)4212-2585 Fax: (03)3556-1916

{ide,satoh}@nii.ac.jp

= > ? @ AB������C�DE��FG�E�H DE��IJKL �M��������NO�����P�����QR��C�ST UV������WX������C !YZ�[\"�]^�_#$DE%�M `&'(��abc���WX������C�def�ZV��ghi�jk�lm��)%���ghno�n��]^ YZ��*�_#('�C�DE%�M p�+qr s,���

tuvuw �����C�DE��M�ghno�-./012�ghi

Retrieval and browsing of news video by human relationship

Ichiro Ide Shin’ichi Satoh

National Institute of Informatics

2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo 101-8430, Japan

Tel: +81-3-4212-2585 Fax: +81-3-3556-1916

{ide,satoh}@nii.ac.jp

Abstract Most conventional news video retrieval and browsing was based on simple

keyword retrieval and browsing of the retrieved segment. We are considering

a retrieval and browsing method that utilizes structured information derived

from the target video contents. In this presentation, we will introduce a re-

trieval and browsing method based on human relationship that is derived from

cooccurence of human names in large news video corpus.

key words news video, retrieval, browsing, human relationship, caption, human name

Page 2: Retrieval and browsing of news video by human relationshipide/res/paper/J01... · The Institute of Electronics, Information and Communication Engineers Technical Report of IEICE.

1 3456

1.1 789:;<=>

�?�@������A�����B�@��������� �CD��E������������F ��G������ �������������H����� �!�"#$%��&'(� ��I�����J��)��A�����"#$%����� �F&�(�*���+$,%�!������K&D��L&�-������)�"#$%��+$,%��.�� /0123%4�!!MN�"#�K&$%1�O!��"#&&!M& 5�1�6'71�(8���� �F&)*��� &I+��

1.2 PQ9RS

90�:(��T��D,&-.��;

• <=�� )�U>�?.�?V/@�@$A�!���B� ���CDEF@GH�K�� ���

• IJKLMN$O )�U>��01234P�Q�56>��� �NTSC@�W7�)�U>�)XYZ 30MN$O�23����

• RSTUVWTUXYZ4 )�>��T8�MN$O�$9�� �[Z4 )�XYZ4:�[T8�;�� �

• RK\]^_`X$a )�b\�T8��XYZ4�$9�� �"#$%����)�X$a)]<=c���

��T1��> 1���M^�23&?��

> 1: ���M^�23

2 _`abcdefg

2.1 hijklmnopqrst

�A�"#$%���!��K��&�(�*�u@!)��AUBO?��X%2O&2C1�� [5]�> 2d�23&�> 3d�DE&?����X%2O�)�U>�?F.� V/@�/vefg$hi3jkXYal&mHDAnO�AU�BO?����:(�������:��Gw���+$,%�AU�2C���x-�"#$%yH1���

&^z�)?: 180I:J{���/�� ���

> 2: >�AUBO?��X%2O�23

> 3: >�AUBO?��X%2O�DE

2.2 |}~r��9����

��+$,%�2C �K������ 1�)�K�!:��� [7]�Q�������)����M^

1opqrstuvwxyz{|}~�������t����������������w���~��������������w���~���w���� ¡¢��£¤��s�¥ [1]�¦§w¨©�

Page 3: Retrieval and browsing of news video by human relationshipide/res/paper/J01... · The Institute of Electronics, Information and Communication Engineers Technical Report of IEICE.

�23 U>���ª�>�xLA=���M&�����1�1�J���)d�:(����!O!������xL&�M=N����)�«����b\����5O�¬�K��&I+�����)�P�QR "#$%������^­�

�SE T�����®� 5W1HeWhen, Where,Who, What, Why, Howl�K�(¯�x°U�eWhol±�²���V!M�I1��K��&�(���:(�W-)B����=1�)�³1��(�)���1����"#$%����)�!:

:�!:��*�!:���K��X�FY�Q�� &Z´� �[��Q� Z´��

3 ����6��������

3.1 � 

9\�:(�Z´]µ/�"#$%�����V!M�I1��K��&��d�^8&CD1������ �F���L&�-���_¡�!�"#$%���V��!M�I1��

�G��L [3] ���?�F�L [6]��-�������!)�U>���V��!M� !MN &=cG����¶·F@GH:�B`��=cG��L�Q��1�1�J���)��V!M�"# (¢`��

ab�²��!M& �� &=cG��� &Z´�����(!M& )�> 4£?���2N¸cd�¹ef���� �Q��iºC����V!M&>?����V!M:�&�:(����Q����:(!M&&6'7��� :��x-�!M d�g»�!M&&�¹���!����&hi��� ��/�:�]<��j1���?�F�6¤������)90��k�!M&���&����

�=cG�&�(�

1. .� /0123%4�!!MN&��

2. U>��!]<l¼&��

3. ]<��!MN�"#�!!M&&��

90�mn�m�k²�op&q½��

3.2 ¥¦§9¨�

23%4�!!MN&�����L)�r��*��©<eNamed entity taskl[8, 9]�Bª 1�����

> 4: iºC����V!M&>�£

����J���)�u@!�s«1��Ntu�¬­�NtvI1�u�,.wx&­M��®y�4z��L [2]&{D1�����L)�¯J,��B��Ntu�¬­�Nt:�uD��,.�|-���}~��� (x�&CD1���90d�£&?��

1��� ¾ 1�¿���x À !M1�� ¾ 1�¿�yz{|} À V�?H�

��:(�,.|-&�(�*�Ntu�¬­��1�!M&��:(�Nt&°RWC23%4+$,%±[10] °�°��23%4+$,%±[11]�!B-���_��,&��1��!°Á�±,²�Â�7:�,²�±�e�³´l±[13]&D�,²���&��� &�31� [2]�9\��L&D��90��k�!MN���&�(�

1. 23%4&���­M¯J,���­MX%2O JUMAN[12]&CD

2. NtµeNtu�(¯Nt�T8����l&��

3. r�Nt������&�¶

4. ¬­�NtvI1�,.wx&­M

Page 4: Retrieval and browsing of news video by human relationshipide/res/paper/J01... · The Institute of Electronics, Information and Communication Engineers Technical Report of IEICE.

5. wx�!M�Q� ·������&��

3.3 ¸¹º»9¨�

]<l¼���)�"#$%���>��23&CD1��(�B��"#$%��)> 5?�:(�23& ��]<�¼½)3j%A��V��°%AÃÄXYZ4±��*������)���x�&CD1��%AÃÄXYZ4&>������� :��]<l¼���&�(�

> 5: "#$%���23�£

%AÃÄXYZ4�xL)�90����Q��

• �/*��¾Åe3j%A��l�B²��

• ¿E¾Åe%AÃÄl����xL�ÀU)��

��:(�xL&Z´�90��k�%AÃÄXYZ4���&�(�

1. �/*��¾Å�B²����XYZ4e!MXYZ4l&��

2. !MXYZ4���MN$OeXYZ4��½MN$O&{Dl��Æ%4ǺO&xLAÈf4D 5�1�xLAÈf4D�:���� �ÁÂ�fº%A&�3����&%AÃÄXYZ4 ·�

1�1���:(1���1�%AÃÄXYZ4)�

]<�Ã��*��� �Q��d���]<�¼½�%AÃÄXYZ4��(B²�xL 1��90�xL&�\���

• An4D/v��*

An4D/v��*&�����*��!90��k&G��´��

3. MN$OD��ÉZÃ��&��ÉZÃ�{�ÀU&��

0.9

1.0

1.1

1.2

1.3

1 2 3 4 5 6 7 8 9 10 11フレーム番号

エッジ強度の相対的変化

タイトル字幕出現

> 6: An4D/v�*B(ÉZÃ�{��=�À7

��)�An4D/v��*� ��ÉZÃ�{�> 6?�:(�=��/«À7��� &CD1���9\��k_�75�:e% 468XYZ4l�"#$%��=1���)¡&��� [3]���)¡���¾Å��)¢£HSI��¤ [4]&CD1�Ä�¾Å�� 2akN$4CZEaÇ&CD1��L&D�����[Z4��)£1«������ 1��d�^8�

� 1?�:(Å¥�^8�O!������%AÃÄXYZ4e¼½l�����Ê�)�]<�¼½����!³�An4D/v��*1����V9�Q��

� 1: %AÃÄXYZ4��)¡�^8ËÌÍÎ ÏÐÑ ÒÐÑ ÐÑÓÔ ÕÖ× ØÙ×ÚÛÜÝ 53 0 0 100% 100%Þßàáâ 37 0 6 86% 100%

3.4 ¥¦ÆÇ9��

!M&���)�3.2 ���1�!MN�"#&�3.3���1�]<���¦´�D]<=1�$%��� :��(���:(1���1�!M&)�> 4?1�iºC����V!M�?���:(�!M&�b\G�e°§È±°ÉT±�¹l&�(��)�!���¹�:(�]<�"#1���:��&&�cʨ-��� )�/�¬ã(�

3.5 ¨ËnÌÍÎÏÐÑÒ}r

9\��k_�������"#$%�����V!M�!M&&������&��?�F��na

Page 5: Retrieval and browsing of news video by human relationshipide/res/paper/J01... · The Institute of Electronics, Information and Communication Engineers Technical Report of IEICE.

> 7: !M&�6'7;!M&�FnaAMä$%

AMä$%&)©1��X���> 7!M&&6'71�!M&��FnaAMä$%&?�����)��å����!M�I1��æg\Óª1�!M �B«¬�!M&&6'71���!M&&�¹��*�naAMä$% 1��æg\Óª1�!MN��åQ�­®�&fmZf��� :���I��!M&¯�°´�� ��/�����&���eÔB]<���"#Õ{�±�l®£1�!MN&�/«�?1�çè�kÖ½�Q��`�> 7�!M&�(¯�x-����I1�

��&�F���*�naAMä$%&> 8?�������FnaAMä$%)�> 7�!M&�FnaAMä$%�!M:�mafe�²�!M æg\�!M&³8��%é$f´�µ�l\Q�­®�e��V9)> 7��ê�»���­®�l&fmZf��� :��?������naAMä$%:���I��!M&�O!����&�F�/�¬���«�d��¹�:(�!M&�Q���&U>��:ëV/@�/v��K�!O�� ��/���?������>�)�mª�]<��æg\�!MNe��£�V9)°×¶��±l�V/@�/v��*1�I·�MN$O�Q�� > 8: ���FnaAMä$%;> 7��ê�»���

maf&fmZf1��?���^8&£?

Page 6: Retrieval and browsing of news video by human relationshipide/res/paper/J01... · The Institute of Electronics, Information and Communication Engineers Technical Report of IEICE.

4 ØÙ

�¸�q½��k�:ë�L]µ/�)¹�"#$%���!!M&&��1���?�FnaAMä$%&�31��)¡)�Ú3 13? 3º 16¯�! 5º31¯�»:�� 53¯��x-�"#$%��eX¯ 30�l&D������¼½���V/@�/v)9% 2,807V�]<¦)9% 560�Q������¾è�)¡�)�]<l¼���)ìí£¿AU7�/�� &î�´�3.3�q½��L&D³�!��¯����^8&CD1��d�^8��½ 2,799��"#�·��¦ 292��!M&�O!���O!��!M&&À� ��Q�·��ÔB!M�

"#�!M& 5�1����������)4zÔ.,:�·�Qe°1���± °1�Á^�±�¹l�ì��Ô.,�)��b\�Ô.�Q�·�Qe°1�ñ °1�ÄÅ�±�¹lV9�½������:(�·�Q�NÆï&�(� )Û��¿Eab&�E ���*ÇÜ�Q������!M&�BÈeÝ�e is-al�&l 5��� :�����¹�©9ï,��Q�?b\�±�x&ÉÊ��� ��/� Z´!���

5 Þßà6

J��)��Gw"#$%��+$,%�!�]<��"#&���� 1�!M&&��1�d�&�� 1�����?�F�L&�-1�)©1�^8&Ëð1��

¾ )�:��Gw���+$,%&=N1��!>���!M���CDWL�¹��á&Ì*�:�����j1���?�F�L�)*&I+�����"#�Íñ�]<�ȱeÎÏ�R��¹l&·¶��� :��!M&��cÊ�b\G�e!M:�maf�b\l 6'7²���á1�/��

âãäå

[1] ò�ÐÑ�¾óôÒ�1æÐç�J:ÓB� �ôÔ�Õ¶Ö£�×èôØ�1ÙÚÛ�K�Üõ�­\éê�ÝëÞì�¾óôí�ßóôÔ��àô¸�á�öâ�°ëôã�äå÷B�øîæï;“.�ðb&CD1�@�D"#$%/vç�X%2O”�Ö�z�vol.J84-D-II, no.6, pp.877-887eJune 2001l�

[2] ó�Bñ�á�öò;“¬­�NtvI1�2N¸"#$%/v�,.­M”��è�z�vol.39, no.8,

pp.2543-2546eAug. 1998l�

[3] ó�Bñ�¶Jéê�óáôë�á�öò;“XYZ4�±]µ«����AU���G��L”�Ö�z�vol.J82-D-II, no.10, pp.1543-1551 eOct.1999l�

[4] ßìôí��J�ê��àôõ�öô÷;“�¾Å���(�¢£HSV�?¤��-”�2N¸d�vol.49, no.6, pp.787-797eJune 1995l

[5] K�ÓB�øôÚ�ùôú�ɶîÍ�óáôï;“��na@fXaÇ�)ð�����*���@$A

È$%X%2O�ñ�”�2001?Ö�Á� SD-5-9�vol.�KX%2O 2�pp.367-368eMarch 2001l�

[6] S. Satoh, Y. Nakamura, and T. Kanade: “Name-It: Naming and detecting faces in news videos”,IEEE Multimedia, vol.6, no.1, pp.22-35 (March1999).

[7] K�ôû�òèÏò�ùóôü!�õàúö;“��+$,%�2C �M”�Ö�z�vol.J82-D-II,no.10, pp.1552-1560eOct. 1999l�

[8] B. M. Sundheim: “Named entity task definition,version 2.1”, Proc. Sixth Message UnderstandingConf. (MUC-6), pp.317-332 (Nov. 1995).

[9] “GýH;NE-.”�IREXû$fXYZkþ�$�pp.264-273eSep. 1999l�

[10] ÷ø��H9P�Kè^s«ù2eRWCPl;“RWC23%4@$AÈ$%ú 1´”�

[11] �°����ü�K���ýa¤�K�ûüý,F@GH��þ;“�°��23%4+$,%ú 2.0 ´”�http://www-lab25.kuee.kyoto-u.ac.jp/nl-resource/corpus.html:����

[12] �°����ü�K���ýa¤�K�ûüý,F@GH��þ;“¯J,���­MX%2O JU-MANú 3.6´”�http://www-lab25.kuee.kyoto-u.ac.jp/nl-resource/juman.html:����

[13] ���,���;“°�±,²�±�7:�,²�±�e�³´lþ�³�"AD�ë7@$A�”�