広島大学工学部における医用画像処理の取り組み:放射線科,眼科,内科
-
Upload
toru-tamaki -
Category
Health & Medicine
-
view
5.348 -
download
7
description
Transcript of 広島大学工学部における医用画像処理の取り組み:放射線科,眼科,内科
ZUQRVR¬��� C��9=��F��� �nT��¸��¸<���
�rb ¶ZUQR·�
h"p://kids.wanpug.com/top_person.html�
h"p://www.sciencekids.co.nz/pictures/humanbody/braintomography.html�
h"p://www.sciencekids.co.nz/pictures/humanbody/heartsurfaceanatomy.html�
h"p://sozai.rash.jp/medical/p/000154.htmlAh"p://sozai.rash.jp/medical/p/000152.htmlA
h"p://medical.toykikaku.com/ǡ�ɞɍɇɦ/Ʃ/Ah"p://www.sciencekids.co.nz/pictures/humanbody/humanorgans.html�
ÅƬƈ�
• ÅƬCTŗ+�
ŨŒƈ�
• Ɩ4Ȕ�• Ţ4Ȕ�
�Ʃƈ�
• �ƩNBI4ǃȆ�
h"p://sozai.rash.jp/medical/p/000154.htmlAh"p://sozai.rash.jp/medical/p/000152.htmlAh"p://medical.toykikaku.com/ǡ�ɞɍɇɦ/Ʃ/�
Non-rigid Registration for Medical Images Using a Free-form Deformation
with Multiple grids
6
ToruAHigaki,AKazufumiAKaneda,AToruATamaki,ANobutakaADate,AShogoAAzemotoA:A"NonMrigidAImageARegistraPonAforAMedicalADiagnosisAUsingAFreeMformADeformaPonAwithAMulPpleAGrids,"Aŗ+ȗ���ǐ,AVol.37,ANo.3,App.286M292A(2008A05).AToruAHigaki,AToruATamaki,AKazufumiAKaneda,ANobutadaADate,AShogoAAzemoto:A"NonMrigidAImageARegistraPonAforAMedicalAImagingAusingAaAFreeMformADeformaPon"AProc.AofAIEVC2007;AImageAElectronicsAandAVisualACompuPngAWorkshop,AInsPtuteAofAImageAElectronicsAEngineersAofAJapanA(2007A11).A�
7
Medical Imaging Technology
• Growth of medical imaging devices • Diagnosis using medical images
– Visualization – Computer Aided Diagnosis
ImportanceAofAAMedicalAImagingATechnologiesA
8
ObservaPonAusingAmulPMmodalAimagesAProvidesAmoreAinformaPon!A
Diagnosis using multi-modal images • Modality
– Medical imaging Devices CT, MRI, PET, etc...
• Features of each modality
CT X-Ray Structure MRI H atoms Tissue PET FDG-tracer Cancers
9
Alignment of Multi modal images
• Superimpose display – CT + PET
Defining the locations of the cancers
CTAimageA PETAimageA
+A =A
14
Proposed deformation model
• Multiple control grid: – Global grid
• entire image • rough alignment
– Local grid • observation area • accurate alignment
15
Interaction between global and local control grids
• Sequential operation <Step1> 2adjusting a global grid 2align the global area
<Step2> 2adjusting a local grid 2align the local area
registraPonA
registraPonA
17
Sampled images
ProposedA
BMSplineAbasedAFFDA
Modality CT (mono-modal) taken at different times
Resolution 152×200 Observation
area 79×94
Proposed B-Spline Control
grid Global 6×6 Local 6×6 11×11
order 5 3
ImagesA
ControlAgridsA
ɲʍʤʃɱŕɀɓÅƬCTŗ+ɞ ƮIʅɽʟʩʋʫʁʣʩ�
Automatic Heart Segmentation in CT images by Using Atlas�
đŖȣO ,AĢwȣÄ,ABisserARaytchev,AŐĄȣÄ,AȂŖȣfò:AȽ�Ǯù¸ɞǓñɝɫɮ8IƦDž�ɞɓɨɞÅƬƽűİȾȣž62lȣȗĩʪÍzȋDz���rìǼDz_��,µ¥©ě��,Aµ¥ʬ2011A10/22ʭ.AƆ¦¹²,AĢwÄ,ABisserARaytchev,AŐĄÄ,AȂŖfòȣ:AȽɲʍʤʃɱŕɀɓÅƬCTŗ+ɞƮIʅɽʟʩʋʫʁʣʩȾ,AÍz9œ��ȣɽʤʕɳɼʃɚCADŮŹ�ȣž141lŮŹšƽ�ȣʗʨɽʤʞ,Aµ¥¬Ż��,Aµ¥(2010A11).AĢwÄ,Ɔ¦¹²,ABisserARaytchev,AŐĄÄ,AȂŖfòȣ:AȽɲʍʤʃɱŕɀɓÅƬɞśŷʠʈʥʋɳŗ+ȊɞƮI�ƚ_ɰɐȾ,AMIRU2010Aŗ+ɞǑǖʪœLJʁʩʛʂɵʞǔòȕ,App.894M897,AȄǠ¬Dž0rȓʅʩʇʫ,ALĵǵȣ(2010A07).A
ŮŹƤþ�ƷƹÉÅŜË ʪ÷ć�ɞĥ�Tmɞžʰ� ʪÅƬɞeɭɞŎźɍɓƹƂ(ǜ�ƹƂ)ɅTm�
2�
�IƦ�
ª�´Ǽ�
ªÖ�\Ö�
ƢIƦ�
\8IƦ� ªC�ƺĎ�
ǜ�ƹƂ�
ʗʤʫɼ�
ªlöĎ�Ŏź�
ʃʒʃʞ�
Xh��Y4GAf�¢��s�v��
h"p://www.mhlw.go.jp/toukei/saikin/hw/jinkou/geppo/nengai11/kekka03.html#k3_2�
p5Ⱥ�ɜĥm?ĥ�ðɞD_ʬ²Õ23³ʭ�
ÅƬCTŗ+ɞªÅ4ƫɞ ʅɽʟʩʋʫʁʣʩ�
gȞŅȺȣ�ƚȿɰɐɞƇ¸Ʌ���
Ȩ�
ĨǦţŦȋɞȡɀªÅ�4ƫɱʅɽʟʩʋʫʁʣʩɍ�ƚȿɰɐ�
ªÅ�4ƫ(endo)ȻÅƬ2�(WH), ªÅ��ƫ(epiLV) , \Å�(RV)ɞ4ɖɞțxɞʅɽʟʩʋʫʁʣʩɅÆǀ
ʅɽʟʩʋʫʁʣʩ�
Ǎóìéɞɓɨɝ�
ƒ_ŗ+�
CTɚSPECTŗ+ɞƒ_ŗ+ɅĂH�
�ƚȿɰɐ�
CTŗ+�ĜDZÍz�
SPECTŗ+�ġƥÍz�
ŗ+ȊɞŦȋɅ�ɀɓɨťä�ƚȿɰɐɟnȖ�
ƹij�
SingleMphotonAemissionAcomputedAtomographyA(SPECT)A�
h"p://en.wikipedia.org/wiki/File:SPECT_CT.JPG� h"p://en.wikipedia.org/wiki/File:Rosies_ct_scan.jpg�
XMrayAcomputedAtomographyA(CT)A
SIMENS,ASymbiaATASeriesASPECT•CTAh"p://healthcare.siemens.com/molecularMimaging/spectMandMspectMct/symbiaMt�
GOAL�
ÅƬCTŗ+Ʉɬ4ɖɞțxɱȡƇ¸ɝʅɽʟʩʋʫʁʣʩ�
ÅƬCTŗ+� WH� epiLV� endoLV� RV�
ʪɲʍʤʃɱÅƬCTŗ+ɝ�ƚȿɰɐɍʅɽʟʩʋʫʁʣʩ ʪ|ŘÍzɞ�ƚȿɰɐɚŗƌÍzɞ�ƚȿɰɐɱ ŕ ʪƙɭǫɍ9œɞ�1�
Idea�
ɲʍʤʃ�
• ÅƬCTŗ+ɱWHʮepiLVʮendoLVʮRVɝ ʅɽʟʩʋʫʁʣʩɍɓɩɞ
• ŗƌÍzɚ|ŘÍzɱ#á
ɲʍʤʃ� ŗƌÍz� |ŘÍz�
�Ƞõİɚʋʃʍʌʫʇ�
�Ƞõİ • 6Čɞʋʃʍŗ+ɞ4Ȼ1ČɱɲʍʤʃĦɭɞ5Čɱʋʃʍŗ+ɚɎɮ Lj6ʒʇʫʩɱ:E �ʋʃʍʌʫʇ�
PatientŚ]1�
PatientŚ]4�
PatientŚ]2�
PatientŚ]5�
PatientŚ]3�
PatientŚ]6�
ŗƌɞ�ƚȿɰɐɞɦȺVS.ȺæĖØİ�
1�
1�
2�
2� 3� 4� 5� 6�
3�
4�
5�
6�
Patient Ś]�
�ƚ_ɰɐɝŕɀɓɲʍʤʃ�
����
1�
1�
2�
2� 3� 4� 5� 6�
3�
4�
5�
6�
Patient Ś]�
�ƚ_ɰɐɝŕɀɓɲʍʤʃ�
����
æĖØİ�ŗƌɞ�ƚȿɰɐɞɦ�
āƏţɝ�¼ɍɓɲʍʤʃ�
æĖØİɟŗƌɞ�ƚȿɰɐɞɦɝĨɣ�ƚȿɰɐɞ�îɱȍɀəɀɮ�
ŗƌɞ�ƚȿɰɐɞɦȺVS.ȺæĖØİ�
1�
1�
2�
2� 3� 4� 5� 6�
3�
4�
5�
6�
Patient Ś]�
�ƚ_ɰɐɝŕɀɓɲʍʤʃ�
����
1�
1�
2�
2� 3� 4� 5� 6�
3�
4�
5�
6�
Patient Ś]�
�ƚ_ɰɐɝŕɀɓɲʍʤʃ�
����
āƏţɝ�¼ɍɓɲʍʤʃ�
æĖØİ�ŗƌɞ�ƚȿɰɐɞɦ�
æĖØİɟŗƌɞ�ƚȿɰɐɞɦɝĨɣ�ƚȿɰɐɞ�îɱȍɀəɀɮ�
ÅƬƈ�
• ÅƬCTŗ+�
ŨŒƈ�
• Ɩ4Ȕ�• Ţ4Ȕ�
�Ʃƈ�
• �ƩNBI4ǃȆ�
h"p://sozai.rash.jp/medical/p/000154.htmlAh"p://sozai.rash.jp/medical/p/000152.htmlAh"p://medical.toykikaku.com/ǡ�ɞɍɇɦ/Ʃ/�
��ªA"�g����\o²�9�6�I!�jx�
Registration method for Cross-Sectional Images of the Fundus considering Eye Movement
¤ċÔ,AĢwÄ,ABisserARaytchev,AŐĄÄ,AȂŖfò,AĀĔȐÇ,AĄ4ưùȣ:AȽŨŒǴIɱƠÓɍɓŨ·óȘŗ+ɞ�ƚ_ɰɐØİȾ,Aŗ+ȗ���ǐ,Avol.A40,Ano.A4,App.A609M614A(2011A07).A¤ċȣÔ,AĢwȣÄ,ABisserARaytchev,AŐĄȣÄ,AȂŖȣfò,AĀĔȣȐÇ,AĄ4ȣưùȣ:AȽŨŒǴIɱƠÓɍɓŨ·óȘŗ+ɞ�ƚ_ɰɐØİȾ,Aŗ+ȗ���ȣž246lŮŹ�Ǖł�Ÿ,A09M02M01,pp.1M6,A£X��,A£X(2009A10).AĢwÄ,AŐĄÄ,AȂŖfò,AĀĔȐÇ,A�§º,AĄ4ưùȣ:AȽĜDZǂBÉɱƠÓɍɓ5Đ¼ŌʚʥʢʫʞʌʫʇɞťäƽűØİȾ,Aȗ�Ízǯ$��ǔòǐD,AVol.J91MD,ANo.7,App.1808M1817A(2008A07).AAŖ�,,AėwÄ,AŐĄÄ,AȂŖfò,AĀĔȐÇ,A�§º,AĄ4ưùȣ:AȽí�ŌóȘŗ+ɄɬɞŨ·ʚʥʢʫʞʌʫʇɞȡƇ¸6ĜÕØİȾȣMIRU2007Aŗ+ɞǑǖʪœLJʁʩʛʂɵʞǔòȕ,App.487M492,Aµ¥¬Ż��,Aµ¥ȣ(2007A07)AĈ�ƋƘ,AėwÄ,AŖ�,,AŐĄÄ,AȂŖfò,AĀĔȐÇ,A�§ºȣ:AȽʃʗʤʊʋɳʩɽİɱŕɀɓ5Đ¼ŌʚʥʢʫʞʌʫʇƽűØİȾ,AVisualACompuPngA/AɽʤʕɳɼʃɚȣCADA_aʁʩʛʂɵʞȣ2006A�Ÿȕ,App.221M226,AĊǻ��ȣƞÇȀɺʡʩʒʃ,ANƵȣ(2006A06).AAȂŖfò,AĄfŖƾò,AĀĔȐÇ,A�§ºȣ:AȽOCTɝɫɮí�ŌóȘŗ+ɄɬɞŨ·ʚʥʢʫʞʌʫʇ6ĜƅØİɞȉšȾ,Aŗ+ȗ���ǐ,AVol.34,ANo.4,App.370M378A(2005A07).�
h"p://www.sciencekids.co.nz/pictures/humanbody/eyediagram.html�
ophthalmoscope�
h"p://medicalMdicPonary.thefreedicPonary.com/ophthalmoscope�
h"p://en.wikipedia.org/wiki/Ophthalmoscopy�
ŮŹƤþ�
Ũ·ęĒ�ŨɞŝĩʪŔIJƞÒŝɞęĒɝĂŕ�
Ɩ4Ȕɞ{_ʪʪʪ�
ǃŲƐ�ȜɅà��
Optical Coherence Tomography (OCT)
ʪ0ɞ±ĺőǙɱÈŕ�
ʪŨ·ɞóȘŗ+ɱWÁ�
Imaging modality �� ��
����
�� ��
����
! Light source… near-infrared light
! Applying coherence of light
OCT3000 Carl Zeiss Meditec Japan
scanning mode
radial circular parallelradial circular parallel
Optical Coherence Tomography
Optical Coherence Tomography
ǃŲƐ�Ȝɞ�ģ-¼Ō�
őŌʪʪʪ�ģ-6ĜÕġƥɱêǧ�
ž2�OCTʪʪʪSpectral-domain OCTȣʬSD-OCTʭ ȹë¾ǰ¸ʪʪʪƉ0.01 [sec./Č]ȣ�ȹ�ƺɆ;LJƥʪʪʪƉ5[µm]�
ȡ"�
%�ɞŨŴəɟʮǝ1ɅnȖ�
Time-domain OCTɝ�ģ-6ĜÕġƥɱêǧ�
ȹë¾ǰ¸ʪʪʪƉ1 [sec./Č] ž1�OCTʪʪʪTime-domain OCTʬTD-OCTʭ
őŌʵʵʵ�ģ-6ĜÕġƥɱąêǧ�
ýU�
ȹ�ƺɆ;LJƥʪʪʪƉ10 [µ m]
Time-domain OCTɝɫɭë¾ɌɯɓóȘŗ+Ɲ�
ǃŲƐ�Ȝɞ�ģ-6ĜÕ�
Goal
gȞŅ�
^óȘŗ+ɞʃɺʡʩɌɯɓ�ƚɱĭ��
Consideration:
ʃɺʡʩɅōɕɓ�ƚɄɬɏɯɮ�
ŨŒǴIɅTm�
œÎɞʃɺʡʩǣǟ�
ő�ɞʃɺʡʩǣǟ�
ȭóȘŗ+Ɲʪʪʪí�ŌɞʃɺʡʩɝɫɭWÁ�
ŮŹŤţ�
Ŗ�ɬȣ[’07]
Õč�
Time-domain OCTɱŕɀʮǃŲƐ�Ȝɞ�ģ-6ĜÕɱ�ő�
gȞŅ�
• ʃɺʡʩǣǟɅWÁəɆɜɀOCTɩ�sɎɮ • ǃŲƐ�Ȝɞ¼ŌɱƠÓɍɘɀɜɀ�
rotation �
u
v
rotation �
rotation �
u
v
rotation �
• 2ǥɞlǤʪ²ƺŵIʒʤʟʫʇ�
í�ŌɝʃɺʡʩɌɯɓóȘŗ+ɞ�ƚ_ɰɐ • ʃɺʡʩǣǟɝ^óȘŗ+ɱǽƚ�
• �«�ƚəɞȡɌɱ�ƯɌɐɮȋð�
ȋDzŮŹ�
óȘŗ+18Č�ʬ512×256[pixel]ʭ�
ʪŨŒǴIɝɫɮɏɯ(d»1, d»2) ʪʃɺʡʩȉ��ƚɞɏɯ(du,dv)
Ũ·ʠʌʦ�
ʳ�Ⱥ2[mm] Ⱥ0[mm]
ŋÃŅʬRPEɞ�ŕʭ�
• �ȁţǎ"ʱĚ5ŏʮȇǥǒ«ʮŭǥǒ«
• ǃDŽţǎ"ʱŨ·ʠʌʦɞ¼ŌɱĤLJ�
Ũ·ʠʌʦɱŕɀɓĨǦ�
�ƚ_ɰɐɜɍ� ÀĉØİ� æĖØİ�
��� ���
����� ���� ����
����� ������ �����
����� ����� �����
����%�
������� ��
������� ��
ÀĉØİȣvs. æĖØİ�
OCTóȘŗ+18Č�
RPEĆżŅ�
0 [deg.] 30 [deg.] 60 [deg.]
90 [deg.] 120 [deg.] 150 [deg.]
• ǃDŽţǎ"ʪʪʪSD-OCTɞ¼ŌɱĤLJ�• �ȁţǎ"ʪʪʪCup-to-Disc ratioʬC/DĨʭ�
OCTóȘŗ+ɱŕɀɓĨǦ�
ΦDisc�
ΦCup� Disc
Cup
Φ
Φ=CDR
�Ǚʱ ȺȺí�ŌɝʥʃʤɴʃɌɯɓóȘŗ+Ɲ
ʬ5¸ȊȒ36Čʭ�
SD-OCTɞC/DĨɱĤLJɚɍʮ^Øİɚɞǒ«ɱƁ:ʪĨǦ�
Cup-to-Disc ratio (C/DĨ)�
æĖØİ� ÀĉØİ�
�ƚ_ɰɐɜɍ�SD-OCT�
ÀĉØİȣvs. æĖØİ�
�\OCT)3'�H�B�
o²�9�$-3,(* ��������������������mujx�� ���s�
1050 1600 22801050 1600 2280
�<01&l;�� %0#� DL¡p ������97hjx�
A method for generating retinal images for investigation of the glare by intraocular lens�
��ı²,ABisserARaytchev,AŐĄÄ,AȂŖfò,AĀĔȐÇ,Ą4ưùʱȣȽŨ4ʧʩʄɝɫɮɽʧɲšŔTmLJùɞɓɨɞƔƫ+ɞ�ÕȾ,AǃDŽɞŴ�,A÷ćŨ0��,AVol.A33,App.77M82(2012A06).A��ı²,ABisserARaytchev,AŐĄÄ,AȂŖfòȣ:AȽɽʧɲšŔüɞƔƫ+�ÕȾ,Až46lȣ÷ćŨ0���ƕ�,A14thATheAIRSJA2010,ʗʨɽʤʞʪÚȅȕ,App.48,AʒʁʕɳɿğĴ(2010A09).AȂŖȣfò,A`ʼnŖȣĸy,Aųīȣ.`,AŐĄȣÄ,AB.ARaytchev,AĀĔȣȐÇ,AĄ4ȣưùȣ:AȽŨ4ʧʩʄɝɫɮɽʧɲšŔTmɞLJùȾ,Aƭ¶ʅʝʏʫ,A÷ćŨ4ʧʩʄ ÝØƻ��ǐȣ(IOL&RS),AVol.A24,ANo.A1,A136M137A(2010A03).A`ʼnŖȣĸy,AŐĄȣÄ,AȂŖȣfò,ABisserARaytchev,AĀĔȣȐÇ,AĄ4ȣưù,Aųīȣ.`ȣ:AȽ0ƗǭǟʁʝʢʧʫʁʣʩɱŕɀɓŨ4ʧʩʄɝɫɮɽʧɲšŔTmɞLJùȾ,Až48l÷ćŢ4Ȕ��ƕ�,Až24l÷ćŨ4ʧʩʄ ÝØƻ��ƕ�,Až45l÷ćŨ0��ƕ�,A22ndAAsiaAPacificAAssociaPonAofACataractAandARefracPveASurgeonsAAnnualAMeePngAǕłÚȅȕ,Ap.77,AĊ�rȓʕɸʫʤʞ,AĊ�(2009A06).Aųī.`ʮŐĄÄʮȂŖfòʮĀĔȐÇʮ�§ºʮĄ4ưùȣ:AȽŨ4ʧʩʄɝɫɮɽʧɲšŔTmLJùɞɓɨɞʧʩʄɞʠʌʦKɚ0ɞʁʝʢʧʫʁʣʩȾȣMIRU2007Aŗ+ɞǑǖʪœLJʁʩʛʂɵʞǔòȕ,App.1171M1176,Aµ¥¬Ż��,Aµ¥ȣ(2007A07)A
h"p://en.wikipedia.org/wiki/Cataract�
¯µ¬M®�ʱŝĩǘūǖ,AŢ4ȔɚɟʲAh"p://www.obihiroMmed.or.jp/blog/2007/12/postM304.html�
�<±j��
h"p://en.wikipedia.org/wiki/File:Cataract_surgery.jpg�
ŮŹƤþ�
ȣ�ȣ�ȣ� ȣ�
ȣ�
ȣ�
ȣ��Ș�
ȣ�
Ũ4ʧʩʄ(IOL)�
Īÿ�ɞ�ǼɅŢŃɍǃEɅ��ɎɮŨŝ�
(Ș�
Ɣƫ�
Īÿ��
â1�
džƫ�
Įşİʱ ØƻɝɫɕɘĪÿ�ɞŃɕɓǼ;ɱWɭȎɆŨ4ʧʩʄɱâ1�
Ţ4Ȕ�
ɽʧɲūDŽ�
Ũ4ʧʩʄɞ¾șɝɫɭ0ľɅ�sɍɜɀõcɄɬɞ0ɞɻʤɖɆ(ɽʧɲ)ɱūDŽɎɮɊɚ�
" ǴǤ�ɞ�cǢɞʘʊʎʤɴʍ
" �ȊɞƼń
šŔŌį!�
QOV(Quality of vision)c�ɞɓɨĝȼɜŮŹɅƺɰɯɘɀɮ�
�ȓɞ0ľ�IOL�
ɽʧɲ�
�sɍɜɀ0ľ�
ȋDzŮŹ�
0�ʁʝʢʧʫʁʣʩɝɫɮ ʧʩʄɷʊʂɞ¼ŌɞǷɀɝɫɮɽʧɲɞ¾șɞǓĒ [Holladay et al. , 1999] [Franchini et al. , 2003]
ɽʧɲūDŽɞTmɚɜɭɁɮ4ŷȟɞ0ƗƐǠ�
# ʧʩʄɷʊʂəV� # ʧʩʄɷʊʂɄɬ1�ɍʮʧʩʄ4(ȘɄɬ:� # ʧʩʄ�(ȘɄɬ1�ɍʮʧʩʄɷʊʂɄɬ:� # ʧʩʄ4əƿðlɞ2V�
0ƗǭǟʁʝʢʧʫʁʣʩɱŕɀɓŨ4ʧʩʄɝɫɮ ɽʧɲšŔTmɞLJùȺ[ȂŖɬ, 2010]
Ťţ�
• Ũ4ʧʩʄɅɽʧɲɞǁɂõɝɛɞɫɁɜ¾șɱɂɘɀɮɄǓĒɎɮɓɨɞƔƫ+�ÕØİɞȉš
• 4ŷȟɞ0ƗƐǠɞɽʧɲɅɑɯɒɯƔƫ+ɝɂɮ¾șɱǓĒ
ɽʧɲūDŽɞTmɚɜɭɁɮ4ŷȟɞ0ƗƐǠɅ�ȓɝ ɛɞɫɁɝǁɂɘɀɮɄɟ�ù�
æĖØİ�
• 1Eŗ+ɝŗdž180¸ɞ�7ŗ+ɱ ŕ • ȂŖɬɞƺɕɓ0ƗǭǟʁʝʢʧʫʁʣʩɞƑčɱ@ŕ • 1Eŗ+ɄɬŨ·õcɢɞʜʊʔʩɽ • Ũ·əɞҸ;ɱ#�ɍʮŗ+ɚɍɘ:E
1Eŗ+� Ũ·�
ŕ�
0ƗǭǟʁʝʢʧʫʁʣʩƑč�
ʜʊʔʩɽ�
:Eŗ+�
ƽű�
0Ɨǭǟʁʝʢʧʫʁʣʩ�
• Ũ4ʧʩʄɟʁʡʫʗɷʊʂ¼Ō • �ðɞ²ƺ0Ɨɱ1�ɌɐʮŨ·ɥəǭǟ • 1�džɋɚɞŨ·�əɞ0Ɨɞ;ɱNJȅ
džƫ�
Ũ·�ʧʩʄ�
Ƹ½�
α�α�
1�džαɞù¸ʋʫʖʦ�
φ�θ�
ǃǥ�
ŗ+ɞ:E�
:Eŗ+�Ũ·�
ƽű�
θ�φ�
0 � 90� 180� 270� 360�30�60�90�
120�150�180�θ[deg.]�
φ[deg.]�
Ƒčʱ�ȊɞƼń(1�dž30¸)�
1Eŗ+� :Eŗ+�
Ƒčʱ�ȊɞƼń(1�dž30¸) 0ƗƐǠɋɚɞƽű�
ʧʩʄɷʊʂəɞV�� ʧʩʄɷʊʂɄɬ1��
ʧʩʄɷʊʂɄɬ:�� ʧʩʄ4əƿðl2V��
Ƒčʱ�ȊɞƼń(1�dž30¸) 0ƗƐǠɋɚɞƽű�
ʧʩʄɷʊʂəɞV�� ʧʩʄɷʊʂɄɬ1��
ʧʩʄɷʊʂɄɬ:�� ʧʩʄ4əƿðl2V��
Ƒčʱ�ȊɞƼń(1�dž50¸)�
1Eŗ+� :Eŗ+�
Ƒčʱ�ȊɞƼń(1�dž50¸) 0ƗƐǠɋɚɞƽű�
ʧʩʄɷʊʂəɞV�� ʧʩʄɷʊʂɄɬ1��
ʧʩʄɷʊʂɄɬ:�� ʧʩʄ4əƿðl2V��
Ƒčʱʧʩʄʠʌʦ�
�ȊɞƼń(1�dž30¸)�
Īÿ�� Ũ4ʧʩʄ�
ÅƬƈ�
• ÅƬCTŗ+�
ŨŒƈ�
• Ɩ4Ȕ�• Ţ4Ȕ�
�Ʃƈ�
• �ƩNBI4ǃȆ�
h"p://sozai.rash.jp/medical/p/000154.htmlAh"p://sozai.rash.jp/medical/p/000152.htmlAh"p://medical.toykikaku.com/ǡ�ɞɍɇɦ/Ʃ/�
Q�kQ<�®�9�NBIkQi�>´ �N�|a2§@K�£¢�
Designing Features and Classifiers for Colorectal Endoscopic Images based on NBI Magnification Findings��
ToruATamaki,AJunkiAYoshimuta,AMisatoAKawakami,ABisserARaytchev,AKazufumiAKaneda,AShigetoAYoshida,AYoshitoATakemura,AKeiichiAOnji,ARieAMiyaki,AShinjiATanaka,AComputerMAidedAColorectalATumorAClassificaPonAinANBIAEndoscopyAUsingALocalAFeatures,AMedicalAImageAAnalysis,AAvailableAonlineA13ASeptemberA2012,AISSNA1361M8415,A10.1016/j.media.2012.08.003.AAYoshitoATakemura,AShigetoAYoshida,AShinjiATanaka,ARieAKawase,AKeiichiAOnji,AShiroAOka,AToruATamaki,ABisserARaytchev,AKazufumiAKaneda,AMasaharuAYoshihara,AKazuakiAChayama,AComputerMaidedAsystemAforApredicPngAtheAhistologyAofAcolorectalAtumorsAbyAusingAnarrowMbandAimagingAmagnifyingAcolonoscopyA(withAvideo),AGastrointesPnalAEndoscopy,AVolumeA75,AIssueA1,AJanuaryA2012,APagesA179M185,AISSNA0016M5107,A10.1016/j.gie.2011.08.051.AKeiichiAOnji,AShigetoAYoshida,AShinjiATanaka,ARieAKawase,AYoshitoATakemura,AShiroAOka,AToruATamaki,ABisserARaytchev,AKazufumiAKaneda,AMasaharuAYoshihara,AKazuakiAChayama,AQuanPtaPveAanalysisAofAcolorectalAlesionsAobservedAonAmagnifiedAendoscopyAimages,AJournalAofAGastroenterology,AVolumeA46,ANumberA12,A1382M1390,A2011.AA`ʼnŖĸy,AŐĄÄ,ABisserARaytchev,AȂŖfò,AŽĈj�,A`ŖÕ�,AŖ�$Įȣ:AȽ�Ʃà�4ǃȆŗ+ɞNBIà�×ǁ;ȟɝyɗɇŋÃȁɚǖ?kɞNjLjȾ,Aȗ�Ízǯ$��ÙƻzdȺʒʇʫʩǑǖʪʟʌɳɲœLJŮŹ�ȣPRMU2011M3,AVol.111,ANo.47,App.13M18,AbY¡��,AÐūȣ(2011A05).AToruATamaki,AJunkiAYoshimuta,ATakahishiATakeda,ABisserARaytchev,AKazufumiAKaneda,AShigetoAYoshida,AYoshitoATakemura,AShinjiATanaka:A"AAsystemAforAColorectalATumorAClassificaPonAinAMagnifyingAEndoscopicANBIAImages,"AProc.AofAACCV2010A;ATheA10thAAsianAConferenceAonAComputerAVision,AVol.2,App.987M998A(2010A11),AQueenstown,ANewAZealand,ANovemberA8M12,A2010.A`ʼnŖĸy,AŽŖ�ů[,AŐĄÄ,ABisserARaytchev,AȂŖfò,AŽĈj�,A`ŖÕ�,AŖ�$Įȣ:AȽDenseASIFTɱŕɀɓ�ƩƨŞNBIŗ+ǑǖȾ,Aȗ�Ízǯ$��ÙƻzdʒʇʫʩǑǖʪʟʌɳɲœLJŮŹ�ȣPRMU2010M73,AVol.110,ANo.187,App.129M134,Aų¤��,Aų¤(2010A09).AAYoshitoATakemura,AShigetoAYoshida,AShinjiATanaka,AKeiichiAOnji,AShiroAOka,AToruATamaki,AKazufumiAKaneda,AMasaharuAYoshihara,AKazuakiAChayama:A"QuanPtaPveAanalysisAandAdevelopmentAofAaAcomputerMaidedAsystemAforAidenPficaPonAofAregularApitApa"ernsAofAcolorectalAlesions,"AGastrointesPnalAEndoscopy,AVol.A72,ANo.A5,App.A1047M1051A(2010A11).AMasashiAHIROTA,AToruATamaki,AKazuhumiAKaneda,AShigetoAYosida,AShinjiATanaka:A"FeatureAextracPonAfromAimagesAofAendoscopicAlargeAintesPne"AProc.AofAFCV2008A;AtheA14thAKoreaMJapanAJointAWorkshopAonAFronPersAofAComputerAVision,App.94M99A(2008A01)A
÷ćəɞ�ƩŠ�
• ƛËơ: 235,000bȣ(²Õ21³ȣSŔG*ŧǓɣ)† – ȡȢơɤɛƛËŏɟȡɀ
• ĥ�ơ: 42,434bȣ(a�)† – 20³CɚĨɣʮ1.7&}F – Šĥmɞž3�ȣ(ž1�: ƢŠʮž2�: ƣŠ) – 7³DzƓəʮ�ÉɞŠĥmž1�
• 5³Ŕ�ŏ: Ćăɔɚ20%��
ȦȥȨ�† http://www.mhlw.go.jp/toukei/saikin/�‡http://www.gunma-cc.jp/sarukihan/seizonritu/index.html�
0AA
20AA
40AA
60AA
80AA
100AA
stageA1A stageA2A stageA3A stageA4A
�ƩŠĥ�ơð³ģåŵ†�
�ƩŠ5³Ŕ�ŏʬʃʋʫʂ;ȟʭ‡�
surv
ival
rate
[%]�
øă� Ćă�
stage 1: �Ʃ~4ɝřɥɮŠ stage 2: �Ʃ~ɱǛɇŠ stage 3: ʥʩʒƃǤŵɍɓŠ stage 4: �ƬkɢǤŵɍɓŠ�
stage 1 (øăŠ)ɜɬɠʮ100%Ǫɇ�ĮZƥ�
0A
10,000A
20,000A
30,000A
40,000A
50,000A
'90A '91A '92A '93A '94A '95A '96A '97A '98A '99A '00A '01A '02A '03A '04A '05A '06A '07A '08A '09A
fatali&
es)of)colorectal)can
cer)
year)
Xh��Y4GAf�¢��s�v��
h"p://www.mhlw.go.jp/toukei/saikin/hw/jinkou/geppo/nengai11/kekka03.html#k3_2�
p8ȺÌÉôŔŊɞ�ɜǼ�?ĥ�ŏʬ�X10��ʭɞ³ģåŵ�
�Ʃ4ǃȆęĒ�
• /żɝCCDɱáɖʃɿʫʗɱ�ǼɄɬâ1ɍʮƩ4ɱęĒ
• �Ʃ4ǃȆęĒʃʋʊʗ
&ŏ100&��ɝà�ɍɓ Ʃ~ƽȘɞĜDZɄɬķǶ¸ɱåļ�
I think this is a cancer…�
����
4ǃȆęĒ�
ƨŞɞ�sŰǑʮë¾�
ĮşõȃɞPǗ�ë¾ɌɯɓƨŞɄɬ ǂĞɪķǶ¸ɱå��
ǯ°ɞ4ǃȆ� à�4ǃȆʬ70&�100&ʭ�
pit-pattern>´�• Q����²�¯G¬��¶pit·�`}�� >´
– �P¬�t�y"J5�¸pit`}"^¦ – z«]��°8d�µ ��[��S¥{��
����
�ǼħɞďƱɟüȊɱǀɍʮËơɢɞǚƳɅ�Ɇɀ�
mŠ� smŠ� dzƺŠ�
smŠʴdzƺŠ�
ƪƨʴsmŠA
(�ɝƪƨÉ)�
Ĥ°� Ȯ� Ě5¼ɞĤ°ƪƂpit�
úƲŌpit�
Ȯvɫɭɩ�ɌɀƂŌʴĚ5¼pit�
Ȯvɫɭɩ�ɆɀƂŌʴĚ5¼pit�
ŀƊĝʮĠĎĝʮƧlǤŌpitA
ȯ�
Ȱ�
Ȱ�
ȱ�
Ȳ�
Ȳ�
S
L
I
N
ȯʮȰAʮȰAʮȱvpitɞȉXǼA�ɆɌɪǽ=ɞ�ñɜpitA
pitɅĻ�ʴĶ�AņĜDZ×ǁɞ:őɱ�ɁŌÑA
SA LA
pitMpa"ern;ȟȣ[S.TanakaAetAal.,A‘06]�
NBIkQi�>´�(NBI: Narrow-band Imaging)�
• pit�`}¸E���w¨�� >´ – ~WM�:"_� �����³M"^¦�� – z«]��°©d�S¥{��
TypeAA�
TypeAB�
TypeAC�
1
2
3
Â�ƹƂɟ�Zǃ�
ƪƂɞeoɱWɭoɧƎɀÂ�ƹƂɱǑɨʮAȊäţɝùŪəñɜpitĝĜDZɅDž�Ɍɯɮ�
�ñɜƔŤĞĝɱĜÕʮAȊäţɝ�ñɜpitĝĜDZɅDž�ZƥʯAƹƂɞ�Ɍ/;ɅĨǦţu�ʯ�
�ñɜƔŤĞĝɱĜÕʮAȊäţɝ�ñɞ»ɀpitĝĜDZɅDž�ZƥʯAƹƂɞ�Ɍ/;Ʌ�u�ʯA
�ñpitĝĜDZɩ�ùŪəʮDž��ƥʯA�ñƹƂɞ�Ɍ/;ɟ�u�ə�ñʯAņƹƂțx(AVA)ɞ:őʯAóňKɍɓÂ�ƹƂɅïsɎɮʯA
NBIà�×ǁ;ȟȣ[H.Kanao et al., ‘09]�
smŠʴdzƺŠ�
ƪƨʴdzƺŠ�
Ĥ°�
ŮŹŤţ�
ǍóìéɱŤţɚɍɓ �ƩƨŞNBIƮIǑǖʁʃʋʞɞȉš�
Our Project’s Goal�
This Presentation’s Objective�
�ƩƨŞNBIŗ+;ȟəȡɀǑǖƇ¸ɱÁɮ�
Local binary patterns [Gross ‘08] : 90[%] Vascularization features [Thomas ‘09]: 89.2[%]�
<NBIŗ+;ȟɞȋDzŮŹ>�
texture analysis approach�
YoshitoATakemura,AShigetoAYoshida,AShinjiATanaka,AKeiichiAOnji,AShiroAOka,AToruATamaki,AKazufumiAKaneda,AMasaharuAYoshihara,AKazuakiAChayama:A"QuanPtaPveAanalysisAandAdevelopmentAofAaAcomputerMaidedAsystemAforAidenPficaPonAofAregularApitApa"ernsAofAcolorectalAlesions,"AGastrointesPnalAEndoscopy,AVol.A72,ANo.A5,App.A1047M1051A(2010A11).A
Outline of Our Approach�
Type A� Type B� Type C3�
12, 55, 63, … 87, 49, 21, … ʱ�
32, 20, 73, … 67, 6, 0, … ʱ�
79, 5, 40, … 11, 36, 87, … ʱ�
27, 64, 25, …, 87 93, 41, 75, …, 8 …
�
12, 55, 63, … 87, 49, 21, … ʱ�
32, 20, 73, … 67, 6, 0, … ʱ�
79, 5, 40, … 11, 36, 87, … ʱ�
65, 33, 19, …, 101 52, 51, 32, …, 89 …
�
12, 55, 63, … 87, 49, 21, … ʱ�
32, 20, 73, … 67, 6, 0, … ʱ�
79, 5, 40, … 11, 36, 87, … ʱ�
66, 95, 47, …, 85 11, 82, 3, …, 124 …
� ȴ�
ȵ�
ȶ�
ȷ ȸ
ȴȣȣȵȣȣȶȣȣȷȣȣȸ�
Type A�
ȴȣȣȵȣȣȶȣȣȷȣȣȸ�
Type B�
ȴȣȣȵȣȣȶȣȣȷȣȣȸ�
Type C3�
84, 99, 40, …, 121 5, 26, 91, …, 150 …
�
ȴȣȣȵȣȣȶȣȣȷȣȣȸ�
Clustering�
Vector quantization�Vector quantization�
Feature space�
Classifier�
Histogram�
Test image�
Learning�
Classification result�
Description of Local features�
�×ŋÃȁȣ+ Bag-of-features�
�×ŋÃȁ: gridSIFT�• Scale Invariant Feature Transform (SIFT) [Lowe, ‘99]
– �×țxɝɃɈɮǨ¸JǽÍzɱ128ģ-ʙɼʍʦəƽő – DoGɝɫɮŋÃŅę:ɝɫɮǑǖŏɟ90[%]C¿
ɽʧʫʃɾʫʦ�ç� ŋÃŅę:ʬDoGʭ� ŋÃȁNJǬ�
• grid samplingɝɫɮSIFTŋÃȁNJǬȣ(gridSIFT) – ǑǖɝɃɀɘʮ�ɝʀʩʗʥʩɽɎɮɚÉƥɅc� – ĕ�ŌɝʀʩʗʥʩɽɍʮSIFTŋÃȁɱNJǬ
grid sampling�
grid space�
scale size�
ǖ?k: Support Vector Machine (SVM)�
• ɹʫʐʦʇɴʗ
– Radial basis function (RBF)
– linear
– χ2
• �ɼʤʃ;ȟȣ: One-Versus-One
vuvu ⋅"=),(lineark)exp(),( 2vuvu −⋅−= γRBFk
( )!!"
#$$%
&
+
−⋅−=
vuvuvu2
2 2exp),(
γχk
ʜʫʂʩā�KɞyĿə ʒʤʟʫʇɱ�ƞɎɮ2ɼʤʃǖ?Øİ�
2
1max ww
€
subject to yiw⋅ φ(xi ) ≥121 w
21 w
ŕŗ+�
• ʤɴʋɳʩɽʪë¾dž¸ʪ&ŏɟaŶ¸ • 4ǃȆɞ-ŗ+Ʉɬ3vţɜTypeɱʍʥʝʩɽ • ŗ+ʀɴʄ: 100×300�900×800[pix.] • �ȈM2b��ɝɫɕɘʤʙʦ�Ɉ
Type A:
Type B:
Type C3:
< ŕŗ+ȣ>�
ǎ"�Ƞ�
• Bag-of-features’s Approach
– �×ŋÃȁȺȺ: gridSIFT
– ɼʤʃʇʥʩɽ: ȑ¢ţk-means
– ǖ?kȺȺȺȺ: SVM
• Dataset
– 908 NBI images (Type A: 359, Type B: 462, Type C3: 87)
• ǎ": 10-fold Cross Validation – Ǐƺħɝʤʩʈʞɝ8ČȎ�ɍʮ900Čɱ ŕ
– # of visual-words: 3×22, 3×23, …, 3×213
ǎ"�ȠɝɫɮŋÃȁʪǖ?kNjLj�• SVM: ɹʫʐʦ
• gridSIFT: ɿʩʍʤʃʍȌ'
• gridSIFT: ʀʩʗʥʩɽȊȒ
• grisSIFT: ʃɾʫʦʀɴʄ
60AA
65AA
70AA
75AA
80AA
85AA
90AA
95AA
100AA
10A 100A 1000A 10000A 100000A
Correct)R
ate)[%
]�
#)of)visual7words)[7]�
gridAspace=5[pix.]A
gridAspace=10[pix.]A
gridAspace=15[pix.]A
60AA
65AA
70AA
75AA
80AA
85AA
90AA
95AA
100AA
10A 100A 1000A 10000A 100000A
Correct)R
ate)[%
]�
#)of)visual7words)[7]�
ContrastThreshold=0.005AContrastThreshold=0A
60AA
65AA
70AA
75AA
80AA
85AA
90AA
95AA
100AA
10A 100A 1000A 10000A 100000A
Correct)R
ate)[%
]�
#)of)visual7words)[7]�
RBFMkernelA
linearMkernelA
χ2MkernelA
60AA
65AA
70AA
75AA
80AA
85AA
90AA
95AA
100AA
10A 100A 1000A 10000A 100000A
Correct)R
ate)[%
]�
#)of)visual7words)[7]�
scale=3A
scale=5A
scale=7A
scale=9A
scale=12A
scale=5,7A
scale=5,9A
scale=5,12A
scale=7,9A
scale=7,12A
scale=9,12A
scale=5,7,12A
scale=5,7,9A
scale=5,9,12A
scale=7,9,12A
$ ǑǖÉƥȣ: χ2 > linear > RBF Ⱥȣȭ ɹʫʐʦɞǷɀɝɫɮǑǖŏɞ«ɟ1[%]Ŷ¸�
$ ǑǖÉƥ: Ȍ'ɜɍȣ> Ȍ'0.005 Ⱥȣ…'ɞ�ɌɜŋÃȁəȿɕɘɩǖ?ƥEɱáɖ
$ ǑǖÉƥ: 5[pix.] > 10[pix.] > 15[pix.]
$ ǑǖÉƥ: (5, 7) [pix.] > other combination
ǎ"�Ƞ�
• Bag-of-features’s Approach
– �×ŋÃȁȺȺ: gridSIFT (threshold: 0, grid space: 5[pix.], scale size: 5, 7[pix.])
– ɼʤʃʇʥʩɽ: ȑ¢ţk-means
– ǖ?kȺȺȺȺ: SVM (linear kernel, -3 ȳȣlog2C ȳȣ19)
• Dataset
– 908 NBI images (Type A: 359, Type B: 462, Type C3: 87)
• ǎ": 10-fold Cross Validation – Ǐƺħɝʤʩʈʞɝ8ČȎ�ɍʮ900Čɱ ŕ
– # of visual-words: 3×22, 3×23, …, 3×213
Results <10-fold Cross Validation>�
60AA
65AA
70AA
75AA
80AA
85AA
90AA
95AA
100AA
10A 100A 1000A 10000A 100000A
Correct)R
ate)[%
]�
#)of)visual7words)[7]�
Correct)Rate)96.00%�
0AA
10AA
20AA
30AA
40AA
50AA
60AA
70AA
80AA
90AA
100AA
10A 100A 1000A 10000A 100000A
Recall)Ra
te)[%
]�
#)of)visual7words)[7]�
Recall)Rate�
TypeAAA
TypeABA
TypeAC3A
0AA
10AA
20AA
30AA
40AA
50AA
60AA
70AA
80AA
90AA
100AA
10A 100A 1000A 10000A 100000A
Precision)Ra
te)[%
]�#)of)visual7words)[7]�
Precision)Rate�
TypeAAA
TypeABA
TypeAC3A
ôǂʌʫʇɝ�Ɏɮǎ"�Ƞ�
• Bag-of-features’s Approach
– �×ŋÃȁȺȺ: gridSIFT (threshold: 0, grid space: 5[pix.], scale size: 5, 7[pix.])
– ɼʤʃʇʥʩɽ: ȑ¢ţk-means
– ǖ?kȺȺȺȺ: SVM (linear kernel)
• Dataset
– 1412 NBI images: 908 training images
504 test images
• ǎ": Holdout Testing – �ƞŗ+ʪʋʃʍŗ+ɱq�ɍɘǎ"
– # of visual-words: 3×22, 3×23, …, 3×213
(Type A: 359, Type B: 462, Type C3: 87)�
(Type A: 156, Type B: 294, Type C3: 54)�
Results <Holdout Testing>�
60AA
65AA
70AA
75AA
80AA
85AA
90AA
95AA
100AA
10A 100A 1000A 10000A 100000A
Correct)R
ate)[%
]�
#)of)visual7words)[7]�
Correct)Rate)
0AA
10AA
20AA
30AA
40AA
50AA
60AA
70AA
80AA
90AA
100AA
10A 100A 1000A 10000A 100000A
Recall)Ra
te)[%
]�
#)of)visual7words)[7]�
Recall)Rate�
TypeAAATypeABATypeAC3A
0AA
10AA
20AA
30AA
40AA
50AA
60AA
70AA
80AA
90AA
100AA
10A 100A 1000A 10000A 100000APrecision)Ra
te)[%
]�#)of)visual7words)[7]�
Precision)Rate�
TypeAAATypeABATypeAC3A
92.86%�
Conclusions�
• ŋÃȁʪǖ?kNjLjɝũŤɍɓǑǖÉƥɞǎ"
$ ŋÃȁgridSIFT: ɿʩʍʤʃʍȌ'ʮʀʩʗʥʩɽȊȒʮʃɾʫʦʀɴʄ
$ ǖ?kSVM: ɹʫʐʦʮʒʤʟʫʇãƍƄo
Future Works�
• Type C3ɝ�ɎɮǑǖÉƥc� $ ɼʤʃɲʩʑʤʩʃɝ�Ɏɮęlj
• Iŗ+əɞǑǖ9œ�
Ǒǖŏȣ96.00[%] (10-fold Cross Validation)�
�ƩȬȪȫ4ǃȆû+ɞʥɲʦʇɴʞǑǖʁʃʋʞ�
Real-Time Recognition System for NBI Video Endoscopy�
¨�ȣƜǾ,A`ʼnŖȣĸy,AŐĄȣÄ,ARaytchevABisser,AȂŖȣfò,A`ŖȣÕ�,AŽĈȣj�,AŖ�ȣ$Įȣ:AȽ�ƩNBI4ǃȆû+ɞʥɲʦʇɴʞǑǖʁʃʋʞɞȉšȾ,Až17lŗ+ʅʩʁʩɽʁʩʛʂɵʞȣSSII2011,App.IS1M09M1MIS1M09M7,AʒʁʕɳɿğĴ,AŲ�¨ȣ(2011A06).A
9œǰ¸:14.7[fps]�
A�B�
C3�
�����
ŗ+WÁ�
• ŗ+ɞ��Ǽ;<ɭ:ɍ�
ǖ?�
Visual Word Histogram �Õ�
• ǖ?Ƒč(A or B or C3) • A, B, C3ɞŰŏ�
ƽű�
• ǖ?k: SVM�
ʥɲʦʇɴʞǑǖʁʃʋʞ�
22 6 … 91 87 …
ŋÃȁLjƁ�
• ɽʥʊʎʀʩʗʥʩɽ • SIFT
120[pix.]�
120[
pix.
]�
ǖ?Ƒč�
AɞŰŏ BɞŰŏ
C3ɞŰŏ
�ȠNj��
• �ƞDataset�
A B C3 Lj�359 461 87 907�
A B C3 Lj� 4 5 3 12�
Iŗʀɴʄȣȣȣȣȣȣȣȣȣȣȣȣȣȣȣȣȣ640*480 [pix.] Ǒǖ�Ǚțxȣȣȣȣȣȣȣȣȣȣȣȣȣ120*120 [pix.]
ŗ+Čð� Iŗćð�
• ʋʃʍDataset�
• SIFT Dense SIFT(VLFeat) — ɽʥʊʎȊȒ: 5[pix.] — ʃɾʫʦʀɴʄ: 5, 7[pix.]�
• ǖ?k SVM(LibSVM) — ɹʫʐʦʇɴʗ: Linear
• Visual wordðȣȣȣȣ768�
^Iŗ200ʕʧʫʞɏɖɞLj2400ʕʧʫʞɱ ŕ
�ȠNj��
Ȧȧȩ�time�
prob
abili
ty�
0�
1�
AɞŰŏ BɞŰŏ C3ɞŰŏ�
ŰŏɝɍɆɀ'ɱNj�
ɍɆɀ'ɝĽɓɜɈɯɠĘR��
MRF"��� Q�NBI<�®A�9�q�?-+.1%�
Temporal labeling NBI Videoendoscopy Using MRF�
ğŖ�t,A²¨Ɵ,AŐĄÄ,ABisserARaytchev,AȂŖfò,A`ŖÕ�,AŽĈj�,Aƶth¬,A�ĄœÊ,AŖ�$ĮʱȣȽ�ƩNBI4ǃȆIŗ+ɞ��KȾ,Až18lŗ+ʅʩʁʩɽʁʩʛʂɵʞȣSSII2012,App.IS1M09M1MIS1M09M5,AʒʁʕɳɿğĴ,AŲ�¨ȣ(2012A06).A
gȞŅȣ[ʥɲʦʇɴʞǑǖʁʃʋʞ]�
0
0.5
1
0 20 40 60 80 100 120 140 160 180 200
Pro
babi
lity�
フレーム番号
Type A
Type B
Type C3
• ʥɲʦʇɴʞǑǖʁʃʋʞɞ:EƑč�
• ^ʕʧʫʞɞǖ?Ɍɯɓʤʙʦ�Type A Type B Type C3�
��ɍɓǑǖƑčɱÁɮɊɚɅəɆɜɀ�
ŮŹŤţ�
SVMɞ:EɎɮ�¿ŰŏɱMRFɞʌʫʇȚɝŕɀɓ Ⱥüƈ=ʤʙʥʩɽØİɞæĖ�
This Presentation’s Objective�
% 4ǃȆęĒM®Ʌŝ�ǼɞŌÑɱÛèɎɮɊɚɱìé�
üȊţɝɫɭŁɬɄɝ ȺȺȺȺȺǑǖƑčɅ�KɎɮɫɁɝƽűɱɎɮ�
Goal
MRFʠʌʦ�
f x y( )∝ exp A xi, yi( )i∑#
$%
&
'(⋅exp I xi, x j( )
j∈Ni
∑#
$%%
&
'((
ʌʫʇȚ� ²ŁKȚ�
x: ^ʕʧʫʞɝɃɈɮå�ɎɣɆʤʙʦƈ= y: SVMɫɭå�Ɍɯɮŗ+ɞŋÃȁƈ=�
x1 x50…………� x100 x150 x200…………� …………� …………�B� B� B�C3� C3�
i0� 50� 100� 200�150�
……� ……� ……� ……�
y1 y50 y100 y150 y200
MRFʠʌʦ�
f x y( )∝ exp A xi, yi( )i∑#
$%
&
'(⋅exp I xi, x j( )
j∈Ni
∑#
$%%
&
'((
ʌʫʇȚ�
0
0.5
1
0 20 40 60 80 100 120 140 160 180 200
Pro
babi
lity�
フレーム番号
Type A
Type B
Type C3
P(x50=A|y50) = 0.004 P(x50=B|y50) = 0.99 P(x50=C3|y50) = 0.006�
exp A xi, yi( )( ) = P xi yi( )
A xi, yi( ) = logP xi yi( )
�ƞĹɦɞSVMɱŕɀɮ�
MRFʠʌʦ�
f x y( )∝ exp A xi, yi( )i∑#
$%
&
'(⋅exp I xi, x j( )
j∈Ni
∑#
$%%
&
'((
²ŁKȚ�
Ǫ)ɞʤʙʦÍzɱŕɀɮ�
• ²ſɜǿɦ�Ɉ ー ðʕʧʫʞɞɦƽűɌɯɓTypeɅĶɂɮZƥÉɅȿɮ
ー Type C3ɚǖ?ɌɯɓǼ;ɱǁƴɚɎQȏÉȿɭ
yiyi−1 yi+1y1
xixi−1 xi+1x1
yn
xn
yiyi−1 yi+1y1
xixi−1 xi+1x1
yn
xn
• C3ɱĦɎɫɁɜǿɦ�Ɉ ー Type C3ɞąę:ɱÜA�
MRFʠʌʦ�
f x y( )∝ exp A xi, yi( )i∑#
$%
&
'(⋅exp I xi, x j( )
j∈Ni
∑#
$%%
&
'((
²ŁKȚ�
B� C3� B�? B�
i
Label�
Time�i−1i− 2 i+1 i+ 2
A�B�
C3�
xixi−1
ǹŵŰŏ�
C3�
²ſɜǿɦ�Ɉ
B� B� B�B B�Label�
MRFʠʌʦ�
Ǫ)ɞʤʙʦÍzɱŕɀɮ�
• ²ſɜǿɦ�Ɉ ー ðʕʧʫʞɞɦƽűɌɯɓTypeɅĶɂɮZƥÉɅȿɮ
ー Type C3ɚǖ?ɌɯɓǼ;ɱǁƴɚɎQȏÉȿɭ
• C3ɱĦɎɫɁɜǿɦ�Ɉ ー Type C3ɞąę:ɱÜA�
exp I xh, xi, x j( )h, j∈Ni
∑#
$%%
&
'((
²ŁKȚ�
f x y( )∝ exp A xi, yi( )i∑#
$%
&
'( exp I xi, x j( )
j∈Ni
∑#
$%%
&
'((
yiyi−1 yi+1y1
xixi−1 xi+1x1
yn
xn
yiyi−1 yi+1y1
xixi−1 xi+1x1
yn
xn
MRFʠʌʦ�
f x y( )∝ exp A xi, yi( )i∑#
$%
&
'(⋅exp I xh, xi, x j( )
h, j∈Ni
∑#
$%%
&
'((
²ŁKȚ�
A�B�
C3�
C3ɱĦɎɫɁɜǿɦ�Ɉ
B� B�? B�
i
Label�
Time�i−1i− 2 i+1 i+ 2
C3�
ǹŵŰŏ�
xixi−1
C3�
xi+1A�
B� C3� B�C3 B�Label�
MRFʠʌʦ�
f x y( )∝ exp A xi, yi( )i∑#
$%
&
'(⋅exp I xi, x j( )
j∈Ni
∑#
$%%
&
'((
• ²ſɜǿɦ�Ɉ ー ðʕʧʫʞɞɦƽűɌɯɓTypeɅĶɂɮZƥÉɅȿɮ
ー Type C3ɚǖ?ɌɯɓǼ;ɱǁƴɚɎQȏÉȿɭ
yiyi−1 yi+1y1
xixi−1 xi+1x1
yn
xn
yiyi−1 yi+1y1
xixi−1 xi+1x1
yn
xn
• C3ɱĦɎɫɁɜǿɦ�Ɉ ー Type C3ɞąę:ɱÜA�
ā��¿Űŏ(MAP)å�ɱǸŕɍʮxɱĬɨɮ�% �xţāǸLJɱÁɮɊɚəʤʙʥʩɽɱ�ő�
IţLjŗİȣ(DP)� ɻʖʃʀʩʗʥʩɽ�
�Ƞȣ[ʌʫʇʅʊʍ]�
�ƞ�
• ŗ+Čðɟ907Č ȺȺ(Type A: 359, Type B: 462, Type C3: 87)
• ʤɴʋɳʩɽʪë¾dž¸ʪ&ŏɟaŶ¸ • 4ǃȆɞ-ŗ+Ʉɬ3vţɜTypeɱʍʥʝʩɽ • �ȈM2b��ɝɫɕɘʤʙʦ�Ɉ
ʋʃʍIŗ+�
• ǑǖƑčɅ���ɜɩɞɱǺß • ^IŗəƨŞɅà�ƽűɌɯɮǼ;ȺȺȺȺȺ
200ʕʧʫʞɱÞɆ:ɍɓɩɞɱ ŕ • Iŗćðɟ4ć
ȺȺ (Type A: 2ćʮType B: 2ć)�
Type B (original)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (DP_0.8)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (DP_0.9)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (DP_0.99)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (DP_0.999)
frame number0 20 40 60 80 100 120 140 160 180 200
������������������
����������������
���������������
���������������
���������������
Type B (original)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (DP_0.8)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (DP_0.9)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (DP_0.99)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (DP_0.999)
frame number0 20 40 60 80 100 120 140 160 180 200
������������������
����������������
���������������
���������������
���������������
Type B (original)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (Gibbs_p4=0.6)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (Gibbs_p4=0.7)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (Gibbs_p4=0.8)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (Gibbs_p4=0.9)
frame number0 20 40 60 80 100 120 140 160 180 200
��������������������
������������������ ����
������������������ ����
������������������ �����
������������������ �����
0
0.5
1
BAC
20 40 60 80 100 120 140 160 180 200
Type BɞIŗɝ�ɎɮǸŕƑč�
0
0.5
1
ABC
20 40 60 80 100 120 140 160 180 200
Type A_1 (original)
frame number0 20 40 60 80 100 120 140 160 180 200
Type A_1 (DP_0.99)
frame number0 20 40 60 80 100 120 140 160 180 200
Type A_1 (Gibbs_p4=0.9)
frame number0 20 40 60 80 100 120 140 160 180 200
��������������������
�����������������
���������� �����������
Type AɞIŗɝ�ɎɮǸŕƑč�
Type A� Type B� Type C3�
ǸŕƑč�
ǸŕƑč�
Type B (original)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (DP_0.8)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (DP_0.9)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (DP_0.99)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (DP_0.999)
frame number0 20 40 60 80 100 120 140 160 180 200
������������������
����������������
���������������
���������������
���������������
Type B (original)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (DP_0.8)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (DP_0.9)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (DP_0.99)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (DP_0.999)
frame number0 20 40 60 80 100 120 140 160 180 200
������������������
����������������
���������������
���������������
���������������
Type B (original)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (Gibbs_p4=0.6)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (Gibbs_p4=0.7)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (Gibbs_p4=0.8)
frame number0 20 40 60 80 100 120 140 160 180 200
Type B (Gibbs_p4=0.9)
frame number0 20 40 60 80 100 120 140 160 180 200
��������������������
������������������ ����
������������������ ����
������������������ �����
������������������ �����
0
0.5
1
BAC
20 40 60 80 100 120 140 160 180 200
Type BɞIŗɝ�ɎɮǸŕƑč�
0
0.5
1
ABC
20 40 60 80 100 120 140 160 180 200
Type A_1 (original)
frame number0 20 40 60 80 100 120 140 160 180 200
Type A_1 (DP_0.99)
frame number0 20 40 60 80 100 120 140 160 180 200
Type A_1 (Gibbs_p4=0.9)
frame number0 20 40 60 80 100 120 140 160 180 200
��������������������
�����������������
���������� �����������
Type AɞIŗɝ�ɎɮǸŕƑč�
Type A� Type B� Type C3�
MAPå�ɱǸŕʮʤʙʦɱƽűɍɓɩɞ�
ʤʙʥʩɽƑčȣ(C3ɱĦɎɫɁɜǿɦ�Ɉ)�
ʤʙʥʩɽƑčȣ(²ſɜǿɦ�Ɉ)�
C3ɅĶɂɘɍɥɕɘɀɮ��Ʌ�C3ɔɕɓ{_ʮƽűəɆɘɀɮ�
Iŗ+��Ǽ;ɱ<ɭ:ɍʮǑǖɱƺɁ�
ǖ?Ƒč AɞŰŏ BɞŰŏ C3ɞŰŏ�
Type A Type B Type C3�
MRFɞǸŕƑčɱŬ¼ɞƱəƽű�
0A
0.5A
1A
0A 50A 100A 150A 200A
Prob
ability�
TypeAAA
TypeABA
TypeAC3A
Type A� Type B�
Type A� Type B�
Type A Type B Type C3�
-+/���³MeO"��� Self-training
~Q�<�®NBI�9¤o��c�~�
Self-training with unlabeled regions and its application to recognition of colorectal NBI
endoscopic images�
ŽŖ�ů[,AŐĄÄ,Aʤɴʉɶʕȣʓʆʦ,AȂŖfò,AēŖ�i�,A`ŖÕ�,AŽĈj�,Aƶth¬,A�ĄœÊ,AŖ�$ĮʱȣȽʤʙʦɞɜɀțxÍzɱŕɀɓSelfMtrainingAɚȣ�Ʃ4ǃȆNBIAŗ+ǍóɢɞÈŕȾ,Aȗ�Ízǯ$��Ùƻzd,AʒʇʫʩǑǖʪʟʌɳɲœLJŮŹ�ȣPRMU2012M11,AVol.112,ANo.37,App.57M62,AbY¡©ě��,AÐūȺ(2012A05).A
MOTIVATION�
• ŗ+ǑǖɞƇ¸c� ! �ǂĞɜ�ƞʌʫʇʅʊʍ
�ŷ�ĝɜąūɞNBIŗ+ɱǑǖɎɮɓɨɞ �ǂĞɜʌʫʇʙʫʃɞĜƅ�
! NBIŗ+ʌʫʇʅʊʍɞ�Ǟ
! �ȁɞŗ+ɝʤʙʦ�Ɉ × ɿʃʍ × ØȊ × �Ȉūǖ
A�B�C3�
ABSTRACT�
• Self-training & ʤʙʦɞɜɀțxɝ�ɍɘʤʙʦ�Ɉ & Ǒǖŏɱ�ɉɮ�ƞʌʫʇʅʊʍɱĜƅ�
ÀĉØİȣ[Yoshimuta et al., ‘10]� æĖØİ�
Key Idea : ʤʙʦɞɜɀțxÍzɱŕɀɮ
Self-training�
• ʤʙʦɜɍʌʫʇɱŕɀɘÉƥɱc� • �ƞʌʫʇɱƮIŔÕ
6�ƞ�
>ă�ƞ�
Ǒǖ�
ʤʙʦ�Ɇʌʫʇ�
ʤʙʦɜɍʌʫʇ�
$ȝÉęĒ�Accept�
Reject�
POINT 1. ʤʙʦɞɂõ
2. ǭFɎɮʀʩʗʦɞǺɡõ
labeled samples�
• �ȈMɝɫɕɘʍʥʝʩɽʪʤʙʥʩɽ • ŗ+ʀɴʄʱ100×300ʴ900×800 [pix.] • ŗ+Čðɞ4nj
Type)A� Type)B� Type)C3� Total�
359� 462� 87� 908�
A�B� C3�
Unlabeled samples �
• ^-ŗ+Ʉɬ10Čɏɖ<ɭ:ɍ • ŗ+ʀɴʄʱ30×30ʴ250×250 [pix.] • <ɭ:ɍõ
– -ŗ+Ʉɬʤʩʈʞɝ<ɭ:ɍ – ʤʙʦ�ɆțxɞeǩɄɬ<ɭ:ɍ
• ŗ+Čðɞ4nj
*ʤʙʦ�Ɇʀʩʗʦɞ4�ɖɔɈ-ŗ+Ʌ�sɍɜɀɓɨȻʤʙʦɜɍʀʩʗʦɅ10Č�ɜɀ
Type)A� Type)B� Type)C3� Total�
3590� 4610� 870� 9070�
<ɭ:ɍ!�
Evaluation�
# 10 hold out testingɝɫɕɘ10lɞ²u'ɱĬɨɮ # tę�ɝɫɭĂÏ«ɱĬɨɮ
ĤƀŏʬǑǖŏʭ�
C3ɞ6őŏ�
Estimated Category�
Type A� Type B� Type C3�True
Category�Type A� (1)� (2)� (3)�Type B� (4)� (5)� (6)�
Type C3� (7)� (8)� (9)�
)9()8()7()6()5()4()3()2()1()9()5()1(
++++++++
++=
)9()8()7()9(++
=
result�
# ÀĉØİʬʤʙʦ�Ɇʀʩʗʦɞɦ ŕʭɚɞǑǖÉƥ
ɞĨǦ & Ǒǖŏ & C36őŏɞĨǦ�
# ʤʙʦɜɍʀʩʗʦɝɫɮǑǖŏɞĨǦ�& ʤʩʈʞɝ<ɭ:ɍɓʤʙʦɜɍʌʫʇɱŕɀɓƑč & ʤʙʦ�Ɇțxɞeǩɱ<ɭ:ɍɓʌʫʇɱŕɀɓƑč
Result�
0.9A
0.91A
0.92A
0.93A
0.94A
0.95A
0.96A
ÀĉØİ� AlgorithmA1A AlgorithmA2A AlgorithmA3A
Recogni&on
)Rate)�
ʤʙʦ�Ɇțxeǩɱ<ɭ:ɍɓʀʩʗʦɱ ŕ�
Ȥȣp=0.013314�
ÅƬƈ�
• ÅƬCTŗ+�
ŨŒƈ�
• Ɩ4Ȕ�• Ţ4Ȕ�
�Ʃƈ�
• �ƩNBI4ǃȆ�
h"p://sozai.rash.jp/medical/p/000154.htmlAh"p://sozai.rash.jp/medical/p/000152.htmlAh"p://medical.toykikaku.com/ǡ�ɞɍɇɦ/Ʃ/�
ZUQRVR¬��� C��9=��F��� �nT��¸��¸<���
�rb ¶ZUQR·�
h"p://kids.wanpug.com/top_person.html�