Warm Up: How does Pharmaceutical Chemist determined on the label, how often and when to take a pill?
Pharmaceutical Pill Recognition Using Computer Vision...
Transcript of Pharmaceutical Pill Recognition Using Computer Vision...
SchoolofInformaticsandComputing,IndianaUniversityBloomington
CharleneTayandMridul Birla
PharmaceuticalPillRecognitionUsingComputerVisionTechniques
Introduction
• Unidentifiedandmisidentifiedprescriptionpillspresentchallengesforpatientsandprofessionals. Takingsuchpillscanresultinadversedrugreactionsthataffecthealthorcouldevencausedeath.Bycomingupwithwaystoeasilyidentifyandverifyprescriptionpills,errorscanbegreatlyreduced.
• Ourgoalistoproduceaframeworkorsetofmethodsthatwilltakeaconsumer-qualityimageofapill(takenfrommobilephones),andhelptoidentifyitbyreturningthemostlikelymatchesfromourdatabasesetofpillimages.
• Eachimageisconvertedtogreyscaleandbackgroundissubtractedtogetanideaofoverallshape.
• Twelveshapefeaturesarealsoextractedforeachimage,including:the7Huinvariantmoments,circularitydegree,rectangledegree,sphericitydegree,concavitydegreeandflatdegree.
• Wecategorizedandlabeledeachpillinourtrainingdatasetbyshapeandtrainedaneuralnetworkontheseimagesandfeatures
Dataset
Overview ofProcessChallenges andFuture Work
References andAcknowledgements
Twoexamplesofconsumer-qualitypillimages Thecorrespondingreferenceimagematchesforthetwopills shownontheleft
TheNationalLibraryofMedicinehasmadepublicasetofpillimagesthatinclude:
• 2000JPEGhigh-resolutionreference images(oneforthefrontandoneforthebackofeachof1000pills)fromtheComputationalPhotographyforPillIdentificationProject.
• 5000JPEGconsumerqualityimagesofthesame1000pills,takenwithdigitalcamerasinvaryinglightconditions ShapeDetermination
Whatifyoucould takeaphotoofanunknown pilland almostimmediatelyfindoutwhatitis?
Marker/ImprintExtraction ShapeClassification
Marker/Imprint Extraction ResultsofShapeDetermination
OriginalImage Imageafterapplyingerosiontechnique Outputaftersubtractingerosionoutputimagefromoriginalimage
• Themostdistinguishingfactorforpillidentificationisthemarkingsorimprintsonthetabletsorcapsules.Toextractthesefeatures,weusedafewmorphologicalimageprocessingoperations toenhancetheimprints.
• Followingthat,weusedopen-sourcedObjectCharacterRecognitionsoftwaretotrytoextractthenumbersandwordsimprinted.
• Scale-InvariantFeatureTransform(SIFT)descriptorsofeachpillwerealsoextractedtotrytocapturearepresentationoftheimage.
“ 3228 25mg “
Lu, W. (2012). Method for Image Shape Recognition wi th Neura l Network. In Adv ances in Computer Scienc e and In formation Engineering (pp. 547-551). Springer Berl in Heide lberg.Cunha, A., Adão, T., & Triguei ros , P. (2014). HelpmePil ls: a mobi le pi ll recognition too l for elderly pers ons. Proc ediaTechnology, 16 , 1523-1532.Us hiz ima, D., Carnei ro, A., Souz a, M., & Medei ros , F. (2015). Investigating Pil l Recogni tion Methods for a New National L ibrary of Medic ine Image Datas et. In Adv ances in Visual Computing (pp. 410-419). Springer In ternational Publ ish ing.
58%
75%70.66%
55%
25%
10%15%
10%
30%
0%
10%
20%
30%
40%
50%
60%
70%
80%Accuracy of Neural Network
Capsule Circle Oblong/Oval/Football
Pentagon Fat Oval Triangle
Square Hexagon Special (all other shapes)
• Whengiventheconsumerqualityimages,theneuralnetworkgivesanoverallof50.7% accuracy.
• Limitedamountofdata- onlyafewimagesforeachspecificpill.Needtogathermoreimagesfromotherresources.
• Lackofdiversityinshape(mostlycircleoroblong)• Imprintschallengingandrequiresmoreresearch.• Colorhardtomatchduetodifferinglight
conditionsandcameralenses.• Createwrappertoclassifyandsuggestbest
matches.
50.68%
74% 75%81% 80%
55%
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
Neura l Network SVM KNN Dec is ion Tree Random Forest Adaboost
Precision of Different Classifiers