Review

Advan

MarziaAnglia Rusk

a r t i c

Article histoReceived 1Received inAccepted 2Available on

Keywords:AutomatedImage procMachine leaMobile-enaClinical dec

1. Introd

The gadvancem(POC) didisease dnonproferequiremto overcofacial expeters for fsimplicity

Tonguuate the cfor examditional hThe diagninspectioof the tonease, tradthe syndr

∗ CorrespRuskin Univ

E-mail a

https://doi.2213-4220/(http://crea

Integrative Medicine Research 8 (2019) 42–56

Contents lists available at ScienceDirect

Integrative Medicine Research

journa l homepage: www. imr- journa l .com

Article

ces in automated tongue diagnosis techniques

Hoque Tania ∗, Khin Lwin, Mohammed Alamgir Hossainin IT Research Institute, Anglia Ruskin University, Chelmsford, UK

l e i n f o

ry:December 2017revised form 18 February 2018March 2018line 8 March 2018

tongue diagnosis

a b s t r a c t

Tongue diagnosis can be an effective, noninvasive method to perform an auxiliary diagnosis any timeanywhere, which can support the global need in the primary healthcare system. This work reviews therecent advances in tongue diagnosis, which is a significant constituent of traditional oriental medicinaltechnology, and explores the literature to evaluate the works done on the various aspects of computer-ized tongue diagnosis, namely preprocessing, tongue detection, segmentation, feature extraction, tongueanalysis, especially in traditional Chinese medicine (TCM). In spite of huge volume of work done on auto-matic tongue diagnosis (ATD), there is a lack of adequate survey, especially to combine it with the current

essingrning

bled systemsision support systems

diagnosis trends. This paper studies the merits, capabilities, and associated research gaps in current workson ATD systems. After exploring the algorithms used in tongue diagnosis, the current trend and globalrequirements in health domain motivates us to propose a conceptual framework for the automatedtongue diagnostic system on mobile enabled platform. This framework will be able to connect tonguediagnosis with the future point-of-care health system.

© 2018 Korea Institute of Oriental Medicine. Publishing services by Elsevier B.V. This is an openNC-N

yelloromas

rders. Elimd ase fortern

estysis,diagonguicalvaila, Ch

titionon

adap

access article under the CC BY-

uction

lobal demand for primary healthcare support and theent of technology enable the platforms for point-of-care

agnostics.1 Despite the recent advances in automatediagnosis tools, the requirement of blood serum via

ssionals, accuracy, reliability, detection time, andent of second confirmatory test are the challenges yetme.2 Thus, temperature, tongue diagnosis, retinopathy,ressions, skin color, and surface could be few vital param-uture smartphone-based clinical expert systems to attain, immediacy, noninvasiveness, and automatic analysis.e diagnosis is an effective noninvasive technique to eval-ondition of a patient’s internal organ in oriental medicine,ple, traditional Chinese medicine (TCM), Japanese tra-erbal medicine, traditional Korean medicine (TKM).3–5

osis process relies on expert’s opinion based on visualn comprising color, substance, coating, form, and motion

andsyndsuchdisoTCMbasescopWeslightanalease

Tmedis, ahandpracsizedthe

gue.5,6 Instead of tongue’s abnormal appearance and dis-itional tongue diagnosis is more inclined to recognizeome.7–9 For an example, the tongue coating white-greasy

onding author at: Anglia Ruskin IT Research Institute (ARITI), Angliaersity, Bishop Hall Lane, Chelmsford CM1 1SQ, UK.ddress: marzia.hoque@pgr.anglia.ac.uk (M.H. Tania).

tal medicpublisheddetailed sand softwmobile ping technto be imthere is amachine

org/10.1016/j.imr.2018.03.001© 2018 Korea Institute of Oriental Medicine. Publishing services by Elsevier B.Vtivecommons.org/licenses/by-nc-nd/4.0/).

D license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

w-dense appearance signifies cold syndrome and hote, respectively, which are tied with health conditionsinfection, inflammation, stress, immune, or endocrine.10 These are two parallel, yet correlated syndromes ininating the dependency on subjective and experience-

sessment of tongue diagnosis may greatly increase thewider use of tongue diagnosis in the world including

medicine. Computerized tongue inspection implicatingimation, color correction, tongue segmentation, imagegeometry analysis, etc. could be an effective tool for dis-nosis aiming to address these concerns.e diagnosis is not a widespread perception in western

practice, which leads to one of the key challenges, thatbility of the authentic classified dataset. On the othereng et al11 questioned about the reliability and validity ofers of western medical system using TCM. They empha-

the importance of the degree of agreement and criticizedtation of ‘proportion of agreement’. Belonging to orien-ine, significant amount of work on tongue diagnosis is

in Chinese. To the authors’ best knowledge, the onlyurvey conducted up to 2012 was motivated by hardwareare of tongue diagnosis.5 There are recent advancement of

hone sensors, dynamics of its application, machine learn-12–18

iques, and expert systems, which has the potential

plemented for automatic tongue diagnosis (ATD). Thus,need to cover the insufficiency of focus on algorithms,

learning techniques, and clinical decision support system

. This is an open access article under the CC BY-NC-ND license

search

(CDSS) regate theresearchhuman intems areThis workthe mechdiagnosis

Thougits applictongue-onot focusin Englishmatic, anthe state-a directiodiagnosisprimarytems in gthem arefeature exquential wlater sectenabled pof the wobasis of ttual diagrthe conce

2. Tongu

Becauimage acqmonly knshaped. J

i) hardwule, co

ii) controsegme

To obtneeds toronmentacameras,ferent conliteratureresolutiodevice22,2

et al,29 Zline tongaround thminationThe workis explore

1 The tonimage analyCTIS. A CTISby a compucamera. Hoimage.

Diffe

pecianformres

, pricms.3

abilitrtionmetoweroved Liuor toiduarform

es. The annd tue anr of hm ws. Zhns oealthugh duatedors sentioa h

ed anhig

i34 cath toanalyg a G

is ansioulateexpeessinhypwhic

ddening

ases.and tt the(PBS

used onence

conhypSAM

M.H. Tania et al / Integrative Medicine Re

lated to tongue diagnosis. This work aims to investi-scope of combining tongue diagnosis with the currentpractice, making it work as an expert system withouttervention. The merits and capabilities of existing sys-

explored to identify the research gap to achieve this goal.will provide an insight into the achievements so far and

anism to achieve the ultimate goal of automated tongue.h few techniques used in case of tongue diagnosis findability in different fields such as speech mechanism,19

perated system for Tetraplegia patients, this paper doeson such cases. This work considers the work publishedonly. Manual, intended for digital diagnosis, semiauto-

d ATD are considered to be reviewed. This work exploresof-the-art works done on tongue diagnosis and presentsn toward a standard and complete work so that tonguecan be included as an auxiliary diagnosis to support the

healthcare system. In Section 2, tongue diagnosis sys-eneral, especially hardware components associated withoverviewed. In Section 3, tongue image segmentation andtraction techniques are critically reviewed. The inconse-orks done on specific disease detections are explored in

ion (Section 4) followed by limited work done on mobile-latform for tongue diagnosis (Section 5). The main focusrk is algorithms used in different steps of ATD. On the

he research gap in the existing works, an overall concep-am is presented in the last section (Section 6) to addressrns undertaking the trend of the demand.

e diagnosis systems

se of advancement in image sensing modules, variousuisition and analysis devices for tongue inspection com-

own as tongue diagnosis systems (TDSs) are surveyed andung et al5 reviewed both type of TDSs:

are consisting image sensing module, illumination mod-mpute, andl module and software for color correction, tongue imagentation, and tongue classification in literature up to 2012.

ain quantitative and objective diagnostic results, the TDSdeliver reproducible tongue images under varying envi-l conditions. In addition to commercially available digitalmany works used charge-coupled devices (CCD) with dif-figuration as the image sensing module.4,6,20–28 In recent

s, both Cibin et al24 and Zhang and Zhang23 used 8-bitn 3-chip CCD cameras. The use of tongue image capturing3 is similar to the use of conventional digital camera. Kim

hang et al,21 and Jiang et al30 used CCD cameras to out-ues, whereas Zhang et al22 placed D65 fluorescent tubese 8-bit resolution CCD camera to maintain uniform illu-

2.1.

Stic ifeatutionsystethedistotaryto pimprLi aning findivis pecessimaging atongfavosystebandnatio75 hthroevalauthconvusedcoatering

LlengandusinfilterdimemodTheprocof aal,35

a hidraindisetionaboutionwerebasepresthanroomused

. Different imaging schemes are described in Section 2.1.performed on color correction and light source estimationd in Section 2.2.

gue image capturing devices are defined as the computerized tonguesis system (CTIS). ISO/TC 249 specified the requirement and scope of thecan be any device or a system that acquires tongue images to be analyzedter, for example, digital camera, bayer CCD camera, even hyperspectralwever, a CITS by definition does not incorporate interpretation of tongue

based oncolor distcolor catand 88%Howeverexploitintongue im

In ord3D tongu3D tongupronunci

8 (2019) 42–56 43

rent computerized tongue image analysis systems1

lized tongue capturing devices1 (e.g., tongue diagnos-ation acquisition system) are often utilized to portray

such as tongue color, moisture, greasiness, indenta-king, fissure in tongue diagnostic information acquisition,8,10,31 The CCD camera is light-sensitive, but it hasy to produce quality images with less visual noise and. When it comes to mobile phone camera, complemen-

al-oxide-semiconductor (CMOS) is replaced by CCD dueconsumption and data-throughput speed. In order to

the performance and gather more information, Zhi et al,32

,33 Yamamoto et al,6 and Li34 utilized hyperspectral imag-ngue diagnosis. With an aim to attain the spectrum of anl pixel in the image of a scene, the hyperspectral imaging

ed to find objects, identify materials, and detect pro-ough RGB is the most influential color model for tongue

alysis, to facilitate better distinguish between tongue coat-ongue body as well as differentiating among closely aliked neighboring tissue colors, Zhi et al32 took the stance inyperspectral imaging. Unlike the traditional 2D imagingith visible light waves, it can collect data from multiplei et al32 performed hyperspectral comparison of combi-f tissue types on 300 chronic Cholecystitis patients andy subjects. Hyperspectral tongue images were analyzedifferent classifiers. The performance of this approach wasin contrast with the traditional RGB model as well. The

tated their suggested method to be more successful thannal RGB approach to a certain extent. Yamamoto et al6

yperspectral camera to quantify color spectrum of bothd uncoated tongue, lips, and periodical areas by consid-

hlight, shadow, and tongue coating.ptured hyperspectral tongue images at a series of wave-segment the tongue image, illustrated tongue colors

zed the tongue texture. A new algorithm was proposedabor filter for the tongue texture analysis. The Gaborlinear filter, similar to human visual system. The two-

nal Gabor filter is a Gaussian kernel function, which isd by a sinusoidal plane wave, useful for edge detection.rimental result in Li34 showed promising for the post-g of computerized tongue diagnosis. Another example

erspectral tongue imaging system is the work of Li eth presented a sublingual vein extraction algorithm usingMorkov model. The sublingual vein is responsible for

the tongue. These veins may be linked with number ofThus, the work involved outlining the spectral correla-he multiband variability to accumulate more informationtongue surface. Pixel-based sublingual vein segmenta-

VS) algorithm and spectral angle mapper (SAM) algorithmd and the performance of their algorithms was analyzed150 scenes of hyperspectral tongue images. Even in theof noise, the result was found to be more satisfactory

ventional algorithms. Li and Liu33 developed a pushb-erspectral tongue imager to capture tongue images. They

on 200 hyperspectral images to analyze tongue colorthe spectral response. The trajectory of tongue surface

ribution was tracked by the presented algorithm. For eachegory, the correctness rate of color recognition was 85%for tongue substance and tongue coating, respectively., none of these works exercised disease classificationg the extracted features obtained from hyperspectral

ages.er to attain three-dimensional geometry of a tongue, the

e diagnostic system could be an effective tool. Thoughe imaging is more popular for visual speech synthesis,36

ation recovery of impaired hearing individuals’, limited

44 searc

work37–3

the viewparticulasified inttriangularesents th

An altThe thickof the ton3D tongudescribedthe basisa frame toadjustedtongue, aated on aa finite-e

Manyble of comin Sectionsystem catems arethe final

2.2. Light

Lightifor tonguthe challsolutionshardwaresition devcamera min the colfection ininconsistdisadvan

Jang elight sourtemperatof an idehue to thance pro(K) on a sof 2000–yellow-wmore neuand abovand creatof inadeqrender cobiased imlonger a c

Resealamp,25 silluminateral, Köhlilluminatto generaability toever, theexpensivenvironmdifferenthalogen i

e thensistpersrtedthatificama

ly invutionimp

e. Beolorvenk26 dectioer w. MuparapendD spfyingtongout fl

er rek doncolorimaghardn thavoron

r, muable

camds. Hase in

age

he mentae sectionric feeque

Tong

hongtion ic tonour fsibleord

isitiosketely, n

ass4 disatabintra

M.H. Tania et al / Integrative Medicine Re

9 has been done in the field of tongue diagnosis. TCM holdsthat the shape of the tongue can change and it indicatesr pathologies.40 In TCM, the shapes of tongues are clas-o seven types: ellipse, square, rectangular, round, acuter, obtuse tri-angular, and hammer. The ellipse tongue rep-e normal tongue.ernative view of tongue diagnosis is thus 3D imaging.ness measurement, impulsively changing the curvaturegue surface angle could be better represented with the

e modeling scheme. This is supported by Liu et al,39 whoa framework to reconstruct bending human tongue on

of multiview geometry. They used two light sources andfix the face-location. With the help of four cameras, they

the exposure time and high speed frame rate. To locate thelaser device was utilized. The artificial features were cre-tongue surface through an adjustable laser beam. Finally,lement-based representation of the tongue was attained.of these devices, utilized by the existing systems, are capa-

pensating lighting environmental condition (discussed2.2), but none of these systems is standalone (i.e., the

nnot provide the diagnosis or syndrome itself). Most sys-portable but they lack a collective intelligence to providediagnosis.

source estimation and color correction

ng condition is considered to be an important prerequisitee diagnosis. Different approaches have been taken to solveenging problem of lighting condition estimation. Theseinclude controlled environment,27,29 color correction,41

based implementation, for example, tongue image acqui-ice20 and ColorChecker.42 An image taken with the sameay provide different tongue diagnosis due to the variationor properties at different lighting conditions. The imper-camera operation may cause some consequences.28 The

ence imaging with different devices may affect as an addedtage making it inconvenient to share or interchange.t al26 and Zhang et al20 used halogen tungsten lamp as thece to capture tongue images. Both of them used low colorures. The color temperature of a lamp is the temperatureal black-body radiator that radiates light of comparableat of the lamp. It is a way to describe the light appear-vided by a light source, measured in degrees of Kelvincale from 1000 to 10,000. Typically, the light appearance3000 K is referred to warm white, producing orange tohite light; 3100–4500 K is cool or bright white emittingtral white light with probable blue tint and 4600–6500 K

e tries to imitate the daylight, producing a blue-white lighting a crisp and invigorating ambience. However, the useuate color temperature in the mentioned work failed tolor in high fidelity. Because of work done with reddish-ages produced under halogen tungsten lamp, this is noonventional method.

rch on tongue diagnosis includes placement of fluorescenttandard light source installed in a dark chest,43 Köhlerion,44 built-in LED lighting for tongue diagnosis. In gen-er illumination is used in optical microscopy as a specimenion method for transmitted and reflected light. It is usefulte an incredibly even illumination of the sample. It has therepress illumination source in the resulting image. How-requirement of additional optical elements makes it more

makincois hyrepoors,classvisedpartexecto beniquthe c

Eworcorrearliwellto seindein 3nulliandwithbettworandtheboth

Oendeissuefiltearebackbrandise

3. Im

Tsegmimagdetemetsubs

3.1.

Zposimaticontinfea

Inacquwasnamwereet alrepeThe

e. Another passing fad is taking images in public officeent.26 Hu et al41 analyzed tongue features observed under

lighting conditions such as fluorescent, incandescent, andlluminant. They used Support Vector Machine (SVM) to

repeatabshape rewhich coworks.

h 8 (2019) 42–56

system independent of lighting environment. To nullifyent or varying lighting conditions, another current trendpectral imaging.33,35 Another SVM-based work is recentlyto attain 94% accuracy to classify four tongue-related col-is, red, light red, and deep red.46 This work encompassedtion in two stages. The first stage involved the unsuper-chine learning, k-means. The background separation wasolved at this stage. The later stage was based on SVM. Thetime was only 48 seconds, which shows the potential

lemented as a real time tongue image segmentation tech-cause of the 2-layer segmentation process, SVM improveddetection, that is, classification by 41.2%.after using low color temperature halogen lamps, someid not involve any color correction method. As a colorn scheme, the printed color card has been used in theork.20,25,43,45,47 Some45,47 used Munsell Color Checker asnsell was considered to be the first color order systemte hue, value, and chroma into perceptually uniform andent dimensions and to systematically illustrate the colors

ace. Hu et al48 used SVM-based color correction matrixthe dependency on controlled lighting environments

ue positions. The lighting environment included with andashlights. They further improved their work and acquired

sult.41 To the best of authors’ knowledge, this is the beste so far in the field of tongue diagnosis for light estimationcorrection. Another popular convention is transforming

e into standard color space which could be achieved byware and software manipulation.e other hand, recent advances of mobile phones areing to address lighting condition and color correctionits own. The camera lens, resolution, aperture, stabilizer,ltiframe imaging, and strong image processing softwareto manage the ambient condition better than ever. Theera-front camera issue41 is also resolved for many phoneowever, as chromatic features of tongue images hold theformation, light estimation can still play a crucial role.

segmentation and feature extraction

ost challenging part of tongue diagnosis is an adequatetion and optimum feature extraction. Before starting thegmentation, few works in literature preferred tongueand positioning. At the next stage, both color and geo-

atures need to be carefully extracted to feed into thent steps.

ue detection and positioning

et al49 used mouth location method to locate dark hole’sn the mouth and active appearance model (AAM) for auto-gue segmentation. The method required a different initialor tongue body segmentation which made it technically.

er to fix the tongue position, a guidance scheme for imagen system which outline the frontal and feedback gridlines,

ched by Jung et al.4 The study included three categories,il, frontal, and frontal and profile gridline and 120 images

essed based on profile angle and tongue area ratio. Jungcussed the relationship between diagnosis accuracy andility of tongue features, for example, color and shape.class correlation coefficient was computed to examine

ility. The result suggested an improvement in color andpeatability. Their study involved only healthy tonguesuld be further used as a calibration model for related

M.H. Tania et al / Integrative Medicine Research

3.2. Colo

The tocolors ontongue imbackgrougray scalbe procesin differeA summafocusing

The RGdiagnosisduce a gitongue ctions of t

Table 1Different Co

Color mod

RGB

HSI

HSV

LAB

seen as trwhich msame imaand HSVcoordinato humanapplicatiomodel, ththe tempground freven forevaluatioJian-Qianwell for ral56 usedof the tonfor tongured, makrated fromspace lesslike Openconsider.

In thehave founceptuallyand compopponenof the whent betw

r chant’s

ontinccuraect ton, th

putacoatlogeds, reandma

estedandahe toet al5

tong

Fig. 1. Tongue image in various color spaces.

r models and chromatic features

ngue diagnosis comprises recognizing the disease-relatedthe tongue surface and analyzing the texture. Theage segmentation steps such as edge detection and

nd–foreground separation are commonly processed ine. However, the overall tongue image analysis needs tosed in color format. The color gamut of a tongue imagent color spaces are portrayed in Fig. 1 as a point cloud.ry of digital camera-based works in different color spacedifferent features is presented in Table 1.B (Red, Green, Blue) is the most popular model for tonguedescribing what kind of light needs to be emitted to pro-

colopatie

Csis ainspregiocomandcatagoriebodyvisedsuggtic stfor tKimThe

ven color.30,50,62 The RGB space was utilized to extractolor features, for example, means and standard devia-he colors per pixel within the region of interest.30 It is

lor Models Used in Tongue Diagnosis

el Feature

Most of the devices take the image in RGB. Then they aretransformed into other color spaces.Tongue image segmentation30,50

Color correction28

Fusion of color and space information51,52

Tongue image extraction51–53

Morphology54,55

Color matching27,56

Contour shape56

Extraction core tongue colors57

Redness measurement58

Color matching56,59,60

Automatic tongue image segmentation61

Guidance system4

index. Asneeded to

Althouture, morhead moubased onall standBlue) is finformat

Extracet al57 ucolor histtexturaldescriptoduced imfeatures iand diseaclassificaened theextractio

8 (2019) 42–56 45

ue color in MATLAB, but the model is device dependent,eans there are many variations of RGB color space for thege using different devices. HSI (Hue, Saturation, Intensity)(Hue, Saturation, Value or Brightness) are two cylindrical-te representations of points in a RGB model. HSI is closer

vision. Zhu et al51 favored HSI due to its specificity of then in tongue image extraction. With the help of HSI color

e greedy rules were utilized to develop the template. Then,late was processed. After successful separation of back-om the raw image, the template was found to be effectivethick tongue coating. It can be useful for the quantitativen of tongue.52 The hue and intensity were exploited byg et al,55 for tongue image segmentation which workedegular tongue surfaces, but not for the irregular ones. Li ethue and brightness of HSV to decide the initial positiongue, whereas Jiang et al30 reported that HSV is unsuitablee diagnosis as it has discontinuities in hue value arounding it sensitive to noise. The color information is sepa-

the energy or image intensity in HSV, making the colornoise-prone than in RGB. HSV being popular in the libraryCV for Android platform makes it a strong candidate to

recently reported article on tongue diagnosis, researchersd to prefer LAB over the other color space as it is more per-linear giving the advantage of effective visual inspectionarison. L stands for lightness and ‘a’ and ‘b’ are the color

t dimensions. Analyzing the tongue indices, the mean a*ole tongue area was found to be distinguishably differ-

een people with and without Yin-deficiency.58 Thus, thennel values can provide significant information about thecondition.uing with LAB color space, Jung et al4 explored diagno-cy and considered the intraclass correlation coefficient tongue feature’s repeatability. The median value of tongueat is, tongue body was taken into account for color feature

tion.63 Kawanabe et al59 examined tongue color, its body,ing by analyzing tongue images of 1080 subjects. They

tongue color and tongue coating into five and six cate-spectively in the L*a*b* color space. To compute tongue

coating information, k-means clustering (an unsuper-chine learning technique) was employed. The authorsthe outcome as a globally suitable tongue color diagnos-

rd. However, the reliability was not quantified, especiallyngue of unhealthy subjects. From the a* value difference,8 separated the tongue coating from the tongue body area.ue coating was quantified by tongue coating percentagethe image was captured by a CTIS, the generated imagebe transformed into L*a*b* from RGB.gh tongue image capturing devices are useful, in litera-e works are done using a CCD camera. Hsu et al54 used anted tongue capture device. Tongue image segmentation

chromatic information is performed in the literature usingard color models. However, sRGB (standard Red Greenound to be more popular choice to fuse color and spaceion.ting primary core colors by an iterative method, Chensed Generalized Lloyd Algorithm (GLA) to obtain theogram. Because of the inadequacy of chromatic features,

features were taken into account by an edge histogramr. This ‘content-based image retrieval technology’ pro-proved result, considering different weights for two

n retrieval. The work focused on feature extractions only,se classification was not considered. Inclusion of disease

tion with same dataset of 268 images would have enlight-

readers more about the result obtained from featuren.

46M

.H.Tania

etal/Integrative

Medicine

Research

8(2019)

42–56

Table 2Neural Network for Chromatic Analysis

Algorithm Camera Input Layer/Structure

Performance Computing cost (s) Dataset Context Ref.

MLP/BP Watec WAT-202D CCDcamera

R, G, B histogram Multistructure192-32-84

Training data set:100%, recall set:58.7%

Learning rate: 0.1 s 31 Color analysis 26

SONY 3-CCD videocamera (SonyDXC-390P)

3 unit for cameraresponse

– – ANN (sigmoid)Training time: 3.3Testing time:0.0277ANN (linear)Training time:2.8511Testing time:0.0257RBF (SVR)Training time: 0.01Testing time:0.0009

>9000 Color correction 47

n/m R, G, B, H, S, I of thepixels

Multi (9)Self-adaptivenetwork structure

Accuracy: 88% – 200 Color recognition 69

RFBNN Hyperspectral camera 120-bandhyperspectralimage

N/m Accuracy: >86.74 844 375 Hyperspectralimaging

44

KNN Accuracy: >85. 93 691LDL Digital camera

(CANON1100D)Tongue body imageH, S, V

Single and multi 68.41% N/m 533 Labeling tonguecolor

70

AA-kNN 86.47%

search

The arfor chromical diagnreducedcomputatsule endolong videhealth syclassify swireless clized for tnetworknetworkthe practZhuo et arent contthe NN-bmethod w

In themore prerecognizianalyzesvolume oblance ofare repor98% of thmentatioof a givevised maand a hiThis segmmodel paManjunawith theby remotor valuethat pixeweight. Tincreasesglobal didow anddiscrimin

TheManjunareduction

lthoucient

wilniqu

Edge

dge de mfilteessingrou73 arointy alg

ases.ize

Xu76

ue sestmeAna

ue sctionof onothes hacom

eodeogylvese info elime anue.

Cont

he centae seused

M.H. Tania et al / Integrative Medicine Re

tificial neural network (ANN) is not an exceptional choiceatic analysis. ANN has been successfully used in med-osis due to its ability to process large volume of data,

probability of ignoring relevant information, and fasterional time.64 For example, the output of a wireless cap-scopy for evaluating the gastrointestinal tract is an 8 hourso. To reduce the inconvenience of already over burdenstem, Barbosa et al65 utilized an ANN to diagnose andmall bowel tumors analyzing the visual information ofapsule endoscopy. An enhanced HSV color model was uti-he segmentation purpose, using the convolutional neural(NN).66 However, Cheung et al67 argued that the neuralis complex and time consuming. Table 2 demonstratesice of neural network for color-based analysis. However,l68 discouraged NN for online color correction in the cur-ext due to the requirement of large sample size to trainased mapping model. The polynomial-based correctionas favored for its lower computational complexity.field of medical image processing, gray scale images are

valent than color images. The tongue diagnosis comprisesng the disease-related colors on the tongue surface andthe texture. Therefore, one needs to address the issue off colors to be managed. The complex part is the resem-the involved colors. Most of the disease-reflection featurested to be located in the 2% region between the 100% ande full tongue color range.47 Thus, J measure based seg-n (JSEG) algorithm is performed to test the homogeneityn color-texture pattern. The method involved unsuper-chine learning techniques such as k-means clusteringerarchical clustering (called agglomerative clustering).entation method is computationally more feasible than

rameter estimation. The method proposed by Deng andth71 is illustrated in Fig. 2. The color quantization startspeer group filtering,72 which provides a smooth image

ving the noise. It also provides a smoothness indica-. The weight to each pixel is assigned in such a wayl at textured area’s weight is less than smooth area’she implementation of agglomerative clustering algorithm

the pixel of approximate same color, decreasing thestribution. The color class-map is applied to local win-then J-image is obtained using Fisher’s multiclass linear

Asuffiworktech

3.3.

Eof thcianprocbackfiltertip pcanndiseoptimandtongadjunenttongdeteuseis anworkal56

as gpholinvospacing timagtong

3.4.

Tsegmimagwas

ation.implementation of JSEG, proposed by Deng andth,71 is shown in Fig. 3 to illustrate the effective color

keeping the useful information intact.

removalbefore imtour. Howto attain

Fig. 2. J measure based segmentatio

8 (2019) 42–56 47

gh chromatic features are essential to analyze, it is notenough to provide the complete information. Thus, this

l continue to discuss about different image processinges in the next section.

detection techniques

etectors are essential tools for tongue diagnosis and oneost popular techniques. The prewitt, sobel, and Lapla-rs are commonly used edge detectors for medical imageg. To separate the tongue from face and mouth (i.e.,nd–foreground separation), different versions of cannye used21,56,74,75. Shi et al75 used canny to find the tongue. Zhang et al21 used CCD camera to outline tongue viaorithm and then utilized feature extraction to diagnoseThe authors transformed the image into gray scale and and canny algorithm to obtain the edge image. Kanawong

presented a complete automatic tongue detection andgmentation framework method excluding any parameternt and initialization. They proposed a Principal Compo-lysis (PCA)-based tongue detection algorithm and hybridegmentation utilizing mean shift algorithm, canny edge

algorithm, and Tensor Voting algorithm. However, thely one sample makes the system incompetent. The Sobelr popular filter to be found in the literature.56,59 Fewve been done using mathematical morphology. Li et

pared different tongue image extraction methods suchsic active contour, HSI color model (mathematical mor-and sequential algorithm), and presented a method thatmultiobjective greedy rules to employ fusion of color andormation. Jian-Qiang et al55 utilized morphological open-

inate the thin link of lower lip attached with the tongued morphological closing to get rid of the small holes on

our shape

ontour detection is quite prevalent for tongue regiontion. The snake model finds its usefulness on tongue

gmentation with modifications.50,75,77–79 The dual snakefor tongue segmentation with 92.89% accuracy.78 Noise

and HIS color model transformation were conductedplementing the algorithm to achieve an accurate con-ever, the method was not feasible due to the hurdle

the initial contour, both inside and outside. Shi et al79

n (JSEG).

48 M.H. Tania et al / Integrative Medicine Research 8 (2019) 42–56

ure ba

used snaksegmentaconformacurve shothe contoof the daten brancing the toof tonguegood trueBut becaureal timeground swork consnake (C2tation, keThe datasvelocity.plexity, iformationlizing the98.4% accTesting oaccuracy.

3.5. Wate

The wby differewater soment basof imagespage, DNis capablthe perfooversegmsegmenta

A poscontrast-cessing),

ownrithmns82

notteresul.nakerodun on

g waused

be athet, Nirshe

hod cfinalsegmstru

Tong

ordourg23 ues ameomeuangetrietrifor

rentmodimpres w

k com

Fig. 3. JSEG of tongue image. JSEG, J meas

e geodesic active contour (GAC) model for tongue imagetion. GAC is also known as geometric active contour orl active contours. The model is inspired by the Euclideanrtening evolution. On the basis of the object of the image,ur splits and merges. Shi et al79 categorized the images

tabase by tongue colors and it contained four classes andhes. The double geodesic flow was applied after divid-ngue image into two parts based on previous knowledge

shape and location. The result showed low error andpositive volume fraction (98%) compared to other works.se of expensive computational time, it is not feasible forsolution. Moreover, the system is not full-proof for back-eparation in case of irregular tongue images. Later, thetinued with color control-geometric and gradient flowG2FSnake) algorithm for fully automated tongue segmen-eping the number of classifications and branches same.75

et included 30 images. This method enhanced the curveThough the work showed good accuracy and less com-t needed four points to be specified for initial contour. Chen et al61 performed tongue image segmentation uti-same dataset via MATLAB and the test result showed

uracy taking 11.46 seconds to compute with C2G2FSnake.n 40 images, recent work has been reported to achieve 75%80

rshed transformation

ord watershed signifies a ridge that divides areas drainednt river systems. The geographical area draining into a

urce, for example, river or reservoir is called the catch-in. The watershed algorithm is adaptable to various kinds, for example, blood cells, stars, toner spots on a printedA microarray elements. This segmentation algorithm

e to differentiate incredibly complex images. However,rmance depends on correct choice of markers to avoidentation. The watershed transform is also utilized as a

as shalgoEddidoesof inusef

Sto pnatiousinwasnotlikeinpuwatemetandoveredge

3.6.

IncontZhanshap13 g

HgeomgeomlizeddiffeAHPandclasswor

tion technique for tongue diagnosis.50,61,81

sible route for watershed transform on tongue could belimited adaptive histogram equalization (after prepro-thresholding, filtering, and then overlaying the perimeter

samplesdeflectionpared thwhere th

sed segmentation.

in Fig. 4. The last image of Fig. 4 is where watershedwas employed in this work and this was introduced by

for cell segmentation with microscopic images. The imagecontain the anticipated textural information. The regiont is cropped and the rest of the image information is not

models are often combined with watershed transformce better edge detection. Wu et al81 used such combi-

120 images, where the initial contour was generatedtershed transform whereas the active contour model

to converge to the exact edge. However, there mayclear edge border for tongue images in natural setting

ones used in this work. Taking RGB tongue images asng et al50 sequentially used gradient vector flow (GVF),d transformation, and then an interactive region mergingalled maximal similarity based region merging (MSRM)ly snake model for refinement. The idea was to avertentation of watershed as GVF can well preserve mainctures while removing trivial structures.

ue shape based analysis

er to analyze the shape of the tongue, one may requireextraction and tongue shape correction. Zhang andsed decision tree to classify five different types of tongueong 130 healthy and 542 diseased people. They included

tric features. Their work outline is sketched in Fig. 5.et al40 corrected the tongue deflection based on three

c categories and analyzed tongue shape based on sevenc features. The Analytic Hierarchy Process (AHP) was uti-tongue classification by computing relative influences ofparameters such as length, area, and angle factor. Eachule represented each tongue shape class. The ambiguityession between the quantitative features and the shapeere presented by the fuzzy fusion framework. This frame-bined seven AHP modules. The study included 362 tongue

and the result showed success to decrease tongue shape. The classification accuracy was 90.3%. The authors com-

e result with kNN83 and linear discriminant analysis,99

e accuracy was 78.2% and 82.9%, respectively. Although

M.H. Tania et al / Integrative Medicine Research 8 (2019) 42–56 49

Fig. 4. Watershed segmentation on Tongue.

sis ba

their studdisease ty

4. Diseas

Tonguitself. Xuniques. AZhao et ahepatitis.the self-rtongue ancontrol gmine theretroflexutilized bpepsia. TIndex scoimages taMore parfind out a

Effectithat CDSSobservatiby clinicispecializefor examresponseor propopopular d

Zhangtem to dpancreatiworks inimages. Ttion andTheir wornosis accresult wadenal perfor Leukeattention

be eing autilipretuloue-ccaninan efores ca

impaobiaobia

colue crecta

impncereenr walativ, Haeatedod t

hangtiate

iagnoetesmon

ncy,nic bto fi

niqug waand

ormsationhang

Fig. 5. Simplified diagram of geometric shape-based tongue diagno

y includes unhealthy tongues, the correlations betweenpes and tongue shapes are not taken into account.

e classification

e diagnosis deals with syndrome, rather than diseaseet al84 classified syndromes using machine learning tech-nother syndrome classification-based work, conducted byl,85 focused on searching for its correlation with viralKim et al58 utilized statistical analysis to prove that

eported Yin deficient patients may have more redness ind less tongue coating in comparison to non-Yin-deficientroup. Statistical analysis was further utilized to deter-relationship between ischemic stroke with and withouttongue among 308 patients.86 A statistical analysis wasy Kim et al87 to track the treatment of functional dys-

his work analyzed the change in the Nepean Dyspepsiares to indicate the primary outcome. The baseline and theken after four weeks were examined to achieve this goal.ameters (such as tongue coating thickness) were added tofew secondary outcomes.

ve CDSS may be yet to be established, but there is no denialhas the potential to improve healthcare by linking health

ons with health knowledge to influence health choicesans. The diagnosis decision support systems (DDSS) is ad CDSS for diagnosis that requests for the patients data,

ple, questionnaires, images, previous medical record. Into the request, the system provides a disease decision

ses a set of appropriate diagnoses. DDSS are becomingay by day and is being tested in dynamic scenario.88,89

et al20 presented a Bayesian network classifier-based sys-iagnose pulmonary heart disease, appendicitis, gastritis,tis, and bronchitis with about 75% accuracy. Their systemreal-time to point out quantitative features from tonguehey used Support Vector Regression (SVR) for color correc-Bayesian classifier for chromatic and textural measuring.k included 544 patients and 56 healthy subjects. The diag-uracy was found to be 100% for the healthy class. The

Acoatwasteriaedentongand

Htoolimagthemicrmicrfromtongcolowithof cabetwothein reseriaindicmeth

Zferento ddiabpneuficiechroSVMtechterinSVMperfsific

Z

s not promising for diseases such as nephritis, gastroduo-foration, and abdominal tuberculosis. Though accuracymia was only 67.6% and the sample size was 37, morecan be drawn toward such cancer diagnosis.

images toto diagnocoronaryChinese M

sed on the work done by Zhang and Zhang.23

t al90 tried to establish the relationship between tonguend aspiration pneumonia. The tongue plaque index (TPI)zed to investigate the association of selected oral bac-sent in saliva and tongue-coating. The subjects were 71us elderly aged over 65 years. The outcome suggested thatoating can be used as a risk indicator for edentate (incisore toothless) subjects to detect aspiration pneumonia.

t al31 investigated tongue coating images as a probablea colorectal cancer diagnosis. They appraised tongueptured by a specialized image acquisition device to studyct on coating thickness in the presence of responsiblel particle. The result showed clear variation betweenl community structures in case of patients sufferingorectal cancer. The presented analysis indicates thatoating analysis may be a novel potential biomarker forl cancer diagnosis. Han et al8 further extended their studyroved sample size for early diagnosis of different types. Their study showed that there was a color variationhealthy and cancer tongues, one was reddish while thes purple respectively. The cancer patients’ tongues lacke abundance of bacterial microorganisms such as Neis-mophilus, Fusobacterium, and Porphyromonas. The worktoward a possible cancer screening and early diagnosis

hrough tongue and tongue coating analysis.et al22 used tongue color gamut on 1045 images to dif-between healthy tongue and tongue with disease and

se different diseases including chronic kidney disease,nephritis, hypertension, verrucous and erosive gastritis,ia, nephritic syndrome, chronic cerebral circulation insuf-

upper respiratory tract infection, coronary heart disease,ronchitis, and mixed hemorrhoid. They used k-NN andnd whether the subject is healthy or not. Both of these

es provided 91.99% classification accuracy. Later on, clus-s performed on 11 diseases with 70% accuracy. Thoughk-NN both show similar outcome for clustering, SVMbetter. For both cases, diabetes and hypertension clas-result are poor.and Zhang23 conducted quantitative analysis on 672

distinguish between healthy and diseased tongue and

se diabetes mellitus, different gastritis, nephritis, andheart disease. Samples were collected from Traditionaledicine Hospital but the images were classified based

50 search 8 (2019) 42–56

on westesify amonto classiffeatures,ward SeleThe averato tonguesuch as tdiagnosis

With atotal 827TDA-1 diand coatined by dother algization, 2tongue csions of tshowed 7diction athe poteninclusion

Inspirextractioworks wobjectiveunhealthform thethe systereal-timeespeciallyprovide d

In recemost of ththe otheron the pible standiagnosisit becombest decireportednot be ac

Theredetection0 and 1)nose BCthe likelisidered fecchymoof the exsuch asand heart

The dple wereBC patientongue feperformthe earlytechniqubetter ac80% to 90features.change in

A sumin Table 3

. Subd

obil

20wid

Health Ast aned ths.95

hougpectworkkno

andruserity,singhe ren untatiortphoJ Z32to pt al ptongnd fiis w

de aphysue vnoseakingd tonen fouseds. Howith

akinge to

matient

etectike ms, iToregisand

. Therwis

M.H. Tania et al / Integrative Medicine Re

rn medical practice. The decision tree was used to clas-g five different types of tongue shapes, SVM was used

y diseases for each tongue shape utilizing 13 geometricand number of features were optimized by Sequential For-ction (SFS). They also used k-NN but it gave poor result.ge accuracy of disease classification is 76.24%. In additionshape and geometric features, fusion of other features

ongue color, fur, and fissure could be explored for betterof the diseases.n aim to diagnose diabetes, tongue image information ofsubjects, both diabetic and non-diabetic obtained from

gital tongue instrument were analyzed.91 Tongue bodying separation, color, and textural features were exam-ivision merging, chrominance-threshold, SVM, and few

orithms. After sample equalization and feature normal-3 input parameters including information of the subject,olor, and texture PCA were used to reduce the dimen-he feature. Optimizing the kernel parameter by GA, SVM8.77% accuracy for diabetes prediction. Though the pre-

ccuracy is low, the work carries clear indication abouttial of tongue diagnosis for diabetes detection. In future,of other tongue features may increase the accuracy.

ed by deep convolutional neural network (CNN), a featuren technique named constrained high dispersal neural net-as implemented on tongue images of 315 people.92 The

of this feature extraction was to separate healthy andy class. A weighted LIBLINEAR SVM was trained to per-task, which showed 91.14% accuracy. The authors claimedm to be faster than CNN; however, to be implemented in, one has to be careful about using deep neural network,

when the accuracy is still low and the system cannotecision on specific disease.nt years, researchers have been working on the ATD. Butem are concerned only up to feature extraction stage. Onhand, CDSS-based works paid an inadequate attention

rior steps involved, which raises the issue of compat-dardization. Besides, the limited work done on tongue-based disease detection is not on the same diseases. Thus,

es hard to compare the accuracy and repeatability of thesion support system found in this field. Moreover, theworks suggest the accuracy slightly under 80%, which mayceptable in many cases of CDSSs.are few works93 done on early stage breast cancer (BC). Nine tongue features of both breast cancer patient (stage

and healthy persons were investigated in a hope to diag-at an early stage to ensure timely treatment to increasehood of recovery, lowering the relapse rate. The con-eatures are tongue color, quality, fur, fissure, red dot,sis, tooth mark, saliva, and tongue shape. The subdivisiontracted features according to the corresponding organsspleen–stomach, liver–gall-left, liver–gall-right, kidney,–lung area are illustrated in Fig. 6.ata of 57 early-stage BC patients and 60 healthy peo-utilized for training, whereas the data of 10 early-stagets and 10 healthy people were applied for testing. Theatures along with Mann–Whitney test were exploited tothe logistic regression. Though the result is not good forstage or potential cancer patient, more feature extractiones as well as different features could be explored to attaincuracy. The accuracy for healthy tongue was varied from%. Higher accuracy was attained using lesser number ofThe work was further upgraded without any significant

Fig. 6

5. M

InagedviaHealrobugainyear

TprosfewbestandBecasecufocu

TtakemensmaandableHu ethanfur ateamincluon atongdiag

Tbasechoswaspartstart

Tphonautoronmon d

Ltionand(cm)tionothe

the result.93

mary of algorithms used in disease diagnosis is provided.

questionUsing theommend

ivision of tongue into areas corresponding to different internal organs.93

e-enabled tongue diagnosis schemes

09, American Recovery and Reinvestment Act encour-espread adoption of health information technology (HIT)th Information Technology for Economic and Clinicalct (HITECH). Gradually these HITs are becoming mored diverse. Disease diagnosis on a mobile platform hase attention of researchers and developers in the last few

h available smartphone technology offers enormousto employ tongue diagnosis for disease classification, very

has been done exploiting existing means. To authors’wledge, only one mobile application is available in iOSoid platform called i-Tongue © University of Missouri.96

of openness, total interface customization, universality,few more tongue diagnosis applications are initiated

android platform than iOS.21,41,48

search work conducted by Otsu97 and Hsu et al54 wasder consideration by Hu et al,41 for tongue region seg-n and tongue diagnosis, and the algorithm was tested innes such as Samsung Galaxy S2 and S3, HTC Sensation,providing similar outputs and server running in PC is

erform analysis of each image taking below 3 seconds.ointed out that most of the mobile phone camera otherue with any kind of deformity this work is promising forssure detection. Though based on this work’s output, theorking on liver disease diagnosis, and this work does not

ny decision support system and thus predominantly reliesician’s opinion. Zhang et al21 used CCD camera to outline

ia canny algorithm and then utilized feature extraction todiseases.teeth color as a standard, Ini98 presented a smartphone-gue color calibration method. 20 female subjects werer the study, continued for a month. K-means algorithmto detect tongue coating differentiating from other body

wever, the distinguishing method itself was unfitting to.lighting condition into account, Hu et al48 used a smart-calculate SVM-based color correction matrix and to

cally detect fur and fissure of tongue. The lighting envi-included with and without flashlights. This work focusedion accuracy, but robustness was not explored.ost diagnosis-based health and fitness mobile applica-

ngue © University of Missouri96 is available for both guesttered users in google play and apple app store. The heightweight (kg) of the user has to be provided during registra-re is a series of questions users must provide answer to;e no result will be generated. There are some additional

s for registered users; few of them are gender-specific.se answers, the application generates the result and rec-ations including constitute of qi asthenia, yang asthenia,

M.H

.Taniaet

al/IntegrativeM

edicineR

esearch8

(2019)42–

5651

Table 3Performance of Different Algorithms Used for Disease Diagnosis

Algorithm/Technique

Ref. Dataset (no. ofimages (I)/subject(S))

No. ofdiseases

Diseases Other techniquesused

Features/Context

Accuracy

Total Healthy Unhealthy

Bayesian 20 14,696 and S-600 56 544 9 Leukemia; pulmonary heartdisease; abdominaltuberculosis; appendicitis;Nephritis; gastroduodenalperforation; gastritis;Pancreatitis; bronchitis

SVR Color correction 75%

SVM 47 >9000 and S-5222a 440 2780 200 n/ab – Tongue colorgamut

N/m

22 I-1045 143 902 13 Chronic kidney; diseasediabetes; nephritis;hypertension; verrucousgastritis; pneumonia; nephriticsyndrome; chronic cerebralcirculation insufficiency; upperrespiratory tract infection;erosive gastritis; coronaryheart disease; chronicbronchitis; mixed hemorrhoid;miscellaneous

k-NN Tongue colorgamut; 12 tonguecolor categories;tongue foregroundpixels

91.99% (healthy vs.disease)≥70% (diseaseidentification)

23 I and S-672 130 542 7 Diabetes mellitus; nephritis;gastritis verrucosa; nephroticsyndrome; erosive gastritis;chronic gastritis; coronaryheart disease

k-NN: poor result 13 geometricfeatures

76.24%

Decision tree 5 tongue shapeSFS To reduce no of

features91 827 531 296 1 Diabetes k-NN

Naive BayesBP-NNGA-SVM

Classification basedon color andtextural features

78.77% (prediction)83.06%(cross-validation)

52M

.H.Tania

etal/Integrative

Medicine

Research

8(2019)

42–56

Table 3 (Continued)

Algorithm/Technique

Ref. Dataset (no. ofimages (I)/subject(S))

No. ofdiseases

Diseases Other techniquesused

Features/Context

Accuracy

Total Healthy Unhealthy

SPSS software;mothur software

8 S-386 100 286 3 Colorectal cancerLung cancerGastric cancer

– Tongue coatingthicknessFisher’s exact test

–

Logistic regression 93 137 – – 1 Breast cancer Tongue color,quality, fur, fissure,red dot,ecchymosis, toothmark, saliva, andtongue shape.Mann–Whitneytest

Healthy class:7 features – 80%, 6features – 80% and5 features – 90%Disease class: 7features – 60%, 6features –60% and 5 features– 50% is

Statistical analysis 90 71 – – 1 Aspiration pneumonia – Tongue plaqueindex (TPI);tongue-coatingoral bacteria

–

ANN (SOMKohonen Classifier)

94 100 – – 1 Diabetes – Tongue color andgist features

–

a 2002 people were grouped in the sub-health category.b The study included analysis of diseases tongue, and the diagnosis itself was not conducted. Diseased class diagnosed using western medicine, not TCM.

M.H. Tania et al / Integrative Medicine Research 8 (2019) 42–56 53

Fig. 7. Schematic diagram of mobile enables automatic tongue diagnosis scheme using clinical decision support system.2

utom

yin asthecan recomguest verthe imageaxis to gucamera. Timages frand machThe userwith a fe14, 2016operatedThe updaintendedreliability

A mohuman insystem wing algorimplemepaper.

ropo

n theaddare

likectice

en acere sthe ognos

mparere ischa

signus, tprobable.ere is an infrequency of interchangeable scheme, making it

Fig. 8. Proposed integrated framework for a

nia, phlegm-dampness, and damp-heat. Each constitutemend environment, exercise, emotion, and diet. The

sion cannot store any images. The application removeswhen the user tries to take another picture. There is (x, y)

ide the user about tongue exposure in front of the phonehe registered users can save the history and use storedom the phone. Image segmentation, feature extraction,ine learning techniques are used to process the result.

can request for manual assessment by TCM professionalse. The current version 2.0.2 is an update released on Seprequires minimum android 2.2 or iOS 5.1.1 and can bein English, Simplified Chinese, and Traditional Chinese.te includes sharing features as well. The application isfor an auxiliary diagnosis with questionable accuracy and.

bile enabled system for tongue diagnosis without anytervention is outlined in Tania et al2 (Fig. 7). Such aould comprise image processing and machine learn-ithms to act as a standalone system. However, the

6. P

Oto beThey

- UnpraopThindiaco

- ThingdeThim

- Th

ntation on mobile platform was not included in the incompstep as

atic tongue diagnosis.

sed framework and discussion

basis of our study, we have identified the research gapsressed to develop an intelligent tongue diagnosis system.summarized as below.

the western research convention, it is not a commonto deposit the classified authentic tongue images in an

cess database, thereby limiting the scope of improvement.hould be benchmark datasets of classified tongue imagespen access database to broaden the horizon of tongueis. Without a benchmark dataset, it is hard to performable experiments, verify, validate, or improve the results.a lack of overall consistency in the ATD systems includ-

racteristics of the input image, for example, image quality,guideline, image segmentation, and feature extraction.he accuracy, robustness, and reliability are inevitably

rehensible to utilize the best found result from the prioran input for the subsequent step for different work.

54 searc

- To thestartingducted

The topotentialsystem. OdiagramCDSS, indincorporadiagnosis

The coapp usuatoms. Thselected dple, TCMmobile puser with

Withments aredevice powith F1.7tures arecolor corrprocessinitself. Theto imageinformatia fusion agram canbe deployhomecare

7. Concl

This pdiagnosisis in favof auxilisystem atexturalindispenswork hatongue imhealthy–works. Thespeciallytrends, th

Whileof CDSS wing a CDSrelying oin the fialgorithmwork andcollectivealso noticto anotheThus, theas everyomost casethe reseaan open a

The vooverall sy

ctedonguaturelop shelpnolo

flicts

he au

owl

he rehipsilityber:

renc

ontreensinttp://aniarocesis schnformttp://im Jis sysunctiottp://

ung Congue014;6ungn tttp://amamegionurg 2

Su W,with lhttp://Han Ssis m

ttp://angasedttp://

iang Bequenicrob

ChengtionalAlternReddinLopezpointhttp://Switzwithhttp://Skanding w

ttp://mitharoevelottp://ogocobile

nfectittp://eters

M.H. Tania et al / Integrative Medicine Re

best of our knowledge, there is no complete work (i.e.,from the light estimation to CDSS) that has been con-

in the literature.

ngue diagnosis requires more autonomy to reach its fulland ensure its widespread application in the digital healthn the basis of the work done in the literature, a conceptualis illustrated in Fig. 8 as a completely ATD system usingependent of human expert involvement. The idea is tote disease detection with extracted features from tonguein a mobile enabled platform (Fig. 7).mmercial health applications, for example, iPhone healthlly provides provision for manual input regarding symp-e manual input of disease symptoms can be based oniseases. Any of the standard questionnaire set, for exam-

, Kampo can be utilized. A polythetic approach on thelatform should consider the symptoms provided by thethe color, texture, and geometric features.

the improvement of hardware, preprocessing require-reducing day by day. The commercially available mobile

ssesses camera resolution as high as 12–16 megapixel,lens. Higher focus and built-in automatic adjustment fea-

narrowing down, and the necessity of light estimation andection is abridging or nullifying. Moreover, a strong imageg algorithm can often manage the lighting condition byextracted features (e.g., fur, texture, shape), subsequent

segmentation, can be utilized for classification. All theseon, including the symptoms, need to be fused together bylgorithm to provide the final result. The conceptual dia-be trained offline and the whole diagnosis system caned as a smartphone application to avail the system as aor mobile setting (Fig. 7).

usion

aper explored the standardized and automated tonguesystems to demonstrate its potential. The investigation

or of tongue diagnosis to make it an integrated partary diagnosis system within the primary healthcarend an early diagnostic tool. Because the chromatic,

and geometric shape based analysis of tongue projectsable information about health status. Remarkable

s already been done on different steps, for example,age segmentation, background–foreground separation,

unhealthy classification, and few specific disease-relatedis work explored each of these steps in the literature,focusing on the algorithms. Identifying some general

is section broadly summarizes the research gap.evaluating the prospect of tongue diagnosis, the absenceas noticeable. Our study indicated that the ATD embrac-S can open the door for standalone systems, rather thann limited TCM experts only. The recent advancementeld of camera of smart devices, classifiers, and CDSSs present the opportunity to develop a real-time frame-standard protocol for everyone to be followed so thatknowledge can be built on existing one. However, it waseable from our survey that the transition from one stepr is incomprehensible due to ‘casual’ diversity in practice.need for standardization is inevitable from the literature,ne is defining their own system for various steps but ins, there is no way to put them together. To proceed with

reflethe tliterdevemaytech

Con

T

Ackn

TnersmObnum

Refe

1. Csh

2. TpsIh

3. Ksfh

4. Jt2

5. Jih

6. YRS

7.

8.

h9. P

bh

10. Jsm

11.

12.

13.

14.

h15. S

ndh

16. BMih

17. P

rch on tongue diagnosis, undeniably there is a need forccess authentic dataset with classified images of tongue.lume of work done in the literature does not combine thestem to make it human intervention-free. The literature

of ahttp://

18. DaaboopticaQuant

h 8 (2019) 42–56

the lack of a complete standalone system from capturinge picture to attain the disease diagnosis. On the basis of thereview, this work proposed an integrated framework to

uch a system. The derivation of general recommendationsthose who intend to bring tongue diagnosis into modern

gy enabled practice.

of interest

thors declare no conflict of interest.

edgements

search work has been funded by Erasmus Mundus Part-Action 2 “FUSION” (Featured eUrope and South asIaNetwork) project of Erasmus Mundus. Grant reference2013-3254 1/001001.

es

ras-naranjo JC, Wei Q, Ozcan A. Mobile phone-based microscopy,g, and diagnostics. IEEE J Sel Top Quantum Electron 2016;22:1–14,dx.doi.org/10.1109/JSTQE.2015.2478657.MH, Lwin KT, Hossain MA. Computational complexity of imagesing algorithms for an intelligent mobile enabled tongue diagno-eme. In: 2016 10th International Conference on Software, Knowledge,ation Management & Applications (SKIMA). Chengdu: IEEE. 2016:29–36,dx.doi.org/10.1109/SKIMA.2016.7916193., Han G, Ko SJ, Nam DH, Park JW, Ryu B, et al. Tongue diagno-tem for quantitative assessment of tongue coating in patients withnal dyspepsia: a clinical trial. J Ethnopharmacol 2014;155:709–13,dx.doi.org/10.1016/j.jep.2014.06.010.J, Kim KH, Jeon YJ, Kim J. Improving color and shape repeatability of

images for diagnosis by using feedback gridlines. Eur J Integr Med:328–36, http://dx.doi.org/10.1016/j.eujim.2014.01.004.

CJ, Jeon YJ, Kim JY, Kim KH. Review on the current trendsongue diagnosis systems. Integr Med Res 2012;1:13–20,dx.doi.org/10.1016/j.imr.2012.09.001.oto S, Tsumura N, Nakaguchi T, Namiki T, Kasahara Y, Terasawa K, et al.

al image analysis of the tongue color spectrum. Int J Comput Assist Radiol011;6:143–52, http://dx.doi.org/10.1007/s11548-010-0492-x.Xu ZY, Wang ZQ, Xu JT. Objectified study on tongue images of patientsung cancer of different syndromes. Chin J Integr Med 2011;17:272–6,dx.doi.org/10.1007/s11655-011-0702-6., Yang X, Qi Q, et al. Potential screening and early diagno-

ethod for cancer: tongue diagnosis. Int J Oncol 2016;48:2257–64,dx.doi.org/10.3892/ijo.2016.3466.B, Zhang D, Li N, Wang K. Computerized tongue diagnosison Bayesian networks. IEEE Trans Biomed Eng 2004;51:1803–10,

dx.doi.org/10.1109/TBME.2004.831534., Liang X, Chen Y, Ma T, Liu L, Li J, et al. Integrating next-generationcing and traditional tongue diagnosis to determine tongue coatingiome. Sci Rep 2012;2:936, http://dx.doi.org/10.1038/srep00936.TL, Lo LC, Huang YC, Chen YL, Wang JT. Analysis of agreement on tradi-

Chinese medical diagnostics for many practitioners. Evid-based ComplemMed 2012;2012, http://dx.doi.org/10.1155/2012/178081.gton AP, Trueb JT, Freedman DS, Tuysuzoglu A, Daaboul GG,CA, et al. An interferometric reflectance imaging sensor for

of care viral diagnostics. IEEE Trans Biomed Eng 2013;60:3276–83,dx.doi.org/10.1109/TBME.2013.2272666.NA, D’Ambrosio MV, Fletcher DA. Low-cost mobile phone microscopya reversed mobile phone camera lens. PLOS ONE 2014;9:e95330,dx.doi.org/10.1371/journal.pone.0095330.arajah A, Reber CD, Switz NA, Fletcher DA. Quantitative imag-

ith a mobile phone microscope. PLOS ONE 2014;9:e96906,dx.doi.org/10.1371/journal.pone.0096906.

ZJ, Chu K, Espenson AR, Rahimzadeh M, Gryshuk A, Moli-M, et al. Cell-phone-based platform for biomedical devicepment and education applications. PLOS ONE 2011;6:e17150,dx.doi.org/10.1371/journal.pone.0017150.h II, Andrews JR, Speich B, Utzinger J, Ame SM, Ali SM, et al.

phone microscopy for the diagnosis of soil-transmitted helminthons: a proof-of-concept study. Am J Trop Med Hyg 2013;88:626–9,dx.doi.org/10.4269/ajtmh.12-0742.en C, Chen T, Ansermino J, Dumont G. Design and evaluation

low-cost smartphone pulse oximeter. Sensors 2013;13:16882–93,dx.doi.org/10.3390/s131216882.ul GG, Lopez CA, Yurt A, Goldberg BB, Connor JH, Ünlü MS. Label-freel biosensors for virus detection and characterization. IEEE J Sel TopicsElectron 2012, http://dx.doi.org/10.1109/JSTQE.2011.2180516.

search

19. Lu J, Yation baEng 20

20. Zhangsis sysference

21. Zhangsis appConferhttp://

22. Zhangical ahttp://

23. Zhangimagehttp://

24. Cibin NdiabetEng Te

25. Chiu CChines2000;6

26. Jang JHtongueConf Anhttp://

27. Li CH,2002;3

28. WangIEEE Tstamp

29. Kim Minspecfor sphttp://

30. Jiang Ltongueand Ap

31. Han Sbiomehttp://

32. Zhi L,medic2007;3

33. Li Q,hypershttp://

34. Li Q.sis. Inhttp://

35. Li Q, Wbased2011;3

36. Jiang Cfor vis2014:2

37. Jian Z,of 3D ping chhttp://

38. Lv H, Cric stehttp://

39. Liu Z,images

40. HuangInf Sci

41. Hu M-smartp2016;4

42. McCam1976;2

43. Wangtraditiand InfDecemberg; 2

44. Liu Zsor forg/10

45. Cai YC19th IEhttp://

46. Kamarbased

olour017;2ang

mage013;2u M-

martetics.hongmagent Syttp://ingased012;2hu Mmagettp://hu Masedationttp://iangenta

ciencettp://su Y

ure ettp://

ian-Qlgoritnginehe 1st008, vi W,iagnoronicsttp://hen Lures eethoim Setweross-ttp://awan

ificatintegrangicitisttp://hen Lenta

n: 201015:9eng014;4ungypettp://matocial nttp://arbosmallis inttp://i J, Xasedditorionalroceettp://heunf theorksttp://huoolorntttp://

M.H. Tania et al / Integrative Medicine Re

ng Z, Okkelberg K, Ghovanloo M. Joint magnetic calibration and localiza-sed on expectation maximization for tongue tracking. IEEE Trans Biomed17;65:52–63, http://dx.doi.org/10.1109/TBME.2017.2688919.HZ, Wang KQ, Zhang D, Pang B, Huang B. Computer aided tongue diagno-tem. In: 2005 IEEE Engineering in Medicine and Biology 27th Annual Con-, vol. 7. 2005:6754–7, http://dx.doi.org/10.1109/IEMBS.2005.1616055.Q, Shang HL, Zhu JJ, Jin MM, Wang WX, Kong QS. A new tongue diagno-lication on android platform. In: Proceedings – 2013 IEEE International

ence on Bioinformatics and Biomedicine, IEEE BIBM 2013. 2013:324–7,dx.doi.org/10.1109/BIBM.2013.6732705.

B, Wang X, You J, Zhang D. Tongue color analysis for med-pplication. Evid-based Complem Alternat Med 2013;2013:264742,dx.doi.org/10.1155/2013/264742.

B, Zhang H. Significant geometry features in tongueanalysis. Evid-based Complem Alternat Med 2015;2015:1–8,

dx.doi.org/10.1155/2015/897580.V, Franklin SW, Nadu T. Diagnosis of diabetes mellitus and NPDR in

ic patient from tongue images using LCA classifier. Int J Adv Res Trendschnol 2015;II:57–62.C. A novel approach based on computerized image analysis for traditionale medical diagnosis of the tongue. Comput Methods Programs Biomed1:77–89, http://dx.doi.org/10.1016/S0169-2607(99)00031-0., Kim JE, Park KM, Park SO, Chang YS, Kim BY. Development of the digitalinspection system with image analysis. In: Proc Second Jt 24th Annunu Fall Meet Biomed Eng Soc [Engineering Med Biol.], vol. 2. 2002:1033–4,

dx.doi.org/10.1109/IEMBS.2002.1106262.Yuen PC. Tongue image matching using color content. Pattern Recognit5:407–19, http://dx.doi.org/10.1016/S0031-3203(01)00021-8.X, Zhang D. An optimized tongue image color correction scheme.

rans Inf Technol Biomed 2010;14:1355–64 http://ieeexplore.ieee.org//stamp.jsp?tp=&arnumber=5570961.

, Cobbin D, Zaslawski C. Traditional Chinese medicine tonguetion: an examination of the inter- and intrapractitioner reliabilityecific tongue characteristics. J Altern Complem Med 2008;14:527–36,dx.doi.org/10.1089/acm.2007.0079., Xu W, Chen J. Digital imaging system for physiological analysis bycolour inspection. In: 2008 3rd IEEE Conference on Industrial Electronics

plications. 2008:1833–6, http://dx.doi.org/10.1109/ICIEA.2008.4582836., Chen Y, Hu J, Ji Z. Tongue images and tongue coating micro-

in patients with colorectal cancer. Microb Pathog 2014;77:1–6,dx.doi.org/10.1016/j.micpath.2014.10.003.Zhang D, Yan Jqi, Li QL, Tang Qlin. Classification of hyperspectral

al tongue images for tongue diagnosis. Comput Med Imaging Graph1:672–8, http://dx.doi.org/10.1016/j.compmedimag.2007.07.008.Liu Z. Tongue color analysis and discrimination based on

pectral images. Comput Med Imaging Graph 2009;33:217–21,dx.doi.org/10.1016/j.compmedimag.2008.12.004.

Hyperspectral imaging technology used in tongue diagno-: 2008 2nd Int Conf Bioinforma Biomed Eng. 2008:2579–81,

dx.doi.org/10.1109/ICBBE.2008.978.ang Y, Liu H, Guan Y, Xu L. Sublingual vein extraction algorithm

on hyperspectral tongue imaging technology. Comput Med Imaging Graph5:179–85, http://dx.doi.org/10.1016/j.compmedimag.2010.10.001., Luo C, Yu J, Li R, Wang Z. Modeling a realistic 3D physiological tongue

ual speech synthesis. In: 2014 IEEE Int Conf Multimed Expo Work, ICMEW.–7, http://dx.doi.org/10.1109/ICMEW.2014.6890595.Lijuan S, Lirong W, Qinsheng D, Yajuan S. Dynamic extraction methodarameters of tongue for pronunciation recovery about impaired hear-

ildren. In: 2012 Fifth Int Conf Intell Networks Intell Syst. 2012:340–3,dx.doi.org/10.1109/ICINIS.2012.59.ai Y, Guo S. 3D reconstruction of tongue surface based on photomet-

reo. In: Int Conf Signal Process Proceedings, ICSP, vol. 3. 2012:1668–71,dx.doi.org/10.1109/ICoSP.2012.6491901.Wang H, Xu H, Song S. 3D tongue reconstruction based on multi-view

and finite element. Adv Inf Sci Serv Sci 2011;3.B, Wu J, Zhang D, Li N. Tongue shape classification by geometric features.

(NY) 2010;180:312–24, http://dx.doi.org/10.1016/j.ins.2009.09.016.C, Cheng M-H, Lan K-C. Color correction parameter estimation on thehone and its application to automatic tongue diagnosis. J Med Syst0:18, http://dx.doi.org/10.1007/s10916-015-0387-z.y C, Marcus H, Davidson J. A color-rendition chart. J Appl Photog Eng:95–9.

Y, Zhou Y, Yang J, Xu Q. An image analysis system for tongue diagnosis inonal Chinese medicine. In: Zhang J, He J-H, Fu Y, editors. Computationalormation Science: First International Symposium, CIS 2004, Shanghai, China,ber 16–18, 2004. Proceedings. Berlin, Heidelberg: Springer Berlin Heidel-005:1181-1186, http://dx.doi.org/10.1007/978-3-540-30497-5 181.

, Li Q, Yan J, Tang Q. A novel hyperspectral medical sen-or tongue diagnosis. Sens Rev 2007;27:57–60, http://dx.doi..1108/02602280710723497.Y. A novel imaging system for tongue inspection. In: IMTC/2002 Proc

c2

47. Wi2

48. Hsn

49. ZiIh

50. Nb2

51. Zih

52. Zbnh

53. LmSh

54. Hth

55. JaEt2

56. Ldth

57. CtM

58. Kbch

59. KtI

60. Pdh

61. CmI2

62. G2

63. Jth

64. Afih

65. Bssh

66. LbetPh

67. Cowh

68. Zceh

EE Instrum Meas Technol Conf (IEEE Cat No00CH37276), vol. 1. 2002:21–3,dx.doi.org/10.1109/IMTC.2002.1006833.udin NDi, Ooi CY, Kawanabe T, Odaguchi H, Kobayashi F. A fast SVM-tongue’s colour classification aided by k-means clustering identifiers and

69. Zhu Mon BP-InternaITME 2

8 (2019) 42–56 55

attributes as computer-assisted tool for tongue diagnosis. J Healthc Eng017, http://dx.doi.org/10.1155/2017/7460168.X, Zhang B, Yang Z, Wang H, Zhang D. Statistical analysis of tongue

s for feature extraction and diagnostics. IEEE Trans Image Process2:5336–47, http://dx.doi.org/10.1109/TIP.2013.2284070.C, Zheng G-Y, Chen Y-T, Lan K-C. Automatic tongue diagnosis using aphone. In: 2014 IEEE International Conference on Systems, Man, and Cyber-2014.

X, Fu H, Yang J, Wang W. Automatic segmentation in tongueby mouth location and active appearance model. In: 8th IEEE

mp Dependable, Auton Secur Comput DASC 2009. 2009:413–7,dx.doi.org/10.1109/DASC.2009.118.J, Zhang D, Wu C, Yue F. Automatic tongue image segmentationon gradient vector flow and region merging. Neural Comput Appl1:1819–26, http://dx.doi.org/10.1007/s00521-010-0484-3., Du J, Ding C. A comparative study of contemporary color tongueextraction methods based on HSI. Int J Biomed Imaging 2014;2014,

dx.doi.org/10.1155/2014/534507.F, Du JQ, Zhang K, Ding CH. A novel approach for tongue image extractionon combination of color and space information. In: 2009 3rd Inter-

al Conference on Bioinformatics and Biomedical Engineering. 2009:1–4,dx.doi.org/10.1109/ICBBE.2009.5162213.C, Shi D. A prior knowledge-based algorithm for tongue body seg-tion. In: Proceedings – 2012 International Conference on Computer

and Electronics Engineering, ICCSEE 2012, vol 2. Hangzhou. 2012:646–9,dx.doi.org/10.1109/ICCSEE.2012.11., Chen Y, Lo L, Chiang JY, Calibration AC. Automatic tongue fea-xtraction. In: 2010 International Comput Symp, ICS2010. 2010:936–41,dx.doi.org/10.1109/COMPSYM.2010.5685377.iang D, Yan-Sheng L, Ming-Feng Z, Kang Z, Cheng-Hua D. A novelhm of color tongue image segmentation based on HSI. In: BioMedicalering and Informatics: New Development and the Future – Proceedings ofInternational Conference on BioMedical Engineering and Informatics, BMEIol. 1. Sanya. 2008:733–7, http://dx.doi.org/10.1109/BMEI.2008.271.Hu S, Wang S, Xu S. Towards the objectification of tongue

sis: Automatic segmentation of tongue image. In: Industrial Elec-, 2009. IECON’09. 35th Annual Conference of IEEE. 2009:2121–4,dx.doi.org/10.1109/IECON.2009.5415334., Wang B, Zhang Z, Lin F, Ma Y. Research on techniques of multifea-xtraction for tongue image and its application in retrieval. Comput Math

ds Med 2017;2017:8064743, http://dx.doi.org/10.1155/2017/8064743., Choi W, Yeo I, Nam D. Comparative analysis of tongue indicesen patients with and without a self-reported yin deficiency: asectional study. Evid-based Complem Altern Med 2017;2017:1279052,dx.doi.org/10.1155/2017/1279052.abe T, Kamarudin ND, Ooi CY, Kobayashi F, Mi X, Sekine M, et al. Quan-on of tongue colour using machine learning in Kampo medicine. Eur JMed 2016, http://dx.doi.org/10.1016/j.eujim.2016.04.002.B, Zhang D, Wang K. Tongue image analysis for appen-

diagnosis. Int J Sci Eng Technol Res 2015;4:2263–8,dx.doi.org/10.1016/j.ins.2005.01.010., Wang D, Liu Y, Gao X, Shang H. A novel automatic tongue image seg-tion algorithm: color enhancement method based on L*a*b* color space.5 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).90–3, http://dx.doi.org/10.1109/BIBM.2015.7359818.X. Label Distribution Learning. IEEE Trans Knowl Data Eng347:1–14, http://dx.doi.org/10.1109/TKDE.2016.2545658.C, Kim K, Ku B, Kim J. Trends in tongue color of sub-patterns on deficiency syndrome. Integr Med Res 2015;4:29,dx.doi.org/10.1016/J.IMR.2015.04.356.

F, López A, Pena-Méndez EM, Vanhara P, Hampl A, Havel J. Arti-eural networks in medical diagnosis. J Appl Biomed 2013;11:47–58,

dx.doi.org/10.2478/v10136-012-0031-x.a DC, Roupar DB, Ramos JC, Tavares AC, Lima CS. Automaticbowel tumor diagnosis by using multi-scale wavelet-based analy-wireless capsule endoscopy images. Biomed Eng Online 2012;11:3,dx.doi.org/10.1186/1475-925X-11-3.u B, Ban X, Tai P, Ma B. A tongue image segmentation methodon enhanced HSV convolutional neural network. In: Luo Y,

. Cooperative Design, Visualization, and Engineering: 14th Interna-Conference, CDVE 2017. Mallorca, Spain, September 17–20, 2017,

dings. Cham: Springer International Publishing; 2017:252–60,dx.doi.org/10.1007/978-3-319-66805-5 32.g V, Westland S, Connah D, Ripamonti C. A comparative study

characterisation of colour cameras by means of neural net-and polynomial transforms. Color Technol 2004;120:19–25,

dx.doi.org/10.1111/j.1478-4408.2004.tb00201.x.L, Zhang P, Qu P, Peng Y, Zhang J, Li X. A K-PLSR-basedcorrection method for TCM tongue images under differ-illumination conditions. Neurocomputing 2016;174:815–21,

dx.doi.org/10.1016/j.neucom.2015.10.008.

, Du J, Zhang K, He Y, Ding C. Research of tongue color recognition basedANN with self-adaptive network structure. In: Proceedings – 2015 7thtional Conference on Information Technology in Medicine and Education,015. 2016:208–11, http://dx.doi.org/10.1109/ITME.2015.13.

56 searc

70. Gui M,tion ba(BMEI

71. Deng Yin imahttp://

72. Dengand pInt Syhttp://

73. CannyMach I

74. Kanawframewnation2011:1

75. Shi Mmentahttp://

76. Xu WmatictongueHealthhttp://

77. Kass MVis 198

78. Zhai XniqueInformhttp://

79. Shi MlizingInternahttp://

80. FaundTonguencehttp://

81. Wu J,ditionaConferneerinhttp://

82. Eddinsmathw

83. Madkofor ac2007;3

u J,romeelevattp://

Zhaoon ma

ttp://uangetwe

retttp://im J,

ionalttp://alldoudbj

isks attp://unjuiseasci 201be S,

rationhttp://

hangbetes2017;2Mengbased2017;2Lo L, csis indhttp://Gabhaand giBourodecisio2014;5Xu Dapp/itOtsu NSyst MIni R. TongueDx: a tongue diagnosis system for personal health care on smart-

M.H. Tania et al / Integrative Medicine Re

Zhang X, Hu G, Zhang C, Zhang Z. A study on tongue image color descrip-sed on label distribution learning. In: 8th Int Conf Biomed Eng Informatics2015). 2015:148–52, http://dx.doi.org/10.1109/BMEI.2015.7401490., Manjunath BS. Unsupervised segmentation of color-texture regionsges and video. IEEE Trans Pattern Anal Mach Intell 2001;23:800–10,dx.doi.org/10.1109/34.946985.YDY, Kenney C, Moore MS, Manjunath BS. Peer group filteringerceptual color image quantization. In: ISCAS’99 Proc 1999 IEEEmp Circuits Syst VLSI (Cat. No. 99CH36349), vol. 4. 1999:21–4,dx.doi.org/10.1109/ISCAS.1999.779933.J. A computational approach to edge detection. IEEE Trans Pattern Analntell 1986;8:679–98, http://dx.doi.org/10.1109/TPAMI.1986.4767851.ong R, Xu D. An automatic tongue detection and segmentationork for computer-aided tongue image analysis. In: 2011 IEEE 13th Inter-

al Conference on E-Health Networking, Applications and Services. IEEE.89–92, http://dx.doi.org/10.1109/HEALTH.2011.6026741.J, Li GZ, Li FF. C2G2FSnake: automatic tongue image seg-

tion utilizing prior knowledge. Sci China Inf Sci 2013;56:1–14,dx.doi.org/10.1007/s11432-011-4428-z., Kanawong R, Xu D, Li S, Ma T, Zhang G, et al. An auto-tongue detection and segmentation framework for computer-aided

image analysis. In: 2011 IEEE 13th International Conference on E-Networking, Applications and Services, HEALTHCOM 2011. 2011:189–92,dx.doi.org/10.1109/HEALTH.2011.6026741., Witkin A, Terzopoulos D. Snakes: active contour models. Int J Comput8;1:321–31, http://dx.doi.org/10.1007/BF00133570.M, Lu HD, Zhang LZ. Application of image segmentation tech-in tongue diagnosis. In: Proceedings – 2009 International Forum onation Technology and Applications, IFITA 2009, vol. 2. 2009:768–71,dx.doi.org/10.1109/IFITA.2009.130., Li GZ, Li F, Xu C. A novel tongue segmentation approach uti-double geodesic flow. In: ICCSE 2012 – Proceedings of 2012 7thtional Conference on Computer Science and Education. 2012:21–5,dx.doi.org/10.1109/ICCSE.2012.6295018.ra MR, Ratna D, Yang X, Liu C, Ringenberg J, Deo M.e segmentation using active contour model. In: IOP Confer-Series: Materials Science and Engineering, vol. 755. 2016:11001,dx.doi.org/10.1088/1742-6596/755/1/011001.Zhang Y, Bai J. Tongue area extraction in tongue diagnosis of tra-l chinese medicine. In: Conference Proceedings: Annual International

ence of the IEEE Engineering in Medicine and Biology Society. IEEE Engi-g in Medicine and Biology Society Conference, vol. 5. 2005:4955–7,dx.doi.org/10.1109/IEMBS.2005.1615586.

84. Xdrh

85.

h86. H

bah

87. Kth

88. HGrh

89. KdS

90. A

91. Z

92.

93.

94.

95.

96.

97.

98.

S. Cell segmentation. MathWorks; 2006. Website: http://blogs.

orks.com/steve/2006/06/02/cell-segmentation [accessed 08.02.17].ur a, Hossain Ma, Dahal KP, Yu H. Intelligent learning algorithmstive vibration control. IEEE Trans Syst Man Cybern C: Appl Rev7:1022–33, http://dx.doi.org/10.1109/TSMCC.2007.900640.

phoneAH’14.

99. Xu Y,Recogn

h 8 (2019) 42–56

Xu Z, Lu P, Guo R, Yan H, Xu W, et al. Classifying syn-s in Chinese medicine using multi-label learning algorithm withnt features for each label. Chin J Integr Med 2016;22:867–71,dx.doi.org/10.1007/s11655-016-2264-0.

Y, He L, Liu B, Li J, Li F, Huo R, et al. Syndrome classification basednifold ranking for viral hepatitis. Chin J Integr Med 2014;20:394–9,dx.doi.org/10.1007/s11655-013-1659-4.

Y, Sun M, Hsu P, Chen Y, Chiang JY, Lo L. The relationshipen ischemic stroke patients with and without retroflex tongue:rospective study. Evid-based Complem Altern Med 2017;2017,dx.doi.org/10.1155/2017/3195749.Kim H, Kim KH. Effects of Bu-Zhong-Yi-Qi-Tang for the treatment of func-dyspepsia: a feasibility study protocol. Integr Med Res 2017;6:317–24,dx.doi.org/10.1016/j.imr.2017.07.003.rsson BV, Bjornsson AH, Gudmundsson HT, Birgisson EO, Ludviksson BR,ornsson B. A clinical decision support system for the diagnosis, fracturend treatment of osteoporosis. Comput Math Methods Med 2015;2015,dx.doi.org/10.1155/2015/189769.nninair AP. Clinical decision support system: risk level prediction of hearte using weighted fuzzy rules and decision tree rules. Cent Eur J Comput2;2, http://dx.doi.org/10.2478/s13537-012-0007-7, 86-86.Ishihara K, Adachi M, Okuda K. Tongue-coating as risk indicator for aspi-pneumonia in edentate elderly. Arch Gerontol Geriatr 2008;47:267–75,dx.doi.org/10.1016/j.archger.2007.08.005.J, Xu J, Hu X, Chen Q, Tu L, Huang J, et al. Diagnostic method of dia-

based on support vector machine and tongue images. Biomed Res Int017:1–9, http://dx.doi.org/10.1155/2017/7961494.

D, Cao G, Duan Y, Zhu M, Tu L, Xu D, et al. Tongue images classificationon constrained high dispersal network. Evid-based Complem Altern Med017, http://dx.doi.org/10.1155/2017/7452427.

hien, Cheng TL, Chen YJ, Natsagdorj S, Chiang JY. TCM tongue diagno-ex of early-stage breast cancer. Complem Ther Med 2015;23:705–13,dx.doi.org/10.1016/j.ctim.2015.07.001.le B, Shinde M, Kamble A, Kulloli M. Tongue image analysis with colorst features for diabetes diagnosis. Int Res J Eng Technol 2017;4:523–6.uis A, Feham M, Hossain MA, Zhang L. An intelligent mobile basedn support system for retinal disease diagnosis. Decis Support Syst9:341–50, http://dx.doi.org/10.1016/j.dss.2014.01.005.. iTongue. iTunes; 2015. Website: https://itunes.apple.com/gb/

ongue/id998044356?mt=8 [accessed 30.12.16].. A threshold selection method from gray-level histograms. IEEE Trans

an Cybern 1979;9:62–6, http://dx.doi.org/10.1109/TSMC.1979.4310076.

s. In: Proceedings of the 5th Augmented Human International Conference,2014:405–7, http://dx.doi.org/10.1145/2582051.2582076.

Yang J-Y, Jin Z. A novel method for fisher discriminant analysis. Patternit 2004;37:381–4, http://dx.doi.org/10.1016/S0031-3203(03)00232-2.