IEEE TRANSACTIONS ON HUMAN-MACHINE … · 2015-10-31 · TSITSOULIS AND BOURBAKIS: A METHODOLOGY...

IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, VOL. 45, NO. 3, JUNE 2015 327

A Methodology for Extracting Standing HumanBodies From Single Images

Athanasios Tsitsoulis, Member, IEEE, and Nikolaos G. Bourbakis, Fellow, IEEE

Abstract—Segmentation of human bodies in images is a chal-lenging task that can facilitate numerous applications, like sceneunderstanding and activity recognition. In order to cope withthe highly dimensional pose space, scene complexity, and varioushuman appearances, the majority of existing works require com-putationally complex training and template matching processes.We propose a bottom-up methodology for automatic extractionof human bodies from single images, in the case of almost uprightposes in cluttered environments. The position, dimensions, andcolor of the face are used for the localization of the human body,construction of the models for the upper and lower body accordingto anthropometric constraints, and estimation of the skin color.Different levels of segmentation granularity are combined toextract the pose with highest potential. The segments that belongto the human body arise through the joint estimation of theforeground and background during the body part search phases,which alleviates the need for exact shape matching. The perfor-mance of our algorithm is measured using 40 images (43 persons)from the INRIA person dataset and 163 images from the “lab1”dataset, where the measured accuracies are 89.53% and 97.68%,respectively. Qualitative and quantitative experimental resultsdemonstrate that our methodology outperforms state-of-the-artinteractive and hybrid top-down/bottom-up approaches.

Index Terms—Adaptive skin detection, anthropometricconstraints, human body segmentation, multilevel imagesegmentation.

I. INTRODUCTION

EXTRACTION of the human body in unconstrained stillimages is challenging due to several factors, including

shading, image noise, occlusions, background clutter, the highdegree of human body deformability, and the unrestricted posi-tions due to in and out of the image plane rotations. Knowledgeabout the human body region can benefit various tasks, such asdetermination of the human layout ([1]–[4]), recognition of ac-tions from static images ([5]–[7]), and sign language recognition([8], [9]). Human body segmentation and silhouette extractionhave been a common practice when videos are available in con-trolled environments, where background information is avail-able, and motion can aid the segmentation through backgroundsubtraction. In static images, however, there are no such cues,and the problem of silhouette extraction is much more chal-lenging, especially when we are considering complex cases.

Manuscript received March 3, 2014; revised July 29, 2014 and December 24,2014; accepted January 4, 2015. Date of publication February 24, 2015; dateof current version May 13, 2015. This paper was recommended by AssociateEditor Y. Yuan.

The authors are with the Department of Engineering, Wright State Uni-versity, Dayton, OH 45435 USA (e-mail: [email protected]; [email protected]).

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/THMS.2015.2398582

Moreover, methodologies that are able to work at a frame levelcan also work for sequences of frames, and facilitate existingmethods for action recognition based on silhouette features andbody skeletonization.

In this study, we propose a bottom-up approach for humanbody segmentation in static images. We decompose the prob-lem into three sequential problems: Face detection, upper bodyextraction, and lower body extraction, since there is a directpairwise correlation among them. Face detection provides astrong indication about the presence of humans in an image,greatly reduces the search space for the upper body, and pro-vides information about skin color. Face dimensions also aid indetermining the dimensions of the rest of the body, according toanthropometric constraints. This information guides the searchfor the upper body, which in turns leads the search for the lowerbody. Moreover, upper body extraction provides additional in-formation about the position of the hands, the detection of whichis very important for several applications. The basic units uponwhich calculations are performed are super pixels from multi-ple levels of image segmentation. The benefit of this approachis twofold. First, different perceptual groupings reveal moremeaningful relations among pixels and a higher, however, ab-stract semantic representation. Second, a noise at the pixel levelis suppressed and the region statistics allow for more efficientand robust computations. Instead of relying on pose estimationas an initial step or making strict pose assumptions, we enforcesoft anthropometric constraints to both search a generic posespace and guide the body segmentation process. An importantprinciple is that body regions should be comprised by segmentsthat appear strongly inside the hypothesized body regions andweakly in the corresponding background. The general flow ofthe methodology can be seen in Fig. 1.

The major contributions of this study address upright and notoccluded poses.

1) We propose a novel framework for automatic segmenta-tion of human bodies in single images.

2) We combine information gathered from different levelsof image segmentation, which allows efficient and robustcomputations upon groups of pixels that are perceptuallycorrelated.

3) Soft anthropometric constraints permeate the whole pro-cess and uncover body regions.

4) Without making any assumptions about the foregroundand background, except for the assumptions that sleevesare of similar color to the torso region, and the lower partof the pants is similar to the upper part of the pants, westructure our searching and extraction algorithm based onthe premise that colors in body regions appear strongly

2168-2291 © 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

328 IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, VOL. 45, NO. 3, JUNE 2015

Fig. 1. Overview of the methodology. Face detection guides estimation of anthropometric constraints and appearance of skin, while image segmentation providesthe image’s structural blocks. The regions with the best probability of belonging to the upper body are selected and the ones that belong to the lower body follow.

inside these regions (foreground) and weakly outside(background).

II. RELATED WORK

We classify approaches for human body segmentation intothe following categories. The first includes interactive methods([10]–[14]) that expect user input in order to discriminate theforeground and background. Interactive segmentation methodsare useful for generic applications, and have the potential toproduce very accurate results in complex cases. However, sincethey rely on low-level cues and do not employ object-specificknowledge, they often require user input to guide their process,and are inappropriate for many real-world problems, where au-tomation is necessary. In general, this category differs from theother two, which are automatic and often task specific.

The second category includes top-down approaches, whichare based upon a priori knowledge, and use the image content tofurther refine an initial model. Top-down approaches have beenproposed ([15]–[17]) as solutions to the problem of segment-ing human bodies from static images. The main characteristic ofthese approaches is that they require high-level knowledge aboutthe foreground, which in the case of humans is their pose. Onemethod for object recognition and pose estimation is the pictorialstructures (PS) model and its variations ([3], [18]–[20]). In gen-eral, human body segmentation approaches based on PS modelscan deal with various poses, but they rely on high-level modelsthat might fail in complex scenarios, restricting the success ofthe end results. Besides, high-level inference is time consumingand, thus, these methods usually are computationally expensive.

Bottom-up approaches ([21] [22]) use low-level elements,such as pixels or superpixels [23], and try to group them intosemantic entities of higher levels. In [21], multiple levels ofa segmentation hierarchy are employed, and the algorithm in-volves a part searching step over all produced segments, whichis computationally expensive. In [24], foreground regions are

sampled using small masks, which is not sufficient to modelthe clothing in complex scenarios (nonuniform clothing, clut-tered background, different poses). In this methodology, wecombine cues from multiple levels of segmentation. In [25], thehuman body is assumed to be inside a large mask, but due tothe variability of human poses, this assumption often fails, andthe sampling may lead to unrecoverable errors. In this study,we propose a more refined searching process for the torso andlegs, where we try to find arbitrary salient regions in the regionsthat correspond to them. Salient regions are comprised of seg-ments that appear strongly inside the hypothesized foregroundand weakly in the background. By considering the foregroundand background conjunctively, we alleviate the need for exactmask fitting and dense searching, and we allow the masks to belarge according to anthropometric constraints so that they mayperform sufficient sampling in fewer steps. Pose estimation canbe considered as a higher level problem compared with bodysegmentation, and many prefer to use a bottom-up approach tofacilitate body part estimation and pose recognition [3].

Hybrid methodologies leverage the advantages of bottom-upand top-down approaches. Perceptual groupings from a bottom-up approach in many cases provide a good foundation to copewith the high number of poses. Shape templates of body parts ap-plied on image segments are a way of solving the segmentationpuzzle ([26]–[28]). In order to alleviate the need for numerousshape templates, [29]–[31] decompose the problem into upperand lower body estimation, similarly to ours, but employ fittingof an accurate torso model. In [29], two deformable models aredesigned to segment the human body on two-scale superpixels,and in [30], a coarse-to-fine strategy is used. Li et al. [29] ex-tend these concepts by combining a kinematic model and datadriven graph cut. Rauschert and Collins [32] present a Bayesianframework for jointly estimating articulated body pose, and thepixel-level segmentation of each body part is proposed. Theresults of the methodology are very promising, but to tacklethe complexity of the problem, even for standing positions, the

TSITSOULIS AND BOURBAKIS: A METHODOLOGY FOR EXTRACTING STANDING HUMAN BODIES FROM SINGLE IMAGES 329

proposed model needs to be optimized over a high number ofparameters.

III. FACE DETECTION

Localization of the face region in our method is performedusing OpenCV’s implementation of the Viola–Jones algorithm[33] that achieves both high performance and speed. The algo-rithm utilizes the Adaboost method on combinations of a vastpool of Haar-like features, which essentially aim in capturing theunderlying structure of a human face, regardless of skin color.Since skin probability in our methodology is learned from theface region adaptively, we prefer an algorithm that is based onstructural features of the face.

The Viola–Jones face detector is prone to false positive detec-tions that can lead to unnecessary activations of our algorithmand faulty skin detections. To refine the results of the algo-rithm, we propose using the skin detection method presentedin [34], and the face detection algorithm presented in [35]. Theskin detection method is based on color constancy and a mul-tilayer perceptron neural network trained on images collectedunder various illumination conditions both indoor and outdoor,and containing skin colors of different ethnic groups. The facedetection method is based on facial feature detection and lo-calization using low-level image processing techniques, imagesegmentation, and graph-based verification of the facial struc-ture.

First, the pixels that correspond to skin are detected usingthe method in [34]. Then, the elliptical regions of the detectedfaces in the image found by the Viola–Jones algorithm are evalu-ated according to the probabilities of the inscribed pixels. Morespecifically, the average skin probability of the pixels X ofpotential face region FRi , for each person i, is compared withthreshold TGlobalSkin (set empirically to 0.7 in our experiments).If it passes the global skin test (greater than TGlobalSkin), it isfurther evaluated by our face detector. If the facial features aredetected, then FRi is considered to be a true positive detection.

After fitting an ellipse in the face region, we are able to definethe fundamental unit with respect to which locations and sizesof human body parts are estimated, according to anthropometricconstraints. This unit is referred to as palm length (PL), becausethe major axis of the ellipse is almost the same size as thedistance from the base of the palm to the tip of the middle finger.Thus, our anthropometric model is adaptive for each person andinvariant to scale.

IV. MULTIPLE-LEVEL IMAGE SEGMENTATION

Relying solely on independent pixels for complicated infer-ence leads to propagation of errors to the high levels of imageprocessing in complex real-world scenarios. There are severaldifferent sources of noise, such as the digital sensors that cap-tured the image, compression, or even the complexity of theimage itself and their effect is more severe at the pixel level.A common practice to alleviate the noise dwelling at the pixellevel is the use of filters and algorithms that extract collectiveinformation from pixels. Moreover, groups of pixels expresshigher semantics. Small groups preserve detail and large groups

Fig. 2. Image segmentation for 100, 200, and 500 superpixels.

tend to capture shape and more abstract structures better. Fi-nally, computations based on superpixels are more efficient andfacilitate more flexible algorithms.

In this study, we propose using an image segmentationmethod, in order to process pixels in more meaningful groups.However, there are numerous image segmentation algorithms,and the selection of an appropriate one was based on the follow-ing criteria. First, we require the algorithm to be able to preservestrong edges in the image, because they are a good indicationof boundaries between semantically different regions. Second,another desirable attribute is the production of segments withrelatively uniform sizes. Studies on image segmentation meth-ods [36], [37] show that although these algorithms approachthe problem in different ways, in general, they utilize low-levelimage cues and, thus, their results cannot guarantee compliancewith the various and subjective human interpretations. Thus, wedeem this step as a high-level filtering process and prefer tooversegment the image; therefore, as not to lose detail. Regionsize uniformity is important because it restrains the algorithmfrom being tricked by oversegmenting local image patches ofhigh entropy (e.g., complex and high detailed textures) at theexpense of more homogeneous regions that could be falselymerged, although they belong to semantically different objects(e.g., human hand over a wooden surface with color similar toskin).

Mori et al. [28] use superpixels instead of single pixels andnormalized cuts [23] for segmenting the image. The method weadopt in this study is the entropy rate superpixel segmentation(ERSS) algorithm [38], which provides a tradeoff between ac-curacy and computational complexity. This approach is basedon optimizing an objective function consisting of two compo-nents: The entropy rate of a random walk on a graph and abalancing term. Results of the ERSS are shown in Fig. 2. Moreimportantly, we propose using multiple levels of segmentation,in order to alleviate the need for selecting an appropriate numberfor the regions to be created and combine information emanat-ing from different perceptual groupings of pixels. Although ourframework can accept any number of segmentation levels, wefind that two segmentation levels of 100 and 200 segments pro-vide accurate results. For the skin detection algorithm, a finersegmentation of 500 superpixels is used, because it manages todiscriminate better between adjacent skin and skin-like regions,and recover skin segments that are often smaller compared withthe rest image regions.


Fig. 3. Skin detection algorithm.

V. SKIN DETECTION

Among the most prominent obstacles to detecting skin regionsin images and video are the skin tone variations due to illumi-nation and ethnicity, skin-like regions and the fact that limbsoften do not contain enough contextual information to discrimi-nate them easily. In this study, we propose combining the globaldetection technique [39] with an appearance model created foreach face, to better adapt to the corresponding human’s skincolor (Fig. 3). The appearance model provides strong discrimi-nation between skin and skin-like pixels, and segmentation cuesare used to create regions of uncertainty. Regions of certaintyand uncertainty comprise a map that guides the GrabCut algo-rithm, which in turn outputs the final skin regions. False posi-tives are eliminated using anthropometric constraints and bodyconnectivity. An overview of the process can be seen in Fig. 4.

Each face region FRi is used to construct an adaptivecolor model for each person’s skin color. In this study, wepropose using the r, g, s, I, Cr, and a channels. In moredetail, r = R/(R + G + B), g = G/(R + G + B), and s =(R + G + B)/3; therefore, r and g are the normalized versionsof the R and G channels, respectively, and s is used instead ofb to achieve channel independence. Channels I, Cr, and a fromYIQ (or NTSC), YCbCr, and Lab colorspaces, respectively, arechosen because skin color is accentuated in them. The skincolor model for each person is estimated after fitting a normaldistribution to each channel, using the pixels in each FRi . Theparameters that represent the model are the mean values μij andstandard deviations σij for each FRi and channel j = 1 . . . 6for channels r, g, s, I, Cr, and a. Each image pixel’s probabilityof being a skin pixel is calculated separately for each channelaccording to a normal probability distribution with the corre-sponding parameters. We expect true skin pixels to have strongprobability response in all of the selected channels. The skinprobability for each pixel X is as follows:

PSkin i(X) =

6∏

j=1

N (X,μij , σij ). (1)

Fig. 4. Skin detection examples.

The adaptive model in general focuses on achieving a highscore of true positive cases. However, most of the time it is too“strict” and suppresses the values of many skin and skin-likepixels that deviate from the true values according to the derivedprobability distribution. At this point, we find that an influence ofthe skin global detection algorithm is beneficial because it aidsin recovering the uncertain areas. Another reason we chooseto extend the skin detection process is that relying solely onan appropriate color space to detect skin pixels is often notsufficient for real-world applications [40]. The two proposals arecombined through weighted averaging (with a weight of 0.25 forthe global model, and 0.75 for the adaptive model). The finestlevel of image segmentation is used at this point to characterizesegments as certain and probable background and foreground.For the certain foreground regions, however, only the pixels withsufficiently high probability in the adaptive model are used asseeds; therefore, as to control their strong influence. In order tocharacterize a region as probable background or foreground, itsmean probability of the combined probability must be above acertain threshold (empirically set to 0.2 and 0.3, respectively).Examples can be seen in Fig. 4.

VI. UPPER BODY SEGMENTATION

In this section, we present a methodology for extraction ofthe whole upper human body in single images, extending [40],which dealt with the case, where the torso is almost upright andfacing the camera. The only training needed is for the initial stepof the process, namely the face detection and a small training setfor the global skin detection process. The rest of the methodol-ogy is mostly appearance based and relies on the assumption thatthere is a connection between the human body parts. Processingusing superpixels instead of single pixels, which are acquired by


an image segmentation algorithm, yield more accurate resultsand allow more efficient computations.

The initial and most crucial step in our methodology is thedetection of the face region, which guides the rest of the process.The information extracted in this step is significant. First, thecolor of the skin in a person’s face can be used to match the restof his or her visible skin areas, making the skin detection processadaptive to each person. Second, the location of the face providesa strong cue about the rough location of the torso. Here, we dealwith cases, where the torso is below the face region, but withoutstrong assumptions about in and out of plane rotations. Third,the size of the face region can further lead to the estimation of thesize of body parts according to anthropometric constraints. Facedetection here is primarily conducted using the Viola–Jones facedetection algorithm for both frontal and side views. Since facedetection is the cornerstone of our methodology, we refine theresults of the aforementioned method using the face detectionalgorithm presented in [35].

Once the elliptical region of the face is known, we proceedto the foreground (upper body) probability estimation. To betterutilize the existing spatial and color relations of the image pixels,we perform multiple level oversegmentation and examine theresulting superpixels. We regard superpixels with color similarto that of the face region as skin and superpixels with colorsimilar to the regions inside torso masks as clothes. With respectto clothes, the size of face’s ellipse guides the constructionof rectangular masks for the foreground using anthropometricconstraints. Our basic assumption is that a good foregroundmask should contain regions that appear mostly inside the maskand not outside (background). In other words, we try to identify“islands of saliency,” in the aforementioned sense. As opposedto approaches based on pose estimation, we employ simpleheuristics to conduct a fast and rough torso pose estimation andguide the segmentation process.

The torso is usually the most visible body part, connectedto the face region and in most cases below it. Using anthro-pometric constraints, one can roughly estimate the size of thetorso and its location. However, different poses and head motionmake torso localization a challenging task, especially when as-sumptions about poses are relaxed. Instead of searching for theexact torso region or using complex pose estimation methods,we propose using a rough approximation of the torso mask inorder to identify the most concentrated island of saliency. Thiscriterion allows for fast inference about the torso’s size and lo-cation, while relieving the need for the complex task of explicittorso estimation, without sacrificing accuracy.

As discussed, different levels of segmentation give rise to dif-ferent perceptual pixel groupings, and each segment is describedby the statistics of its color distribution. In each segmentationlevel, each segment is compared with the rest and its similarityimage is created, depicting the probabilistic similarity of eachpixel to the segment. Similarly to the skin detection process,normal probability distributions according to the mean μi andstandard deviation σi of segment Si are estimated for each chan-nel j = 1, 2, 3 of the Lab color space, and the probability foreach image pixel belonging to this probability is calculated. Weestimate the final probability as the product of the probabilities

Fig. 5. Example of similarity images for random segments.

Fig. 6. Masks used for torso localization.

(1) in each channel separately. Example similarity images areshown in Fig. 5. The resulting image that depicts the probabilitysegment Si that is the same color as the rest of the segments isreferred to as the similarity image. Similarity images are gath-ered for all of the different segmentation levels l. Here, we usetwo segmentation levels in this stage of 100 and 200 superpix-els, because they provide a good tradeoff between perceptualgrouping and computational complexity

PSimIm l i(X) =

3∏

j=1

N (X,μij , σij ). (2)

Sequentially, a searching phase takes place, where a loosetorso mask is used for sampling and rating of regions accordingto their probability of belonging to the torso. Since we assumethat sleeves are more similar to the torso colors than the back-ground, this process combined with skin detection actually leadsto upper body probability estimation. The mask is used for suf-ficient sampling instead of torso fitting; therefore, it is estimatedas a large square with sides of 2.5PL, with the top most sidecentered with respect to the face’s center. In order to relax theassumptions about the position and pose of the torso, the maskis rotated by 30◦ left and right of its initial position (0◦) (see inFig. 6). By using a large square mask and allowing this degreeof freedom, we manage to sample a large area of potential torsolocations. By constraining its size according to anthropometricconstraints, we make the foreground/background hypothesesmore meaningful.

During the search process, the mask is applied to each sim-ilarity image and its corresponding segment is scored. LetTorsoMaskt be a binary image, where pixels are set to 1 (or“on”) inside the square mask and 0 (or “off”) outside so thatSimImli ∩ TorsoMaskt selects the probabilities of the similarityimage that appear inside the mask. Index t = 1, 2, 3 correspondsto a torso mask at angle −30, 0, or 30. Thus, (3) and (4) rateeach segment’s potential of belonging to the foreground andbackground, respectively, and (5) combines the two potentials


Fig. 7. Segments with potential of belonging to torso. (a), (b) For segmentationlevel 1 and 2 and torso mask at 0 ◦. (c), (d) For segmentation level 1 and 2 andtorso mask at 30 ◦. (e) (f) For segmentation level 1 and 2 and torso mask at−30 ◦.

Fig. 8. Aggregation of torso potentials shown in Fig. 7, for torso masks at 0 ◦,30 ◦, and −30 ◦.

in the form of a ratio as follows:

PFG(Stli) =|St l i |∑

SimImli ∩ TorsoMaskt (3)

PBG(Stli) =|St l i |∑

SimImli ∩ TorsoMaskt (4)

TorsoScore(Stli) =PFG(Stli)

PBG(Stli) + ε. (5)

In the end of the previous process, images showing the seg-ments’ potentials for each segmentation level and for each torsohypothesis are created (see Fig. 7). In order to select the torsomask that retrieves the most suitable segments, we accumulatethe results in each segmentation level for each torso mask. Werefer to the normalized result as the aggregated potential torsoimage. As seen in Fig. 8, the final potential maps accentuate suc-cessfully the torso and arms. Our approach has the advantagesof taking different perceptual groupings into account and beingable to alleviate the need for an accurate torso mask estimation,by conjunctively measuring the foreground and background po-tentials. The fact that we use superpixels in the computationsmakes comparisons more meaningful, preserves strong bound-aries, and improves algorithmic efficiency. Results may be im-proved by adding more segmentation levels and masks at dif-ferent sizes and locations, but at the cost of computational com-plexity. Here, we show how even with rough approximations,

Fig. 9. Thresholding of the aggregated potential torso images and final upperbody mask. Note that the masks in the top row are discarded.

we can achieve accurate and robust results without imposingcomputational strain.

The obvious step is to threshold the aggregated potential torsoimages in order to retrieve the upper body mask. In most cases,hands or arms’ skin is not sampled enough during the torsosearching process, especially in the cases, where arms are out-stretched. Thus, we use the skin masks estimated during the skindetection process, which are more accurate than in the case theywere retrieved during this process, since they were calculatedusing the face’s skin color, in a color space more appropriatefor skin and segments created at a finer level of segmentation.These segments are superimposed on the aggregated potentialtorso images and receive the highest potential (1, since the po-tentials are normalized).

Instead of using a simple or even adaptive thresholding, weuse a multiple level thresholding to recover the regions withstrong potential according to the method described, but at thesame time comply with the following criteria: 1) they form aregion size close to the expected torso size (actually bigger inorder to allow for the case, where arms are outstretched), and2) the outer perimeter of this region overlaps with sufficientlyhigh gradients. The distance of the selected region at threshold t(Regiont) to the expected upper body size (ExpUpperBodySize)is calculated as follows:

ScoreSize = e

−|Regiont −ExpUpperBodySize|ExpUpperBodySize (6)

where ExpUpperBodySize = 11 × PL2 . The score for thesecond criterion is calculated by averaging the gradient image(GradIm) responses for the pixels that belong to the perimeter(PRegiont) of Regiont as

ScoreGrad =1

|PRegiont |

|P Regiont |∑GradIm ∩ PRegiont .

(7)Thresholding starts with zero and becomes increasingly

stricter at small steps (0.02). In each thresholding level, thelargest connected component is rated, and the masks withScoreGrad > 0.05 and ScoreSize > 0.6 are accumulated to arefined potential image (see in Fig. 9). Incorporation of this a


priori knowledge to the thresholding process aids the accen-tuation of the true upper body regions (UBR). Accumulationof surviving masks starts when ScoreSize > 0.6 and resultingmasks after this point will keep getting closer monotonically tothe expected region size. Accumulation ends when ScoreSizedrops below 0.6. The rationale behind this process is to bothrestrict and define the thresholding range and focus the inter-est to segments with high potential of forming the upper bodysegment. The aggregate mask (AggregateMask) can now beprocessed easily and produce more meaningful results. Specif-ically, we set a final threshold, which allows only regions thathave survived more than 20% of the accumulation process inthe final mask for the UBR. This process is performed for everyinitial torso hypothesis; therefore, in the end, there are threecorresponding aggregate masks, out of which the one that over-laps the most with the initial torso mask and obtains the highestaggregation score is selected. The aggregation score shows howmany times each pixel has appeared in the accumulation pro-cess, implicitly implying its potential of belonging to the trueupper body segment.

A. Refinement

In many cases, the extracted upper body mask is very accurateand can be used as a final result. However, we choose to addan extra refinement step to cope with probable segmentation er-rors and pixels that manage to survive the multiple thresholdingprocess. One idea that we use here is to give the upper bodymask as input to an interactive foreground/background algo-rithm that requires “seeds” corresponding to the foreground andbackground. GrowCut and GrabCut are used for experiments.

GrowCut expects the RGB image as input and a map denot-ing the seeds for background, foreground, and uncertain pixels,whereas GrabCut can operate on a more refined map contain-ing the certain foreground, certain background, probable fore-ground, and probable background regions. In order to constructthese maps, we employ morphological operations on the up-per body mask, with adaptive square structural elements (SEs)according to anthropometric constraints. For GrowCut, the un-certain region is constructed by dilating the upper body maskwith a SE with sides equal to PL/6, the face’s ellipse with aSE with sides equal to PL/10 and the skin regions with a SEwith sides equal to PL/12. Possible holes between the face andtorso region are also filled. The certain foreground is similarlyconstructed with erosion instead of dilation, where the sides ofthe SEs are now PL/4, PL/4, and PL/10, respectively. The rest ofthe map is classified as background. For the GrabCut algorithm,the possible background ground is constructed by dilating theupper body mask, the face’s ellipse and skin masks using SEswith sides PL/10, PL/2, and PL/12, the probable foreground isconstructed by eroding the masks with SEs with sides PL/4,PL/4, and PL/10, respectively, and the certain foreground byeroding them with SEs with sides PL/1, PL/3, and PL/8, re-spectively. Both algorithms are guided by the extracted upperbody mask; therefore, their results are very similar. Their maindifference is that GrabCut can make better guesses in cases ofuncertainty and segment large regions loosely defined by the

Fig. 10. Example of foreground/background certainty maps and segmenta-tions for (a), (b) GrabCut and (c), (d) GrowCut.

map, whereas GrowCut is more sensitive to the map and moreinfluenced by background seeds. In Fig. 10, for example, bothalgorithms extract the upper body successfully, but GrowCut re-moves the small enclosed regions by the arms, whereas GrabCutincludes them.

VII. LOWER BODY EXTRACTION

The algorithm for estimating the lower body part, in orderto achieve full body segmentation is very similar to the one forupper body extraction. The difference is the anchor points thatinitiate the leg searching process. In the case of upper bodysegmentation, it was the position of the face that aided the es-timation of the upper body location. In the case of lower bodysegmentation, it is the upper body that aids the estimation ofthe lower body’s position. More specifically, the general crite-rion we employ is that the upper parts of the legs should beunderneath and near the torso region. Although the previouslyestimated UBR provides a solid starting point for the leg local-ization, different types of clothing like long coats, dresses, orcolor similarities between the clothes of the upper and lowerbody might make the torso region appear different (usuallylonger) than it should be. To better estimate the torso region,we perform a more refined torso fitting process, which doesnot require extensive computations, since the already estimatedshape provides a very good guide.

The expected dimensions of the torso are again calculatedbased on anthropometric constraints, but in a more accuratemodel. In addition, in order to cope with slight body defor-mations, we allow the rectangle to be constructed accordingto a constrained parameter space of highest granularity and di-mensionality. Specifically, we allow rotations with respect torectangle’s center by angle φ, translations in x- and y-axes, τx

and τy and scaling in x- and y-axes, sx and sy . The initial di-mensions of the rectangle correspond to the expected torso infull frontal and upright view and it is decreased during searchingin order to accommodate other poses. The rationale behind thefitting score of each rectangle is measuring how much it coversthe UBR, since the torso is the largest semantic region of theupper body, defined by potential upper body coverage (UBC),while at the same time covering less of the background region,defined by potential S (for Solidity). Finally, in many cases, therectangle needs to be realigned with respect to the face’s center(FaceCenter) to recover from misalignments caused by differentposes and errors. A helpful criterion is the maximum distance ofthe rectangle’s upper corners (LShoulder, RShoulder) from the


Fig. 11. Best torso rectangle with shoulder and beginning of the legs positions.

face’s center (Dsf ), which should be constrained. Thus, fittingof the torso rectangle is formulated as a maximization problem

θmaxf(θ) = α1 × UBC(θ) + α2 × S(θ) + α3 × Dsf (θ)(8)

where θ = (φ, τx , τy , sx , sy )

UBC(θ) =∑ TorsoMask(θ)∩UBR∑ TorsoMask(θ)

S(θ) =∑ TorsoMask(θ)∑ UBR

Dsf (θ) = e−|MaxD s f −1 . 5×PL|

1 . 5×PLwhere TorsoMask(θ) is the binary image, where pixels in-side the rectangle rTorsoMask(θ) are 1, else 0; UBR is thebinary image, where pixels inside the UBR are 1, else0; a1 , a2 , a3 are weights, set to 0.4, 0.5, and 0.1, respec-tively, and ; MaxDsf = max(d(FaceCenter, RShoulder(θ)) andd(FaceCenter, LShoulder(θ))) is the distance of the furthestshoulder point from the face’s center.

After finding the torso rectangle with the best score, we esti-mate the shoulder positions (top corners of the rectangle), andmore importantly, the waist positions (lower corners of the rect-angle). In turn, waist positions approximately indicate the begin-ning of the right and left leg legBR = (x, y) and legBL = (x, y),respectively. These points are the middle points of the line seg-ments of the waist points and the point in the center of the linethat connects them. Fig. 11 shows a case of a fitted torso and theaforementioned points.

Similarly to upper body extraction and the torso rectanglefitting case, we explore hypotheses about the leg positions us-ing rectangles by first creating rectangle masks for the upperleg parts and using them as samples for the pants color andfinally perform appearance matching and evaluate the result.The assumption we make here is that there is uniformity in thecolor of the upper and lower parts of the pants. In the case ofshort pants, where the lower leg parts are naked, the previouslycalculated skin regions are used to recover them. In order toreduce computational complexity, the size and position of theupper leg rectangles are fixed and adhering to anthropometricconstraints and the only free parameter is their angle of rotationwith respect to their center φright and φleft . Let LegMask(θ)be the binary mask for the two hypothesized leg parts, whereθ = (φright , φleft).

Every possible upper leg mask is used as a sample of the pantsregions, and the leg regions are estimated using the clothes and

Fig. 12. Example legs mask for φright = 0 and φle ft = 0.

Fig. 13. Example of foreground/background certainty maps and segmenta-tions for (a) and (b) GrabCut and (c) and (d) GrowCut.

skin detection process (1)–(5). An example mask can be seen inFig. 12. The hypothesized foreground is the pixels that belongto the leg mask, and background is the rest of the image plusthe pixels of the upper body mask, without the pixels below thewaist line segment (if any). The leg mask retrieved from eachhypothesis is the largest connected component of image seg-ments with color similar to the hypothesis and the skin regionsretrieved in the previous steps. There is no strong need for pre-cise alignment of the masks and the real leg parts, just enoughcoverage is pursued in order to perform a useful sampling. Thus,the algorithm can recover from slight torso misalignment andperforms well in cases of different leg positions, without im-posing the computational strain of dense searching using densemask parameters.

After the leg potentials are found, the same thresholding pro-cess as in the case of the upper body takes place, with thedifference that now the expected lower body size is used in(6) (ExpLowerBodySize instead of ExpUpperBodySize), whereExpLowerBodySize = 6 × PL2 . In order to construct the trimapof GrowCut to perform the refinement process for the leg re-gions, the leg mask is eroded by a square structuring element(SE) with side PL/4 followed by dilation by a SE with sidePL/5 in order to create the uncertainty mask, and for the certainforeground mask it is eroded using a SE with side PL/3. Fig. 13shows an example. In some cases, thin and ambiguous regionslike belts or straps might end up belonging to both the upper andlower body, or in the worst case the background. Most of thetime, however, the refinement of the upper and lower regionsis able to recover them, and during merging of the two regionsthey are included in the final outcome.

VIII. RESULTS

To evaluate our algorithm, we used samples from the publiclyavailable INRIA person dataset [41], which includes people


TABLE ISAMPLE RESULTS OF THE TESTED METHODOLOGIES

TABLE IIEVALUATION RESULTS FOR INRIA DATASET AND COMPARISON WITH

INTERACTIVE (1–5) AND AUTOMATIC (6–8) METHODS

performing everyday activities in outside environments inmostly upright position. This is a challenging dataset, sincethe photos are taken under various illumination conditions, inheavily cluttered environments and people appear in varioustypes of clothing. Our current implementation is based on C++and MATLAB functions and processing (including refinementsteps) lasts on average 4 min per image with size of 640 × 480pixels on a 2.4-GHz Intel Core 2 Duo machine.

We estimated the performance of our algorithm in segmenting43 not occluded persons in 40 images and compared the resultswith those of five generic algorithms (not designed specificallyfor human body segmentation): Proposals [42], GrabCut [12],the original version of GraphCut [14], geodesic star convexity(GSC) [13], and random walker (RW) [11]. Except for the pro-posals method, which is automatic, the rest of the algorithmsare interactive; therefore, they are provided with the foreground

and background seeds. For GraphCut, GSC and RW foregroundseeds are in the form of a straight line from the top of thehead to the tip of the shoe, trying to pass through the most im-portant regions of the body (hair, skin, and different types ofclothing), while background seeds are two lines almost enclos-ing the human body and covering the most important parts ofthe background along their path. For GrabCut foreground andbackground are estimated through a bounding box that tightlyenclosed the human body. Table I shows some samples of theimages used and the corresponding results of each algorithm.

The score EvalScore of each silhouette Ra extracted by thealgorithms is compared with ground truth silhouettes Rgt ac-cording to the formula:

EvalScore =∑

Ra ∩ Rgt∑Ra ∪ Rgt

(9)

where the ∩ and ∪ are the AND and OR operators, respectively.The results for each test are shown in Table II. The segmen-tations that our algorithm produces are accurate, with smoothboundaries most of the time and manage to preserve the skinregions, which are strongly correlated with body parts; there-fore, preserving them should be a priority. The proposals algo-rithm recognizes salient objects from using segments producedby graph cuts, where the seeds are estimated by a hierarchicalsegmentation method. The algorithm is fully automatic; how-ever, in cases like the ones we are interested in the human bodycannot be easily separated from the complex scenery, except fora few cases like in example 4 in Table I. GrabCut produces very


Fig. 14. Examples from the dataset in [43] (the outline of the segmented bodyis superimposed to the images to conserve space).

Fig. 15. Cases of poor segmentation.

good results, which are also appealing to the eye and do notrequire a lot of human effort when the bounding box version isused. On the other hand, in cases were the limbs are outstretchedor enclose background regions, the bounding box may containlarge background portions, which are treated as foreground andseverely harm the algorithm’s capability to segregate them fromthe foreground. GraphCut performs very well, in general, butin many cases it produces many false positives that lower itsscores (hence its high standard deviation value). As for exam-ple, in row 4 of Table I, where the foreground color distributionis adequately different than that of the background, it is theonly one that manages to discard the small regions enclosedby body limbs as opposed to the other algorithms, except forours. However, in complex scenarios there are cases, where theresults are not natural. GSC produces results comparable withGrabCut, and in a few cases, the best among the tested group.Experiments show that it usually has difficulties segmentingnonconvex objects with complex color distributions, which is ausual case with humans due to their articulation. Finally, RWalgorithm’s results are comparable with GrabCut’s, as well, andin a few cases better. RW’s ability to guess and complete edgesproves to be very powerful, but it often has the side effect ofproducing more rugged boundaries.

To further evaluate our method, we include the results of theworks of Li et al. [29], Lu et al. [30], and Li et al. [31], as

reported in [31], which have state-of-the-art performance onthe INRIA dataset. The results can be seen in Table II (6–8).Contrary to the previous generic methods (1–5 in Table II),these methods are designed to specifically solve the problemof human body segmentation in still images without interactionfrom a user. All three methods make use of segments providedby image segmentation and employ an accurate torso detectionstep. Li et al. [31] extend these concepts by incorporating akinematic model, and achieves the best accuracy among thethree methodologies, at the cost of increased computational load.The low scores of standard deviation indicate that these methodsare more stable than our method. However, our method hasthe highest mean accuracy, and demonstrates that even whensubstituting complex pose models with simple anthropometricconstraints performance can still remain high.

In addition to the INRIA dataset, where we demonstrate therobustness of the algorithm, we performed a test to verify the al-gorithm’s ability to segment bodies in more difficult poses. Theexperiment was conducted over 163 images from the “lab1” im-age set in [43]. In these images, the foreground and backgroundare simple, and the main challenges from a color distributionperspective are caused due to the background being similar tothe skin color. However, the actor performs various movements,and we are interested in whether our simple mask hypothesesare able to cope with them. The mean evaluation score reached97.68%, which indicates that the methodology can cope withvarious poses as well. Fig. 14 shows examples of different posesfrom the dataset.

IX. DISCUSSION

The first advantage of our methodology over those tested isthat it can automatically localize and segment the human body.Additionally, the final results achieve very good accuracy, evenin complex scenarios, and the small standard deviation showsthat it is stable. The main advantages of our method are as fol-lows. First, we combine cues from multiple levels of segmen-tation; therefore, to take into consideration different perceptualgroupings from coarse to fine. Second, during our searchingprocess, we try to find arbitrary salient regions that are com-prised by segments that appear strongly inside the (hypothe-sized) foreground rectangles and weakly outside. By consider-ing foreground and background conjunctively, we alleviate theneed for exact mask fitting and dense searching, and we allowthe masks to be large according to anthropometric constraintsso that they may perform sufficient sampling in fewer steps.Third, we demonstrate how soft anthropometric constraints canguide and automate the process in many levels, from efficientmask creation and searching to the refinement of the proba-bilistic map that leads to the final mask for the body regions.Searching for the upper and lower body parts, as well as thesimilar process of torso fitting, however, still remain one of themost computationally expensive steps of the methodology.

There remain prevailing complexities of the main steps. Oneof the important complexities is the complexity of the imagesegmentation algorithm. Here, we use the algorithm proposedin [38], which has an average complexity of O(n log n) and on


average requires 2.5 s for image size 481 × 321. Global andadaptive skin detection can be performed in almost real time, sothis step is performed quickly, excluding the superpixel creation.Face detection is also fast, since the main step [33] runs in realtime and significantly restricts the search space for [35]. Thegraph matching process described in [35] is generally computa-tionally expensive, but considering that it is only performed onthe produced regions of [33], it can be calculated quickly. Themost computationally demanding steps are those of mask val-idation during the upper and lower body extraction processes.More specifically, similarity image creation, as described inSection VI, requires the creation of n similarity images, eachof which requires n normal distribution fittings. This approachcan be viewed as part of scene understanding. Instead of calcu-lating similarities of each pixel to a segment using the normaldistribution, one could use the mean values of segments and per-form segmentwise comparisons. However, while this approachreduces the computational cost, it has an impact on accuracy forcomplex scenes high in texture and detail. The selected numberof segmentation levels and segmentation granularity provide atradeoff between accuracy and computational load. To decreasecomputational load, one can decrease the number of similar-ity images, since most computations use their content, by onlytaking into account superpixels with significant overlap withthe examined mask. Additionally, performing the evaluation ofhypothesized rectangles in parallel is straightforward.

One limitation of the methodology is that it requires the facesto be visible in profile and side views. We believe, however,that the face is a vast source of information that should be usedwhen available. Another limitation is that hair and shoes arenot accounted for explicitly, and there are cases, where they aremerged with the background if their appearance is more similarthan the pants. One way to alleviate this problem would be toadd masks for extreme parts of the limbs, which will essentiallyextend the existing foreground masks. Such a solution, how-ever, would impose additional computational complexity. Poorsegmentation may occur when one of the basic assumptions ofour methodology, namely that there is sufficient discriminationbetween the foreground and background, is violated. Fig. 15shows examples of poor segmentation. As far as the supportedposes are concerned, the methodology deals with almost uprightnatural poses, mostly because the range of degrees of freedomis restricted. For example, although the algorithms can produceaccurate results for different positions of the limbs, frontal, side,and in between body positions, sitting poses, and bent torso, itcannot guarantee success in lying, crawling, and curled positionsor extreme poses, as might occur in the case of gymnastics. Fi-nally, one of our assumptions is that bodies are not occluded bysmall objects. If an object is small enough and in close proxim-ity to the body, i.e., in case of a bag or satchel, it is expected thatit will be included in the final result.

X. CONCLUSION

We presented a novel methodology for extracting human bod-ies from single images. It is a bottom-up approach that combinesinformation from multiple levels of segmentation in order to

discover salient regions with high potential of belonging to thehuman body. The main component of the system is the face de-tection step, where we estimate the rough location of the body,construct a rough anthropometric model, and model the skin’scolor. Soft anthropometric constraints guide an efficient searchfor the most visible body parts, namely the upper and lowerbody, avoiding the need for strong prior knowledge, such as thepose of the body. Experiments on a challenging dataset showedthat the algorithm can outperform state-of-the-art segmentationalgorithms, and cope with various types of standing everydayposes. However, we make some assumptions about the humanpose, which restrict it from being applicable to unusual posesand when occlusions are strong. In the future, we intend to dealwith more complex poses, without necessarily relying on strongpose prior. Problems like missing extreme regions, such as hair,shoes, and gloves can be solved by incorporation of more masksin the search for these parts, but caution should be taken inkeeping the computational complexity from rising excessively.

REFERENCES

[1] M. Andriluka, S. Roth, and B. Schiele, “Pictorial structures revisited:People detection and articulated pose estimation,” in Proc. IEEE Conf.Comput. Vis. Pattern Recog., 2009, pp. 1014–1021.

[2] M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman,“The pascal visual object classes (VOC) challenge,” Int. J. Comput. Vis.,vol. 88, no. 2, pp. 303–338, 2010.

[3] V. Ferrari, M. Marin-Jimenez, and A. Zisserman, “Progressive searchspace reduction for human pose estimation,” in Proc. IEEE Conf. Comput.Vis. Pattern Recog., 2008, pp. 1–8.

[4] M. P. Kumar, A. Zisserman, and P. H. Torr, “Efficient discriminativelearning of parts-based models,” in Proc. IEEE 12th Int. Conf. Comput.Vis., 2009, pp. 552–559.

[5] V. Delaitre, I. Laptev, and J. Sivic, “Recognizing human actions in stillimages: A study of bag-of-features and part-based representations,” inProc. IEEE Brit. Mach. Vis. Conf., 2010.

[6] A. Gupta, A. Kembhavi, and L. S. Davis, “Observing human-object inter-actions: Using spatial and functional compatibility for recognition,” IEEETrans. Pattern Anal. Mach. Intell., vol. 31, no. 10, pp. 1775–1789, Oct.2009.

[7] B. Yao and L. Fei-Fei, “Grouplet: A structured image representation forrecognizing human and object interactions,” in Proc. IEEE Conf. Comput.Vis. Pattern Recog., 2010, pp. 9–16.

[8] P. Buehler, M. Everingham, D. P. Huttenlocher, and A. Zisserman, “Longterm arm and hand tracking for continuous sign language TV broadcasts,”in Proc. 19th Brit. Mach. Vis. Conf., 2008, pp. 1105–1114.

[9] A. Farhadi and D. Forsyth, “Aligning ASL for statistical translation usinga discriminative word model,” in Proc. IEEE Comput. Soc. Conf. Comput.Vis. Pattern Recog., 2006, pp. 1471–1476.

[10] L. Zhao and L. S. Davis, “Iterative figure-ground discrimination,” in Proc.17th Int. Conf. Pattern Recog., 2004, pp. 67–70.

[11] L. Grady, “Random walks for image segmentation,” IEEE Trans. PatternAnal. Mach. Intell., vol. 28, no. 11, pp. 1768–1783, Nov. 2006.

[12] C. Rother, V. Kolmogorov, and A. Blake, “Grabcut: Interactive foregroundextraction using iterated graph cuts,” ACM Trans. Graph., vol. 23, no. 3,pp. 309–314, Aug. 2004.

[13] V. Gulshan, C. Rother, A. Criminisi, A. Blake, and A. Zisserman,“Geodesic star convexity for interactive image segmentation,” in Proc.IEEE Conf. Comput. Vis. Pattern Recog., 2010, pp. 3129–3136.

[14] Y. Y. Boykov and M.-P. Jolly, “Interactive graph cuts for optimal boundary& region segmentation of objects in ND images,” in Proc. IEEE 8th Int.Conf. Comput. Vis., 2001, pp. 105–112.

[15] M. P. Kumar, P. H. S. Ton, and A. Zisserman, “Obj cut,” in Proc. IEEEComput. Soci. Conf. Comput. Vision Pattern Recog., 2005, pp. 18–25.

[16] S. Li, H. Lu, and L. Zhang, “Arbitrary body segmentation in static images,”Pattern Recog., vol. 45, no. 9, pp. 3402–3413, 2012.

[17] L. Huang, S. Tang, Y. Zhang, S. Lian, and S. Lin, “Robust human bodysegmentation based on part appearance and spatial constraint,” Neurocom-puting, vol. 118, pp. 191–202, 2013.


[18] P. F. Felzenszwalb and D. P. Huttenlocher, “Pictorial structures for objectrecognition,” Int. J. Comput. Vis., vol. 61, no. 1, pp. 55–79, 2005.

[19] D. Ramanan, “Learning to parse images of articulated bodies,” Adv. Neur.Inf. Process. Sys., pp. 1129–1136, 2006.

[20] M. Eichner and V. Ferrari, “Better appearance models for pictorial struc-tures,” in Proc. Brit. Mach. Vis. Conf., 2009.

[21] Y. Bo and C. C. Fowlkes, “Shape-based pedestrian parsing,” in Proc. IEEEInt. Conf. Comput. Vis. Pattern Recog., 2011, pp. 2265–2272.

[22] Z. Hu, G. Wang, X. Lin, and H. Yan, “Recovery of upper body poses instatic images based on joints detection,” Pattern Recog. Lett., vol. 30, no.5, pp. 503–512, 2009.

[23] J. Malik, S. Belongie, T. Leung, and J. Shi, “Contour and texture analysisfor image segmentation,” Int. J. Comput. Vis., vol. 43, no. 1, pp. 7–27,2001.

[24] M. Yao and H. Lu, “Human body segmentation in a static image withmultiscale superpixels,” in Proc. 3rd Int. Conf. Awareness Sci. Technol.,2011, pp. 32–35.

[25] Y. Hu, “Human body region extraction from photos,” in Proc. Mach. Vis.Appl., 2007, pp. 473–476.

[26] Z. Lin, L. Davis, D. Doermann, and D. DeMenthon, “An interactive ap-proach to pose-assisted and appearance-based segmentation of humans,”in Proc. IEEE 11th Int. Conf. Comput. Vis., Oct. 2007, pp. 1–8.

[27] P. Kohli, J. Rihan, M. Bray, and P. Torr, “ Simultaneous segmentation andpose estimation of humans using dynamic graph cuts,” Int. J. Comput.Vis., vol. 79, no. 3, pp. 285–298, 2008.

[28] G. Mori, X. Ren, A. A. Efros, and J. Malik, “Recovering human bodyconfigurations: Combining segmentation and recognition,” in Proc. IEEEComput. Soc. Conf. Comput. Vis. Pattern Recog., 2004, pp. II-326–II-333.

[29] S. Li, H.-C. Lu, X. Ruan, and Y.-W. Chen, “ Human body segmentationbased on deformable models and two-scale superpixel,” Pattern Anal.Appl., vol. 15, no. 4, pp. 399–413, 2012.

[30] H. Lu, G. Fang, X. Shao, and X. Li, “Segmenting human from photo im-ages based on a coarse-to-fine scheme,” IEEE Trans. Syst., Man, Cybern.B, Cybern., vol. 42, no. 3, pp. 889–899, Jun. 2012.

[31] S. Li, H. Lu, and X. Shao, “Human body segmentation via data-drivengraph cut,” IEEE Trans. Cybern., vol. 44, no. 11, pp. 2099–2108, Nov.2014.

[32] I. Rauschert and R. T. Collins, “A generative model for simultaneousestimation of human body shape and pixel-level segmentation,” in Proc.Eur. Conf. Comput. Vis., 2012, pp. 704–717.

[33] P. Viola and M. Jones, “Rapid object detection using a boosted cascade ofsimple features,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. PatternRecog., 2001, pp. I-511–I-518.

[34] P. Kakumanu, S. Makrogiannis, R. Bryll, S. Panchanathan, and N. Bour-bakis, “Image chromatic adaptation using ANNs for skin color adapta-tion,” in Proc. 16th IEEE Int. Conf. Tools Artif. Intell. Conf., 2004, pp.478–485.

[35] A. Tsitsoulis and N. Bourbakis, “A methodology for detecting faces fromdifferent views,” in Proc. IEEE 24th Int. Conf. Tools Artif. Intell., 2012,pp. 238–245.

[36] A. Tsitsoulis and N. Bourbakis, “An LG-graph-based early evaluationof segmented images,” Meas. Sci. Technol., vol. 23, no. 11, p. 114007,2012.

[37] H. Zhang, J. E. Fritts, and S. A. Goldman, “Image segmentation evaluation:A survey of unsupervised methods,” Comput. Vis. Image Understanding,vol. 110, no. 2, pp. 260–280, 2008.

[38] M.-Y. Liu, O. Tuzel, S. Ramalingam, and R. Chellappa, “Entropy rate su-perpixel segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog.,2011, pp. 2097–2104.

[39] C. O. Conaire, N. E. O’Connor, and A. F. Smeaton, “Detector adaptationby maximising agreement between independent data sources,” in Proc.IEEE Conf. Comput. Vis. Pattern Recog., 2007, pp. 1–6.

[40] M. C. Shin, K. I. Chang, and L. V. Tsap, “Does color space transformationmake any difference on skin detection?” in Proc. IEEE 6th Workshop Appl.Comput. Vis., 2002, pp. 275–279.

[41] N. Dalal and B. Triggs, “Histograms of oriented gradients for humandetection,” in Proc. IEEE Comput. Vision Pattern Recog., Comput. Soc.Conf., Jun. 2005, pp. 886–893.

[42] I. Endres and D. Hoiem, “Category independent object proposals,” in Proc.Eur. Conf. Comput. Vis., 2010, pp. 575–588.

[43] H. Jiang, “Human pose estimation using consistent max covering,” IEEETrans. Pattern Anal. Mach. Intell., vol. 33, no. 9, pp. 1911–1918, Sep.2011.

Athanasios Tsitsoulis (M’12) received the Diplomadegree from the Computer Engineering and Informat-ics Department, University of Patras, Patras, Greece,in 2010, and the Ph.D. degree from the Departmentof Computer Science and Engineering, Wright StateUniversity, Dayton, OH, USA.

He was a Member of the Assistive TechnologiesResearch Center. His research interests include com-puter vision, data analysis and modeling, and naturallanguage processing.

Nikolaos G. Bourbakis (F’96) received the Ph.D. de-gree in computer engineering and informatics fromthe University of Patras, Patras, Greece.

He is currently the OBR Distinguished Profes-sor of information technology with the EngineeringCollege, a J. A. Professor with the Geriatrics Depart-ment School of Medicine, the Director of the Assis-tive Technologies Research Center, Wright State Uni-versity, Dayton, OH, USA, and a Visiting ResearchProfessor with Ohio State University, Columbus OH,USA. His research interests include applied AI, ma-

chine vision, bioengineering, assistive technologies, cyber security.

IEEE TRANSACTIONS ON HUMAN-MACHINE … · 2015-10-31 · TSITSOULIS AND BOURBAKIS: A METHODOLOGY...

Documents

Transcript of IEEE TRANSACTIONS ON HUMAN-MACHINE … · 2015-10-31 · TSITSOULIS AND BOURBAKIS: A METHODOLOGY...