Automatic Extraction of a Kidney Region by Using the Q-learning

Yoshiki Kubota

Yasue Mitsukura

Minoru Fukumi

INTRODUCTION

In these years, owing to aging population and Western style food, kidney disease patients are increasing.

Because CT image data has a huge quantity, the doctor needs a long time and a heavy labor for diagnosis.

INTRODUCTION

This paper regards a kidney contour extraction as a benchmark problem, a maze learning, used in the field of reinforcement learning, and a kidney outline is extracted by solving this maze.

MEDICAL IMAGES

Abdominal X-ray CT Image– Usually expressed as the DICOM image with 100

0 gradation levels from 0 to 1000.

Hounsfield Number– It expresses the X-ray wane coefficient in an orga

nization..– Water is set to 0 as a standard and bone is set to

1000.

Hounsfield Number

Contrast medium

THE FLOW OF THE PROPOSED METHOD

Step1: Backbone domain extraction– Pinpoint the position of the backbone.

Step2: Specification of rough kidney position– A rough kidney position is estimated using the

position of the backbone.

Step3: Contour extraction of the kidney– Q-learning is used within the rough kidney region.

Step1: Backbone domain extraction

binary image– Pixels with 230 or greater gray scale values in an image are

displayed as a binarized image.

Domain limitation– A diagonal line is drawn from a center in the image and only

pixels in the part below the diagonal line are displayed.

Labeling– labeling is performed, and a region with the largest area is e

xtracted.

Step1: Backbone domain extraction

Step2: Specification of rough kidney position

Q-learning

Q-learning is a reinforcement learning method that deals with the problem of learning to control autonomous agents is used.

Agent– A subject which solves a problem.

Environment– All things other than an agent.

The basic concept of Q-learning

An agent observes environment, takes an

action and receives the remuneration according to state changes.

Q value is updated by

Agent's action pattern

An agent can move to a pixel in neighboring eight, and has Q values (Q table) according to each moving direction.

Agent's action pattern

An agent chooses one of the following action methods.

– Probability ε : An agent moves along the greatest Q value direction.

– Probability (1- ε ): An agent moves randomly to the neighboring eight pixels.

An agent can move, if the pixel of an action place has a concentration value between T1 and T2.

The remuneration according to an action is received and Q value is updated.

Step3: Contour extraction of the kidney

The pixels along which the agent can pass are considered to be kidney edge.

Narrowed the gray scale value range of kidney edge’s pixels .

DYNAMIC GRAY SCALE VALUE REFINEMENT

STEP1:– As an initial state, a gray scale value range along which an agent

can pass is set up with T1 to T2. (*Initial state: 50 to 200) STEP2:

– When an agent reaches the goal, T1’ to T2’ which is a gray scale value range of the pixel along which it passed is evaluated.

STEP3:– If T1’ > T1 and T2’ < T2, the gray scale value range of the pixel al

ong which the agent in the next learning can pass is updated into T1’ to T2’.

STEP4: – STEP2 and STEP3 are repeated.

DYNAMIC GRAY SCALE VALUE REFINEMENT

Learning conditions:

The number of learning cycles:– 10,000 times

One learning cycle: – an agent reaches the goal or an agent moves for

100,000 steps

One step: – an agent can move to one of neighboring eight pi

xels

Learning coefficients

α (Learning coefficient):– 0.1

τ (Discount coefficient): – 0.9

ε (Probability witch random action takes place): – 0.5

Initial setting

A start, a goal (determined manually) Initial gray scale value range:

– 50-200

Reward conditions:

Arrival at the goal: 10 Not arrival at the goal: 0

Experiment image

Patient A: 15 images Patient B: 15 images Patient C: 11 images

Experiment result

The examples of success of extraction

The examples of failure

Consideration

The success example

The failure of training

Region extraction failure