Multidisciplinary Senior Design Conference Kate Gleason College of Engineering

Rochester Institute of Technology Rochester, New York 14623

Project Number: P14546

VIRTUAL REALITY DATA GLOVE

Corey Rothfuss – Project Lead

Rochester Institute of Technology

Rochester, NY, USA

Josh Horner – Mechanical Engineer

Rochester Institute of Technology

Rochester, NY, USA Kayla King – Mechanical Engineer Cody Stevens – Electrical Engineer Mathew Nealon – Electrical

Rochester Institute of Technology Rochester Institute of Technology Engineer Rochester, NY, USA

Rochester, NY, USA Rochester Institute of Technology

Rochester, NY, USA

Ryan Dunn – Electrical Engineer David Yoon – Electrical Engineer

Rochester Institute of Technology Rochester Institute of Technology

Rochester, NY, USA Rochester, NY, USA

Faculty Advisor Customer

Ed Hanzlik Dr. Gabriel Diaz

Rochester Institute of Technology Rochester Institute of Technology

Rochester, NY, USA Rochester, NY, USA

Copyright © 2014 Rochester Institute of Technology

CONTENTS

Contents ......................................................................................................................................................................................... 2

Abstract .......................................................................................................................................................................................... 2

Introduction (or background) ......................................................................................................................................................... 3

Process (or methodology) .............................................................................................................................................................. 3

The design ...................................................................................................................................................................................... 4

Benchmarking ................................................................................................................................................................................. 6

Budget and market analysis ........................................................................................................................................................... 7

Results and Recommendations ...................................................................................................................................................... 7

References .................................................................................................................................................................................... 11

ABSTRACT

Virtual Reality is used to simulate 3D environments using multiple cameras, sensors, and immersive

displays. It is also a growing value to the world of technology and for research. Currently calculating hand

movements to the virtual reality environments is used using motion tracking cameras. This can often result in poor

data because of positions of the hand that the cameras cannot see. The objective of this project is to create a glove

that collects data from the hand's movements without relying on motion capturing cameras where data can be

hindered by certain hand movement. The project will focus on providing a functional prototype that is lightweight,

durable, inexpensive, and does not interfere with the user's natural movements. This glove, when functioning,

should be a strong competitor for the current commercial models of virtual reality gloves.

Page 3

Copyright © 2014 Rochester Institute of Technology

INTRODUCTION (OR BACKGROUND)

Virtual reality is an exciting way for a user to experience 3D environments that may be difficult or dangerous

to replicate in the real world. Although many people see virtual reality as a fun way for gaming, it also has other

applications in research of the human body. Currently there is research being done on relating eye movements with

corresponding body movements. The current system for collecting this data uses cameras and sensors to relay the

information to the researcher for analysis. This works well for most of the body, but becomes problematic when

focusing the cameras' attention on the hands. Due to the many movements of the hands it is hard for the cameras

to see all of the sensors they need for accurate data.

PROCESS (OR METHODOLOGY)

As prescribed by the general senior design procedure, the entire first semester of the two-semester timeline

was a planning and design phase. Much of this planning involved input from the customer (namely, Dr. Gabriel

Diaz from the Center of Imaging Science) in order to draft a list of customer requirements and draw from that, a

list of engineering requirements. Input during this customer phase was also given by our always fearless and

optimistic guide, Ed Hanzlik, using his

experience with senior design teams and the

design process.

Table 1 shows the final customer

requirements that were reached after

multiple iterations. Through multiple

meetings and discussions with our customer

and guide, clear customer requirements

were laid out. The importance category on

the table is rated on a 1-3-9 scale with 9

being of the utmost importance and 1 being

the least critical. Twenty-four total customer

requirements were created, translating into

34 engineering requirements. The

engineering requirements, shown in table 2,

are technical specifications that need to be met in order to

ensure that the customer requirement was met. Table 1 – Customer Requirements

For example, if the question “how” was asked about a given customer requirement, the corresponding engineering

requirement would provide the answer the “how” question. For each engineering requirement, a marginal and

target goal was established. In order to make sure the engineering requirements are met, a detailed set of tests

and test plans were developed. And shown in the engineering requirements excel file located on the EDGE

website, http://edge.rit.edu/edge/P14546/public/Home.

Table 2 – Engineering Requirements

After creating the engineering requirements, potential risks were brainstormed and compiled in a Risk Analysis

Table. Analyzing and understanding these potential risks and drafting mitigation strategies are key steps of any

project. Since the Data Glove is to be used multiple times by many different people, safety and durability were a

key concern for us to evaluate in the risk analysis. In order to assign a more empirical value system to these risks,

an “importance score” was used. An importance score is produced by multiplying the risk’s severity by its

likelihood of happening. The severity and likelihood score used the same 1-3-9 scale that was used for the

customer and engineering requirements. Multiplying across a risk’s severity and likelihood gives us the hazard

score, which can then be used to determine its relative importance on a more empirical scale than completely

relying on assumptions.

THE DESIGN

The Glove

Since the design is focused on the hand and wrist, the main component of the assembly was done by

placing peripherals on a glove. The design originally started out as to be a custom created glove, the team decided

that it would be best to purchase a baseball glove. This glove not only is of high quality and durability, but could

also stretch in the case of larger hand sizes. Thus, we purchased an Adidas baseball glove was purchased that was

sized to a medium hand but was found through testing of multiple hand sizes that it worked well with other hand

sizes to deliver accurate results.

Knuckle Assembly

The “knuckle” assembly was made for a variety of reasons. Placing the

flex sensors used for data collection directly on the hand caused multiple issues

including but not limited to unsmooth bend radii of the flex sensor, twisting and

bending of the flex sensor due to curvature of the fingers, collision between

front and back sensor for the two sensors used on each finger, and overall

safety of the flex sensors to be more durable. What was created was using a

three-knuckle design for each finger which is shown in figure 1 (to the right).

The knuckle closest the palm and the knuckle at the end of the finger were

created to allow for the sensor to stay in place and the curved top allowed it

Figure 1 – Knuckle Design

Page 5

Copyright © 2014 Rochester Institute of Technology

to create a smoother bend radius to contribute to more accurate results. The middle knuckle has two slots so the

front flex sensor and the back flex sensor did not collide during movement and sliding of the flex sensor. Trying to

minimize total size of the glove to eliminate the pieces being cumbersome was a key consideration in the design.

All parts were 3D printed using the Brinkman Lab of Rochester Institute of Technology.

Microcontroller and Amplification PCB Assembly

The microcontroller and amplification PCB assembly serves to capture, process, and transmit the electrical

signals generated by the flex sensors. The Flexpoint sensors serve as varying resistances in a voltage divider. As the

sensor bends, the change in resistance results in a change in output voltage. The output voltage is used as the raw

electrical signal. To condition the signal, amplification is needed. The printed circuit board (PCB) serves as the

electrical groundwork and mounting point for all electrical components. The

components are made up of 5 dual operational amplifier packages and resistors. The

signal is amplified to utilize the full 0-3.3 VDC input range of the microcontroller.

The TM4c123GXL is the microcontroller used in the system. It is capable of sampling up

to 11 input signals. For the purposes of this project, only 9 are utilized. The code on the

microcontroller translates the measured signals into angles and transmits them to a

host computer. To accurately translate voltage to bend angles, a calibration step is

needed. The user simply needs to press switch #2 (SW2) on the microcontroller to

bring the system into the calibration routine, which is indicated by the LED changing

from green to blue. The calibration routine will capture the voltage values associate

with each angle captured and interpolate the data. This interpolation

will generate a polynomial approximation for the output characteristics of each

Flexpoint sensor. Figure 2 – PCB Layout

Wrist Assembly

Since the PCB board, microcontroller, and wiring all needed to be close to the hand for accuracy of the flex

sensor, a wrist assembly was made to hold all of the associated accessories necessary for data collection. Figure 3

below shows the front half of the wrist plate, it is about 3 inches wide and extends up to half way up the forearm.

The initial PCB board design was larger so a large wrist plate was needed but after iterations it turned out much

smaller than expected. Through empirical evidence, it was found that the large wrist plate was more secure when

the arm and hand were in motion which was crucial so the wires did not break apart. In figure 3, a “slot” is shown

in the front that allows all the wiring to be relocated into one spot to reduce excess wiring from being exposed

outside of the glove. Velcro slots were made and Velcro was used to wrap the arm around the wrist plate. All 3D

printing was done through Brinkman Lab at Rochester Institute of Technology.

Figure 2 – Wrist Assembly

BENCHMARKING

Research was an important part of this project because this is something that most of us have never came

across through school or through work experience. Learning how to be compatible with python so Dr. Diaz can use

the data for his VR software and all the associated accessories that would be necessary to construct the data glove.

Also, to find a reasonable and feasible requirements based on the budget of other companies and their research

capabilities was important. Table 3 below shows the benchmarking analysis of all the companies compared with

our engineering goals of the project.

Table 3 – Benchmarking analysis

Page 7

Copyright © 2014 Rochester Institute of Technology

BUDGET AND MARKET ANALYSIS

The design team was granted a budget of $1000, 50 percent of the budget was through the RIT

Multidisciplinary Design Fund and the final $500 was generously given to us by our customer, Dr. Diaz through the

CIAS department and his startup grant fund to help him with his research. However, due to the fact that the

design, build plans, and assembly instructions will be freely available to the public, the goal was to keep the system

cost as low as possible. This would increase the number of potential users which the glove can create a variety of

other uses as well. Market for Virtual Reality (VR) is slightly less than a 1 billion dollar business and projected to

grow with companies such as Oculus and Sony making a large impact lately. Having things such as the data glove

will increase help connect the eyes through the VR with the hands making it even more lifelike. Unfortunately for

us, competitors are charging tens of thousands of dollars for highly accurate gloves so keeping ours accurate and

affordable created quite a challenge. Nearly the entire budget was spent for various items, the most expensive

item being the flex sensors and iterations of the PCB boards. The team wanted to make the PCB board correct so in

the future; all repairs or improvements were to be low cost and easy to repair.

RESULTS AND RECOMMENDATIONS

This project was intended to give additional data

with the use of active camera markers to give accurate

results of finger and hand movements in the use of a Virtual

Reality environment. Through feasibility, correlating hand

movements were determined to be not a scope of this

project and only the finger data was to be recorded. This can

also be used for other scenarios.

All components were successfully created and

connected to the wrist assembly. Figure 3 shows the finished

product as a whole assembly. When plugged into a

computer, the glove can give real-time results of finger

movements through voltage differentials of the sensors. Figure 3 – Fully Assembled Glove

The flex sensors and lightweight glove assembly provides a cost efficient data glove that can be used if given the

proper training and calibration which is provided in an instruction manual.

Figure 4 – The Calibration Chart

Figure 4 shows the graph of the results of calibration. Calibration was accomplished by having the subject hold their hand steady at

three different fixed angles, 0°, 45°, and 90°. 0° was realized by the subject holding their hand against a flat surface. The other angles

were measured with a goniometer.

3200.00

3300.00

3400.00

3500.00

3600.00

3700.00

3800.00

3900.00

4000.00

0 10 20 30 40 50 60 70 80 90 100

Angle

Calibration Chart

Sensor 1

Sensor 2

Sensor 3

Sensor 4

Sensor 5

Sensor 6

Sensor 7

Sensor 8

Sensor 9

Page 9

Copyright © 2014 Rochester Institute of Technology

Sensor Number Voltage Calibration Equation Angle Calibration Equation

Sensor 1 y = -0.0215x2 + 6.6139x + 3431.7 X=1/430*(66139-(33886987321-8600000*y)^

(0.5))

Sensor 2 y = -0.0735x2 + 11.4x + 3221.8 X=(-20/147)*( (2692923-735*y)^(1/2)-570)

Sensor 3 y = 0.07x2 - 2.3378x + 3561.6 X=-1/700*(11689-(7000000*y-24794567279)^(1/2))

Sensor 4 y = -0.0558x2 + 9.6141x + 3288.9 X=1/372*(32047-(9183482209-2480000*y)^(1/2))

Sensor 5 y = 0.0397x2 - 0.4837x + 3691 X=1/794*((15880000*y-58589683431)^(1/2)+4837)

Sensor 6 y = 0.0102x2 + 1.7944x + 3299.2 X=2/51*((63750*y-205292951)^(1/2)-2243)

Sensor 7 y = 0.0449x2 + 0.0154x + 3555 X=1/449*(-77+(4490000*y-15961944071)^(1/2))

Sensor 8 y = 0.0538x2 - 1.5199x + 3295.2 X= (-1/1076)*(15199-(21520000*y-

70681694399)^(1/2))

Sensor 9 y = 0.007x2 + 6.0172x + 3368.6 X=1/35*((7)^(1/2)*(25000*y-51887593)^(1/2)-15043)

Table 4 – The Calibration Equations

From the calibration chart, a second order polynomial was used to linearize the equations. This is shown in the Voltage Calibration

Equation column of Table 4. These equations were then flipped so that an input of the micro controller tick was the input and a bend

angle was the output. These new equations are shown in the Angle Calibration Equation column of the same figure. These were what

were used to generate the bend angle graphs. It is worth mentioning that the direct equations for sensors 3 and 8 were giving the

negative angle of what was expected. To counteract this a negative sign was placed at the beginning of the equation. This, coupled

with the varying directions of initial curves imply strongly that the calibration method needs improvement. For testing, the following

procedure was utilized. The calibrated subject was asked to have their hand flat for two seconds, grasp the cylinder or sphere in

question for two seconds, and then put their hand back to being flat. This process was repeated for three trials per object. The objects

included four cylinders ranging from 3.5 cm up to 8.75 cm in diameter, and four spheres ranging in size from a golf ball to a whiffle ball.

Table 5 – Test Results of Grasping Cylinders

Figure 5 – Voltages of Sensors in Ticks

Shown in Table 5 are the results from the largest and smallest cylinders. Sensor 8 was omitted from the 8.75 cm cylinder as the results

show that the sensor was likely twisted or kinked, as after the initial grasp the output was stuck at roughly 26°. Sensor 3 had similar

though less pronounced problem as it briefly dropped to the base voltage when the subject’s hand was flattened. The voltages are

reported not as true voltages, but rather as the number of ticks given by the micro controller. This was done as there was no

conversion factor given by the data sheets and these are the numbers that the end user will be working with if they choose to use the

raw data. The ranges of ticks varied from sensor to sensor, but were up to 400 ticks for the 8.75 cm cylinder and up to 600 ticks for the

3.5 cm cylinder. Figure 5 shows the three bends in each trial and their average per sensor in graphical form. The 8.75 cm cylinder graph

is missing its last “trough” as that was sensor 8 and it did not give graphable results.

Table 6 – Mapping of Test Results

Figure 6 – Angles of Sensors

Each of the data points was converted via their calibration equation in Table 4 to its equivalent angle. These numbers were then

processed the same way as the voltage tick levels. For the 8.75 cm cylinder trial, sensors 3 and 8 are still the ones giving less than ideal

data. For the 3.5 cm cylinder trial none of the sensors were behaving unexpectedly. For the larger cylinder the highest inaccuracy was

in sensor 7 with 7.02°, and for the smaller cylinder the largest inaccuracy was in sensor 8 with 5.17°. However most of the sensors, with

a small sample size of three trials, were within 5° of the average at each constant amount of finger bending. Figure 6 shows the

graphical representation equivalent of Figure 5 with the standard deviations represented as error bars on the graph at each point. An

Page 11

Copyright © 2014 Rochester Institute of Technology

important note is that for sensors 2 and 4 (data points 1 and 3) on the 3.5cm cylinder actually exceed the limits of the calibration

equation and are slightly skewed by MS Excel’s inability to process imaginary numbers.

If given more time and resources, more research would be done towards different type of sensors for the

glove. The flex sensors from FlexPoint were the most accurate sensors to be used for the budget given. Through

looking at the benchmarking table, Cyberglove III, which is the most accurate glove cost around $15,000 to create.

So for only a $1000 budget we provided the most accurate results that we could. Also more time to record data

and test results for more repeatability results could have provided more accurate results.

REFERENCES

"Arduino Playground - Python." Arduino Playground - Python. Arduino, n.d. Web. 30 Sept. 2014.

Kessler, Drew, Larry Hodges, and Neff Walker. "Evaluation of the CyberGlove™ as a Whole Hand Input Device."

Https://smartech.gatech.edu/bitstream/handle/1853/3550/95-

05.pdf;jsessionid=9B43B2C34F54B10130D585D5BD2B6E5C.smart2?sequence=1. Georgia Tech, n.d.

Web. 10 Feb. 2014.

Klowery. "Sign Language Glove V1." Clloks.com. N.p., n.d. Web. 24 Feb. 2014.

"Navigation." Welcome · SensorWiki.org. SensorWiki, n.d. Web. 25 Jan. 2014.

"SPLINE Interpolation and Approximation of Data." SPLINE. Florida State University, n.d. Web. 15 Oct. 201