Tremor Detection UsingSmartphone-based Acoustic Sensing

Wei WangState Key Laboratory for NovelSoftware Technology, NanjingUniversityNanjing, 210023, chinaww@nju.edu.cn

Xun WangState Key Laboratory for NovelSoftware Technology, NanjingUniversityNanjing, 210023, china131220134@smail.nju.edu.cn

Lei XieState Key Laboratory for NovelSoftware Technology, NanjingUniversityNanjing, 210023, chinalxie@nju.edu.cn

Paste the appropriate copyright statement here. ACM now supports three differentcopyright statements:

• ACM copyright: ACM holds the copyright on the work. This is the historicalapproach.

• License: The author(s) retain copyright, but ACM receives an exclusivepublication license.

• Open Access: The author(s) wish to pay for the work to be open access. Theadditional fee must be paid to ACM.

This text field is large enough to hold the appropriate release statement assuming it issingle spaced in a sans-serif 7 point font.Every submission will be assigned their own unique DOI string to be included here.

AbstractIn this demo, we present a tremor detection application onsmartphones for early diagnosis of Parkinson’s disease.Our tremor detection scheme uses inaudible sound emit-ted by the built-in speaker on the smartphone. We measurethe movement of the hand by extracting the phase of soundwaves reflected by the hand. Our application can classifywhether the hand is static, moving, or trembling, without re-quiring the patient to hold the phone. The application canalso determine the intensity of the tremor by measuring pa-rameters such as the trembling frequency and the tremblingmagnitude.

Author KeywordsUltrasound; Tremor Detection; Smartphone; Device-free

ACM Classification KeywordsH.5.m [Information interfaces and presentation (e.g., HCI)]:Miscellaneous

IntroductionTremor is an important symptom associated with disorderssuch as Parkinson’s Disease (PD) or Alzheimer’s disease(AD). Tremor detection and classification are important forearly diagnosis of these disorders. Therefore, it is importantto develop a convenient way to monitor and record tremorbehaviors that can be used by anyone at any place.

Speaker

Microphone

Figure 1: Demonstration scenario

Existing tremor detection schemes either use Laser based[1], electromyography (EMG) based [2], or accelerometerbased measurements [3, 6]. Laser and EMG based tremordetection require specialized equipment that is only avail-able in hospitals and clinics. Accelerometer based schemesutilize the Inertial Measurement Unit (IMU) in smartphonesor smartwatches to detect hand tremor so that users canperform the test at almost everywhere. However, the pa-tient needs to hold the phone or wear a smartwatch whileperforming the test. The weight and pressure introducedby the smartphone/smartwatch may significantly impact thetesting results. Furthermore, it is difficult to detect intentiontremor and task-specific tremor when the user is holding asmartphone.

Start Test

Norm

al

Light

Severe

Tremor Detection

History

Figure 2: User interface design

Hand

Tracking

Sound

Reflection

Tremor

Detection

Static Moving

Movement

distance

Tremble

Frequency

Magnitude

Figure 3: Tremor detection systemdesign

In this demo, we present a tremor detection system thatuses acoustic signals emitted by smartphones, as illus-trated in Figure 1. Our tremor detection system is device-free so that users do not need to hold or wear any devices.They just need to put their hand/arm close to the mobilephone (within 30cm) to perform the test. Our system worksfor various use cases, including the case that the user isholding an object, such as a pen or a spoon.

Our hand movement tracking system is based on the LowLatency Acoustic Phase (LLAP) technology [5]1. By mea-suring the phase of sound waves reflected by the hand,LLAP provides millimeter level accuracy in movement dis-tances. In this way, our tremor detection application candetermine whether the hand is static, moving, or trembling.Our application also measures the frequency and magni-tude of the tremor. With these measurements, our appli-cation can classify the tremor to light and severe tremors,as shown in Figure 2. For example, the PD tremor is a typ-ical resting tremor that has frequencies of 4∼6Hz [1]. Toour best knowledge, this application is the first contact-freetremor detection scheme that can run on commercial smart-phones.

System DesignOur tremor detection system runs as an iOS application oncommercial iPhones without any hardware modifications.Our system contains two components: Hand tracking andtremor detection, as shown in Figure 3.

Hand TrackingThe hand tracking component uses sound wave reflectedby the hand to measure hand movements [5]. It first sendshuman-inaudible sound through the speaker of the smart-phone in eight frequencies from 16,800Hz to 21,700Hz,with a frequency interval of 700Hz. It then uses the built-inmicrophone to record the reflected sound with a samplingrate of 48kHz. After converting the received sound into thebaseband signal by mixing it with the sending frequencies,LLAP calculates the phase change caused by hand move-ments by subtracting the static signals caused by Line-Of-Sight path and reflections of static objects, such as wallsand tables. The key advantage of LLAP is that the phase

1The LLAP system has been presented and demonstrated in Mobi-Com 2016 [5].

0 2 4 6 8 10 12 14 16 18 20

Time (second)

-60

-40

-20

0

20

Dis

tan

ce (

mm

)

Normal Movements Trembling

Figure 4: Distance measurement for normal movement andtrembling

measurements lead to very accurate distance measure-ments since a distance change equal to the sound wave-length of 1.8cm introduces a phase change of 2π. There-fore, LLAP can track the hand movements with millimeteraccuracy. Furthermore, LLAP provides low latency distanceinformation with a small delay of 10.7 milliseconds. In thisdemo, we use our open source LLAP implementation whichis available at [4].

Tremor DetectionWe detect hand tremor using the high precision distancemeasurements provided by LLAP. Figure 4 shows the dis-tance output for normal movement and trembling. We ob-serve that normal movements have smooth distance changesover time. When the hand starts trembling after 12 sec-onds, we can clearly observe the vibrations of the hand.However, the tremor signals are mixed with hand movementtrends, as the user’s hand moves around slowly during thetrembling period.

We use a high-pass filter to remove the smooth movementsof the hand and extract the tremor signal. This high-passfilter contains two parts. The first part is a moving average

0 2 4 6 8 10 12 14 16 18 20

Time (second)

-5

0

5

10

Dis

tan

ce

(m

m)

Normal Movements Trembling

Figure 5: Distance measurement after high-pass filtering

estimator for the hand movement trend, which averages thedistance measurements over a time period of 200ms. Thesecond part extracts the tremor signal by subtracting thehand movement trend from the distance measurements.Figure 5 shows the tremor signal after the high-pass filterprocessing for the same sequence shown in Figure 4.

Our tremor detection component uses the variation of thefiltered tremor signal and the movement trend to determinewhether the hand is static, moving or trembling. Under thecase of a tremor, our system will measure both the trem-bling frequency and the trembling magnitude using the fil-tered tremor signal. For example, from Figure 5, the trem-bling frequency (which is about 7Hz) can be measured bycounting the peaks during a time period of one second. Wecan also obtain the trembling magnitude, which is around8mm (peak-to-peak) in this case. The application then de-termines the tremor intensity using these measurements.

Implementation and Demo SetupThe tremor detection application prototype is implementedusing Objective-C on the iOS platform. The application canbe installed on common iPhone models, such as iPhone6/6s/7.

Our application provides a convenient way to measure andrecord different types of tremors. The use cases of the ap-plication are listed in Table 1.

We will bring our smartphones to demonstrate the tremordetection application. The demonstration can be setupwithin 5 minutes.

Use Cases Method

Rest tremormonitoring

Recording tremoractivities multipletimes everyday forPD diagnosis.

Dailyactivitymonitoring

Performing dailyactivities, such aseating and writing,while putting thephone on the table.

Task-specifictremor

Measuring hand-writing tremor byholding a pen inthe hand.

Table 1: Use cases for tremordetection.

The conference attendees can interact with our demo in thefollowing ways.

• The conference attendees can have hands-on experi-ence on the application, as shown in Figure 1.

• To help the attendees to understand the underlyingtechnology of acoustic sensing, we will also demon-strate a simple gesture-based game, called “LLAPFly”.Interacting with this simple game will provide a chancefor to evaluate the sensitivity and robustness of ouracoustic sensing scheme.

• For interested attendees, we can also show a real-time baseband plot on our laptop. This will help theattendees to understand how the baseband phase isrelated to hand movements.

• We will also provide opportunities to install our ap-plications on the attendees’ smartphones so thatthey can interact with it on their own devices at anytime/place.

Conclusion and Future WorkIn this demo, we have shown the feasibility of device-freetremor detection based on sound signals. In the future, wewill conduct clinical research and develop tremor classifica-tion algorithms that help early diagnosis of different types of

disorders. Furthermore, we plan to use laser or accelerom-eter based systems to calibrate the tremor frequency andmagnitude measurements and further improve the accuracyof our measurements.

REFERENCES1. Anne Beuter, Michèle S Titcombe, François Richer,

Christian Gross, and Dominique Guehl. 2001. Effect ofdeep brain stimulation on amplitude and frequencycharacteristics of rest tremor in Parkinson’s disease.Thalamus & Related Systems 1, 3 (2001), 203–211.

2. Carmen Camara, Pedro Isasi, Kevin Warwick, VirginieRuiz, Tipu Aziz, John Stein, and Eduard Bakštein.2015. Resting tremor classification and detection inParkinson’s disease patients. Biomedical SignalProcessing and Control 16 (2015), 88–97.

3. Iván García-Magariño, Carlos Medrano, InmaculadaPlaza, and Bárbara Oliván. 2016. A smartphone-basedsystem for detecting hand tremors in unconstrainedenvironments. Personal and Ubiquitous Computing 20,6 (2016), 959–971.

4. Ke Sun, Wei Wang, and Alex X. Liu. 2017. LLAPopensource platform.https://github.com/Samsonsjarkal/LLAP/. (2017).

5. Wei Wang, Alex X. Liu, and Ke Sun. 2016. Device-FreeGesture Tracking Using Acoustic Signals. InProceedings of ACM MobiCom.

6. Daryl J Wile, Ranjit Ranawaya, and Zelma HT Kiss.2014. Smart watch accelerometry for analysis anddiagnosis of tremor. Journal of neuroscience methods230 (2014), 1–4.