EMG-Based Human Machine Interface System By:...
Transcript of EMG-Based Human Machine Interface System By:...
EMG-Based Human Machine Interface SystemBy: Jonathan Morón, Thomas DiProva, and John CochraneAdvisors: Drs. In Soo Ahn and Yufeng Lu
Department of Electrical and Computer EngineeringBradley University5/2/2017
1
Overview
1. Introduction2. Objectives 3. System Design and Implementations4. Results and Discussion5. Project Management6. Conclusion and Future Work7. References8. Questions
2
IntroductionBackground: EMG Signals
● Electromyogram (EMG) signals are electric potential generated by muscles when activated by
the nervous system.
● Flexion of muscles produce EMG signals with a typical frequency range from 10 Hz to 500 Hz
● Raw EMG signals have a peak to peak voltage of -5 to +5 mV.
● Adhesive electrode pads are commonly used in recording EMG signals
Figure 1. Example of a recorded EMG signal 3
IntroductionRelated Work
● In 2015 Harvard made an assisted motion glove designed to give grip strength
and lifting support[1]
● Prosthetic limbs
● EMG from eye movement is used for HMI[2]
○ Path planning algorithm○ Feature detection through threshold○ Approximately 95.71% accuracy
● EMG is used to determine muscle damage [3]
● EMG is used to determine location of nerve damage [3]Figure 2. EMG soft glove design by Harvard[1]
4
Overview
1. Introduction2. Objectives3. System Design and Implementations4. Results and Discussion5. Project Management6. Conclusion and Future Work7. References8. Questions
5
ObjectivesThis project aims to develop a wearable EMG-based human machine interface system for in home assistance.
● The system shall read EMG signals from the user.● The system shall be calibrated to provide four control commands left, right,
forward, and stop based on EMG patterns from different movements.● The system shall send out commands wirelessly to control a service robot.● The system shall generate PWM signals for motor control.● The system shall provide live video for the purpose of monitoring.
6
Overview
1. Introduction2. Objectives 3. System Design and Implementations4. Results and Discussion5. Project Management6. Conclusion and Future Work7. References8. Questions
7
● Microcontroller (MCU) Board○ A low-power and small-size MCU board is preferred.○ It should have WiFi capability.○ 10 bit or above A/D converter is preferred.
● Wearable EMG sensor board○ A development muscle sensor board with documentation is preferred.○ It should provide EMG signal for interfacing.
● Microcomputer ○ It should have WiFi capability.○ It should host a webserver and provide live video feed.○ It should be able to generate Pulse Width Modulation (PWM) signals.
8
System Design and ImplementationsSystem Constraints/Design Considerations
System Design and ImplementationsSystem Block Diagram
9Figure 3. System Block Diagram
System Design and ImplementationsHardware: EMG Control Subsystem
● EMG Control Subsystem○ Particle Photon MCU Board○ MyoWare Muscle Sensor Board○ 3M Red Dot Electrodes○ Lithium ion battery
■ Output: 3.7v ■ Capacity: 2500mAH
10
System Design and ImplementationsHardware: Wheel Service Robot and Video Feedback
● Wheel Service Robot Subsystem○ Raspberry Pi 3 Model B○ Power Bank
■ Output: 5v, 2.4A ■ Capacity: 4000mAH
○ Logitech C260 webcam
● Video Feedback subsystem○ Computer
11
System Design and ImplementationsSystem Block Diagram (EMG Control System)
12Figure 4. EMG Control Subsystem
Figure 5. EMG Control Subsystem on John Cochrane 13
System Design and Implementations
MyoWare Muscle Sensor Board
Particle Photon MCU Board
Battery
System Design and ImplementationsMyoWare Muscle Sensor Board
● Output signal mode○ Raw EMG signal○ EMG signal envelope
■ Amplified■ Rectified■ Integrated
14
Figure 6. MyoWare Sensor Board
Figure 7. MyoWare Output Signals
System Design and ImplementationsMuscle Sensor Placement
● Sensor Placement○ Forearm Anterior
● Targeted muscles○ Brachioradialis○ Palmaris longus
Figure 9. Sensor Placement in Our Project (subject #1)
Figure 8.EMG Sensor Placements from Research Study[4]
15
System Design and ImplementationsParticle Photon
● STM32F205 ARM Cortex M3● 128 KB RAM● A/D converter
○ 12 bit resolution ○ 0.8 mV per level (Vref = 3.3 V)○ Maximum fadc = 30 MHz○ fadc clock Prescaler:(2 , 4 , 6 , 8)○ Minimum conversion time: 3 cycles
Figure 10. Particle Photon Board
Figure 11. ADC Timing Diagram 16
System Design and ImplementationsEMG Classification: Signal and Feature Extraction
17
Command 1
Command 2
Command 3
Command 4
Figure 12. Artificial Neural Network
Features derived from the input data
Recognized movement / command
Non-linear functions
System Design and ImplementationsEMG Classification: Artificial Neural Network
● Backpropagation
18
Figure 13. ANN Network Diagram: Backpropagation Algorithm
● Cost Function
Figure 14. Gradient Descent Graphical Representation
System Design and ImplementationsDescription of Pattern Recognition
1) In the MCU, the event of muscle movement is identified by thresholding. 2) After the occurrence of event, maximum 15000 samples (about 400 ms duration) are
captured to calculate the features for pattern recognition.3) After the training of Neural Network, the weight and bias of each neuron is determined
based on the optimization of cost function.4) The Neural Network structure with calibrated weight and bias is used for pattern
recognition.
19
System Design and ImplementationsSystem Block Diagram (Robot and Video Feedback)
20Figure 15. System Block Diagram
System Design and ImplementationsStinger Robot
● Used for System Validation○ Communication between subsystems○ PWM signal generation○ Webserver
● Previous experience
Figure 17. Service Robot: Top ViewFigure 16. Service Robot: Front View 21
Raspberry Pi 3 Transistor Circuit
Interface Board on Stinger Robot
System Design and ImplementationsRaspberry Pi 3 Model B
● Processor○ 1.2 GHZ quad-core ARM Cortex A53
● RAM○ 1 GB
● 40 GPIO Pins● 2 - PWM channels● WiFi
○ Network: 10/100 Mbps Ethernet○ 802.11n Wireless LAN
22
Figure 18. Raspberry Pi 3 Model B
System Design and ImplementationsPWM Signal Generation
● WiringPi Library (C Library)
○ GPIO interface library
○ PWM specific functions
● PWM output voltage
○ 5 V
23Figure 19. GPIO Pin Header
System Design and ImplementationsTransistor Circuit
24
System Design and ImplementationsVideo Feedback
25
Figure 21. Logitech C260
● Motion ○ Open source live video library ○ Frame rate: 30 FPS○ Resolution: 640 x 480
● Logitech C260
System Design and ImplementationsWireless Communication Chart
● Website○ Embedded Motion live video
○ User instructions
■ Movement configurations
○ Local Raspberry Pi IP address
26
Mobile Hotspot
Figure 22. Communications Network
Overview
1. Introduction2. Objectives 3. System Design and Implementations4. Results and Discussion5. Project Management6. Conclusion and Future Work7. References8. Questions
27
Results
Figure 23. EMG Signal: Wrist Movement Figure 24. EMG Signal: Fist Movement
Figure 26. EMG Signal: Middle Finger MovementFigure 25. EMG Signal: Ring Finger Movement
EMG Signals (Subject 1)
28
Results
Figure 28. EMG Signal: Fist Movement
Figure 30. EMG Signal: Middle Finger Movement
Figure 27. EMG Signal: Ring Finger Movement
Figure 29. EMG Signal: Wrist Movement29
EMG Signals (Subject 2)
ResultsPWM Output
30
Figure 31. PWM Output - Straight
Figure 32. PWM Output - Left Turn Figure 33. PWM Output - Right Turn
Results
● Fist○ Left Motor - on
● Wrist Flexion ○ Stop Motor
31
● Ring Finger○ Right Motor - on
● Middle Finger○ Straight - both motors on
Overview
1. Introduction2. Objectives3. System Design and Implementations4. Results and Discussion5. Project Management6. Conclusion and Future Work7. References8. Questions
32
Project ManagementDivision of Labor
Jon Morón1. Wheel Service Robot Subsystem 2. Video Monitoring System
a. Server
John Cochrane1. EMG-Control Subsystem
a. MyoWare Placementb. EMG DSP
Thomas DiProva1. EMG-Control Subsystem
a. EMG classificationb. Wireless Communication
33
Project ManagementSchedule
34
Project ManagementBill of Materials
35
Overview
1. Introduction2. Objectives3. System Design and Implementations4. Results and Discussion5. Project Management6. Conclusion and Future Work7. References
36
Conclusion
● A wearable EMG human machine interface system has been developed. ● The system has successfully acquired EMG signal. ● The system can calibrate and classify EMG signals from different movements.● Control commands have been sent wirelessly to control a service robot.● A web server has been implemented to provide video feedback.
37
Future Work
● A more powerful embedded system could be used to implement advanced
signal processing algorithms for pattern recognition.
● System sensitivity and accuracy could be further improved by including more
EMG sensors.
● A “real” service robot could be used to demonstrate practical applications.
38
Overview
1. Introduction2. Objectives 3. System Design and Implementations4. Results and Discussion5. Project Management6. Conclusion and Future Work7. References
39
References[1] Walsh, Connor J. Herman, Maxwell. Sanah, Siddharth. Galloway, Kevin C. Polygerinos, Panagiotis. “EMG Controlled Soft Robotic Glove For Assistance
During Activities of Daily Living.” Harvard Biodesign Laboratory, August 2015.
[2] Ferreira, Andre, Wanderley C. Celeste, Fernando A. Cheein, Teodiano F. Bastos-Filho, Mario Sarcinelli-Filho, and Ricardo Carelli. "Human-machine Interfaces
Based on EMG and EEG Applied to Robotic Systems." Journal of NeuroEngineering and Rehabilitation. BioMed Central, March 2008.
[3] Keating, Jennifer. "Relating Forearm Muscle Electrical Activity To Finger Forces." Worcester Polytechnic Institute, May 2014.
[4] Stergiou, Christos. "Neural Networks." Department of Computing. Imperial College London, September 2011.
[5]"What Is EMG (Electromyography) and How Does It Work?" IMotions, September 2016.
[6] Wu, Yunfen, María Ángeles Martínez Martínez, and Pedro Orizaola Balaguer. "Overview of the Application of EMG Recording in the Diagnosis and
Approach of Neurological Disorders."Overview of the Application of EMG Recording in the Diagnosis and Approach of Neurological Disorders | InTechOpen.
InTech, May 2013.
[7]Gil, Jamie Gomez, Israel San-Jose-Gonzalez, Luis Fernando Nicolas-Alonso, and Sergio Alonso Garcia. Steering a Tractor by Means of an EMG-Based
Human-Machine Interface. Tech. Department of Signal Theory, Communications and Telematics Engineering, University of Valladolid, July 2011.
40
Questions?
41