Multispectral Imaging DroneP17231Handoff Document____________________________________________________________________________

Table of Contents

1. Bill of Materials2. Manufacturing and Assembly Documents

a. Gimbalb. Electronics

3. Operator Manual4. Service Manual5. Software6. Recommendations for future work

1. Bill of Materials

Item No. Description Price/Unit Quantity Price Vendor Part Number

1SF11/C Rangefinder $279.00 1 $279.00 Parallax 28054

2 Rangefinder Mount $24.99 1 $24.99 Parallax 725-28920

3 Raspberry Pi $74.99 1 $74.99 Amazon B01C6Q2GSY

4Storm32 Gimbal Controller $29.99 1 $29.99 Amazon B011THSHIY

5 Video Capture Card $12.99 1 $12.99 Amazon B015J9LBTK

6Pixhawk Flight Controller $200.99 1 $200.99 Amazon B01HERRHM0

7 Gimbal Motors $49.99 3 $149.97 Amazon B01I6MU2SI

8 3 Cell LiPo Battery $32.98 1 $32.98 Amazon B0072AFARE

9Video Downlink Receiver $20.98 1 $20.98 Amazon B01H36AIGY

107-pin connector for Rangefinder $0.70 2 $1.40 Digikey WM10142-ND

116-pin connector for Pixhawk $0.19 12 $2.29 Digikey H2183-ND

1216-pin circular GPIO connector $35.89 1 $35.89 Digikey HR1430

13 DC power plug $4.01 1 $4.01 Digikey SC1149-ND

14Socket for Rangefinder $0.21 20 $4.12 Digikey WM10145CT-ND

15 Socket for Pixhawk $0.04 100 $3.83 Digikey DF13-2630SCF

16Carbon Fiber ~ 1/8" x 12" x 24" $120.00 1 $120.00 DragonPlate n/a

17Double Sealed Needle Bearing $11.92 1 $11.92 McMaster 5905K131

18

Aluminum Unthreaded Spacers $0.34 2 $0.68 McMaster 92510A424

19One-End Threaded Stud $4.74 1 $4.74 McMaster 97042A216

20 Backup 3S Battery $46.95 1 $46.95Performance

Hobbies n/a

21 XT-60 Connector $3.99 1 $3.99Performance

Hobbies n/a

22 FAA Test Cost $150.00 1 $150.00 n/a n/a

23 Spray Max 2K Clear $23.99 1 $23.99 Eastwood 50772Z

Aerosol

24 Two Part Epoxy $20.73 1 $20.73 MSC 30079340293534

25USB Extension Cable $3.99 3 $11.97 Newegg

N82E16812119338

26 Velcro $8.00 1 $8.00 NeweggN82E1684803556

6

27Waterproof SiRFIV GPS Rx $39.99 1 $39.99 robotshop RB-Usg-02

28 Female Housing Kit $14.17 1 $14.17 robotshop RB-Dfr-268

TOTAL $1,335.55

2. Manufacturing and Assembly

a. Gimbal

Required materials:

- ⅛” Carbon Fiber Plate (12x24”) x 1- Spray Max 2k Glamour Aerosol clear coat x 1 - Loctite E120-HP epoxy x 1- ¼ - 20 bolt (¾” length) x 1- 4-40 socket cap bolt (length) x 12- M3 socket cap bolt (8 mm length) x 24- iPower gimbal motors x 3

Carbon fiber will need to be machined into (qty) individual parts, the details of each is documented in the .pdf files attached.

CF_Sensor_Bracket_Rev5 x 1 CF_PitchBaseRev5 x 1CF_PitchConnection1 x 1CF_PitchConnection2 x 1Carbon_Tab x 26New_BracketConnection x 1New_BracketConnection_Bearing x 1Rib x 4Carbon_VertPlate_eng x 1Carbon_Yaw_Side_eng x 2Carbon_HorzPlate_eng x 1Carbon_AngPlat_eng x 1

The quality of these parts is paramount to proper alignment and fit-up of the finished product. In order to achieve the tolerances required for this, the Brinkman Lab water jet was used for all carbon fiber part production. The .dxf files can be used to submit for water jet machining.

Following the water jet, some additional sanding may be needed to remove burrs and ensure proper fit up of the mating parts. Assemble the gimbal per the assembly images below.

Figure 2.1: 3D modeled gimbal components

In order to permanently fix the gimbal design together, Loctite E120-HP epoxy was applied to all mating parts. Allow the epoxy to set for at least 24 hours. Afterwards, some additional sanding may be needed to smooth the surfaces. To finish the assembly, Spray Max 2k Glamour Aerosol clear coat was applied to all surfaces. The iPower motors are to be installed in the locations shown in the Gimbal Assembly with M3 socket cap bolts - four bolts to each side of the motors. The bolt pattern on the Mounting Plate may need to be modified to suit the correct model drone rails.

b. Electronics

Once the gimbal assembly has been completed please reference document Gimbal Plate Layout.pdf to begin to layout the electronic devices. The initial electronic devices required for this process will be:

● Raspberry Pi● Video Capture Card● Tetracam GPS● Pixhawk Flight Controller● Power Module

Note: The battery is shown in the Gimbal Plate Layout.pdf document, but is not necessary for placement on plate until the time of a flight.

Plate Layout1. Align the Raspberry Pi over the predrilled holes in the plate and fasten down using M2.5

screws and locknuts through the case. 2. For the Video Card, Tetracam GPS and Battery use velcro strips in relation to where the

components will lay per the drawing.3. Lastly, place the power module using velcro on the underside of the gimbal plate.

Note: Be mindful of the mounting screw for the yaw axis of the gimbal in the middle of the plate.

Gimbal Layout1. Align 1” M2.5 standoffs over predrilled holes on the back side of the gimbal and fasten

them down. 2. Place Storm32 Gimbal Control in its appropriate case and fasten down to standoffs

using M2.5 screws.Note: Do not overtighten screws due to the possibility of cracking the Storm32 Gimbal Controller case

3. Use rubber cement to mount camera IMU on the concave side of the pitch plate with plug side facing upNote: See gimbal configuration instructions to program IMU orientation into controller

4. Using range finder bracket fasten into the base of the gimbal arm where pre-drilled holes are using provided hardware from Lightware manufacturer

5. After the rubber cement has dried for camera IMU align the tetracam on the concave side of the pitch plate with the cameras facing down line the mounting holes up with the slots on plate

6. Use two M2.5 screws to fasten down the tetracam to the pitch plate.7. On the convex side of the pitch plate line up the mounting hole of the thermal camera

with the larger center slot of the plate and fasten down with a ¼-20 screw and straighten as much as possible using a level with camera facing down.

Wiring

For all wiring follow document Harnessing Layout Rev D.pdf for guidance. For wiring specifications use these guidelines:

● +16.5VDC Power - Red, 12AWG● -16.5VDC Power - Black, 12AWG● USB Communication/Power Black Cable● +5VDC Power - Red, 26AWG● +5VDC Power - Black, 26AWG● 5VDC Signal - White, 26AWG● Tx - Yellow, 26AWG● Rx - Orange, 26 AWG

Note: These specifications can and may be disregarded when using pre-harnessed materials (i.e. PixHawk GPS) or multiconductor cables.

To ensure the durability and longevity of all harnessing and wiring please be sure to use shrink tube around any exposed wiring when possible. In order to maintain a product that is easily maintenanced be sure to use crimp connections where possible and avoid using soldering connections. There will be some cases where soldering cannot be avoided however (i.e. tetracam multi I/O connector). Using the properly sized tools for crimping will also ensure the longevity of all connections.

3. Operator Manual

Operation of the gimbal and drone is fairly straightforward. Data collection missions will be planned both Universal Ground-Control Software (UGCS) and Mission Planner, both of which are free applications available for download on the internet. All flight of the flight planning, including choosing where to fly and how fast is done in UGCS. Flight planning for the data collection system is done in Mission Planner. In order for automated data capture to occur, both flight plans in both software must align within approximately 5m accuracy. For data capture to trigger properly, waypoints issuing data collection commands must issue the following commands with proper delays,

The trigger output for the thermal camera is set to Relay 0 in the Mission Planner software and Relay 1 is set to trigger the Tetracam captures. Care must be taken to plan flights in order to ensure quick and accurate data collection. Flight planning can be done remotely anywhere in

the world and the missions sent via email and uploaded to the drone through the mission planning software.

Upon arrival at the desired data collection site the system must be thoroughly checked to ensure proper functionality. Auxiliary gimbal battery must be plugged in, powering up the data collection system and gimbal. A series of beeps will be heard coming from the Pixhawk speaker. This is normal and indicates standard operation. If an alarm buzzing sound is heard, this means the auxiliary battery being used is low and must be replaced. Ensure the drone is rested in a flat space and not disturbed during the startup process as IMUs for both the gimbal and flight controller are being calibrated. Upon proper initialization the gimbal controller will be indicating the startup process has run successfully. The Pixhawk RGB LED in the center of the flight controller will flash a green color indicating that the Pixhawk has a 3D GPS lock and is ready for operation. The status LED on the Tetracam will also turn green indicating it is ready for image capture.

The sensors should be tested on the ground prior to the first flight to ensure functionality. This can be done using the Pixhawk transmitter. Toggle the CH8 switch on the controller to initiate a thermal video capture. The data storage space on the Pi should be checked to ensure a video file was properly written. Toggling the CH7 switch on the controller will initiate a tetracam capture. The green status LED will turn red for approximately half a second as the camera captures and stores its data to the onboard SD cards. Upon verification of functionality of the data collection system the standard setup procedure for the DJI S1000/S600 can be performed.

The gimbal can be tested by lifting up the drone using two of the motor arms on opposite sides and tilting it in various directions. The gimbal should operate properly and keep the imaging sensors facing the ground.

Once all the preflight checks have been completed, all that remains is the operator clicking go in UGCS to begin the flight. The drone will take off, fly, capture data, and land all autonomously. Care should be taken to monitor the flight status using the telemetry downlink in both mission planning softwares to ensure the flight is going as planned and there is enough battery life remaining. A registered FAA drone pilot should also be present with controller in hand to take over the automated flight should an issue arise.

Data can then be taken off the Pi and tetracam for later processing.

4. Service Manual

This portion of the document highlights potential problems that the user may have in the field and the solutions for those specific problems.

Gimbal* IMPORTANT * - When a solution for the problem is found and the o323BGCTool_v090 application was used to mitigate the issue, the user should select Store to EEPROM under the Settings dropdown menu to ensure all parameters were written to the board.

Problem: Gimbal cannot level itself during the initial start up.

Solution: 1. Ensure that all wires/cords and miscellaneous objects are not impeding the movement of the gimbal and the sensors and that the motors are connected to the board, along with external IMU.

2. Disconnect the power to the gimbal and verify that the resting position of the gimbal with the sensors connected is roughly in the position that is desired for data capture (level and pointing downwards).2.1. If the sensors are resting unleveled in the Roll direction then the

entire lower portion of the gimbal connected to the Roll motor needs to slide along along the slots into the proper position. To do this, loosen the screws that connect the Roll motor to the sensor plate slightly. Slide the entire lower portion of the gimbal to the

balanced position and re-tighten the screws.2.2. If the sensors are resting unleveled in the Pitch direction then their

positions need to be adjusted on the sensor plate. This is done by first slightly loosening the screws that hold the sensor to the sensor plate, sliding the sensor to the balanced position, and re-tightening the screws. If no position appears to be leveled adequately, counterweights must be used to reach a desired position. Note, if counterweights are used, the increased weight may lead to other issues covered in this section.

3. If these are not the cause of the issue then the next step would be to reset the gimbal. This is done by pressing the RESET button on the STorM32 controller mounted on the back of the gimbal.

4. If the problem persists the next step would be to connect the STorM32 to a laptop with a USB mini cable and run the o323BGCTool_v090 application. Select Read to connect to the board.

5. Under the Dashboard tab, ensure that the motors are listed as ACTIVE and not OFF. If the motors are OFF, disconnect the board from power and reconnect it. Click Read again to reconnect. If the motors are still OFF it is possible that the Battery Voltage is too low.

6. If the voltage level is below ~9.5V, the battery needs to be switched. If the battery is not the issue, check to ensure that there are no I2C Errors.

7. If all of the above are not the issue, it is possible that the voltage set for the motors is too low and they are struggling to get the gimbal into position. Please note that if the gimbal appears to be turning in the wrong direction while trying to level itself, the voltage level on each motor is not the problem and the user should move on to item number 7. Under the PID tab, set the voltage for the appropriate motor 10-15 units higher. Note, this may lead to other issues covered in this section.

8. Regardless of the state of the motors or Battery Voltage, if there are I2C Errors, this is most likely the cause of the issues. There are a number of possible reasons why that must be fixed in order for the gimbal to level itself.8.1. First, ensure that the external IMU wires are not near any motor or

sensor wires; the EM fields produced by the motors and/or sensors can cause interference in the I2C communication.

8.2. Next try twisting the external IMU wires.8.3. After performing the above I2C potential fixes, restart the gimbal

using the Tools dropdown menu.8.4. If there are still I2C Errors lower the voltage of the motors. Note,

this may fix the I2C Errors but can lead to other issues covered in this section.

8.5. Check for a loose pin(s) on the connection from the external IMU to the STorM32 board. The pin needs to be reset into place within the connector housing in order for the I2C errors to potentially go away.

8.6. If all of the option above were exhausted and the I2C Errors persist, it is very possible that there is an unfixable hardware issue with either the external IMU or the STorM32, in which case one or both may need to be replaced.

9. If the Motors are ACTIVE, the Battery Voltage is OK, and there are no I2C

Errors, go to the Gimbal Configuration tab and select the Configure Gimbal Tool on the left. This will deactivate the motors.9.1. On the right hand side of the window that pops up, deselect all of

the Step I check marks, the Yaw Motor Position check mark, and the Enable Motors and Restart Controller check mark.

9.2. Select Continue and run through the rest of the tool.9.3. The Gimbal should produce and audible *beep* and should be

ready to use.9.4. If the problem persists, it may be a calibration issue with the IMU,

in which case, please refer to the Calibrating IMUs section under the Appendix.

Problem: Gimbal is shaking uncontrollably.

Solution: 1. This error is caused by grossly incorrect PID values. In order to mitigate this please refer to the PID Tuning section of the Appendix.

Problem: Gimbal reaction time is slow and does not keep up with the movement of the drone.

Solution: 1. Ensure that all wires/cords and miscellaneous objects are not impeding the movement of the gimbal and the sensors and that the motors are connected to the board, along with external IMU.

2. Ensure that the Battery Voltage is not below ~9.5V. This can be done by disconnecting the battery and probing its positive and negative terminals with an multimeter.

3. If there are no impedances and the Battery Voltage is OK then the cause of the issue is most likely the PID and voltage/motor settings. In order to fix this please refer to the PID Tuning section of the Appendix. Note, adjusting for this problem may lead to another one listed under this section. It is possible that the user will bounce back and forth between two problems to fix the initial issue.

Problem: Gimbal balanced itself and beeped but the sensors are not level.

Solution 1. The offsets need to be adjusted. To do this, connect the board to a computer and run the o323BGCTool_v090 application. Press Read to connect to the board and under the Setup tab adjust the Rc Pitch Offset or Rc Roll Offset accordingly. At the bottom of the window select Write and continue to repeat this process until the sensors are leveled.

5. Software

Obtaining Data from Raspberry Pi:

The Raspberry Pi is responsible for capturing videos from the thermal camera and copying the files from the Tetracam. The output is one folder named “yyyymmdd-HHMMSS” where yyyy is the year, mm is the month, dd is the day, HH is the hour, MM is the minute, and SS is the second of when the data capture was initiated (e.g., 20170407-151222 is April 7, 2017 at 3:12pm). This timestamp is obtained via a GPS receiver.

The easiest way to pull data from the Raspberry Pi is to remote into it and copy the desired data to a PC. This can be done through the Verizon MiFi supplied by the UAS Laboratory in the College of Imaging Science. The Raspberry Pi will automatically connect to the Verizon MiFi once it is turned on. Check the connected devices on the Verizon MiFi to find the IP address of the Raspberry Pi. Next, open remote desktop on a Windows computer and type the IP address of the Raspberry Pi into the box labeled “Computer:” as seen below:

Open a file browser and navigate to: /home/pi/RaspberryPi/thermalVideo

Plug a flash drive into the Pi and copy the files over. Alternatively, you could use a cloud service such as Google Drive to upload the data from the Pi and then download it on a PC for post-processing.

Changing the duration of a thermal video capture:

Setting up a new Raspberry Pi:

If a new Raspberry Pi needs to be installed and configured, open a terminal and type:

sudo apt-get install subversion

svn co --username (e.g., rpm1652) https://edge.rit.edu/dav/web/public/RaspberryPi (it will prompt you for your password)

cd RaspberryPi/

sudo ./install_dependencies.sh

This will install all drivers, software dependencies, and make the necessary changes to the operating system such that the Raspberry Pi will be ready to capture data when it starts up.

6. Recommendations for Future Work

a. Gimbal Mounting i. In the future, more damping needs to be added in the interface between the

gimbal and DJI Matrice 600. The current, hard-mounted setup may contribute to blurriness of images and compromise the structural integrity of the gimbal.

ii. A crack appeared on the carbon fiber horizontal plate attached to the yaw section of the gimbal as a result of the crash. It is recommended that this plate be removed and replaced with a carbon fiber plate of the same dimensions (see Carbon_HorzPlate_eng in the Gimbal CAD Files).

b. Flight Controller Height Controli. Using the pixhawk as the main flight controller was originally desired, but due to

time constraints the idea was disregarded. If possible, the integration of the pixhawk as the main flight controller which includes automatic integration of the laser range finder to control the height of the drone, would result in an ideal data collection scenario.

Appendix

Calibration of the Storm32 Gimbal Controller

1. Open Storm32 GUI on Windows PC2. Plug USB Mini into the controller3. Make sure battery is sufficiently charged (recommended >12V)4. Look in Control Panel on your PC and under devices and Universal Bus Devices find the

COM number that the controller is assigned5. In GUI select the appropriate COM number on main page and then read6. Under the Gimbal Configuration tab select gimbal configuration7. Make sure all items are checked along the right side of the window that pops up8. Follow through the instructions until configuration is completed

Tuning

Tune in the order of pitch, roll and then yaw (if necessary) 1. Set Vmax with P, I & D to zero

a. Adjust so that the gimbal can withstand its own weight and camerab. Should feel the motors resist if trying to physically turn themc. Turn Vmax down if motors start to get too hot

2. Set Da. Keep P & I zero at this time.b. The D value should be as large as possible without any noise occurring. Once a good value is found, one should move the motor by hand to double-check that no noise appears for any gimbal position. c. Move the gimbal by hand to make sure no noise or “shakes occur during correction

3. Set Pa. Set I to a very low, non-zero level (i.e. 5)b. Get P as high as possible before the gimbal begins to shake, lower until shakes stopc. The goal is to have this as high as possible with smooth movement and no erratic behavior

4. Set Ia. Same idea adjust I to as large of a value as possible until unwanted behavior occurs

Once tuning is completed click on Settings in the menu at the top and then select store to EEProm