Sensor Platforms for Multimodal Underwater...

7
Sensor Platforms for Multimodal Underwater Monitoring Ryan Kastner * , Albert Lin , Curt Schurgers , Jules Jaffe , Peter Franks , and Brent S. Stewart § * Computer Science and Engineering Department, University of California, San Diego California Institute of Telecommunications and Information Technology, University of California, San Diego Scripps Institution of Oceanography, University of California, San Diego § Hubbs-SeaWorld Research Institute, San Diego Abstract—Surprisingly little is known about underwater ma- rine environments and ecosystems owing to several factors. One important factor has been the general inability to observe and monitor these massive subsurface ocean spaces. Despite significant advances in sensor, communication and computing technology, underwater sensor platforms are generally inferior to their terrestrial counterparts. Here we highlight several sensor platforms that are currently being used and developed for underwater monitoring applications. We focus on two monitoring applications: 1) observation of whale sharks (Rhincodon typus); and 2) transport, accumulation and dispersion of plankton. In both cases, we describe the need for multimodal underwater sen- sor platforms to work in a cyber-physical mode, communicating a variety of different sensor data among them systematically to maximize acquisition of data. We think that this will substantially enhance understanding underwater ecosystems. I. I NTRODUCTION The aquatic research community is rapidly equipping eco- logical sites with a broad range of sensors and data recording instruments. Advanced sensing systems facilitate real-time, large-scale experiments and monitoring of complex ecosystem processes. Fundamental to these efforts is a broad suite of environmental observations ranging from basic environmental factors (e.g., water temperature, salinity and bio-optical vari- ables) to more complex measurements (e.g., nutrient concen- tration and other measures of the status and vitality of marine ecosystems). Underwater monitoring is burdened by significant difficul- ties, particularly by technological challenges of packaging sensors to resist saltwater intrusion and hydrostatic pressure and deploying them in dynamic aquatic environments. Though deployment of some sensor platforms have greatly enhanced ocean observation, they have significant room for improve- ment. In particular, the individual sensors typically work in isolation. They log data to local storage, and those data are often only retrieved after the monitoring is complete. A much more powerful way to leverage these platforms would be to close the loop, and work in a true cyber-physical fashion where a system of multimodal sensor platforms learn from and adapt to all of the sensor data in a real-time manner. We address here two underwater monitoring applications to motivate the need for true cyber-physical monitoring solutions: Monitoring of whale sharks; and, transport of plankton. In 978-1-4673-2154-9/12/$31.00 c 2012 IEEE. both scenarios, we provide an overview of the goals for monitoring, describe the current technology used in these applications, and then suggest some directions to develop sensor platforms that would significantly advance exploration and monitoring underwater environments. They highlight the need for multimodal monitoring using of a variety of sensor platforms working in a cyber-physical format. We describe a number of sensor platforms that are cur- rently under development. These include aerial and underwater vehicles, and acoustic imaging and wireless communication. These platforms are being developed to enhance underwater monitoring. We describe exactly how they are now being deployed, and how they can be modified to be more useful. The remainder of the article is organized as follows. We pro- vide an overview of the whale shark and plankton monitoring applications in the next section. Section III provides details on a number of sensor platforms that are useful in these mon- itoring applications. Section IV suggests potential directions for improving these platforms to apply them effectively in the two monitoring applications. We conclude in Section V. II. MOTIVATING APPLICATIONS We motivate our work through two underwater monitoring applications. These applications lie at opposite ends of the spectrum. The first aims at classifying the movements and behaviors of whale sharks (Rhincodon typus), the largest fish on earth. The second focuses on substantially smaller organisms, phyto- and zoo-plankton. In both cases, we pro- vide an overview, describe existing monitoring solutions, and finally discuss ideas on how to increase the effectiveness and efficiency of the monitoring. A. Whale Shark Monitoring 1) Overview: There is no other fish that grows as large (the largest whale shark is over 12 meters and 20,000 kg), yet there are few fishes of any size that are so poorly known as the whale shark. These enormous filter-feeding sharks have become known to appear for brief periods near a number of widely scattered neritic coastal and coral reef habitats among three of the Earths primary oceans, the Atlantic, Pacific, and Indian. These seasonal aggregations are composed almost ex- clusively of young, sexually immature males who reappear in some years before vanishing for the rest of their lives. Where whale sharks mate and give birth are unknown. Likewise, the

Transcript of Sensor Platforms for Multimodal Underwater...

Page 1: Sensor Platforms for Multimodal Underwater Monitoringcseweb.ucsd.edu/~kastner/papers/igcc12-sensor_platforms.pdfSensor Platforms for Multimodal Underwater Monitoring Ryan Kastner ,

Sensor Platforms for Multimodal Underwater Monitoring

Ryan Kastner∗, Albert Lin†, Curt Schurgers†, Jules Jaffe‡, Peter Franks‡, and Brent S. Stewart§

∗Computer Science and Engineering Department, University of California, San Diego†California Institute of Telecommunications and Information Technology, University of California, San Diego

‡Scripps Institution of Oceanography, University of California, San Diego§Hubbs-SeaWorld Research Institute, San Diego

Abstract—Surprisingly little is known about underwater ma-rine environments and ecosystems owing to several factors.One important factor has been the general inability to observeand monitor these massive subsurface ocean spaces. Despitesignificant advances in sensor, communication and computingtechnology, underwater sensor platforms are generally inferiorto their terrestrial counterparts. Here we highlight several sensorplatforms that are currently being used and developed forunderwater monitoring applications. We focus on two monitoringapplications: 1) observation of whale sharks (Rhincodon typus);and 2) transport, accumulation and dispersion of plankton. Inboth cases, we describe the need for multimodal underwater sen-sor platforms to work in a cyber-physical mode, communicatinga variety of different sensor data among them systematically tomaximize acquisition of data. We think that this will substantiallyenhance understanding underwater ecosystems.

I. INTRODUCTION

The aquatic research community is rapidly equipping eco-logical sites with a broad range of sensors and data recordinginstruments. Advanced sensing systems facilitate real-time,large-scale experiments and monitoring of complex ecosystemprocesses. Fundamental to these efforts is a broad suite ofenvironmental observations ranging from basic environmentalfactors (e.g., water temperature, salinity and bio-optical vari-ables) to more complex measurements (e.g., nutrient concen-tration and other measures of the status and vitality of marineecosystems).

Underwater monitoring is burdened by significant difficul-ties, particularly by technological challenges of packagingsensors to resist saltwater intrusion and hydrostatic pressureand deploying them in dynamic aquatic environments. Thoughdeployment of some sensor platforms have greatly enhancedocean observation, they have significant room for improve-ment. In particular, the individual sensors typically work inisolation. They log data to local storage, and those data areoften only retrieved after the monitoring is complete. A muchmore powerful way to leverage these platforms would be toclose the loop, and work in a true cyber-physical fashion wherea system of multimodal sensor platforms learn from and adaptto all of the sensor data in a real-time manner.

We address here two underwater monitoring applications tomotivate the need for true cyber-physical monitoring solutions:Monitoring of whale sharks; and, transport of plankton. In

978-1-4673-2154-9/12/$31.00 c© 2012 IEEE.

both scenarios, we provide an overview of the goals formonitoring, describe the current technology used in theseapplications, and then suggest some directions to developsensor platforms that would significantly advance explorationand monitoring underwater environments. They highlight theneed for multimodal monitoring using of a variety of sensorplatforms working in a cyber-physical format.

We describe a number of sensor platforms that are cur-rently under development. These include aerial and underwatervehicles, and acoustic imaging and wireless communication.These platforms are being developed to enhance underwatermonitoring. We describe exactly how they are now beingdeployed, and how they can be modified to be more useful.

The remainder of the article is organized as follows. We pro-vide an overview of the whale shark and plankton monitoringapplications in the next section. Section III provides detailson a number of sensor platforms that are useful in these mon-itoring applications. Section IV suggests potential directionsfor improving these platforms to apply them effectively in thetwo monitoring applications. We conclude in Section V.

II. MOTIVATING APPLICATIONS

We motivate our work through two underwater monitoringapplications. These applications lie at opposite ends of thespectrum. The first aims at classifying the movements andbehaviors of whale sharks (Rhincodon typus), the largestfish on earth. The second focuses on substantially smallerorganisms, phyto- and zoo-plankton. In both cases, we pro-vide an overview, describe existing monitoring solutions, andfinally discuss ideas on how to increase the effectiveness andefficiency of the monitoring.

A. Whale Shark Monitoring

1) Overview: There is no other fish that grows as large(the largest whale shark is over 12 meters and 20,000 kg),yet there are few fishes of any size that are so poorly knownas the whale shark. These enormous filter-feeding sharks havebecome known to appear for brief periods near a number ofwidely scattered neritic coastal and coral reef habitats amongthree of the Earths primary oceans, the Atlantic, Pacific, andIndian. These seasonal aggregations are composed almost ex-clusively of young, sexually immature males who reappear insome years before vanishing for the rest of their lives. Wherewhale sharks mate and give birth are unknown. Likewise, the

Page 2: Sensor Platforms for Multimodal Underwater Monitoringcseweb.ucsd.edu/~kastner/papers/igcc12-sensor_platforms.pdfSensor Platforms for Multimodal Underwater Monitoring Ryan Kastner ,

whereabouts of males are unknown from the time they areborn until they are around three meters long and then longerthan six to eight meters long. Moreover, the whereabouts offemales at any times in their lives are virtually unknown.

Uncovering these mysteries has been a key initiative ofDr. Brent Stewart’s research program at Hubbs-SeaWorldResearch Institute for the past two decades. Those effortshave used multidisciplinary approaches and collaborative part-nerships to probe population genetics, movements, divingpatterns, aggregation characteristics, and behavior of whalesharks that are accessible for study to yield clues about thewhereabouts of the substantial missing elements of whaleshark populations, their mating areas, and their pupping areasto help promote conservation of these charismatic marinemega-vertebrates and their ocean habitats.

2) Existing Monitoring Solutions: Long-term studies ofwhale sharks have typically depended on labor- and energy-intensive surveillance of relatively small marine zones by boat-based or aircraft-based observers. Despite their remarkablesize, whale sharks are not easily spotted at sea (particularlyfrom boats) owing to their cryptic body coloration and theirtendency to infrequently and briefly approach the sea surface.The use of aircraft can substantially enhance the chances ofdetecting and studying whale sharks that are near the surface.Portable sonar has been used in recent pilot studies to detectwhale sharks at depth and results suggest that it could bea useful study approach with better sensing and imagingequipment (B. Stewart, unpublished observations). Expandingthe use of both passive and active underwater acoustics couldalso increase the likelihood of observing and studying whalesharks.

Because these systems are developed individually, they arecreated for a specific sensing modality. They work relativelywell in isolation (though they could be improved substantially)but only poorly take advantage of potential synergies. Forexample, the aerial and underwater sensors each can indi-vidually detect the presence of a whale shark, though theymiss important opportunities for inter-sensor collaboration andmutual enhancement.

Moreover, the systems operate in a purely open-loop setup.Knowledge of the environment is used during study planningand data analysis, but there only very limited abilities foradaptive modification during the actual experiments. Anyadaptive response requires a human in the loop (e.g., someonemight modify the course of the boat when the sonar detectssomething in a particular area). Thus, while the cyber andphysical elements are clearly present, the current state of theart systems are not true cyber-physical systems, as the com-putation and sensing elements do do not integrate functionally(i.e., where the analysis of past data shapes or drives futurecollection of data).

3) Ideal Monitoring Scenario: There is an urgent need todevelop and test other detection and surveillance methods tocomplement, supplement, and extend those traditional methodsto document the habitats that are key to the vitality andconservation of whale sharks and, more broadly, coral reef

ecosystems and to discover the locations where whale sharksgive birth and mate.

An ideal whale shark observation mission would involveseveral sensing modalities working in a true cyber-physicalmanner. The sensor data would be analyzed immediately andinfluence the further collection of data. Data from individualsensors would be combined into an integrated frameworkthat uses data from one sensing modality to influence theoperation of another. For example, the data collected fromthe underwater acoustic sensors could be transmitted to theaerial vehicles, allowing them to adjust their movements tomore productive observation locations.

B. Plankton Transport, Accumulation and Dispersion

1) Overview: One of the fundamental problems in studyingthe biology of the ocean is that some of the smallest organisms(e.g., larvae, algae) are constantly moving, but methods totrack them or predict where they will go over their lifetimes arelimited. This lack of knowledge is troubling, as the swimmingbehavior of these organisms significantly affects their localconcentration [9], which in turn is directly correlated witha great deal of biological activity in the ocean like primaryproduction, grazing and foraging of fish. These organismsare weak swimmers, and their geographic movements areprincipally determined by ocean currents while they have somecontrol on vertical position by varying their buoyancy. Ourresearch shows that plankton with depth-keeping behavior, forexample, will tend to accumulate in regions of ocean currentconvergence, like the troughs of internal waves [10], [18], [12].

A specific application of plankton movement is for detect-ing, monitoring and predicting the presence and dynamicsdense blooms of toxic phytoplankton (e.g., dinoflagellates, ordiatoms like pseudo-nitzshcia). Many of these harmful algalblooms appear suddenly at coastal sites, often over a tidalcycle. Dinoflagellates are rather slow-growing, so these densered tides must have formed below the surface, where theycould have been growing for weeks prior to surfacing. Theycan cause damage to regional fisheries and tourism, as wellas pose a significant human health hazard [21], thus a betterunderstanding would have significant societal benefits.

2) Existing Monitoring Solutions: The few existing plank-ton monitoring systems generally consist of moored opticalsensors, sometimes complemented by remote sensing by earth-orbiting satellites. Both of these methods have poor spatialacuity: the moored sensors may miss thin layers that do notpass by the sensor, while satellite remote sensing cannot detectlayers more than one optical depth below the surface. Forexample, dye-release experiments allow us to map circulationpatterns, which give us a better understanding of the movementof plankton. But, the dye must be sampled using profiling ormoored instruments. Satellite readings of the dye are possible,but again this is restricted to the surface layers.

Subsurface floats provide another method to monitor plank-ton. They can closely approximate Lagrangian behavior, whileat the same time gathering a wide variety of sensor data.These types of vehicles have been used for over 50 years

Page 3: Sensor Platforms for Multimodal Underwater Monitoringcseweb.ucsd.edu/~kastner/papers/igcc12-sensor_platforms.pdfSensor Platforms for Multimodal Underwater Monitoring Ryan Kastner ,

(a) (b)

Fig. 1. Part (a) shows the gimbal supporting a Canon EOS Rebel T2i. Part (b) shows the stabilized aerial camera platform (SACP) suspending from atethered balloon.

[5]. But, even in deployments with multiple vehicles, eachone essentially acts independently [23], [19]. For example,the Argo profiling floats [2], which operate at ocean-widescales, sample individually and have to resurface periodicallyto localize via GPS and report information via satellite links.

3) Ideal Monitoring Scenario: An inexpensive way toreproduce the behaviors of plankton, permitting the mea-surement of trajectories as well as environmental changes isclearly needed. There is a need for technologies that helpunderstand the physical-biological interactions that underliepatch formation, the connectivity of spatially separated re-gions, and the potential for planktonic transport, accumulationand dispersion.

This requires a highly spatially and temporally resolvedmonitoring system capable of generating frequent 4D mapsof their locations and their physical environment (water ve-locities, temperature, irradiance, etc.), for the price of one“regular” mooring generating time series of data from a singlelocation. Moreover, the system should function as a network toallow the individual sensor platforms to take actions based ontheir local sensor data and also on global data collected by allof the sensor platforms. Ideally, they will not have to surfaceto communicate but rather be able to communicate whenevernecessary to work as a functional, adaptive and distributedsystem.

III. SENSOR PLATFORMS

We are developing and beginning to apply a number ofsensor platforms to address the monitoring needs of thescenarios summarized above. These include aerial imagingplatforms, underwater vehicles, and acoustic imaging and com-munications solutions. Each platform provides unique benefitsfor monitoring while coming with a set of constraints on theirusage.

A. Stabilized Aerial Camera Platform (SACP)

There is often a need to survey a monitoring site fromabove. An ideal system is low cost, deployable for a longperiod of time, and carries a large payload consisting of high

definition cameras and/or other sensors. Furthermore, it shouldbe operated by an untrained individual in the field.

There are several options for aerial surveillance. Mannedvehicles like small planes or helicopters are one possibility.Unfortunately they are quite expensive and therefore can onlybe used for limited amounts of time if at all. Unmanned vehi-cles like planes and helicopters are another option. However,they have limited payloads and relative small flight times –heavier and larger payloads limit the flight time and requirebigger, more expensive vehicles. Furthermore, these require anexpert to pilot them.

Tethered balloons are relatively easy to deploy and canoperate continuously for several weeks or more. They arerelatively inexpensive and they can carry a large payload. Thecost and payload are largely a function of the size of theballoon. Small balloons (ca $50) have a payload of 1-2 kg. Thelargest balloons cost $100,000+, but can carry several thousandkilograms.. The balloon can be deployed from anywhere thathas a relatively open aerial clearance (e.g., a boat).

We have developed a stabilized aerial camera platform(SACP) that enables more effective aerial surveillance. SACPuses an actively stabilized gimbal with wirelessly or au-tonomously controlled pitch, roll and yaw. Figure 1 showsthe SACP holding a Canon EOS Rebel T2i and the SACPsuspended from the blimp. It is relatively inexpensive andsupports a large payload. The pictured balloon is a KingfisherAerostat K12U-SC balloon. It is about 2 m high with a 3.5 mdiameter and carries a 7 kg. The balloon has a sail that helpstremendously with directional stability.

The SACP uses an actively stabilized gimbal to control acamera. It has three separate mechanical joints that indepen-dently control pitch, roll and yaw. An inertial measurementunit (IMU) and a PID control loop stabilize the camera. Itcan be programmed to orient in any sequence of arbitrarydirections. This can be done wirelessly by a remote operator,in a pre-determined manner or autonomously. The camerasettings (e.g., shutter speed, aperture and ISO) can be alsobe controlled. This enables a variety of functions including

Page 4: Sensor Platforms for Multimodal Underwater Monitoringcseweb.ucsd.edu/~kastner/papers/igcc12-sensor_platforms.pdfSensor Platforms for Multimodal Underwater Monitoring Ryan Kastner ,

quickly taking a picture in a region of interest or creating gi-gapan images by taking a sequence of pictures in a panoramicmanner and stitching them together (e.g., a gigapan imageof the La Jolla coastline taken from the UCSD Scripps Pierusing the SACP) [17]. Camera images can also be wirelesslystreamed in real-time from the camera to another place (e.g.,the boat or an UAV).

B. Multirotor UAVA multirotor unmanned aerial vehicle (UAV) offers an

inexpensive way to capture unique points-of-view by carryingboth video and still cameras in the air. They are popular amongmodel aircraft hobbyists and even used in commercial andmilitary applications, thanks to their omnidirectional maneu-vering and hovering capabilities and inherent ability to avoidhazards on the ground. They have garnered significant researchinterest in the past decade [7], [6], [20], [16]. The abilities theyoffer enable mapping, surveying, and other scientific sensingactivities. Figure 2 shows the multirotor UAV in flight.

We have developed several versions of multirotor UAVsand used them as stable aerial platforms capable of carryingadvanced executing out missions autonomously. The currentUAV can carry a payload of several kg (e.g., a DSLR camera),and can navigate autonomously to a set of GPS waypoints.We have wireless communications with the UAV via a Zigbeetransceiver and a six to eight-channel radio receiver andtransmitter for standard first person view controllers. We canalso send data directly from the camera through an analogor digital video wireless transmitter. We built a custom two-axis gimbal for roll and pitch. The yaw is controlled directlythrough the UAV itself and we have developed a two-controllersystem. One person controls the UAV using a “virtual” nosewhile the other controller controls the direction of the camera.The pitch and roll commands go directly to the gimbal whilethe yaw is controlled through the AUV itself. Consequently,the two controllers work together to command the flight of thevehicle.

Fig. 2. A custom quadrotor UAV being flown by two UCSD undergraduatestudents. One is controlling the vehicle and the other is controlling the gimbalwhich is carrying a GoPro Hero2.

C. DrifterAn autonomous buoyancy-controlled drifter is a vehicle that

can control its depth by changing its buoyancy, but is otherwise

carried entirely by the ocean currents. As such, drifters aresubjected to the same dynamics as ocean phenomena them-selves. When equipped with appropriate sensors, an ensembleof drifters forms a dynamic underwater sampling system.It fits within a recent transformation in ocean exploration,where instruments are becoming part of a true networkedsensing collective rather than being controlled individually [1],[13], [15], [14]. Essentially, the vision of sensor networking,advocating a tight coupling between the cyber and physicalworld, is being extended to the oceanic domain.

Fig. 3. An Autonomous Underwater Explorer (AUE) or Drifter. The pistonunderneath changes the volume/density to control the vehicle’s buoyancy.

Several generations of underwater drifters, nicknamed AUEs(Autonomous Underwater Explorers), have been developed inJaffe’s lab [14]. The version shown in Figure 3 (about the sizeof a basketball) has been equipped with the acoustic Micro-modem from WHOI (Woods Hole Oceanographic Institution)[11]. It has a piston for buoyancy control, sensors, processorand batteries, and is able to follow depth, temperature ordensity surfaces, or implement other sensor-driven tasks. Ithas been operational for several years and is depth-rated toover 100 m. Additional design modifications are underwayto include satellite communications and GPS tracking for useon the surface. We have also designed a surface version ofthe AUE. This surface drifter cannot submerge, but it doeshave a GPS receiver and a transmission link to earth-orbitingsatellites. Consequently, it can serve the role of a surfacestation.

We have also designed a mini-AUE, shown in Figure 4.The body is 5 inches in height, and is equipped with depthand temperature sensors, an accelerometer, satellite commu-nications and a dual ballasting system. It has a hydrophoneto detect and record acoustic signals from the surface drifters.This allows them to be localized via the differential traveltime from an array of accurately synched surface drifters.One of their great advantages is that they can form cloudsof underwater microphones over ranges of kilometers that canbe used to localize underwater sound sources such as whalesin addition to being transported with the local flow patterns.

Page 5: Sensor Platforms for Multimodal Underwater Monitoringcseweb.ucsd.edu/~kastner/papers/igcc12-sensor_platforms.pdfSensor Platforms for Multimodal Underwater Monitoring Ryan Kastner ,

Fig. 4. A fleet of mini-AUEs. These smaller drifters are equipped with sensorsand hydrophones. The hydrophones allows them to act as a distributed under-water acoustic array for localizing sounds and receiving wireless commands.

The unique power of the networked drifter swarm is two-fold. First, it is able to monitor oceanographic phenomenawithin their inherently mobile frame of reference, the coastalcirculation. Carried by tides and currents, the biological andphysical ocean processes operate in a highly dynamic envi-ronment. Our drifter ensemble is therefore able to captureLagrangian behavior (i.e., following the moving frame ofreference). This is vital for enhancing understanding of manyoceanographic phenomen [14]. Second, our system enablesthis sampling at spatio-temporal densities that have to datebeen far beyond practical reach, as our drifters sample contin-uously and are spaced only a few hundred meters apart [15].This 4D sampling density is possible due to the fact that thedrifters come a fraction of the cost of existing technologies,such as gliders and propelled underwater vehicles. Such denseLagrangian sampling promises to deliver a wealth of new data,ranging from applications in physical oceanography (mapping3D currents), biology (observing the dispersion of larvae andnutrients), environmental science (tracking coastal pollutantsand effluents from storm drains), and security (monitoringharbors and ports). We envision swarms of tens to hundredsof drifters as the technology matures.

D. Acoustic Sensor Platforms

Acoustic monitoring has a long history and wide use inocean studies for imaging and communication. Sonar imagersare used on boats and underwater vehicles to map bathymetryand to detect fishes and other animals and objects. These sonardevices range from fish finders and side scan sonars to moresophisticated imaging and 3D sonars that use multiple beamsand frequencies. Acoustic signals are also the primary modefor wireless underwater communication. These devices cantransmit over tens of kilometers at over 10,000 bits/second.We discuss sonar imaging and acoustic communication in thefollowing sections.

1) Sonar Imaging: The detection and classification of un-derwater objects using acoustics is difficult but promising

nonetheless. Sonar-based methods are used to map ocean floorstructure and to detect and classify a variety of objects. Sonarcan greatly expand the area that can be effectively surveyedand can also document sub-surface characteristics that can notbe detected by surface observers (i.e., deeper than just a coupleof meters) or satellite and aircraft-based observers (i.e., deeperthan about 4-5 meters).

However, sonar provides much lower resolution images thando optical cameras. Optical cameras can be used underwaterbut their range is limited to relatively short distances evenin clear water. In contrast, sonar can potentially detect objectsover significantly long distances (10-100 km) regardless of theunderwater environment.

Sonar imaging systems are becoming significantly moreadvanced providing the equivalent of low quality video. Thus,we believe that these systems are ripe for computer visionoriented detection algorithms, e.g., using Haar, HOG, andSURF features, to detect objects of interest.

2) Underwater Wireless Communication: One of the keyreasons for the success of terrestrial sensor networks is theavailability of low cost radio platforms. We think that anaquatic counterpart to these radios is needed to truly enablesophisticated underwater ecological analyses.

Much of the currently available acoustic modem technologyis based on frequency shift keying (FSK) and frequency-hopping FSK (FH-FSK). This is likely due to the fact thatFSK is a relatively simple protocol that provides decentperformance. However, FSK-based modems are sensitive toDoppler shifts and typically have data rates on the orderof 1000 bits/second at relatively short ranges. Higher datarate/longer-range acoustic communication has been achievedusing phase-shift keying (PSK), spread-spectrum, and morerecently orthogonal frequency division multiplexing (OFDM).These more advanced modulation techniques increase the datarates to over 10,000 bits/second.

Most of these modems, however, are prohibitively expen-sive, costing thousands to tens of thousands of dollars. In manyapplication scenarios, including the two that we highlight inthis work, cost plays a significant role in the ability to usewireless underwater networking.

The key to a low cost modem is the use of an inexpensivetransducer. The transducers used on many commercial andresearch acoustic modems cost thousands of dollars alone.To prove that low cost modems are feasible, we designed amodem around a $50 homemade omni-directional transducermade from an inexpensive piezoelectric ceramic ring, pottingcompound, and shielded cable. Of course, this is not an idealtransducer, lacking the quality control, bandwidth, superioracoustic coupling and beam pattern to its commercial coun-terparts. However, we have shown that a modem, carefullydesigned around this transducer, can meet our design goals,which sacrifice data rate and range in order to significantlyreduce the cost [3], [4].

Page 6: Sensor Platforms for Multimodal Underwater Monitoringcseweb.ucsd.edu/~kastner/papers/igcc12-sensor_platforms.pdfSensor Platforms for Multimodal Underwater Monitoring Ryan Kastner ,

IV. RESEARCH DIRECTIONS

Now that we have described both the applications andsensor platforms, we discuss how to use the platforms in amultimodal, cyber-physical manner to enhance the acquisitionof data, and ultimately better understand these and otherunderwater ecosystems.

A. Whale Shark Monitoring

Studies of whale sharks is generally labor intensive. Theyinvolve human observers in boats or manned aircrafts. Be-fore a mission, a careful plan is designed, often composedof preset experiments that the observation platform follows,designed to yield the most statistically relevant data. Forexample, a mobile aerial vehicle flies a series of transectsto uniformly cover a predetermined area. After the survey iscompleted, the resulting data are analyzed through statisticaltechniques to determine the distribution of detected sharksand perhaps estimate abundance. The challenge here is thatthe observations provide limited coverage of cryptic animalsthat spend very little time near the sea-surface where theymight be detected. Documenting local abundance would bemore effective by using sonar from survey boats to locate thegeographic and vertical dispersion of sharks. These data couldalso be used to direct particular efforts by swimmers, divers,ROVs, AUVs or drifters to approach, tag, photograph, or justobserve sharks. This approach is adaptive and fully exploitsthe interaction between cyber and physical modes in a tightcontrol loop. We observe, generate initial estimates, and directour sensing resources in a continuing feedback system. Withinthis feedback, the algorithms and strategies, as well as thephysical models of animal and ocean behavior operate on eachother.

Observation and classification should yield not only bestestimate outcomes, but also provide feedback on how toimprove the observation strategy itself. So, in our example, afar more capable system would take observations, extract theuseful information in real-time (the presence of whale sharks,their positions and identification of specific individuals) andthen use this to actively guide the deployed assets. Creatingthis novel closed-loop strategy presents a set of technologicalchallenges in the area of cyber-physical systems, robotics, andsensor fusion.

First, the observation platforms need to be highly interac-tive, such that they are able to handle real-time control. Forexample, we would use a balloon equipped with our stabilizedaerial camera platform, a multirotor UAV, a beamformingarrays of acoustic sensor from the boat or a set of mini-AUEs.The observation strategies must be redesigned to provide theadaptivity required for specific observation goals, while takinginto account the constraints from the vehicles as well as thephysical environment. For example, instead of traditional back-and-forth paths used in open-loop observations, we need tocreate solutions that find the best paths to follow given thefeedback provided by our classification, as well as the physicallimitation of our vehicles in combination with the underwaterand aerial environments in which they operate.

Second, the classification algorithms need not only be ableto provide best estimates, but also generate feedback onwhat inputs would be desired to improve such estimates.For example, in addition to finding the best estimates ofpositions of whale sharks and classification of individuals,the algorithms need to generate information on the accuracyof such estimates and provide indicators on how to improvesuch estimates, e.g., where we would need the copter to fly inorder to get as much reduction in our estimation uncertainty aspossible. These must incorporate both sensor readings aboutthe physical environment (e.g., wind speeds, currents), as wellas a priori knowledge of the physical environment, such asprevailing tides or statistics on animal behavior.

1) Plankton Transport, Accumulation and Dispersion:Studies of both oceanic and atmospheric currents via La-grangian motions have been widely documented and usedfor modeling and research purposes [8], [22]. Historically,these methods have relied on passive motion of buoyantvehicles, but more recently drifters with controllable buoyancyhave been used to perform Lagrangian and mass-transportstudies on global scales (such as Argo). The ability to activelychange buoyancy (and, thus, vertical position) of a smallrobotic system opens the door to interesting new experimentalparadigms by making data available on ocean phenomena intheir own moving frame of reference, at relatively high spatio-temporal scales.

A drifter swarm equipped with inexpensive fluorometersprovides a highly spatially and temporally resolved systemfor monitoring plankton such as harmful algal blooms. Itcan generate 4D maps of their locations and their physicalenvironment (water velocities, temperature, irradiance, etc.),for the price of one mooring generating time series of datafrom a single location. The drifter swarm must operate as anintelligent distributed collective. This is possible by network-ing the drifters together, which, for dense coastal settings, hasto rely on underwater communications.

To harvest the power of a swarm of drifters, a number ofcapabilities need to be supported by the system. The drifterswarm should utilize underactuated control algorithms for theswarm to self-organize in advantageous configurations (e.g.,to maximize the information related to the sensor field ofthe algal bloom), while only leveraging buoyancy control.This requires communication between the drifters, and inturn networking protocols optimized for the specifics of theunderwater acoustic channel. Furthermore, the drifters requirecontinuous location estimates. These should work in a dis-tributed fashion and combine local information, sensor dataand correlations from current-induced motion behavior.

V. CONCLUSION

We described two monitoring applications – whale sharkidentification and classification and modeling plankton trans-port, accumulation and dispersion. In each case, we describethe current state of the art and discuss opportunities for futureimprovements. These center around the use of advanced sensorplatforms working in a cyber-physical fashion.

Page 7: Sensor Platforms for Multimodal Underwater Monitoringcseweb.ucsd.edu/~kastner/papers/igcc12-sensor_platforms.pdfSensor Platforms for Multimodal Underwater Monitoring Ryan Kastner ,

REFERENCES

[1] I. F. Akyildiz, D. Pompili, and T. Melodia. Underwater acoustic sensornetworks: research challenges. Ad hoc networks, 3(3):257–279, 2005.

[2] ARGO home page. http://www-argo.ucsd.edu/.[3] B. Benson, G. Chang, D. Manov, B. Graham, and R. Kastner. Design

of a low-cost acoustic modem for moored oceanographic applications.International Workshop on Underwater Networks (WUWNet), 2006.

[4] B. Benson, Y. Li, R. Kastner, B. Faunce, K. Domond, D. Kimball, andC. Schurgers. Design of a low-cost, underwater acoustic modem forshort-range sensor networks. IEEE Oceans, 2010.

[5] W. Boicourt. The recent history of ocean current measurement: Anunderview. IEEE Second Working Conference on Current Measurement,2:9–18, 1982.

[6] S. Bouabdallah, P. Murrieri, and R. Siegwart. Design and control ofan indoor micro quadrotor. IEEE International Conference on Roboticsand Automation, 5:4393–4398 Vol. 5, 2004.

[7] P. Castillo, R. Lozano, and A. E. Dzul. Modelling and control of mini-flying machines. Springer-Verlag New York Inc, 2005.

[8] E. A. D’Asaro, D. M. Farmer, J. T. Osse, and G. T. Dairiki. A lagrangianfloat. Journal of Atmospheric and Oceanic Technology, 13:1230, 1996.

[9] P. J. S. Franks. Sink or swim: Accumulation of biomass at fronts. Marineecology progress series. Oldendorf, 82(1):1–12, 1992.

[10] P. J. S. Franks. Spatial patterns in dense algal blooms. Limnology andOceanography, 42:1297–3105, 1997.

[11] L. Freitag, M. Grund, S. Singh, J. Partan, P. Koski, and K. Ball. Thewhoi micro-modem: an acoustic communications and navigation systemfor multiple platforms. OCEANS, pages 1086–92 Vol, 2005.

[12] A. Genin, J. S. Jaffe, R. Reef, C. Richter, and P. J. S. Franks. Swimmingagainst the flow: a mechanism of zooplankton aggregation. Science,(308):860–862, 2005.

[13] J. Heidemann, W. Ye, J. Wills, A. Syed, and Y. Li. Research challengesand applications for underwater sensor networking. IEEE WirelessCommunications and Networking Conference, 1:228–235, 2006.

[14] J. S. Jaffe, R. Glatts, C. Schurgers, D. Mirza, P. J. S. Franks, P. Roberts,and F. Simonet. Aue: An autonomous float for monitoring the upperwater column. OCEANS, pages 1–4, 2007.

[15] J. S. Jaffe and C. Schurgers. Sensor networks of freely driftingautonomous underwater explorers. ACM International Workshop onUnderwater Networks, pages 93–96, 2006.

[16] V. Kumar and N. Michael. Opportunities and challenges with au-tonomous micro aerial vehicles. International Symposium on RoboticsResearch, 2011.

[17] La Jolla Shores Gigapan. http://ngs.ucsd.edu/stabilized-aerial-camera-platform.html.

[18] C. E. Lennert-Cody and P. J. S. Franks. Fluorescence patches in high-frequency internal waves. Marine Ecology Progress Series, 235:29–42,2002.

[19] R. C. Lien and E. A. D’Asaro. Measurement of turbulent kinetic energydissipation rate with a lagrangian float. Journal of Atmospheric andOceanic Technology, 23(7):964–976, 2006.

[20] N. Michael, D. Mellinger, Q. Lindsey, and V. Kumar. The grasp multiplemicro-uav testbed. IEEE Robotics & Automation Magazine, 17(3):56–65, 2010.

[21] Monitoring and Event Response for Harmful Algal Blooms (MERHAB).http://www.cop.noaa.gov/stressors/extremeevents/hab/current/fact-merhab.aspx.

[22] T. Rossby, D. Dorson, and J. Fontaine. The RAFOS system. Journal ofAtmospheric and Oceanic Technology, 3(4):672–679, 1986.

[23] T. Rossby, M. D. Prater, and H. Siland. Pathways of inflow anddispersion of warm waters in the nordic seas. Journal of GeophysicalResearch, 114(C4):C04011, 2009.