1

mPING: Crowd-Sourcing Weather Reports for Research

Kimberly L. Elmore1

Cooperative Institute for Mesoscale Meteorological Studies, University of OklahomaNOAA/National Severe Storms Laboratory

Norman, Oklahoma

Z. L. Flamig

University of Oklahoma School of MeteorologyNorman, Oklahoma

V. Lakshmanan1

Cooperative Institute for Mesoscale Meteorological Studies, University of OklahomaNOAA/National Severe Storms Laboratory

Norman, Oklahoma

B. T. Kaney1

Cooperative Institute for Mesoscale Meteorological Studies, University of OklahomaNOAA/National Severe Storms Laboratory

Norman, Oklahoma

V. Farmer

INDUS Corp.Norman, OK

and

Lans P. Rothfusz

NOAA National Severe Storms LaboratoryNorman, OK

Submitted to:

Bulletin of the American Meteorological Society

April 24, 2013

1. Also affiliated with the NOAA/National Severe Storms LaboratoryCorresponding author address: Dr. Kimberly L. Elmore, NSSL, 120 David L Boren Blvd, Norman, OK 73072. Email: kim.elmore@noaa.gov

2

ABSTRACT

The Weather Service Radar 88 Doppler (WSR-88D) network within the United States has

recently been upgraded to include dual-polarization capability. One of the expectations that have

resulted from the upgrade is the ability to discriminate between different precipitation types in

winter precipitation events. To know how well any such algorithm performs, and whether new

algorithms are an improvement, observations of winter precipitation type are needed. Unfortu-

nately, the automated observing systems cannot discriminate between some of the more important

types. Thus human observers are needed. Yet, to deploy dedicated human observers is impractical

and unnecessary because the knowledge needed to identify the various precipitation types is com-

mon among the public. To most efficiently gather such observations would require the public to

be engaged as Citizen Scientists using a very simple, convenient, non-intrusive method. To

achieve this, a simple “app” called mPING (meteorological Phenomena Identification Near the

Ground) was developed to run on “smart” phones or, more generically, web-enabled devices with

GPS location capabilities. Using mPING, anyone with a smart phone can pass observations to

researchers at no additional cost to their phone service or to the research project. Deployed in

mid-December 2012, mPING has proven to be not only very popular, but also capable of provid-

ing consistent, accurate observational data.

3

1. Introduction or Why is this such a great idea?

Late in 2011 a planned upgrade of the Weather Service Radar 1988 Doppler (WSR-88D)

radar network began in earnest (http://www.roc.noaa.gov/WSR88D/PublicDocs/DualPol/DPsta-

tus.pdf). This upgrade adds vertical polarization moments to the already-available horizontal

polarization moments. The over-arching focus for the dual-pol upgrade remains improvement in

quantitative precipitation estimation (QPE) and, in this, there has been some success (Cocks et al.

2011; Berkowitz et al. 2013). But dual-pol radar also offers far more capabilities, especially when

merged with environmental data.

Among these capabilities is the ability to help discriminate between different precipitation

species or types in winter weather. Precipitation type information is useful for various reasons.

For example, forecasters need knowledge of winter precipitation type because it helps inform

them whether or not the thermodynamic profiles are developing as expected. Winter weather pre-

cipitation type affects surface transportation support and road maintenance since precipitation

type affects decisions about whether to treat roads and, if treatment is needed, what process to use.

Aviation ground deicing operations are heavily affected by precipitation type, but certain types of

precipitation, e.g., ice pellets, also indicates freezing rain aloft and so flight conditions that should

be avoided. Electric utility infrastructure suffers during freezing precipitation events, which

means that knowledge of where freezing precipitation is occurring helps utilities plan how best to

maintain the power grid.

Within the suite of dual-pol algorithms fielded with the upgraded radars is the hydrome-

teor classification algorithm (HCA), which is intended primarily for QPE enlacement (Straka et

al., 2000; Berkowitz et al. 2013). Because the HCA is fundamentally intended to provide hydrom-

eteor type within the radar pulse volume, applying it to precipitation type at the surface has pre-

4

dictably unsatisfactory results (Elmore, 2011). These results have also been noted by operational

meteorologists of the NWS and the broadcast media, with the strong desire for improved surface

HCA output expressed by both groups. To address the specific need for surface hydrometeor type

information in winter weather, the winter surface hydrometeor classification algorithm

(WiSHCA) is being developed. In order to both develop and also validate such algorithms, and

other dual-pol algorithms such as Lakshmanan et al. (2013), high-quality surface observations of

precipitation type are needed. The current automated observing systems do not provide informa-

tion about some types, such as ice pellets. Yet, these types have important operational ramifica-

tions. So, a better source of precipitation type data is needed.

People are ideal observers and observing precipitation requires no advanced education in

meteorology. The general public can distinguish between different forms of frozen precipitation

(e.g., snow vs. ice pellets) and the difference between non-freezing and freezing precipitation.

Because this knowledge is common, it seems only natural to use it. But to do so effectively

requires something that is available essentially anywhere. The new generation of web-enabled

portable devices (“smart” devices) offers the ideal platform.

To employ these devices requires an application, or “app” that reports back only the

required data. Because there are so many devices and potentially so many available observations,

the observations must be compact – free-form comments and photographs of precipitation fail in

this regard because of their sheer volume but also because photographs, in particular, cause an

enormous increase in required bandwidth. Another requirement is to use the device’s intrinsic

GPS location and time for tagging observations. Finally the app should keep the reporter anony-

mous to ensure privacy.

5

2. Architecture

The meteorological Phenomena Identification Near the Ground (mPING) project origi-

nated in 2006 as a way to gather validation information to assess the performance of the HCA as a

surface precipitation type classifier (Elmore 2011). In its initial form, observations were entered

through a web page interface (Fig. 1). Observations were requested within a 150 km radius of the

KOUN testbed radar because, at the time, it was the sole WSR-88D-based dual pol prototype.

Users provided their latitude and longitude, based on either their own knowledge or through any

of a number of web-based geolocation services, the time of the observation and, through radio

buttons, the precipitation type. The resulting data were added to a large database system main-

tained at NSSL. While data collection through the web form continues, it has become clear that

with the nationwide dual pol upgrade to the WSR-88D, a more effective data gathering means is

needed.

This led to a program based on the Severe Hazards and Verification Experiment (SHAVE,

Ortega et al. 2009) wherein students actively probe areas of winter weather via telephone calls,

seeking observations of precipitation type. While successful, it became clear that targeting areas

of transitional precipitation types, such as mixes, freezing precipitation and ice pellets is not

straightforward; standard surface observations are inadequate; radar clues are ambiguous; and

such regions are relatively small and transient in nature.

One of us (Flamig) has substantial experience developing weather-based apps for iOS

devices and offered to help develop one that would support widespread, easy submission of pre-

cipitation type observations. The iOS development of mPING and the Android version are func-

tionally identical, but follow different operating system guidelines and so look very different. So

6

far, apps exist only for the iOS and Android platforms, as these make up about 80% of the devices

currently in use. Versions for other platforms may be developed in the future.

Among the key features of mPING are immediate feedback to users that their submission

has been accepted and the ability to display and even download all submissions using a web-

based display (viewable from within the apps). Up to 24 hours of reports from across the conti-

nental United States and for any day back to November 2006 can be displayed. While users

remain anonymous, the report density and frequency is such that when the display is centered on

the users location and magnified (zoomed in), individual reports are easily seen when they appear.

The display can be seen using a desktop browser at http://www.nssl.noaa.gov/projects/ping/dis-

play/ (Fig. 2). A simplified display is used for mobile devices (http://www.nssl.noaa.gov/projects/

ping/display/phone.php).

3. Considerations

We paid particular attention to simplicity. The user interface had to be very simple (Fig.

3), and data entry had to also be simple and intuitive, not because users lack sophistication, but

because the app must remain unobstrusive. Users are extremely concerned about battery life, so

the app has to be “smart” about the way it uses the GPS engine, which is a significant power

drain. To both avoid confusion and to standardize the various types that can be reported, users

choose from a limited number of precipitation types with a pull-down menu (Fig. 4). These types

are: Test, None, Hail, Rain, Drizzle, Freezing Rain, Freezing Drizzle, Snow, Wet Snow, Mixed

Rain and Snow, Mixed Rain and Ice Pellets, Mixed Ice Pellets and Snow, Ice Pellets/Sleet and

Graupel/Snow Grains. For Hail only, an additional parameter (size to the nearest 0.6 cm) is also

required. Location and observation time (in UTC) are gathered from the device’s internal GPS

7

engine. Thus, only the precipitation type is provided by the user; all else is automatic. The

WiSHCA research at NSSL is focused exclusively on precipitation type so no intensity estimates

are requested.

To avoid rapid, inadvertent data submission while the device is being carried in a pocket

or purse, a 5 minute lockout timer is enforced so that observations can be entered at most every 5

min. The 5 minute lockout timer also suppresses malicious attempts to rapidly enter misleading

data.

Both the mobile apps and the web page submit information via HTTP to a common data-

base that validates user input (to prevent malicious attacks, but not to quality-control the observa-

tions) and provides persistent storage of the public reports. All quality control is done in post

processing. We have so far found that these crowdsourced data are very high quality. It is clear to

us that the vast majority of entries are made with the best intentions. Even so, mistakes occur and

the occasional misleading report appears. Fortunately, misleading reports in particular are very

obvious, e.g., 20 cm hail reports in the absent any convection, rain in midst of large-scale snow,

reports of precipitation in areas known to be clear, etc.

4. What we have learned and where we may go next

We have become convinced that immediate feedback to the user is very important and fig-

ures largely in the success of the mPING app. Not only are users rewarded by seeing that their

data are actually being ingested, but feedback from users expresses an overall increased interest in

weather and the project by simply watching the reports as they come in and change with time. In

addition, the data are open and publicly available for text download in 24 h increments via the

main display web page.

8

When these apps were initially released, the announcement was limited to only social

media, i.e., Facebook™, Twitter™, etc. The formal press release occurred much later, on 6 Feb

2013. Yet, we found that once mention was made within social networks, word spread rapidly

about both the apps and the mPING project among those who are interested in weather but are not

necessarily professional meteorologists; evidence is apparent in the download history of both the

iOS and Android versions of the app (Fig. 5) and in the ~209,000 reports recieved between 19

December 2102 and 23 April 2013 (Fig. 6, Table 1). During this time, we have occasionally seen

occasional locations around cities become very active within the span of about an hour following

mention of the app and project, often in cooperation with a local National Weather Service Fore-

cast Office. Even before the formal media announcement, several media articles were published

about mPING and at least one favorable editorial (http://idealab.talkingpointsmemo.com/2013/

02/mping-noaa-storm-app.php, http://www.npr.org/blogs/alltechconsidered/2013/02/25/

171715999/this-app-uses-the-power-of-you-to-report-the-weather, http://www.bostonglobe.com/

editorials/2013/02/08/the-folks-behind-national-weather-service-are-now-crowdsourcing-nemo/

5MD65k88EfDUA30iV8DY3K/story.html).

Among the informal comments made on various social networks and in e-mails to the

authors, users find two favorable characteristics that stand out. In no particular order, the first is

the simplicity of the interface. Users appreciate the simplicity and how quickly they can enter

observations and then be about their business. The second is immediate, uncluttered feedback,

which is is typically mentioned as being very important to users because it both satisfies their

basic curiosity and helps retain their interest, even when weather is not occurring in their immedi-

ate vicinity. Both of these characteristics, taken together, may constitute a fundamental require-

9

ment for future efforts like mPING. The simple observation entry interface avoids tedium and

immediate feedback keeps user’s interest.

We suspect that allowing users to submit a “test” report and then see the report appear on

the real-time display satisfies a reasonable desire to use and test the app immediately upon instal-

lation. Test report submission also strengthens users’ confidence that the app does what is

claimed. While we have no proof, we also suspect that the ability to submit “test” reports helps

users resist the temptation to falsely report precipitation to test the app and see a report when no

precipitation is occurring.

Finally, even though the vast majority of observers are not trained in meteorological

observations, we find that the observations appear to of remarkably consistent and of high quality.

In several instances, professors of meteorology in regions experiencing complex winter precipita-

tion, such as ice pellets, freezing precipitation, or mixed precipitation have been polled. In every

case, these trained meteorologists validate the reports that are nearest to them in both time and

space (Reeves, personal communication).

The app itself is not static: enhancements are forthcoming and additional platforms will be

added. These data will prove to be invaluable for the development of precipitation type algorithms

that work with the upgraded dual-pol WSR-88D radars and also for hail-size algorithms planned

for the WSR-88D dual pol radars. These data will also no doubt be useful for additional studies

and works, including (but not limited to) precipitation type algorithms for numerical models,

ground icing for road maintenance and aviation operations, and even aviation in-flight icing work.

Precipitation type and hail size reports received via mPING will also be relayed electronically and

in real-time to NWS field offices so as to provide data that otherwise can be challenging to collect

via standard communication methods and difficult to discern using remote sensing technologies.

10

5. Acknowledgements

This work was supported by the NEXRAD Product Improvement Program, by NOAA/

Office of Oceanic and Atmospheric Research under NOAA-University of Oklahoma Cooperative

Agreement NA17RJ1227, U.S. Department of Commerce and the University of Oklahoma Coop-

erative Institute for Mesoscale Meteorological Studies. The statements, findings, conclusions, and

recommendations are those of the authors and do not necessarily reflect the views of NOAA, the

U.S. DOC, or the University of Oklahoma.

11

References

Berkowitz, D. S., J. A. Schultz, S. Vasiloff, K. L. Elmore, C. D. Payne, and J. B. Boettcher, 2013:

Status of dual pol QPE in the WSR-88D network. 27th Conf. on Hydrology, Austin, TX.,

6 pp.

Cocks, S. B., D. S. Berkowitz, R. Murnan, J. A. Schultz, S. Castleberry, K. Howard, K. L. Elmore,

and S. Vasiloff, 2012: Initial assessment of the dual-polarization quantitative precipitation

estimate algorithm's performance for two dual-polarization WSR-88Ds. 28th Conf. on

Interactive Information Processing Systems (IIPS), New Orleans, LA, 15 pp.

Elmore, K. L. 2011: The NSSL hydrometeor classification algorithm in winter surface precipita-

tion: Evaluation and future development. Wea. and Forecasting, 26, 756–765.

Lakshmanan, V., C. Karstens, J. Krause, and L. Tang, 2013: Quality control of weather radar data

using polarimetric variables, J. Atm. Ocea. Tech., In Review.

Ortega, K. L., T. M. Smith, K. L. Manross, K. A. Scharfenberg, A. Witt, A. G. Kolodziej, J. J.

Gorley, 2009: The severe hazards analysis and verification experiment. Bull. Amer.

Meteor. Soc., 90, 1519–1530.

Straka, J. M., D. S. Zrnic, A. V Ryzhkov, 2000: Bulk hydrometeor classification and quantifica-

tion using polarimetric radar data: synthesis of relations. J. of Appl. Meteorology, 39,

1341–1372.

12

Tables

Table 1. mPING reports received starting 19 Dec 2012 and ending 23 April 2013.

Table 1.

Type Number

Test 13188

None 45348

Rain 38713

Drizzle 17485

Freezing Rain 3234

Freezing Drizzle 1909

Snow 50470

Wet Snow 13474

Ice Pellets 5574

Graupel 4091

Rain and Snow mixed 6533

Rain and Ice Pellets mixed

4644

Snow and Ice Pellets mixed

4123

13

Figure Captions

FIGURE 1. The original web page interface used to enter mPING observations. Submissions had to

include the observer’s location in decimal latitude and longitude, as well as time and time zone.

Precipitation type is selected via radio buttons.

FIGURE 2. Display of all observations submitted in a single hour spanning 0000 UTC to 0100

UTC on 22 Feb 2013. The display can loop over selected periods showing the spatiotemporal pro-

gression of precipitation and precipitation type. In addition, a rectangle can be created by a mouse

click-and-drag operation such that any subregion can be zoomed and displayed. Text versions of

reports, accurate to two decimal places, can be displayed in a new tab using the “Text Reports”

button and saved with a cut-and-paste operation.

FIGURE 3. The mPING interface is shown for the iOS™ systems on the left and the Android™

systems on the right. The interfaces are kept intentionally simple and relatively uncluttered.

FIGURE 4. Precipitation type choice is made via a drop-down list or menu. Users simply select the

observed precipitation type, at which point the app returns to the submit page. Two taps of the

screen are all that is needed to submit an observation once the app is opened,. An extra tap is

needed for hail because the user must select the (preferably measured) hail size using a slider bar.

FIGURE 5. Download history for the combined iPhone and Android mPING versions. Increases in

download rates are typically the result of media attention. In particular, the increased download

rates spanning 6 Feb 2013 through 13 Feb 2013 are due to the NOAA press release followed on

12 Feb 2013 by a brief feature on the National Public Radio “All Things Considered” newscast.

14

FIGURE 6. Spatial distribution of mPING observations submitted between 19 Dec 2012 and 23

April 2013. Dueto poulation distribution, coverage over the eastern half of the CONUS is much

better thna the western half. Reports that appear outside of the CONUS are legitimate.

15

FIGURE 1. The original web page interface used to enter PING observations. Submissions had to include theobserver’s location in decimal latitude and longitude, as well as time and time zone. Precipitation type is selected viaradio buttons.

16

FIGURE 2. Display of all observations submitted in a single hour spanning 0000 UTC to 0100 UTC on 22 Feb 2013.The display can loop over selected periods showing the spatiotemporal progression of precipitation and precipitationtype. In addition, a rectangle can be created by a mouse click-and-drag operation such that any subregion can bezoomed and displayed. Text versions of reports, accurate to two decimal places, can be displayed in a new tab usingthe “Text Reports” button and saved with a cut-and-paste operation.

17

FIGURE 3. The mPING interface is shown for the iOS™ systems on the left and the Android™ systems on the right.The interfaces are kept intentionally simple and relatively uncluttered.

18

FIGURE 4. Precipitation type choice is made via a drop-down list or menu. Users simply select the observed precipita-tion type, at which point the app returns to the submit page. Two taps of the screen are all that is needed to submit anobservation once the app is opened,. An extra tap is needed for hail because the user must select the (preferably mea-sured) hail size using a slider bar.

19

FIGURE 5. Download history for the combined iPhone and Android mPING versions. Increases in download rates aretypically the result of media attention. In particular, the increased download rates spanning 6 Feb 2013 through 13Feb 2013 are due to the NOAA press release followed on 12 Feb 2013 by a brief feature on the National Public Radio“All Things Considered” newscast.

20

FIGURE 6. Spatial distribution of mPING observations submitted between 19 Dec 2012 and 23 April 2013. Coverageover the eastern half of the CONUS is mucg better thna the wetsren half, a result of population distribution.