Once the NWEM has been issued via HazCollect, the only way to verify that it has been disseminated is to monitor NOAA Weather Radio, NWS text products, and the Emergency Alert System broadcast over local television and radio. This type of monitoring also provides the first opportunity to check whether the audio message has been rendered accurately from your text.
However, beyond verifying delivery of the NWEM to the public, a monitoring system to gather feedback as to how the public is responding to the emergency message is important to evaluating the effectiveness of the message. According to Mileti and Sorensen:
A public monitoring system is an important part of a comprehensive warning plan even though it may not seem relevant before a disaster. A number of postdisaster audits show that if officials had known what was happening, a revised message or a different warning strategy could have produced a more effective response or, in some cases, saved lives. Yet, few emergency plans have adopted the concept of a monitoring system.Analysis of the data collected from monitoring efforts can identify trends in the public response, and if the response is inadequate, can indicate what further warning or alerting efforts may be required.
Methods of Monitoring
The methods used for monitoring will vary depending on the hazard and the type of protective instructions issued. Monitoring methods may include:
reviewing news media reports;
- reviewing comments from the public published through social media (Facebook, Twitter, etc.) ;
- reviewing reports of observations from first responders in the field;
- reviewing activity at any phone bank, Information and Referral Service (211), or 911 call center;
- reviewing reports of misdirected calls from the public;
- reviewing data from any available traffic counters or cameras; and/or,
- reviewing activity reports from shelters, if opened.
- Another type of evaluation activity that may prove useful over a longer term is a post-event survey, whereby a representative sampling of the public is asked whether they received the warning, believed the warning, and took action in response.
- The National Weather Service conducts service assessments for major weather events that review NWS issued warnings. (See Net Links below to access NWS Service Assessment reports.)
- If this type of research activity is beyond the capacity of your organization, consider partnering with a college emergency management program that may have students willing and able to do survey design and data collection. (See Net Links below to access the FEMA Higher Éducation Program lists of colleges with emergency management related degree programs.)
Planning Considerations
- Review the suggested monitoring methods, and any others you may have available, and where feasible, incorporate the method into your Warning Annex and/or Standard Operating Procedures (SOP).
- For each method selected, identify the source of the monitoring data by responsible official.
- Consider developing job aids such as reporting forms, or adding sections to existing reporting forms, for summarizing monitoring data.
- Determine to whom monitoring reports will be routed for analysis and with what frequency (e.g., once per day, etc.).
- Consider adding a section to daily, summary level incident report forms for an assessment of the public response.
Please see the following links for further information on NWS service assessments and potential resources for post-event research. Then check your knowledge via the self-assessment quiz on the next page.
