Global Flood Detection and Monitoring using Social Media
A new tool for disaster response and validation of flood risk models.
02/02/2018 | 5:50 PM
As part of ongoing research into the use of online media in flood monitoring, researchers at the Institute for Environmental Studies (IVM - VU Amsterdam) and FloodTags released a new tool, available at www.globalfloodmonitor.org, that globally detects and monitors flood events. It provides a real-time overview of ongoing flood events based on filtered Twitter data. Specifically, the global flood monitor (GFM) detects, in real-time, regions with enhanced flood-related Twitter activity and classifies these as flood events. Then, it generates a world-map visualizing these events and their relevant tweets. The platform also provides access to historical events and their relevant tweets dating back to July 2014.
Data collection and filtering
FloodTags collects, among other data, real-time Twitter data using Twitter’ streaming API. The GFM analyzes this data in 12 languages using the keywords as specified in Table 1.
Table 1: Languages and keywords used for the global flood monitor.
||flood, floods, flooding, flooded, inundation, inundations, inundated
||banjir, banjirjkt, bantubanjir
||baha, bumabaha, pagbaha
||flut, hochwasser, Überflutung
||inondazione, inondacioni, alluvione
||poplava, poplave, поплава, поплаве
||inundação, inundacão, inundaçao, inundacao, inundações
||inundación, inundacion, inundar, inundaciones
||su taşkın, su baskını, sel bastı, sel suyu, sel yüzünden, taşkın oldu, sel suyunun
On average this amounts to roughly 75,000 flood-related tweets a day. Naturally, the number of tweets highly varies depending on the characteristics of currently ongoing flood events. For example, when Hurricane Harvey made landfall in the USA, upwards of 600,000 tweets were posted within 24 hours. First, these tweets are filtered using a blacklist, discarding all tweets mentioning words such as “protests”, “smuggled” and “timeline”.
To detect enhanced Twitter activity in regions, locations need to be attached to tweets. Unfortunately, merely ~2% of tweets have the GPS location of the user at the time of posting available. An additional problem in using these GPS locations is that when a major flood event occurs, such as the hurricanes that hit several countries around the Caribbean Sea and the Gulf of Mexico, these events might receive news coverage from all around the world. This might result in enhanced flood-related activity in many locations around the world.
Therefore, we created the TAGGS-algorithm1,2 (Toponym-based Algorithm for Grouped Geoparsing of Social media) to find mentions of locations (i.e., countries, administrative subdivision, cities, towns and villages) in tweets. This roughly employs two steps: 1) toponym recognition and 2) toponym disambiguation. In the first step the sentence is split up into individual words (unigram) as well sequences of individual words up to a length of 3 (bigrams and trigrams). These n-grams are then matched to the near-comprehensive set of geographical locations (gazetteer) as created using the GeoNames database3 (Figure 2).
Unfortunately, many place names (toponyms) can refer to multiple locations (e.g., Boston, UK and Boston, Massachusetts, USA). To disambiguate the toponyms, the algorithm first groups all tweets mentioning the same toponyms within a 24-hour timeframe. Then for all tweets within these groups, additional spatial indicators, such as user time zone, user home town, GPS location and other location mentions in a tweet’s text are analyzed. Based on these indicators the most likely location is selected for all tweets within the group (Figure 3).
"text": "Red River at Grand Forks is 18.53 feet, -9.47 feet of flood stage, -35.82 feet of 1997 crest. #RRVFlood14",
"time zone": "Central Time (US & Canada)",
"location": "Grand Forks, ND",
"toponym": "grand forks",
The GFM conducts event detection at the level of a country and their first order administrative subdivisions (e.g., provinces in the Netherlands and states in the USA). Based on the locations mentioned, tweets are assigned to these regions. Tweets mentioning a country are assigned to the country and tweets mentioning a first order administrative subdivision or a geographic entity therein are assigned to the first order administrative subdivisions.
Then, burst detection is performed by analyzing the time difference between several consecutive tweets assigned to a region. When the time difference between several consecutive tweets falls below a region-specific threshold, this burst is classified as a flood event. An example thereof is given in Figure 4 for the Rift Valley Province in Kenya.
- Flood awareness
The GFM demonstrates the prevalence of floods in the world and their impact on communities. The tweets, often sent by affected people, show that almost daily people need to be evacuated, lose their homes and even lose their lives due floods. Even though many people work towards reducing flood risk and mitigating their impact when a flood hits, further efforts to reduce the impact of flood events on people’s lives are required.
- Disaster response
Disaster relief organizations increasingly use online media to improve their situation awareness. The FloodTags dashboard uses, after careful validation within a specific region, parts of the GFM. In the dashboard and corresponding API’s, localized tweets are classified (result of natural language processing) and combined with other information (e.g., rainfall measurements, river discharge data, maps of likely flooded and impacted areas) to create a tool that can be enhance the situation awareness of local aid organizations. This dashboard is currently operational at the Philippine and Tanzanian Red Cross.
- Reference database
Many minor flooding events remain unreported. Although social media cannot provide an extensive overview of all flood events, many events that are not available in other disaster databases are detected. The platform also provides access to these historic events going back to July 2014. These historic events can be used, for example, as a reference for validation of various flood risk models and historic flood mapping. It should be noted, that the available events are not manually validated and are incomplete. Before using the data, the user should carefully assess the quality of the data for their application (or contact FloodTags or IVM for support in this).
- Social media guided satellite tasking
Finally, when satellites observe the earth, their cameras can be pointed towards areas of interest. When a flood event is detected using, for example, social media, these satellites can be tasked to observe the impacted area and thus provide more information about the event.
1 de Bruijn, J.A., de Moel, H., Jongman, B., Wagemaker, J. & Aerts, .J.C.J.H. (2018). TAGGS: Grouping Tweets to Improve Global Geoparsing for Disaster Response. Journal of Geovisualization and Spatial Analysis, 2, 2. https://doi.org/10.1007/s41651-017-0010-6
2TAGGS source code on GitHub