Showing 1–14 of 14 results for author: Brody, S

FloodDamageCast: Building Flood Damage Nowcasting with Machine Learning and Data Augmentation

Authors: Chia-Fu Liu, Lipai Huang, Kai Yin, Sam Brody, Ali Mostafavi

Abstract: Near-real time estimation of damage to buildings and infrastructure, referred to as damage nowcasting in this study, is crucial for empowering emergency responders to make informed decisions regarding evacuation orders and infrastructure repair priorities during disaster response and recovery. Here, we introduce FloodDamageCast, a machine learning framework tailored for property flood damage nowca… ▽ More Near-real time estimation of damage to buildings and infrastructure, referred to as damage nowcasting in this study, is crucial for empowering emergency responders to make informed decisions regarding evacuation orders and infrastructure repair priorities during disaster response and recovery. Here, we introduce FloodDamageCast, a machine learning framework tailored for property flood damage nowcasting. The framework leverages heterogeneous data to predict residential flood damage at a resolution of 500 meters by 500 meters within Harris County, Texas, during the 2017 Hurricane Harvey. To deal with data imbalance, FloodDamageCast incorporates a generative adversarial networks-based data augmentation coupled with an efficient machine learning model. The results demonstrate the model's ability to identify high-damage spatial areas that would be overlooked by baseline models. Insights gleaned from flood damage nowcasting can assist emergency responders to more efficiently identify repair needs, allocate resources, and streamline on-the-ground inspections, thereby saving both time and effort. △ Less

Submitted 24 May, 2024; v1 submitted 23 May, 2024; originally announced May 2024.

Comments: 20 pages, 6 figures

Non-locality and Spillover Effects of Residential Flood Damage on Community Recovery: Insights from High-resolution Flood Claim and Mobility Data

Authors: Junwei Ma, Russell Blessing, Samuel Brody, Ali Mostafavi

Abstract: Examining the relationship between vulnerability of the built environment and community recovery is crucial for understanding disaster resilience. Yet, this relationship is rather neglected in the existing literature due to previous limitations in the availability of empirical datasets needed for such analysis. In this study, we combine fine-resolution flood damage claims data (composed of both in… ▽ More Examining the relationship between vulnerability of the built environment and community recovery is crucial for understanding disaster resilience. Yet, this relationship is rather neglected in the existing literature due to previous limitations in the availability of empirical datasets needed for such analysis. In this study, we combine fine-resolution flood damage claims data (composed of both insured and uninsured losses) and human mobility data (composed of millions of movement trajectories) during the 2017 Hurricane Harvey in Harris County, Texas, to specify the extent to which vulnerability of the built environment (i.e., flood property damage) affects community recovery (based on the speed of human mobility recovery) locally and regionally. We examine this relationship using a spatial lag, spatial reach, and spatial decay models to measure the extent of spillover effects of residential damage on community recovery. The findings show that: first, the severity of residential damage significantly affects the speed of community recovery. A greater extent of residential damage suppresses community recovery not only locally but also in the surrounding areas. Second, the spatial spillover effect of residential damage on community recovery speed decays with distance from the highly damaged areas. Third, spatial areas display heterogeneous spatial decay coefficients, which are associated with urban structure features such as the density of points-of-interest facilities and roads. These findings provide a novel data-driven characterization of the spatial diffusion of residential flood damage effects on community recovery and move us closer to a better understanding of complex spatial processes that shape community resilience to hazards. This study also provides valuable insights for emergency managers and public officials seeking to mitigate the non-local effects of residential damage. △ Less

Submitted 8 May, 2024; v1 submitted 6 May, 2024; originally announced May 2024.

MaxFloodCast: Ensemble Machine Learning Model for Predicting Peak Inundation Depth And Decoding Influencing Features

Authors: Cheng-Chun Lee, Lipai Huang, Federico Antolini, Matthew Garcia, Andrew Juanb, Samuel D. Brody, Ali Mostafavi

Abstract: Timely, accurate, and reliable information is essential for decision-makers, emergency managers, and infrastructure operators during flood events. This study demonstrates a proposed machine learning model, MaxFloodCast, trained on physics-based hydrodynamic simulations in Harris County, offers efficient and interpretable flood inundation depth predictions. Achieving an average R-squared of 0.949 a… ▽ More Timely, accurate, and reliable information is essential for decision-makers, emergency managers, and infrastructure operators during flood events. This study demonstrates a proposed machine learning model, MaxFloodCast, trained on physics-based hydrodynamic simulations in Harris County, offers efficient and interpretable flood inundation depth predictions. Achieving an average R-squared of 0.949 and a Root Mean Square Error of 0.61 ft on unseen data, it proves reliable in forecasting peak flood inundation depths. Validated against Hurricane Harvey and Storm Imelda, MaxFloodCast shows the potential in supporting near-time floodplain management and emergency operations. The model's interpretability aids decision-makers in offering critical information to inform flood mitigation strategies, to prioritize areas with critical facilities and to examine how rainfall in other watersheds influences flood exposure in one area. The MaxFloodCast model enables accurate and interpretable inundation depth predictions while significantly reducing computational time, thereby supporting emergency response efforts and flood risk management more effectively. △ Less

Submitted 11 August, 2023; originally announced August 2023.

ELEV-VISION: Automated Lowest Floor Elevation Estimation from Segmenting Street View Images

Authors: Yu-Hsuan Ho, Cheng-Chun Lee, Nicholas D. Diaz, Samuel D. Brody, Ali Mostafavi

Abstract: We propose an automated lowest floor elevation (LFE) estimation algorithm based on computer vision techniques to leverage the latent information in street view images. Flood depth-damage models use a combination of LFE and flood depth for determining flood risk and extent of damage to properties. We used image segmentation for detecting door bottoms and roadside edges from Google Street View image… ▽ More We propose an automated lowest floor elevation (LFE) estimation algorithm based on computer vision techniques to leverage the latent information in street view images. Flood depth-damage models use a combination of LFE and flood depth for determining flood risk and extent of damage to properties. We used image segmentation for detecting door bottoms and roadside edges from Google Street View images. The characteristic of equirectangular projection with constant spacing representation of horizontal and vertical angles allows extraction of the pitch angle from the camera to the door bottom. The depth from the camera to the door bottom was obtained from the depthmap paired with the Google Street View image. LFEs were calculated from the pitch angle and the depth. The testbed for application of the proposed method is Meyerland (Harris County, Texas). The results show that the proposed method achieved mean absolute error of 0.190 m (1.18 %) in estimating LFE. The height difference between the street and the lowest floor (HDSL) was estimated to provide information for flood damage estimation. The proposed automatic LFE estimation algorithm using Street View images and image segmentation provides a rapid and cost-effective method for LFE estimation compared with the surveys using total station theodolite and unmanned aerial systems. By obtaining more accurate and up-to-date LFE data using the proposed method, city planners, emergency planners and insurance companies could make a more precise estimation of flood damage. △ Less

Submitted 5 June, 2023; originally announced June 2023.

FuseCap: Leveraging Large Language Models for Enriched Fused Image Captions

Authors: Noam Rotstein, David Bensaid, Shaked Brody, Roy Ganz, Ron Kimmel

Abstract: The advent of vision-language pre-training techniques enhanced substantial progress in the development of models for image captioning. However, these models frequently produce generic captions and may omit semantically important image details. This limitation can be traced back to the image-text datasets; while their captions typically offer a general description of image content, they frequently… ▽ More The advent of vision-language pre-training techniques enhanced substantial progress in the development of models for image captioning. However, these models frequently produce generic captions and may omit semantically important image details. This limitation can be traced back to the image-text datasets; while their captions typically offer a general description of image content, they frequently omit salient details. Considering the magnitude of these datasets, manual reannotation is impractical, emphasizing the need for an automated approach. To address this challenge, we leverage existing captions and explore augmenting them with visual details using "frozen" vision experts including an object detector, an attribute recognizer, and an Optical Character Recognizer (OCR). Our proposed method, FuseCap, fuses the outputs of such vision experts with the original captions using a large language model (LLM), yielding comprehensive image descriptions. We automatically curate a training set of 12M image-enriched caption pairs. These pairs undergo extensive evaluation through both quantitative and qualitative analyses. Subsequently, this data is utilized to train a captioning generation BLIP-based model. This model outperforms current state-of-the-art approaches, producing more precise and detailed descriptions, demonstrating the effectiveness of the proposed data-centric approach. We release this large-scale dataset of enriched image-caption pairs for the community. △ Less

Submitted 15 November, 2023; v1 submitted 28 May, 2023; originally announced May 2023.

On the Expressivity Role of LayerNorm in Transformers' Attention

Authors: Shaked Brody, Uri Alon, Eran Yahav

Abstract: Layer Normalization (LayerNorm) is an inherent component in all Transformer-based models. In this paper, we show that LayerNorm is crucial to the expressivity of the multi-head attention layer that follows it. This is in contrast to the common belief that LayerNorm's only role is to normalize the activations during the forward pass, and their gradients during the backward pass. We consider a geome… ▽ More Layer Normalization (LayerNorm) is an inherent component in all Transformer-based models. In this paper, we show that LayerNorm is crucial to the expressivity of the multi-head attention layer that follows it. This is in contrast to the common belief that LayerNorm's only role is to normalize the activations during the forward pass, and their gradients during the backward pass. We consider a geometric interpretation of LayerNorm and show that it consists of two components: (a) projection of the input vectors to a $d-1$ space that is orthogonal to the $\left[1,1,...,1\right]$ vector, and (b) scaling of all vectors to the same norm of $\sqrt{d}$. We show that each of these components is important for the attention layer that follows it in Transformers: (a) projection allows the attention mechanism to create an attention query that attends to all keys equally, offloading the need to learn this operation by the attention; and (b) scaling allows each key to potentially receive the highest attention, and prevents keys from being "un-select-able". We show empirically that Transformers do indeed benefit from these properties of LayeNorm in general language modeling and even in computing simple functions such as "majority". Our code is available at https://github.com/tech-srl/layer_norm_expressivity_role . △ Less

Submitted 11 May, 2023; v1 submitted 4 May, 2023; originally announced May 2023.

Comments: Accepted as a short paper in Findings of ACL 2023

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

Authors: BigScience Workshop, :, Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, Jonathan Tow, Alexander M. Rush, Stella Biderman, Albert Webson, Pawan Sasanka Ammanamanchi, Thomas Wang, Benoît Sagot, Niklas Muennighoff, Albert Villanova del Moral, Olatunji Ruwase, Rachel Bawden, Stas Bekman, Angelina McMillan-Major , et al. (369 additional authors not shown)

Abstract: Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access… ▽ More Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License. △ Less

Submitted 27 June, 2023; v1 submitted 9 November, 2022; originally announced November 2022.

Unveiling Vulnerability and Inequality in Disrupted Access to Dialysis Centers During Urban Flooding

Authors: Faxi Yuan, Hamed Farahmand, Russell Blessing, Samuel Brody, Ali Mostafavi

Abstract: Despite the criticality of dialysis facilities, limited knowledge exists regarding the extent and inequality of disrupted access caused by weather events. This study uses mobility data in the context of the 2017 Hurricane Harvey in Harris County to examine the impact of flooding on access to dialysis centers. We examined access dimensions using multiple static and dynamic metrics. By analyzing flu… ▽ More Despite the criticality of dialysis facilities, limited knowledge exists regarding the extent and inequality of disrupted access caused by weather events. This study uses mobility data in the context of the 2017 Hurricane Harvey in Harris County to examine the impact of flooding on access to dialysis centers. We examined access dimensions using multiple static and dynamic metrics. By analyzing fluctuations in dynamic metrics during the flood event, we derived spatiotemporal patterns of access disruptions and inequity in mobility from CBGs to dialysis centers. The results show that: (1) the extent of dependence of CBGs on dialysis centers varies; (2) flooding significantly reduces access redundancy and frequency of dialysis center visits; access disruptions persist for more than one month after the flood event; (3) CBGs with a greater minority percentage and lower household income were more likely to experience more extensive access disruptions; (4) high-income CBGs more quickly revert to their pre-disaster levels; (5) larger dialysis centers located in non-flooded areas are critical to absorbing the unmet demand from disrupted facilities; these larger facilities contribute to the absorptive capacity of the network of dialysis facilities in the region. △ Less

Submitted 25 August, 2022; v1 submitted 4 August, 2022; originally announced August 2022.

Examining Data Imbalance in Crowdsourced Reports for Improving Flash Flood Situational Awareness

Authors: Miguel Esparza, Hamed Farahmand, Samuel Brody, Ali Mostafavi

Abstract: The use of crowdsourced data has been finding practical use for enhancing situational awareness during disasters. While recent studies have shown promising results regarding the potential of crowdsourced data for flood map**, little attention has been paid to data imbalances issues that could introduce biases. We examine biases present in crowdsourced reports to identify data imbalances with a g… ▽ More The use of crowdsourced data has been finding practical use for enhancing situational awareness during disasters. While recent studies have shown promising results regarding the potential of crowdsourced data for flood map**, little attention has been paid to data imbalances issues that could introduce biases. We examine biases present in crowdsourced reports to identify data imbalances with a goal of improving disaster situational awareness. Sample bias, spatial bias, and demographic bias are examined as we analyzed reported flooding from 3-1-1, Waze reports, and FEMA damage data collected in the aftermaths of Tropical Storm Imelda in 2019 and Hurricane Ida in 2021. Integrating other flooding related topics from 3-1-1 reports into the Global Moran's I and Local Indicator of Spatial Association (LISA) test revealed more communities that were impacted by floods. To examine spatial bias, we perform the LISA and BI-LISA tests on the three datasets at the census tract and census block group level. By looking at two geographical aggregations, we found that the larger spatial aggregations, census tracts, show less data imbalance in the results. Finally, one-way analysis of Variance (ANOVA) test performed on the clusters generated from the BI-LISA shows that data imbalance exists in areas where minority populations reside. Through a regression analysis, we found that 3-1-1 and Waze reports have data imbalance limitations in areas where minority populations reside. The findings of this study advance understanding of data imbalances and biases in crowdsourced datasets that are growingly used for disaster situational awareness. △ Less

Submitted 12 July, 2022; originally announced July 2022.

Comments: 28 Pages, 12 Figures 9 Tables

Predicting Road Flooding Risk with Machine Learning Approaches Using Crowdsourced Reports and Fine-grained Traffic Data

Authors: Faxi Yuan, William Mobley, Hamed Farahmand, Yuanchang Xu, Russell Blessing, Shangjia Dong, Ali Mostafavi, Samuel D. Brody

Abstract: The objective of this study is to predict road flooding risks based on topographic, hydrologic, and temporal precipitation features using machine learning models. Predictive flood monitoring of road network flooding status plays an essential role in community hazard mitigation, preparedness, and response activities. Existing studies related to the estimation of road inundations either lack observe… ▽ More The objective of this study is to predict road flooding risks based on topographic, hydrologic, and temporal precipitation features using machine learning models. Predictive flood monitoring of road network flooding status plays an essential role in community hazard mitigation, preparedness, and response activities. Existing studies related to the estimation of road inundations either lack observed road inundation data for model validations or focus mainly on road inundation exposure assessment based on flood maps. This study addresses this limitation by using crowdsourced and fine-grained traffic data as an indicator of road inundation, and topographic, hydrologic, and temporal precipitation features as predictor variables. Two tree-based machine learning models (random forest and AdaBoost) were then tested and trained for predicting road inundations in the contexts of 2017 Hurricane Harvey and 2019 Tropical Storm Imelda in Harris County, Texas. The findings from Hurricane Harvey indicate that precipitation is the most important feature for predicting road inundation susceptibility, and that topographic features are more essential than hydrologic features for predicting road inundations in both storm cases. The random forest and AdaBoost models had relatively high AUC scores (0.860 and 0.810 for Harvey respectively and 0.790 and 0.720 for Imelda respectively) with the random forest model performing better in both cases. The random forest model showed stable performance for Harvey, while varying significantly for Imelda. This study advances the emerging field of smart flood resilience in terms of predictive flood risk map** at the road level. For example, such models could help impacted communities and emergency management agencies develop better preparedness and response strategies with improved situational awareness of road inundation likelihood as an extreme weather event unfolds. △ Less

Submitted 14 September, 2021; v1 submitted 30 August, 2021; originally announced August 2021.

Comments: 17 pages, 7 figures

How Attentive are Graph Attention Networks?

Authors: Shaked Brody, Uri Alon, Eran Yahav

Abstract: Graph Attention Networks (GATs) are one of the most popular GNN architectures and are considered as the state-of-the-art architecture for representation learning with graphs. In GAT, every node attends to its neighbors given its own representation as the query. However, in this paper we show that GAT computes a very limited kind of attention: the ranking of the attention scores is unconditioned on… ▽ More Graph Attention Networks (GATs) are one of the most popular GNN architectures and are considered as the state-of-the-art architecture for representation learning with graphs. In GAT, every node attends to its neighbors given its own representation as the query. However, in this paper we show that GAT computes a very limited kind of attention: the ranking of the attention scores is unconditioned on the query node. We formally define this restricted kind of attention as static attention and distinguish it from a strictly more expressive dynamic attention. Because GATs use a static attention mechanism, there are simple graph problems that GAT cannot express: in a controlled problem, we show that static attention hinders GAT from even fitting the training data. To remove this limitation, we introduce a simple fix by modifying the order of operations and propose GATv2: a dynamic graph attention variant that is strictly more expressive than GAT. We perform an extensive evaluation and show that GATv2 outperforms GAT across 11 OGB and other benchmarks while we match their parametric costs. Our code is available at https://github.com/tech-srl/how_attentive_are_gats . GATv2 is available as part of the PyTorch Geometric library, the Deep Graph Library, and the TensorFlow GNN library. △ Less

Submitted 31 January, 2022; v1 submitted 30 May, 2021; originally announced May 2021.

Comments: Published in ICLR 2022

A Structural Model for Contextual Code Changes

Authors: Shaked Brody, Uri Alon, Eran Yahav

Abstract: We address the problem of predicting edit completions based on a learned model that was trained on past edits. Given a code snippet that is partially edited, our goal is to predict a completion of the edit for the rest of the snippet. We refer to this task as the EditCompletion task and present a novel approach for tackling it. The main idea is to directly represent structural edits. This allows u… ▽ More We address the problem of predicting edit completions based on a learned model that was trained on past edits. Given a code snippet that is partially edited, our goal is to predict a completion of the edit for the rest of the snippet. We refer to this task as the EditCompletion task and present a novel approach for tackling it. The main idea is to directly represent structural edits. This allows us to model the likelihood of the edit itself, rather than learning the likelihood of the edited code. We represent an edit operation as a path in the program's Abstract Syntax Tree (AST), originating from the source of the edit to the target of the edit. Using this representation, we present a powerful and lightweight neural model for the EditCompletion task. We conduct a thorough evaluation, comparing our approach to a variety of representation and modeling approaches that are driven by multiple strong models such as LSTMs, Transformers, and neural CRFs. Our experiments show that our model achieves a 28% relative gain over state-of-the-art sequential models and 2x higher accuracy than syntactic models that learn to generate the edited code, as opposed to modeling the edits directly. Our code, dataset, and trained models are publicly available at https://github.com/tech-srl/c3po/ . △ Less

Submitted 12 October, 2020; v1 submitted 27 May, 2020; originally announced May 2020.

Comments: Accepted to OOPSLA 2020

code2seq: Generating Sequences from Structured Representations of Code

Authors: Uri Alon, Shaked Brody, Omer Levy, Eran Yahav

Abstract: The ability to generate natural language sequences from source code snippets has a variety of applications such as code summarization, documentation, and retrieval. Sequence-to-sequence (seq2seq) models, adopted from neural machine translation (NMT), have achieved state-of-the-art performance on these tasks by treating source code as a sequence of tokens. We present ${\rm {\scriptsize CODE2SEQ}}$:… ▽ More The ability to generate natural language sequences from source code snippets has a variety of applications such as code summarization, documentation, and retrieval. Sequence-to-sequence (seq2seq) models, adopted from neural machine translation (NMT), have achieved state-of-the-art performance on these tasks by treating source code as a sequence of tokens. We present ${\rm {\scriptsize CODE2SEQ}}$: an alternative approach that leverages the syntactic structure of programming languages to better encode source code. Our model represents a code snippet as the set of compositional paths in its abstract syntax tree (AST) and uses attention to select the relevant paths while decoding. We demonstrate the effectiveness of our approach for two tasks, two programming languages, and four datasets of up to $16$M examples. Our model significantly outperforms previous models that were specifically designed for programming languages, as well as state-of-the-art NMT models. An interactive online demo of our model is available at http://code2seq.org. Our code, data and trained models are available at http://github.com/tech-srl/code2seq. △ Less

Submitted 21 February, 2019; v1 submitted 3 August, 2018; originally announced August 2018.

Comments: Accepted to ICLR'2019

Moving the Needle: What Works Cities and the use of data and evidence

Authors: Simone Brody, Andel Koester, Zachary Markovits, Jacob Phillips

Abstract: Bloomberg Philanthropies launched What Works Cities (WWC) in 2015 to help cities better leverage data and evidence to drive decision-making and improve residents' lives. Over three years, WWC will work with 100 American cities with populations between 100,000 and 1,000,000 to measure their state of practice and provide targeted technical assistance. This paper uses the data obtained through the WW… ▽ More Bloomberg Philanthropies launched What Works Cities (WWC) in 2015 to help cities better leverage data and evidence to drive decision-making and improve residents' lives. Over three years, WWC will work with 100 American cities with populations between 100,000 and 1,000,000 to measure their state of practice and provide targeted technical assistance. This paper uses the data obtained through the WWC discovery process to understand how 67 cities are currently using data to deliver city services. Our analysis confirms that while cities possess a strong desire to use data and evidence, government leaders are constrained in their ability to apply these practices. We find that a city's stated commitment to using data is the strongest predictor of overall performance and that strong practice in almost any one specific technical area of using data to inform decisions is an indicator of strong practices in other areas. The exception is open data; we find larger cities are more adept at adopting open data policies and programs, independent of their performance using data overall. This paper seeks to develop a deeper understanding of the issues underlying these findings and to continue the conversation on how to best support cities' efforts in this work. △ Less

Submitted 27 September, 2016; originally announced September 2016.

Comments: Presented at the Data For Good Exchange 2016