-
SmartCS: Enabling the Creation of ML-Powered Computer Vision Mobile Apps for Citizen Science Applications without Coding
Authors:
Fahim Hasan Khan,
Akila de Silva,
Gregory Dusek,
James Davis,
Alex Pang
Abstract:
It is undeniable that citizen science contributes to the advancement of various fields of study. There are now software tools that facilitate the development of citizen science apps. However, apps developed with these tools rely on individual human skills to correctly collect useful data. Machine learning (ML)-aided apps provide on-field guidance to citizen scientists on data collection tasks. How…
▽ More
It is undeniable that citizen science contributes to the advancement of various fields of study. There are now software tools that facilitate the development of citizen science apps. However, apps developed with these tools rely on individual human skills to correctly collect useful data. Machine learning (ML)-aided apps provide on-field guidance to citizen scientists on data collection tasks. However, these apps rely on server-side ML support, and therefore need a reliable internet connection. Furthermore, the development of citizen science apps with ML support requires a significant investment of time and money. For some projects, this barrier may preclude the use of citizen science effectively. We present a platform that democratizes citizen science by making it accessible to a much broader audience of both researchers and participants. The SmartCS platform allows one to create citizen science apps with ML support quickly and without coding skills. Apps developed using SmartCS have client-side ML support, making them usable in the field, even when there is no internet connection. The client-side ML helps educate users to better recognize the subjects, thereby enabling high-quality data collection. We present several citizen science apps created using SmartCS, some of which were conceived and created by high school students.
△ Less
Submitted 23 May, 2024;
originally announced May 2024.
-
VortexViz: Finding Vortex Boundaries by Learning from Particle Trajectories
Authors:
Akila de Silva,
Nicholas Tee,
Omkar Ghanekar,
Fahim Hasan Khan,
Gregory Dusek,
James Davis,
Alex Pang
Abstract:
Vortices are studied in various scientific disciplines, offering insights into fluid flow behavior. Visualizing the boundary of vortices is crucial for understanding flow phenomena and detecting flow irregularities. This paper addresses the challenge of accurately extracting vortex boundaries using deep learning techniques. While existing methods primarily train on velocity components, we propose…
▽ More
Vortices are studied in various scientific disciplines, offering insights into fluid flow behavior. Visualizing the boundary of vortices is crucial for understanding flow phenomena and detecting flow irregularities. This paper addresses the challenge of accurately extracting vortex boundaries using deep learning techniques. While existing methods primarily train on velocity components, we propose a novel approach incorporating particle trajectories (streamlines or pathlines) into the learning process. By leveraging the regional/local characteristics of the flow field captured by streamlines or pathlines, our methodology aims to enhance the accuracy of vortex boundary extraction.
△ Less
Submitted 1 April, 2024;
originally announced April 2024.
-
RipViz: Finding Rip Currents by Learning Pathline Behavior
Authors:
Akila de Silva,
Mona Zhao,
Donald Stewart,
Fahim Hasan Khan,
Gregory Dusek,
James Davis,
Alex Pang
Abstract:
We present a hybrid machine learning and flow analysis feature detection method, RipViz, to extract rip currents from stationary videos. Rip currents are dangerous strong currents that can drag beachgoers out to sea. Most people are either unaware of them or do not know what they look like. In some instances, even trained personnel such as lifeguards have difficulty identifying them. RipViz produc…
▽ More
We present a hybrid machine learning and flow analysis feature detection method, RipViz, to extract rip currents from stationary videos. Rip currents are dangerous strong currents that can drag beachgoers out to sea. Most people are either unaware of them or do not know what they look like. In some instances, even trained personnel such as lifeguards have difficulty identifying them. RipViz produces a simple, easy to understand visualization of rip location overlaid on the source video. With RipViz, we first obtain an unsteady 2D vector field from the stationary video using optical flow. Movement at each pixel is analyzed over time. At each seed point, sequences of short pathlines, rather a single long pathline, are traced across the frames of the video to better capture the quasi-periodic flow behavior of wave activity. Because of the motion on the beach, the surf zone, and the surrounding areas, these pathlines may still appear very cluttered and incomprehensible. Furthermore, lay audiences are not familiar with pathlines and may not know how to interpret them. To address this, we treat rip currents as a flow anomaly in an otherwise normal flow. To learn about the normal flow behavior, we train an LSTM autoencoder with pathline sequences from normal ocean, foreground, and background movements. During test time, we use the trained LSTM autoencoder to detect anomalous pathlines (i.e., those in the rip zone). The origination points of such anomalous pathlines, over the course of the video, are then presented as points within the rip zone. RipViz is fully automated and does not require user input. Feedback from domain expert suggests that RipViz has the potential for wider use.
△ Less
Submitted 24 February, 2023;
originally announced February 2023.
-
Authoring Platform for Mobile Citizen Science Apps with Client-side ML
Authors:
Fahim Hasan Khan,
Akila de Silva,
Gregory Dusek,
James Davis,
Alex Pang
Abstract:
Data collection is an integral part of any citizen science project. Given the wide variety of projects, some level of expertise or, alternatively, some guidance for novice participants can greatly improve the quality of the collected data. A significant portion of citizen science projects depends on visual data, where photos or videos of different subjects are needed. Often these visual data are c…
▽ More
Data collection is an integral part of any citizen science project. Given the wide variety of projects, some level of expertise or, alternatively, some guidance for novice participants can greatly improve the quality of the collected data. A significant portion of citizen science projects depends on visual data, where photos or videos of different subjects are needed. Often these visual data are collected from all over the world, including remote locations. In this article, we introduce an authoring platform for easily creating mobile apps for citizen science projects that are empowered with client-side machine learning (ML) guidance. The apps created with our platform can help participants recognize the correct data and increase the efficiency of the data collection process. We demonstrate the application of our proposed platform with two use cases: a rip current detection app for a planned pilot study and a detection app for biodiversity-related projects.
△ Less
Submitted 11 December, 2022;
originally announced December 2022.
-
Automated Rip Current Detection with Region based Convolutional Neural Networks
Authors:
Akila de Silva,
Issei Mori,
Gregory Dusek,
James Davis,
Alex Pang
Abstract:
This paper presents a machine learning approach for the automatic identification of rip currents with breaking waves. Rip currents are dangerous fast moving currents of water that result in many deaths by swee** people out to sea. Most people do not know how to recognize rip currents in order to avoid them. Furthermore, efforts to forecast rip currents are hindered by lack of observations to hel…
▽ More
This paper presents a machine learning approach for the automatic identification of rip currents with breaking waves. Rip currents are dangerous fast moving currents of water that result in many deaths by swee** people out to sea. Most people do not know how to recognize rip currents in order to avoid them. Furthermore, efforts to forecast rip currents are hindered by lack of observations to help train and validate hazard models. The presence of web cams and smart phones have made video and still imagery of the coast ubiquitous and provide a potential source of rip current observations. These same devices could aid public awareness of the presence of rip currents. What is lacking is a method to detect the presence or absence of rip currents from coastal imagery. This paper provides expert labeled training and test data sets for rip currents. We use Faster-RCNN and a custom temporal aggregation stage to make detections from still images or videos with higher measured accuracy than both humans and other methods of rip current detection previously reported in the literature.
△ Less
Submitted 4 February, 2021;
originally announced February 2021.
