Showing 1–7 of 7 results for author: Wahlstrom, J

Distributed Black-box Attack against Image Classification Cloud Services

Authors: Han Wu, Sareh Rowlands, Johan Wahlstrom

Abstract: Black-box adversarial attacks can fool image classifiers into misclassifying images without requiring access to model structure and weights. Recent studies have reported attack success rates of over 95% with less than 1,000 queries. The question then arises of whether black-box attacks have become a real threat against IoT devices that rely on cloud APIs to achieve image classification. To shed so… ▽ More Black-box adversarial attacks can fool image classifiers into misclassifying images without requiring access to model structure and weights. Recent studies have reported attack success rates of over 95% with less than 1,000 queries. The question then arises of whether black-box attacks have become a real threat against IoT devices that rely on cloud APIs to achieve image classification. To shed some light on this, note that prior research has primarily focused on increasing the success rate and reducing the number of queries. However, another crucial factor for black-box attacks against cloud APIs is the time required to perform the attack. This paper applies black-box attacks directly to cloud APIs rather than to local models, thereby avoiding mistakes made in prior research that applied the perturbation before image encoding and pre-processing. Further, we exploit load balancing to enable distributed black-box attacks that can reduce the attack time by a factor of about five for both local search and gradient estimation methods. △ Less

Submitted 21 August, 2023; v1 submitted 28 October, 2022; originally announced October 2022.

Comments: 8 pages, 10 figures

Adversarial Detection: Attacking Object Detection in Real Time

Authors: Han Wu, Syed Yunas, Sareh Rowlands, Wenjie Ruan, Johan Wahlstrom

Abstract: Intelligent robots rely on object detection models to perceive the environment. Following advances in deep learning security it has been revealed that object detection models are vulnerable to adversarial attacks. However, prior research primarily focuses on attacking static images or offline videos. Therefore, it is still unclear if such attacks could jeopardize real-world robotic applications in… ▽ More Intelligent robots rely on object detection models to perceive the environment. Following advances in deep learning security it has been revealed that object detection models are vulnerable to adversarial attacks. However, prior research primarily focuses on attacking static images or offline videos. Therefore, it is still unclear if such attacks could jeopardize real-world robotic applications in dynamic environments. This paper bridges this gap by presenting the first real-time online attack against object detection models. We devise three attacks that fabricate bounding boxes for nonexistent objects at desired locations. The attacks achieve a success rate of about 90% within about 20 iterations. The demo video is available at https://youtu.be/zJZ1aNlXsMU. △ Less

Submitted 12 December, 2023; v1 submitted 5 September, 2022; originally announced September 2022.

Comments: Accepted by IEEE Intelligent Vehicle Symposium, 2023

Journal ref: IEEE Intelligent Vehicle Symposium, 2023

A Man-in-the-Middle Attack against Object Detection Systems

Authors: Han Wu, Sareh Rowlands, Johan Wahlstrom

Abstract: Object detection systems using deep learning models have become increasingly popular in robotics thanks to the rising power of CPUs and GPUs in embedded systems. However, these models are susceptible to adversarial attacks. While some attacks are limited by strict assumptions on access to the detection system, we propose a novel hardware attack inspired by Man-in-the-Middle attacks in cryptography… ▽ More Object detection systems using deep learning models have become increasingly popular in robotics thanks to the rising power of CPUs and GPUs in embedded systems. However, these models are susceptible to adversarial attacks. While some attacks are limited by strict assumptions on access to the detection system, we propose a novel hardware attack inspired by Man-in-the-Middle attacks in cryptography. This attack generates an Universal Adversarial Perturbation (UAP) and then inject the perturbation between the USB camera and the detection system via a hardware attack. Besides, prior research is misled by an evaluation metric that measures the model accuracy rather than the attack performance. In combination with our proposed evaluation metrics, we significantly increases the strength of adversarial perturbations. These findings raise serious concerns for applications of deep learning models in safety-critical systems, such as autonomous driving. △ Less

Submitted 21 August, 2023; v1 submitted 15 August, 2022; originally announced August 2022.

Comments: 6 pages, 7 figures

Adversarial Driving: Attacking End-to-End Autonomous Driving

Authors: Han Wu, Syed Yunas, Sareh Rowlands, Wenjie Ruan, Johan Wahlstrom

Abstract: As research in deep neural networks advances, deep convolutional networks become promising for autonomous driving tasks. In particular, there is an emerging trend of employing end-to-end neural network models for autonomous driving. However, previous research has shown that deep neural network classifiers are vulnerable to adversarial attacks. While for regression tasks, the effect of adversarial… ▽ More As research in deep neural networks advances, deep convolutional networks become promising for autonomous driving tasks. In particular, there is an emerging trend of employing end-to-end neural network models for autonomous driving. However, previous research has shown that deep neural network classifiers are vulnerable to adversarial attacks. While for regression tasks, the effect of adversarial attacks is not as well understood. In this research, we devise two white-box targeted attacks against end-to-end autonomous driving models. Our attacks manipulate the behavior of the autonomous driving system by perturbing the input image. In an average of 800 attacks with the same attack strength (epsilon=1), the image-specific and image-agnostic attack deviates the steering angle from the original output by 0.478 and 0.111, respectively, which is much stronger than random noises that only perturbs the steering angle by 0.002 (The steering angle ranges from [-1, 1]). Both attacks can be initiated in real-time on CPUs without employing GPUs. Demo video: https://youtu.be/I0i8uN2oOP0. △ Less

Submitted 12 December, 2023; v1 submitted 16 March, 2021; originally announced March 2021.

Comments: Accepted by IEEE Intelligent Vehicle Symposium, 2023

Journal ref: IEEE Intelligent Vehicle Symposium, 2023

DeepTIO: A Deep Thermal-Inertial Odometry with Visual Hallucination

Authors: Muhamad Risqi U. Saputra, Pedro P. B. de Gusmao, Chris Xiaoxuan Lu, Yasin Almalioglu, Stefano Rosa, Changhao Chen, Johan Wahlström, Wei Wang, Andrew Markham, Niki Trigoni

Abstract: Visual odometry shows excellent performance in a wide range of environments. However, in visually-denied scenarios (e.g. heavy smoke or darkness), pose estimates degrade or even fail. Thermal cameras are commonly used for perception and inspection when the environment has low visibility. However, their use in odometry estimation is hampered by the lack of robust visual features. In part, this is a… ▽ More Visual odometry shows excellent performance in a wide range of environments. However, in visually-denied scenarios (e.g. heavy smoke or darkness), pose estimates degrade or even fail. Thermal cameras are commonly used for perception and inspection when the environment has low visibility. However, their use in odometry estimation is hampered by the lack of robust visual features. In part, this is as a result of the sensor measuring the ambient temperature profile rather than scene appearance and geometry. To overcome this issue, we propose a Deep Neural Network model for thermal-inertial odometry (DeepTIO) by incorporating a visual hallucination network to provide the thermal network with complementary information. The hallucination network is taught to predict fake visual features from thermal images by using Huber loss. We also employ selective fusion to attentively fuse the features from three different modalities, i.e thermal, hallucination, and inertial features. Extensive experiments are performed in hand-held and mobile robot data in benign and smoke-filled environments, showing the efficacy of the proposed model. △ Less

Submitted 19 January, 2020; v1 submitted 16 September, 2019; originally announced September 2019.

Comments: Accepted to IEEE Robotics and Automation Letters (RAL)

Map-aided Dead-reckoning --- A Study on Locational Privacy in Insurance Telematics

Authors: Johan Wahlström, Isaac Skog, João G. P. Rodrigues, Peter Händel, Ana Aguiar

Abstract: We present a particle-based framework for estimating the position of a vehicle using map information and measurements of speed. Two measurement functions are considered. The first is based on the assumption that the lateral force on the vehicle does not exceed critical limits derived from physical constraints. The second is based on the assumption that the driver approaches a target speed derived… ▽ More We present a particle-based framework for estimating the position of a vehicle using map information and measurements of speed. Two measurement functions are considered. The first is based on the assumption that the lateral force on the vehicle does not exceed critical limits derived from physical constraints. The second is based on the assumption that the driver approaches a target speed derived from the speed limits along the upcoming trajectory. Performance evaluations of the proposed method indicate that end destinations often can be estimated with an accuracy in the order of $100\,[m]$. These results expose the sensitivity and commercial value of data collected in many of today's insurance telematics programs, and thereby have privacy implications for millions of policyholders. We end by discussing the strengths and weaknesses of different methods for anonymization and privacy preservation in telematics programs. △ Less

Submitted 14 November, 2016; originally announced November 2016.

Smartphone-based Vehicle Telematics - A Ten-Year Anniversary

Authors: Johan Wahlström, Isaac Skog, Peter Händel

Abstract: Just like it has irrevocably reshaped social life, the fast growth of smartphone ownership is now beginning to revolutionize the driving experience and change how we think about automotive insurance, vehicle safety systems, and traffic research. This paper summarizes the first ten years of research in smartphone-based vehicle telematics, with a focus on user-friendly implementations and the challe… ▽ More Just like it has irrevocably reshaped social life, the fast growth of smartphone ownership is now beginning to revolutionize the driving experience and change how we think about automotive insurance, vehicle safety systems, and traffic research. This paper summarizes the first ten years of research in smartphone-based vehicle telematics, with a focus on user-friendly implementations and the challenges that arise due to the mobility of the smartphone. Notable academic and industrial projects are reviewed, and system aspects related to sensors, energy consumption, cloud computing, vehicular ad hoc networks, and human-machine interfaces are examined. Moreover, we highlight the differences between traditional and smartphonebased automotive navigation, and survey the state-of-the-art in smartphone-based transportation mode classification, driver classification, and road condition monitoring. Future advances are expected to be driven by improvements in sensor technology, evidence of the societal benefits of current implementations, and the establishment of industry standards for sensor fusion and driver assessment △ Less

Submitted 11 November, 2016; originally announced November 2016.