-
AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model
Authors:
Seungwhan Moon,
Andrea Madotto,
Zhaojiang Lin,
Tushar Nagarajan,
Matt Smith,
Shashank Jain,
Chun-Fu Yeh,
Prakash Murugesan,
Peyman Heidari,
Yue Liu,
Kavya Srinet,
Babak Damavandi,
Anuj Kumar
Abstract:
We present Any-Modality Augmented Language Model (AnyMAL), a unified model that reasons over diverse input modality signals (i.e. text, image, video, audio, IMU motion sensor), and generates textual responses. AnyMAL inherits the powerful text-based reasoning abilities of the state-of-the-art LLMs including LLaMA-2 (70B), and converts modality-specific signals to the joint textual space through a…
▽ More
We present Any-Modality Augmented Language Model (AnyMAL), a unified model that reasons over diverse input modality signals (i.e. text, image, video, audio, IMU motion sensor), and generates textual responses. AnyMAL inherits the powerful text-based reasoning abilities of the state-of-the-art LLMs including LLaMA-2 (70B), and converts modality-specific signals to the joint textual space through a pre-trained aligner module. To further strengthen the multimodal LLM's capabilities, we fine-tune the model with a multimodal instruction set manually collected to cover diverse topics and tasks beyond simple QAs. We conduct comprehensive empirical analysis comprising both human and automatic evaluations, and demonstrate state-of-the-art performance on various multimodal tasks.
△ Less
Submitted 27 September, 2023;
originally announced September 2023.
-
Forensic Data Analytics for Anomaly Detection in Evolving Networks
Authors:
Li Yang,
Abdallah Moubayed,
Abdallah Shami,
Amine Boukhtouta,
Parisa Heidari,
Stere Preda,
Richard Brunner,
Daniel Migault,
Adel Larabi
Abstract:
In the prevailing convergence of traditional infrastructure-based deployment (i.e., Telco and industry operational networks) towards evolving deployments enabled by 5G and virtualization, there is a keen interest in elaborating effective security controls to protect these deployments in-depth. By considering key enabling technologies like 5G and virtualization, evolving networks are democratized,…
▽ More
In the prevailing convergence of traditional infrastructure-based deployment (i.e., Telco and industry operational networks) towards evolving deployments enabled by 5G and virtualization, there is a keen interest in elaborating effective security controls to protect these deployments in-depth. By considering key enabling technologies like 5G and virtualization, evolving networks are democratized, facilitating the establishment of point presences integrating different business models ranging from media, dynamic web content, gaming, and a plethora of IoT use cases. Despite the increasing services provided by evolving networks, many cybercrimes and attacks have been launched in evolving networks to perform malicious activities. Due to the limitations of traditional security artifacts (e.g., firewalls and intrusion detection systems), the research on digital forensic data analytics has attracted more attention. Digital forensic analytics enables people to derive detailed information and comprehensive conclusions from different perspectives of cybercrimes to assist in convicting criminals and preventing future crimes. This chapter presents a digital analytics framework for network anomaly detection, including multi-perspective feature engineering, unsupervised anomaly detection, and comprehensive result correction procedures. Experiments on real-world evolving network data show the effectiveness of the proposed forensic data analytics solution.
△ Less
Submitted 17 August, 2023;
originally announced August 2023.
-
A Study on the Efficiency and Generalization of Light Hybrid Retrievers
Authors:
Man Luo,
Shashank Jain,
Anchit Gupta,
Arash Einolghozati,
Barlas Oguz,
Debojeet Chatterjee,
Xilun Chen,
Chitta Baral,
Peyman Heidari
Abstract:
Hybrid retrievers can take advantage of both sparse and dense retrievers. Previous hybrid retrievers leverage indexing-heavy dense retrievers. In this work, we study "Is it possible to reduce the indexing memory of hybrid retrievers without sacrificing performance"? Driven by this question, we leverage an indexing-efficient dense retriever (i.e. DrBoost) and introduce a LITE retriever that further…
▽ More
Hybrid retrievers can take advantage of both sparse and dense retrievers. Previous hybrid retrievers leverage indexing-heavy dense retrievers. In this work, we study "Is it possible to reduce the indexing memory of hybrid retrievers without sacrificing performance"? Driven by this question, we leverage an indexing-efficient dense retriever (i.e. DrBoost) and introduce a LITE retriever that further reduces the memory of DrBoost. LITE is jointly trained on contrastive learning and knowledge distillation from DrBoost. Then, we integrate BM25, a sparse retriever, with either LITE or DrBoost to form light hybrid retrievers. Our Hybrid-LITE retriever saves 13X memory while maintaining 98.0% performance of the hybrid retriever of BM25 and DPR. In addition, we study the generalization capacity of our light hybrid retrievers on out-of-domain dataset and a set of adversarial attacks datasets. Experiments showcase that light hybrid retrievers achieve better generalization performance than individual sparse and dense retrievers. Nevertheless, our analysis shows that there is a large room to improve the robustness of retrievers, suggesting a new research direction.
△ Less
Submitted 23 May, 2023; v1 submitted 4 October, 2022;
originally announced October 2022.
-
Spach Transformer: Spatial and Channel-wise Transformer Based on Local and Global Self-attentions for PET Image Denoising
Authors:
Se-In Jang,
Tinsu Pan,
Ye Li,
Pedram Heidari,
Junyu Chen,
Quanzheng Li,
Kuang Gong
Abstract:
Position emission tomography (PET) is widely used in clinics and research due to its quantitative merits and high sensitivity, but suffers from low signal-to-noise ratio (SNR). Recently convolutional neural networks (CNNs) have been widely used to improve PET image quality. Though successful and efficient in local feature extraction, CNN cannot capture long-range dependencies well due to its limit…
▽ More
Position emission tomography (PET) is widely used in clinics and research due to its quantitative merits and high sensitivity, but suffers from low signal-to-noise ratio (SNR). Recently convolutional neural networks (CNNs) have been widely used to improve PET image quality. Though successful and efficient in local feature extraction, CNN cannot capture long-range dependencies well due to its limited receptive field. Global multi-head self-attention (MSA) is a popular approach to capture long-range information. However, the calculation of global MSA for 3D images has high computational costs. In this work, we proposed an efficient spatial and channel-wise encoder-decoder transformer, Spach Transformer, that can leverage spatial and channel information based on local and global MSAs. Experiments based on datasets of different PET tracers, i.e., $^{18}$F-FDG, $^{18}$F-ACBC, $^{18}$F-DCFPyL, and $^{68}$Ga-DOTATATE, were conducted to evaluate the proposed framework. Quantitative results show that the proposed Spach Transformer framework outperforms state-of-the-art deep learning architectures. Our codes are available at https://github.com/sijang/SpachTransformer
△ Less
Submitted 10 December, 2023; v1 submitted 7 September, 2022;
originally announced September 2022.
-
Multi-Perspective Content Delivery Networks Security Framework Using Optimized Unsupervised Anomaly Detection
Authors:
Li Yang,
Abdallah Moubayed,
Abdallah Shami,
Parisa Heidari,
Amine Boukhtouta,
Adel Larabi,
Richard Brunner,
Stere Preda,
Daniel Migault
Abstract:
Content delivery networks (CDNs) provide efficient content distribution over the Internet. CDNs improve the connectivity and efficiency of global communications, but their caching mechanisms may be breached by cyber-attackers. Among the security mechanisms, effective anomaly detection forms an important part of CDN security enhancement. In this work, we propose a multi-perspective unsupervised lea…
▽ More
Content delivery networks (CDNs) provide efficient content distribution over the Internet. CDNs improve the connectivity and efficiency of global communications, but their caching mechanisms may be breached by cyber-attackers. Among the security mechanisms, effective anomaly detection forms an important part of CDN security enhancement. In this work, we propose a multi-perspective unsupervised learning framework for anomaly detection in CDNs. In the proposed framework, a multi-perspective feature engineering approach, an optimized unsupervised anomaly detection model that utilizes an isolation forest and a Gaussian mixture model, and a multi-perspective validation method, are developed to detect abnormal behaviors in CDNs mainly from the client Internet Protocol (IP) and node perspectives, therefore to identify the denial of service (DoS) and cache pollution attack (CPA) patterns. Experimental results are presented based on the analytics of eight days of real-world CDN log data provided by a major CDN operator. Through experiments, the abnormal contents, compromised nodes, malicious IPs, as well as their corresponding attack types, are identified effectively by the proposed framework and validated by multiple cybersecurity experts. This shows the effectiveness of the proposed method when applied to real-world CDN data.
△ Less
Submitted 23 July, 2021;
originally announced July 2021.
-
Best Practices for Data-Efficient Modeling in NLG:How to Train Production-Ready Neural Models with Less Data
Authors:
Ankit Arun,
Soumya Batra,
Vikas Bhardwaj,
Ashwini Challa,
Pinar Donmez,
Peyman Heidari,
Hakan Inan,
Shashank Jain,
Anuj Kumar,
Shawn Mei,
Karthik Mohan,
Michael White
Abstract:
Natural language generation (NLG) is a critical component in conversational systems, owing to its role of formulating a correct and natural text response. Traditionally, NLG components have been deployed using template-based solutions. Although neural network solutions recently developed in the research community have been shown to provide several benefits, deployment of such model-based solutions…
▽ More
Natural language generation (NLG) is a critical component in conversational systems, owing to its role of formulating a correct and natural text response. Traditionally, NLG components have been deployed using template-based solutions. Although neural network solutions recently developed in the research community have been shown to provide several benefits, deployment of such model-based solutions has been challenging due to high latency, correctness issues, and high data needs. In this paper, we present approaches that have helped us deploy data-efficient neural solutions for NLG in conversational systems to production. We describe a family of sampling and modeling techniques to attain production quality with light-weight neural network models using only a fraction of the data that would be necessary otherwise, and show a thorough comparison between each. Our results show that domain complexity dictates the appropriate approach to achieve high data efficiency. Finally, we distill the lessons from our experimental findings into a list of best practices for production-level NLG model development, and present them in a brief runbook. Importantly, the end products of all of the techniques are small sequence-to-sequence models (2Mb) that we can reliably deploy in production.
△ Less
Submitted 7 November, 2020;
originally announced November 2020.
-
Cost-optimal V2X Service Placement in Distributed Cloud/Edge Environment
Authors:
Abdallah Moubayed,
Abdallah Shami,
Parisa Heidari,
Adel Larabi,
Richard Brunner
Abstract:
Deploying V2X services has become a challenging task. This is mainly due to the fact that such services have strict latency requirements. To meet these requirements, one potential solution is adopting mobile edge computing (MEC). However, this presents new challenges including how to find a cost efficient placement that meets other requirements such as latency. In this work, the problem of cost-op…
▽ More
Deploying V2X services has become a challenging task. This is mainly due to the fact that such services have strict latency requirements. To meet these requirements, one potential solution is adopting mobile edge computing (MEC). However, this presents new challenges including how to find a cost efficient placement that meets other requirements such as latency. In this work, the problem of cost-optimal V2X service placement (CO-VSP) in a distributed cloud/edge environment is formulated. Additionally, a cost-focused delay-aware V2X service placement (DA-VSP) heuristic algorithm is proposed. Simulation results show that both CO-VSP model and DA-VSP algorithm guarantee the QoS requirements of all such services and illustrates the trade-off between latency and deployment cost.
△ Less
Submitted 14 October, 2020;
originally announced October 2020.
-
Machine Learning for Performance-Aware Virtual Network Function Placement
Authors:
Dimitrios Michael Manias,
Manar Jammal,
Hassan Hawilo,
Abdallah Shami,
Parisa Heidari,
Adel Larabi,
Richard Brunner
Abstract:
With the growing demand for data connectivity, network service providers are faced with the task of reducing their capital and operational expenses while simultaneously improving network performance and addressing the increased connectivity demand. Although Network Function Virtualization (NFV) has been identified as a solution, several challenges must be addressed to ensure its feasibility. In th…
▽ More
With the growing demand for data connectivity, network service providers are faced with the task of reducing their capital and operational expenses while simultaneously improving network performance and addressing the increased connectivity demand. Although Network Function Virtualization (NFV) has been identified as a solution, several challenges must be addressed to ensure its feasibility. In this paper, we address the Virtual Network Function (VNF) placement problem by develo** a machine learning decision tree model that learns from the effective placement of the various VNF instances forming a Service Function Chain (SFC). The model takes several performance-related features from the network as an input and selects the placement of the various VNF instances on network servers with the objective of minimizing the delay between dependent VNF instances. The benefits of using machine learning are realized by moving away from a complex mathematical modelling of the system and towards a data-based understanding of the system. Using the Evolved Packet Core (EPC) as a use case, we evaluate our model on different data center networks and compare it to the BACON algorithm in terms of the delay between interconnected components and the total delay across the SFC. Furthermore, a time complexity analysis is performed to show the effectiveness of the model in NFV applications.
△ Less
Submitted 13 January, 2020;
originally announced January 2020.
-
Edge-enabled V2X Service Placement for Intelligent Transportation Systems
Authors:
Abdallah Moubayed,
Abdallah Shami,
Parisa Heidari,
Adel Larabi,
Richard Brunner
Abstract:
Vehicle-to-everything (V2X) communication and services have been garnering significant interest from different stakeholders as part of future intelligent transportation systems (ITSs). This is due to the many benefits they offer. However, many of these services have stringent performance requirements, particularly in terms of the delay/latency. Multi-access/mobile edge computing (MEC) has been pro…
▽ More
Vehicle-to-everything (V2X) communication and services have been garnering significant interest from different stakeholders as part of future intelligent transportation systems (ITSs). This is due to the many benefits they offer. However, many of these services have stringent performance requirements, particularly in terms of the delay/latency. Multi-access/mobile edge computing (MEC) has been proposed as a potential solution for such services by bringing them closer to vehicles. Yet, this introduces a new set of challenges such as where to place these V2X services, especially given the limit computation resources available at edge nodes. To that end, this work formulates the problem of optimal V2X service placement (OVSP) in a hybrid core/edge environment as a binary integer linear programming problem. To the best of our knowledge, no previous work considered the V2X service placement problem while taking into consideration the computational resource availability at the nodes. Moreover, a low-complexity greedy-based heuristic algorithm named "Greedy V2X Service Placement Algorithm" (G-VSPA) was developed to solve this problem. Simulation results show that the OVSP model successfully guarantees and maintains the QoS requirements of all the different V2X services. Additionally, it is observed that the proposed G-VSPA algorithm achieves close to optimal performance while having lower complexity.
△ Less
Submitted 13 January, 2020;
originally announced January 2020.
