Showing 1–12 of 12 results for author: Ionescu, A

Weight Copy and Low-Rank Adaptation for Few-Shot Distillation of Vision Transformers

Authors: Diana-Nicoleta Grigore, Mariana-Iuliana Georgescu, Jon Alvarez Justo, Tor Johansen, Andreea Iuliana Ionescu, Radu Tudor Ionescu

Abstract: Few-shot knowledge distillation recently emerged as a viable approach to harness the knowledge of large-scale pre-trained models, using limited data and computational resources. In this paper, we propose a novel few-shot feature distillation approach for vision transformers. Our approach is based on two key steps. Leveraging the fact that vision transformers have a consistent depth-wise structure,… ▽ More Few-shot knowledge distillation recently emerged as a viable approach to harness the knowledge of large-scale pre-trained models, using limited data and computational resources. In this paper, we propose a novel few-shot feature distillation approach for vision transformers. Our approach is based on two key steps. Leveraging the fact that vision transformers have a consistent depth-wise structure, we first copy the weights from intermittent layers of existing pre-trained vision transformers (teachers) into shallower architectures (students), where the intermittence factor controls the complexity of the student transformer with respect to its teacher. Next, we employ an enhanced version of Low-Rank Adaptation (LoRA) to distill knowledge into the student in a few-shot scenario, aiming to recover the information processing carried out by the skipped teacher layers. We present comprehensive experiments with supervised and self-supervised transformers as teachers, on five data sets from various domains, including natural, medical and satellite images. The empirical results confirm the superiority of our approach over competitive baselines. Moreover, the ablation results demonstrate the usefulness of each component of the proposed pipeline. △ Less

Submitted 17 April, 2024; v1 submitted 14 April, 2024; originally announced April 2024.

TEE4EHR: Transformer Event Encoder for Better Representation Learning in Electronic Health Records

Authors: Hojjat Karami, David Atienza, Anisoara Ionescu

Abstract: Irregular sampling of time series in electronic health records (EHRs) is one of the main challenges for develo** machine learning models. Additionally, the pattern of missing data in certain clinical variables is not at random but depends on the decisions of clinicians and the state of the patient. Point process is a mathematical framework for analyzing event sequence data that is consistent wit… ▽ More Irregular sampling of time series in electronic health records (EHRs) is one of the main challenges for develo** machine learning models. Additionally, the pattern of missing data in certain clinical variables is not at random but depends on the decisions of clinicians and the state of the patient. Point process is a mathematical framework for analyzing event sequence data that is consistent with irregular sampling patterns. Our model, TEE4EHR, is a transformer event encoder (TEE) with point process loss that encodes the pattern of laboratory tests in EHRs. The utility of our TEE has been investigated in a variety of benchmark event sequence datasets. Additionally, we conduct experiments on two real-world EHR databases to provide a more comprehensive evaluation of our model. Firstly, in a self-supervised learning approach, the TEE is jointly learned with an existing attention-based deep neural network which gives superior performance in negative log-likelihood and future event prediction. Besides, we propose an algorithm for aggregating attention weights that can reveal the interaction between the events. Secondly, we transfer and freeze the learned TEE to the downstream task for the outcome prediction, where it outperforms state-of-the-art models for handling irregularly sampled time series. Furthermore, our results demonstrate that our approach can improve representation learning in EHRs and can be useful for clinical prediction tasks. △ Less

Submitted 9 February, 2024; originally announced February 2024.

TimEHR: Image-based Time Series Generation for Electronic Health Records

Authors: Hojjat Karami, Mary-Anne Hartley, David Atienza, Anisoara Ionescu

Abstract: Time series in Electronic Health Records (EHRs) present unique challenges for generative models, such as irregular sampling, missing values, and high dimensionality. In this paper, we propose a novel generative adversarial network (GAN) model, TimEHR, to generate time series data from EHRs. In particular, TimEHR treats time series as images and is based on two conditional GANs. The first GAN gener… ▽ More Time series in Electronic Health Records (EHRs) present unique challenges for generative models, such as irregular sampling, missing values, and high dimensionality. In this paper, we propose a novel generative adversarial network (GAN) model, TimEHR, to generate time series data from EHRs. In particular, TimEHR treats time series as images and is based on two conditional GANs. The first GAN generates missingness patterns, and the second GAN generates time series values based on the missingness pattern. Experimental results on three real-world EHR datasets show that TimEHR outperforms state-of-the-art methods in terms of fidelity, utility, and privacy metrics. △ Less

Submitted 9 February, 2024; originally announced February 2024.

Amalur: Data Integration Meets Machine Learning

Authors: Rihan Hai, Christos Koutras, Andra Ionescu, Ziyu Li, Wenbo Sun, Jessie van Schijndel, Yan Kang, Asterios Katsifodimos

Abstract: The data needed for machine learning (ML) model training, can reside in different separate sites often termed data silos. For data-intensive ML applications, data silos pose a major challenge: the integration and transformation of data demand a lot of manual work and computational resources. With data privacy and security constraints, data often cannot leave the local sites, and a model has to be… ▽ More The data needed for machine learning (ML) model training, can reside in different separate sites often termed data silos. For data-intensive ML applications, data silos pose a major challenge: the integration and transformation of data demand a lot of manual work and computational resources. With data privacy and security constraints, data often cannot leave the local sites, and a model has to be trained in a decentralized manner. In this work, we present a vision on how to bridge the traditional data integration (DI) techniques with the requirements of modern machine learning. We explore the possibilities of utilizing metadata obtained from data integration processes for improving the effectiveness and efficiency of ML models. We analyze two common use cases over data silos, feature augmentation and federated learning. Bringing data integration and machine learning together, we highlight the new research opportunities from the aspects of systems, representations, factorized learning and federated learning. △ Less

Submitted 1 March, 2023; v1 submitted 19 May, 2022; originally announced May 2022.

Comments: Accepted at ICDE2023 -- Special track (Vision)

Coupled VO2 oscillators circuit as analog first layer filter in convolutional neural networks

Authors: Elisabetta Corti, Joaquin Antonio Cornejo Jimenez, Kham M. Niang, John Robertson, Kirsten E. Moselund, Bernd Gotsmann, Adrian M. Ionescu, Siegfried Karg

Abstract: In this work we present an in-memory computing platform based on coupled VO2 oscillators fabricated in a crossbar configuration on silicon. Compared to existing platforms, the crossbar configuration promises significant improvements in terms of area density and oscillation frequency. Further, the crossbar devices exhibit low variability and extended reliability, hence, enabling experiments on 4-co… ▽ More In this work we present an in-memory computing platform based on coupled VO2 oscillators fabricated in a crossbar configuration on silicon. Compared to existing platforms, the crossbar configuration promises significant improvements in terms of area density and oscillation frequency. Further, the crossbar devices exhibit low variability and extended reliability, hence, enabling experiments on 4-coupled oscillator. We demonstrate the neuromorphic computing capabilities using the phase relation of the oscillators. As a application, we propose to replace digital filtering operation in a convolutional neural network with oscillating circuits. The concept is tested with a VGG13 architecture on the MNIST dataset, achieving performances of 95% in the recognition task. △ Less

Submitted 18 November, 2020; originally announced November 2020.

Valentine: Evaluating Matching Techniques for Dataset Discovery

Authors: Christos Koutras, George Siachamis, Andra Ionescu, Kyriakos Psarakis, Jerry Brons, Marios Fragkoulis, Christoph Lofi, Angela Bonifati, Asterios Katsifodimos

Abstract: Data scientists today search large data lakes to discover and integrate datasets. In order to bring together disparate data sources, dataset discovery methods rely on some form of schema matching: the process of establishing correspondences between datasets. Traditionally, schema matching has been used to find matching pairs of columns between a source and a target schema. However, the use of sche… ▽ More Data scientists today search large data lakes to discover and integrate datasets. In order to bring together disparate data sources, dataset discovery methods rely on some form of schema matching: the process of establishing correspondences between datasets. Traditionally, schema matching has been used to find matching pairs of columns between a source and a target schema. However, the use of schema matching in dataset discovery methods differs from its original use. Nowadays schema matching serves as a building block for indicating and ranking inter-dataset relationships. Surprisingly, although a discovery method's success relies highly on the quality of the underlying matching algorithms, the latest discovery methods employ existing schema matching algorithms in an ad-hoc fashion due to the lack of openly-available datasets with ground truth, reference method implementations, and evaluation metrics. In this paper, we aim to rectify the problem of evaluating the effectiveness and efficiency of schema matching methods for the specific needs of dataset discovery. To this end, we propose Valentine, an extensible open-source experiment suite to execute and organize large-scale automated matching experiments on tabular data. Valentine includes implementations of seminal schema matching methods that we either implemented from scratch (due to absence of open source code) or imported from open repositories. The contributions of Valentine are: i) the definition of four schema matching scenarios as encountered in dataset discovery methods, ii) a principled dataset fabrication process tailored to the scope of dataset discovery methods and iii) the most comprehensive evaluation of schema matching techniques to date, offering insight on the strengths and weaknesses of existing techniques, that can serve as a guide for employing schema matching in future dataset discovery methods. △ Less

Submitted 13 February, 2021; v1 submitted 14 October, 2020; originally announced October 2020.

Reconfigurable radiofrequency electronic functions designed with 3D Smith Charts in Metal-Insulator-Transition Materials

Authors: Andrei Muller, Alin Moldoveanu, Victor Asavei, Riyaz Khadar, Esther Sanabria Codesal, Anna Krammer, Montserrat Fernandez-Bolaños, Matteo Cavalleri, Junrui Zhang, Emanuele Casu, Andreas Schuler, Adrian Mihai Ionescu

Abstract: Recently, the field of Metal-Insulator-Transition (MIT) materials has emerged as an unconventional solution for novel energy efficient electronic functions, such as steep slope subthermionic switches, neuromorphic hardware, reconfigurable radiofrequency functions, new types of sensors, teraherz and optoelectronic devices. Designing radiofrequency (RF) electronic circuits with a MIT material like v… ▽ More Recently, the field of Metal-Insulator-Transition (MIT) materials has emerged as an unconventional solution for novel energy efficient electronic functions, such as steep slope subthermionic switches, neuromorphic hardware, reconfigurable radiofrequency functions, new types of sensors, teraherz and optoelectronic devices. Designing radiofrequency (RF) electronic circuits with a MIT material like vanadium dioxide, VO2, requires the understanding of its physics and appropriate models and tools, with predictive capability over large range of frequency (1-100GHz). Here, we develop 3D Smith charts for devices and circuits having complex frequency dependences, like the ones resulting by the use of MIT materials. The novel foundation of a 3D Smith chart involves here the geometrical fundamental notions of oriented curvature and variable homothety in order to clarify first theoretical inconsistencies in Foster and Non Foster circuits, where the driving point impedances exhibit mixed clockwise and counter-clockwise frequency dependent paths on the Smith chart as frequency increases. We show here the unique visualization capability of a 3D Smith chart, which allows to quantify orientation over variable frequency. The new 3D Smith chart is applied as a 3D multi-parameter modelling and design environment for the complex case of Metal-Insulator-Transition (MIT) materials where their permittivity is dependent on the frequency. In this work, we apply 3D Smith charts to on Vanadium Dioxide (VO2) reconfigurable Peano inductors. We report fabricated inductors with record quality factors using VO2 phase transition to program multiple tuning states, operating in the range 4 GHz to 10 GHz. Finally, we fabricate new Peano curves filters used to extract the frequency-dependent dielectric constant of VO2 within 1 GHz-50 GHz for the accurate design of RF electronic applications with phase change materials △ Less

Submitted 23 May, 2019; originally announced May 2019.

Comments: 36 pages, 13 article pages, 23 supplementary pages, 5 main article figures, 32 additional figures

Report number: 18346

Journal ref: Nature Scientific Reports 2019

Page Cache Attacks

Authors: Daniel Gruss, Erik Kraft, Trishita Tiwari, Michael Schwarz, Ari Trachtenberg, Jason Hennessey, Alex Ionescu, Anders Fogh

Abstract: We present a new hardware-agnostic side-channel attack that targets one of the most fundamental software caches in modern computer systems: the operating system page cache. The page cache is a pure software cache that contains all disk-backed pages, including program binaries, shared libraries, and other files, and our attacks thus work across cores and CPUs. Our side-channel permits unprivileged… ▽ More We present a new hardware-agnostic side-channel attack that targets one of the most fundamental software caches in modern computer systems: the operating system page cache. The page cache is a pure software cache that contains all disk-backed pages, including program binaries, shared libraries, and other files, and our attacks thus work across cores and CPUs. Our side-channel permits unprivileged monitoring of some memory accesses of other processes, with a spatial resolution of 4KB and a temporal resolution of 2 microseconds on Linux (restricted to 6.7 measurements per second) and 466 nanoseconds on Windows (restricted to 223 measurements per second); this is roughly the same order of magnitude as the current state-of-the-art cache attacks. We systematically analyze our side channel by demonstrating different local attacks, including a sandbox bypassing high-speed covert channel, timed user-interface redressing attacks, and an attack recovering automatically generated temporary passwords. We further show that we can trade off the side channel's hardware agnostic property for remote exploitability. We demonstrate this via a low profile remote covert channel that uses this page-cache side-channel to exfiltrate information from a malicious sender process through innocuous server requests. Finally, we propose mitigations for some of our attacks, which have been acknowledged by operating system vendors and slated for future security patches. △ Less

Submitted 4 January, 2019; originally announced January 2019.

Web Publishing of the Files Obtained by Flash

Authors: Virgiliu Streian, Adela Ionescu

Abstract: The aim of this article is to familiarize the user with the Web publishing of the files obtained by Flash. The article contains an overview of Macromedia Flash 5, as well as the running of a Playing Flash movie, information on Flash and Generator, the publishing of Flash movies, a HTLM publishing for Flash Player files and publishing by Generator templates. The aim of this article is to familiarize the user with the Web publishing of the files obtained by Flash. The article contains an overview of Macromedia Flash 5, as well as the running of a Playing Flash movie, information on Flash and Generator, the publishing of Flash movies, a HTLM publishing for Flash Player files and publishing by Generator templates. △ Less

Submitted 4 June, 2009; originally announced June 2009.

Comments: 10 pages, exposed on 5th International Conference "Actualities and Perspectives on Hardware and Software" - APHS2009, Timisoara, Romania

Journal ref: Ann. Univ. Tibiscus Comp. Sci. Series VII(2009),349-358

Token Ring Project

Authors: Virgiliu Streian, Adela Ionescu

Abstract: Ring topology is a simple configuration used to connect processes that communicate among themselves. A number of network standards such as token ring, token bus, and FDDI are based on the ring connectivity. This article will develop an implementation of a ring of processes that communicate among themselves via pipe links. The processes are nodes in the ring. Each process reads from its standard… ▽ More Ring topology is a simple configuration used to connect processes that communicate among themselves. A number of network standards such as token ring, token bus, and FDDI are based on the ring connectivity. This article will develop an implementation of a ring of processes that communicate among themselves via pipe links. The processes are nodes in the ring. Each process reads from its standard input and writes in its standard output. N-1 process redirects the its standard output to a standard input of the process through a pipe. When the ring-structure is designed, the project can be extended to simulate networks or to implement algorithms for mutual exclusion. △ Less

Submitted 25 March, 2009; originally announced March 2009.

Comments: 6 pages, exposed on 4th International Conferences "Actualities and Perspectives on Hardware and Software" - APHS2007, Timisoara, Romania

Journal ref: Ann. Univ. Tibiscus Comp. Sci. Series V (2007), 85-90

0-level Vacuum Packaging RT Process for MEMS Resonators

Authors: N. Abelé, D. Grogg, C. Hibert, F. Casset, P. Ancey, A. Ionescu

Abstract: A new Room Temperature (RT) 0-level vacuum package is demonstrated in this work, using amorphous silicon (aSi) as sacrificial layer and SiO2 as structural layer. The process is compatible with most of MEMS resonators and Resonant Suspended-Gate MOSFET [1] fabrication processes. This paper presents a study on the influence of releasing hole dimensions on the releasing time and hole clogging. It d… ▽ More A new Room Temperature (RT) 0-level vacuum package is demonstrated in this work, using amorphous silicon (aSi) as sacrificial layer and SiO2 as structural layer. The process is compatible with most of MEMS resonators and Resonant Suspended-Gate MOSFET [1] fabrication processes. This paper presents a study on the influence of releasing hole dimensions on the releasing time and hole clogging. It discusses mass production compatibility in terms of packaging stress during back-end plastic injection process. The packaging is done at room temperature making it fully compatible with IC-processed wafers and avoiding any subsequent degradation of the active devices. △ Less

Submitted 21 February, 2008; originally announced February 2008.

Comments: Submitted on behalf of EDA Publishing Association (http://irevues.inist.fr/EDA-Publishing)

Journal ref: Dans Symposium on Design, Test, Integration and Packaging of MEMS/MOEMS - DTIP 2007, Stresa, lago Maggiore : Italie (2007)

Fabrication of MEMS Resonators in Thin SOI

Authors: D. Grogg, Nicoleta Diana Badila-Ciressan, Adrian Mihai Ionescu

Abstract: A simple and fast process for micro-electromechanical (MEM) resonators with deep sub-micron transduction gaps in thin SOI is presented in this paper. Thin SOI wafers are important for advanced CMOS technology and thus are evaluated as resonator substrates for future co-integration with CMOS circuitry on a single chip. As the transduction capacitance scales with the resonator thickness, it is imp… ▽ More A simple and fast process for micro-electromechanical (MEM) resonators with deep sub-micron transduction gaps in thin SOI is presented in this paper. Thin SOI wafers are important for advanced CMOS technology and thus are evaluated as resonator substrates for future co-integration with CMOS circuitry on a single chip. As the transduction capacitance scales with the resonator thickness, it is important to fabricate deep sub-micron trenches in order to achieve a good capacitive coupling. Through the combination of conventional UV-lithography and focused ion beam (FIB) milling the process needs only two lithography steps, enabling therefore a way for fast prototy** of MEM-resonators. Different FIB parameters and etching parameters are compared in this paper and their effect on the process are reported. △ Less

Submitted 21 February, 2008; originally announced February 2008.

Comments: Submitted on behalf of EDA Publishing Association (http://irevues.inist.fr/EDA-Publishing)

Journal ref: Dans Symposium on Design, Test, Integration and Packaging of MEMS/MOEMS - DTIP 2007, Stresa, lago Maggiore : Italie (2007)