Showing 1–8 of 8 results for author: She, T

Leveraging Large Language Models for Concept Graph Recovery and Question Answering in NLP Education

Authors: Rui Yang, Boming Yang, Sixun Ouyang, Tianwei She, Aosong Feng, Yuang Jiang, Freddy Lecue, **ghui Lu, Irene Li

Abstract: In the domain of Natural Language Processing (NLP), Large Language Models (LLMs) have demonstrated promise in text-generation tasks. However, their educational applications, particularly for domain-specific queries, remain underexplored. This study investigates LLMs' capabilities in educational scenarios, focusing on concept graph recovery and question-answering (QA). We assess LLMs' zero-shot per… ▽ More In the domain of Natural Language Processing (NLP), Large Language Models (LLMs) have demonstrated promise in text-generation tasks. However, their educational applications, particularly for domain-specific queries, remain underexplored. This study investigates LLMs' capabilities in educational scenarios, focusing on concept graph recovery and question-answering (QA). We assess LLMs' zero-shot performance in creating domain-specific concept graphs and introduce TutorQA, a new expert-verified NLP-focused benchmark for scientific graph reasoning and QA. TutorQA consists of five tasks with 500 QA pairs. To tackle TutorQA queries, we present CGLLM, a pipeline integrating concept graphs with LLMs for answering diverse questions. Our results indicate that LLMs' zero-shot concept graph recovery is competitive with supervised methods, showing an average 3% F1 score improvement. In TutorQA tasks, LLMs achieve up to 26% F1 score enhancement. Moreover, human evaluation and analysis show that CGLLM generates answers with more fine-grained concepts. △ Less

Submitted 22 February, 2024; originally announced February 2024.

Lips Are Lying: Spotting the Temporal Inconsistency between Audio and Visual in Lip-Syncing DeepFakes

Authors: Weifeng Liu, Tianyi She, Jiawei Liu, Run Wang, Dongyu Yao, Ziyou Liang

Abstract: In recent years, DeepFake technology has achieved unprecedented success in high-quality video synthesis, whereas these methods also pose potential and severe security threats to humanity. DeepFake can be bifurcated into entertainment applications like face swap** and illicit uses such as lip-syncing fraud. However, lip-forgery videos, which neither change identity nor have discernible visual art… ▽ More In recent years, DeepFake technology has achieved unprecedented success in high-quality video synthesis, whereas these methods also pose potential and severe security threats to humanity. DeepFake can be bifurcated into entertainment applications like face swap** and illicit uses such as lip-syncing fraud. However, lip-forgery videos, which neither change identity nor have discernible visual artifacts, present a formidable challenge to existing DeepFake detection methods. Our preliminary experiments have shown that the effectiveness of the existing methods often drastically decreases or even fails when tackling lip-syncing videos. In this paper, for the first time, we propose a novel approach dedicated to lip-forgery identification that exploits the inconsistency between lip movements and audio signals. We also mimic human natural cognition by capturing subtle biological links between lips and head regions to boost accuracy. To better illustrate the effectiveness and advances of our proposed method, we curate a high-quality LipSync dataset by employing the SOTA lip generator. We hope this high-quality and diverse dataset could be well served the further research on this challenging and interesting field. Experimental results show that our approach gives an average accuracy of more than 95.3% in spotting lip-syncing videos, significantly outperforming the baselines. Extensive experiments demonstrate the capability to tackle deepfakes and the robustness in surviving diverse input transformations. Our method achieves an accuracy of up to 90.2% in real-world scenarios (e.g., WeChat video call) and shows its powerful capabilities in real scenario deployment. To facilitate the progress of this research community, we release all resources at https://github.com/AaronComo/LipFD. △ Less

Submitted 28 January, 2024; originally announced January 2024.

Comments: The first two authors contributed equally to this work

Large Language Models on Wikipedia-Style Survey Generation: an Evaluation in NLP Concepts

Authors: Fan Gao, Hang Jiang, Rui Yang, Qingcheng Zeng, **ghui Lu, Moritz Blum, Dairui Liu, Tianwei She, Yuang Jiang, Irene Li

Abstract: Educational materials such as survey articles in specialized fields like computer science traditionally require tremendous expert inputs and are therefore expensive to create and update. Recently, Large Language Models (LLMs) have achieved significant success across various general tasks. However, their effectiveness and limitations in the education domain are yet to be fully explored. In this wor… ▽ More Educational materials such as survey articles in specialized fields like computer science traditionally require tremendous expert inputs and are therefore expensive to create and update. Recently, Large Language Models (LLMs) have achieved significant success across various general tasks. However, their effectiveness and limitations in the education domain are yet to be fully explored. In this work, we examine the proficiency of LLMs in generating succinct survey articles specific to the niche field of NLP in computer science, focusing on a curated list of 99 topics. Automated benchmarks reveal that GPT-4 surpasses its predecessors, inluding GPT-3.5, PaLM2, and LLaMa2 by margins ranging from 2% to 20% in comparison to the established ground truth. We compare both human and GPT-based evaluation scores and provide in-depth analysis. While our findings suggest that GPT-created surveys are more contemporary and accessible than human-authored ones, certain limitations were observed. Notably, GPT-4, despite often delivering outstanding content, occasionally exhibited lapses like missing details or factual errors. At last, we compared the rating behavior between humans and GPT-4 and found systematic bias in using GPT evaluation. △ Less

Submitted 23 May, 2024; v1 submitted 20 August, 2023; originally announced August 2023.

Journal ref: ACL 2024 Findings

Free Fine-tuning: A Plug-and-Play Watermarking Scheme for Deep Neural Networks

Authors: Run Wang, Jixing Ren, Boheng Li, Tianyi She, Chenhao Lin, Liming Fang, **g Chen, Chao Shen, Lina Wang

Abstract: Watermarking has been widely adopted for protecting the intellectual property (IP) of Deep Neural Networks (DNN) to defend the unauthorized distribution. Unfortunately, the popular data-poisoning DNN watermarking scheme relies on target model fine-tuning to embed watermarks, which limits its practical applications in tackling real-world tasks. Specifically, the learning of watermarks via tedious m… ▽ More Watermarking has been widely adopted for protecting the intellectual property (IP) of Deep Neural Networks (DNN) to defend the unauthorized distribution. Unfortunately, the popular data-poisoning DNN watermarking scheme relies on target model fine-tuning to embed watermarks, which limits its practical applications in tackling real-world tasks. Specifically, the learning of watermarks via tedious model fine-tuning on a poisoned dataset (carefully-crafted sample-label pairs) is not efficient in tackling the tasks on challenging datasets and production-level DNN model protection. To address the aforementioned limitations, in this paper, we propose a plug-and-play watermarking scheme for DNN models by injecting an independent proprietary model into the target model to serve the watermark embedding and ownership verification. In contrast to the prior studies, our proposed method by incorporating a proprietary model is free of target model fine-tuning without involving any parameters update of the target model, thus the fidelity is well preserved. Our research findings reveal that model fine-tuning with poisoned data is not prepared for the IP protection of DNN models deployed in real-world tasks and poses a new research direction toward a more thorough understanding and investigation of adopting the proprietary model for DNN watermarking. The source code and models are available at https://github.com/AntigoneRandy/PTYNet. △ Less

Submitted 16 October, 2022; v1 submitted 14 October, 2022; originally announced October 2022.

On Hermitian Adjacency Matrices for Mixed Graphs

Authors: Tao She, Chunxiang Wang

Abstract: We study the spectra of mixed graphs about its Hermitian adjacency matrix of the second kind (i.e. N-matrix) introduced by Mohar [1]. We extend some results and define one new Hermitian adjacency matrix, and the entry corresponding to an arc from $u$ to $v$ is equal to the $k$-th( or the third) root of unity, i.e. $ω = cos(2π/k) + \textbf{i} \ sin(2π/k), k {\geq} 3$; the entry corresponding to an… ▽ More We study the spectra of mixed graphs about its Hermitian adjacency matrix of the second kind (i.e. N-matrix) introduced by Mohar [1]. We extend some results and define one new Hermitian adjacency matrix, and the entry corresponding to an arc from $u$ to $v$ is equal to the $k$-th( or the third) root of unity, i.e. $ω = cos(2π/k) + \textbf{i} \ sin(2π/k), k {\geq} 3$; the entry corresponding to an undirected edge is equal to 1, and 0 otherwise. In this paper, we characterize the cospectrality conditions for a mixed graph and its underlying graph. In section 4, we determine a sharp upper bound on the spectral radius of mixed graphs, and provide the corresponding extremal graphs. △ Less

Submitted 7 June, 2022; v1 submitted 31 May, 2022; originally announced May 2022.

Comments: 14 pages, 5 figures

MSC Class: 05C50; 15A18

Application of Principal Component Analysis in Chinese Sovereign Bond Market and Principal Component-Based Fixed Income Immunization

Authors: Lim Tze Yee, Tony She, Kezia Irene

Abstract: This paper analyses the Chinese Sovereign bond yield to find out the principal factors affecting the term structure of interest rate changes. We apply Principal Component Analysis (PCA) on our data consisting of the Chinese Sovereign bond from January 2002 till May 2018 with the different yield to maturity. Then we will discuss the multi-factor immunization model (method on hedging market risk) on… ▽ More This paper analyses the Chinese Sovereign bond yield to find out the principal factors affecting the term structure of interest rate changes. We apply Principal Component Analysis (PCA) on our data consisting of the Chinese Sovereign bond from January 2002 till May 2018 with the different yield to maturity. Then we will discuss the multi-factor immunization model (method on hedging market risk) on a bond portfolio. △ Less

Submitted 17 November, 2019; originally announced November 2019.

Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model

Authors: Alexander R. Fabbri, Irene Li, Tianwei She, Suyi Li, Dragomir R. Radev

Abstract: Automatic generation of summaries from multiple news articles is a valuable tool as the number of online publications grows rapidly. Single document summarization (SDS) systems have benefited from advances in neural encoder-decoder model thanks to the availability of large datasets. However, multi-document summarization (MDS) of news articles has been limited to datasets of a couple of hundred exa… ▽ More Automatic generation of summaries from multiple news articles is a valuable tool as the number of online publications grows rapidly. Single document summarization (SDS) systems have benefited from advances in neural encoder-decoder model thanks to the availability of large datasets. However, multi-document summarization (MDS) of news articles has been limited to datasets of a couple of hundred examples. In this paper, we introduce Multi-News, the first large-scale MDS news dataset. Additionally, we propose an end-to-end model which incorporates a traditional extractive summarization model with a standard SDS model and achieves competitive results on MDS datasets. We benchmark several methods on Multi-News and release our data and code in hope that this work will promote advances in summarization in the multi-document setting. △ Less

Submitted 19 June, 2019; v1 submitted 4 June, 2019; originally announced June 2019.

Comments: ACL 2019, 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, 2019

Unsupervised Learning from Video with Deep Neural Embeddings

Authors: Chengxu Zhuang, Tianwei She, Alex Andonian, Max Sobol Mark, Daniel Yamins

Abstract: Because of the rich dynamical structure of videos and their ubiquity in everyday life, it is a natural idea that video data could serve as a powerful unsupervised learning signal for training visual representations in deep neural networks. However, instantiating this idea, especially at large scale, has remained a significant artificial intelligence challenge. Here we present the Video Instance Em… ▽ More Because of the rich dynamical structure of videos and their ubiquity in everyday life, it is a natural idea that video data could serve as a powerful unsupervised learning signal for training visual representations in deep neural networks. However, instantiating this idea, especially at large scale, has remained a significant artificial intelligence challenge. Here we present the Video Instance Embedding (VIE) framework, which extends powerful recent unsupervised loss functions for learning deep nonlinear embeddings to multi-stream temporal processing architectures on large-scale video datasets. We show that VIE-trained networks substantially advance the state of the art in unsupervised learning from video datastreams, both for action recognition in the Kinetics dataset, and object recognition in the ImageNet dataset. We show that a hybrid model with both static and dynamic processing pathways is optimal for both transfer tasks, and provide analyses indicating how the pathways differ. Taken in context, our results suggest that deep neural embeddings are a promising approach to unsupervised visual learning across a wide variety of domains. △ Less

Submitted 10 March, 2020; v1 submitted 28 May, 2019; originally announced May 2019.

Comments: To appear in CVPR 2020