Showing 1–11 of 11 results for author: Jie, L

LLM-jp: A Cross-organizational Project for the Research and Development of Fully Open Japanese LLMs

Authors: LLM-jp, :, Akiko Aizawa, Eiji Aramaki, Bowen Chen, Fei Cheng, Hiroyuki Deguchi, Rintaro Enomoto, Kazuki Fujii, Kensuke Fukumoto, Takuya Fukushima, Namgi Han, Yuto Harada, Chikara Hashimoto, Tatsuya Hiraoka, Shohei Hisada, Sosuke Hosokawa, Lu Jie, Keisuke Kamata, Teruhito Kanazawa, Hiroki Kanezashi, Hiroshi Kataoka, Satoru Katsumata, Daisuke Kawahara, Seiya Kawano , et al. (57 additional authors not shown)

Abstract: This paper introduces LLM-jp, a cross-organizational project for the research and development of Japanese large language models (LLMs). LLM-jp aims to develop open-source and strong Japanese LLMs, and as of this writing, more than 1,500 participants from academia and industry are working together for this purpose. This paper presents the background of the establishment of LLM-jp, summaries of its… ▽ More This paper introduces LLM-jp, a cross-organizational project for the research and development of Japanese large language models (LLMs). LLM-jp aims to develop open-source and strong Japanese LLMs, and as of this writing, more than 1,500 participants from academia and industry are working together for this purpose. This paper presents the background of the establishment of LLM-jp, summaries of its activities, and technical reports on the LLMs developed by LLM-jp. For the latest activities, visit https://llm-jp.nii.ac.jp/en/. △ Less

Submitted 4 July, 2024; originally announced July 2024.

SuperEdge: Towards a Generalization Model for Self-Supervised Edge Detection

Authors: Leng Kai, Zhang Zhijie, Liu Jie, Zed Boukhers, Sui Wei, Cong Yang, Li Zhijun

Abstract: Edge detection is a fundamental technique in various computer vision tasks. Edges are indeed effectively delineated by pixel discontinuity and can offer reliable structural information even in textureless areas. State-of-the-art heavily relies on pixel-wise annotations, which are labor-intensive and subject to inconsistencies when acquired manually. In this work, we propose a novel self-supervised… ▽ More Edge detection is a fundamental technique in various computer vision tasks. Edges are indeed effectively delineated by pixel discontinuity and can offer reliable structural information even in textureless areas. State-of-the-art heavily relies on pixel-wise annotations, which are labor-intensive and subject to inconsistencies when acquired manually. In this work, we propose a novel self-supervised approach for edge detection that employs a multi-level, multi-homography technique to transfer annotations from synthetic to real-world datasets. To fully leverage the generated edge annotations, we developed SuperEdge, a streamlined yet efficient model capable of concurrently extracting edges at pixel-level and object-level granularity. Thanks to self-supervised training, our method eliminates the dependency on manual annotated edge labels, thereby enhancing its generalizability across diverse datasets. Comparative evaluations reveal that SuperEdge advances edge detection, demonstrating improvements of 4.9% in ODS and 3.3% in OIS over the existing STEdge method on BIPEDv2. △ Less

Submitted 4 January, 2024; originally announced January 2024.

Comments: 7pages

Secure Software Development: Issues and Challenges

Authors: Sam Wen **, Jeffrey Cheok Jun Wah, Lee Wen Jie, Jeremy Bong Yong Han, Saira Muzafar

Abstract: In recent years, technology has advanced considerably with the introduction of many systems including advanced robotics, big data analytics, cloud computing, machine learning and many more. The opportunities to exploit the yet to come security that comes with these systems are going toe to toe with new releases of security protocols to combat this exploitation to provide a secure system. The digit… ▽ More In recent years, technology has advanced considerably with the introduction of many systems including advanced robotics, big data analytics, cloud computing, machine learning and many more. The opportunities to exploit the yet to come security that comes with these systems are going toe to toe with new releases of security protocols to combat this exploitation to provide a secure system. The digitization of our lives proves to solve our human problems as well as improve quality of life but because it is digitalized, information and technology could be misused for other malicious gains. Hackers aim to steal the data of innocent people to use it for other causes such as identity fraud, scams and many more. This issue can be corrected during the software development life cycle, integrating security across the development phases, and testing of the software is done early to reduce the number of vulnerabilities that might or might not heavily impact an organisation depending on the range of the attack. The goal of a secured system software is to prevent such exploitations from ever happening by conducting a system life cycle where through planning and testing is done to maximise security while maintaining functionality of the system. In this paper, we are going to discuss the recent trends in security for system development as well as our predictions and suggestions to improve the current security practices in this industry. △ Less

Submitted 18 November, 2023; originally announced November 2023.

Comments: 20 Pages, 4 Figures

AdapterShadow: Adapting Segment Anything Model for Shadow Detection

Authors: Lei** Jie, Hui Zhang

Abstract: Segment anything model (SAM) has shown its spectacular performance in segmenting universal objects, especially when elaborate prompts are provided. However, the drawback of SAM is twofold. On the first hand, it fails to segment specific targets, e.g., shadow images or lesions in medical images. On the other hand, manually specifying prompts is extremely time-consuming. To overcome the problems, we… ▽ More Segment anything model (SAM) has shown its spectacular performance in segmenting universal objects, especially when elaborate prompts are provided. However, the drawback of SAM is twofold. On the first hand, it fails to segment specific targets, e.g., shadow images or lesions in medical images. On the other hand, manually specifying prompts is extremely time-consuming. To overcome the problems, we propose AdapterShadow, which adapts SAM model for shadow detection. To adapt SAM for shadow images, trainable adapters are inserted into the frozen image encoder of SAM, since the training of the full SAM model is both time and memory consuming. Moreover, we introduce a novel grid sampling method to generate dense point prompts, which helps to automatically segment shadows without any manual interventions. Extensive experiments are conducted on four widely used benchmark datasets to demonstrate the superior performance of our proposed method. Codes will are publicly available at https://github.com/Lei**Jie/AdapterShadow. △ Less

Submitted 15 November, 2023; originally announced November 2023.

Bi-VLGM : Bi-Level Class-Severity-Aware Vision-Language Graph Matching for Text Guided Medical Image Segmentation

Authors: Chen Wenting, Liu Jie, Yuan Yixuan

Abstract: Medical reports with substantial information can be naturally complementary to medical images for computer vision tasks, and the modality gap between vision and language can be solved by vision-language matching (VLM). However, current vision-language models distort the intra-model relation and mainly include class information in prompt learning that is insufficient for segmentation task. In this… ▽ More Medical reports with substantial information can be naturally complementary to medical images for computer vision tasks, and the modality gap between vision and language can be solved by vision-language matching (VLM). However, current vision-language models distort the intra-model relation and mainly include class information in prompt learning that is insufficient for segmentation task. In this paper, we introduce a Bi-level class-severity-aware Vision-Language Graph Matching (Bi-VLGM) for text guided medical image segmentation, composed of a word-level VLGM module and a sentence-level VLGM module, to exploit the class-severity-aware relation among visual-textual features. In word-level VLGM, to mitigate the distorted intra-modal relation during VLM, we reformulate VLM as graph matching problem and introduce a vision-language graph matching (VLGM) to exploit the high-order relation among visual-textual features. Then, we perform VLGM between the local features for each class region and class-aware prompts to bridge their gap. In sentence-level VLGM, to provide disease severity information for segmentation task, we introduce a severity-aware prompting to quantify the severity level of retinal lesion, and perform VLGM between the global features and the severity-aware prompts. By exploiting the relation between the local (global) and class (severity) features, the segmentation model can selectively learn the class-aware and severity-aware information to promote performance. Extensive experiments prove the effectiveness of our method and its superiority to existing methods. Source code is to be released. △ Less

Submitted 20 May, 2023; originally announced May 2023.

When SAM Meets Shadow Detection

Authors: Lei** Jie, Hui Zhang

Abstract: As a promptable generic object segmentation model, segment anything model (SAM) has recently attracted significant attention, and also demonstrates its powerful performance. Nevertheless, it still meets its Waterloo when encountering several tasks, e.g., medical image segmentation, camouflaged object detection, etc. In this report, we try SAM on an unexplored popular task: shadow detection. Specif… ▽ More As a promptable generic object segmentation model, segment anything model (SAM) has recently attracted significant attention, and also demonstrates its powerful performance. Nevertheless, it still meets its Waterloo when encountering several tasks, e.g., medical image segmentation, camouflaged object detection, etc. In this report, we try SAM on an unexplored popular task: shadow detection. Specifically, four benchmarks were chosen and evaluated with widely used metrics. The experimental results show that the performance for shadow detection using SAM is not satisfactory, especially when comparing with the elaborate models. Code is available at https://github.com/Lei**Jie/SAMSh. △ Less

Submitted 19 May, 2023; originally announced May 2023.

Comments: Technical Report

A Prompt Log Analysis of Text-to-Image Generation Systems

Authors: Yutong Xie, Zhaoying Pan, **ge Ma, Luo Jie, Qiaozhu Mei

Abstract: Recent developments in large language models (LLM) and generative AI have unleashed the astonishing capabilities of text-to-image generation systems to synthesize high-quality images that are faithful to a given reference text, known as a "prompt". These systems have immediately received lots of attention from researchers, creators, and common users. Despite the plenty of efforts to improve the ge… ▽ More Recent developments in large language models (LLM) and generative AI have unleashed the astonishing capabilities of text-to-image generation systems to synthesize high-quality images that are faithful to a given reference text, known as a "prompt". These systems have immediately received lots of attention from researchers, creators, and common users. Despite the plenty of efforts to improve the generative models, there is limited work on understanding the information needs of the users of these systems at scale. We conduct the first comprehensive analysis of large-scale prompt logs collected from multiple text-to-image generation systems. Our work is analogous to analyzing the query logs of Web search engines, a line of work that has made critical contributions to the glory of the Web search industry and research. Compared with Web search queries, text-to-image prompts are significantly longer, often organized into special structures that consist of the subject, form, and intent of the generation tasks and present unique categories of information needs. Users make more edits within creation sessions, which present remarkable exploratory patterns. There is also a considerable gap between the user-input prompts and the captions of the images included in the open training data of the generative models. Our findings provide concrete implications on how to improve text-to-image generation systems for creation purposes. △ Less

Submitted 16 March, 2023; v1 submitted 8 March, 2023; originally announced March 2023.

Is Complexity Required for Neural Network Pruning? A Case Study on Global Magnitude Pruning

Authors: Manas Gupta, Efe Camci, Vishandi Rudy Keneta, Abhishek Vaidyanathan, Ritwik Kanodia, Chuan-Sheng Foo, Wu Min, Lin Jie

Abstract: Pruning neural networks has become popular in the last decade when it was shown that a large number of weights can be safely removed from modern neural networks without compromising accuracy. Numerous pruning methods have been proposed since, each claiming to be better than prior art, however, at the cost of increasingly complex pruning methodologies. These methodologies include utilizing importan… ▽ More Pruning neural networks has become popular in the last decade when it was shown that a large number of weights can be safely removed from modern neural networks without compromising accuracy. Numerous pruning methods have been proposed since, each claiming to be better than prior art, however, at the cost of increasingly complex pruning methodologies. These methodologies include utilizing importance scores, getting feedback through back-propagation or having heuristics-based pruning rules amongst others. In this work, we question whether this pattern of introducing complexity is really necessary to achieve better pruning results. We benchmark these SOTA techniques against a simple pruning baseline, namely, Global Magnitude Pruning (Global MP), that ranks weights in order of their magnitudes and prunes the smallest ones. Surprisingly, we find that vanilla Global MP performs very well against the SOTA techniques. When considering sparsity-accuracy trade-off, Global MP performs better than all SOTA techniques at all sparsity ratios. When considering FLOPs-accuracy trade-off, some SOTA techniques outperform Global MP at lower sparsity ratios, however, Global MP starts performing well at high sparsity ratios and performs very well at extremely high sparsity ratios. Moreover, we find that a common issue that many pruning algorithms run into at high sparsity rates, namely, layer-collapse, can be easily fixed in Global MP. We explore why layer collapse occurs in networks and how it can be mitigated in Global MP by utilizing a technique called Minimum Threshold. We showcase the above findings on various models (WRN-28-8, ResNet-32, ResNet-50, MobileNet-V1 and FastGRNN) and multiple datasets (CIFAR-10, ImageNet and HAR-2). Code is available at https://github.com/manasgupta-1/GlobalMP. △ Less

Submitted 7 January, 2024; v1 submitted 29 September, 2022; originally announced September 2022.

LMBAO: A Landmark Map for Bundle Adjustment Odometry in LiDAR SLAM

Authors: Letian Zhang, **** Wang, Lu Jie, Nanjie Chen, Xiaojun Tan, Zhifei Duan

Abstract: LiDAR odometry is one of the essential parts of LiDAR simultaneous localization and map** (SLAM). However, existing LiDAR odometry tends to match a new scan simply iteratively with previous fixed-pose scans, gradually accumulating errors. Furthermore, as an effective joint optimization mechanism, bundle adjustment (BA) cannot be directly introduced into real-time odometry due to the intensive co… ▽ More LiDAR odometry is one of the essential parts of LiDAR simultaneous localization and map** (SLAM). However, existing LiDAR odometry tends to match a new scan simply iteratively with previous fixed-pose scans, gradually accumulating errors. Furthermore, as an effective joint optimization mechanism, bundle adjustment (BA) cannot be directly introduced into real-time odometry due to the intensive computation of large-scale global landmarks. Therefore, this letter designs a new strategy named a landmark map for bundle adjustment odometry (LMBAO) in LiDAR SLAM to solve these problems. First, BA-based odometry is further developed with an active landmark maintenance strategy for a more accurate local registration and avoiding cumulative errors. Specifically, this paper keeps entire stable landmarks on the map instead of just their feature points in the sliding window and deletes the landmarks according to their active grade. Next, the sliding window length is reduced, and marginalization is performed to retain the scans outside the window but corresponding to active landmarks on the map, greatly simplifying the computation and improving the real-time properties. In addition, experiments on three challenging datasets show that our algorithm achieves real-time performance in outdoor driving and outperforms state-of-the-art LiDAR SLAM algorithms, including Lego-LOAM and VLOM. △ Less

Submitted 19 September, 2022; originally announced September 2022.

Comments: 9 pages, 3 tables, 6 figures

Learning to Prune Deep Neural Networks via Reinforcement Learning

Authors: Manas Gupta, Siddharth Aravindan, Aleksandra Kalisz, Vijay Chandrasekhar, Lin Jie

Abstract: This paper proposes PuRL - a deep reinforcement learning (RL) based algorithm for pruning neural networks. Unlike current RL based model compression approaches where feedback is given only at the end of each episode to the agent, PuRL provides rewards at every pruning step. This enables PuRL to achieve sparsity and accuracy comparable to current state-of-the-art methods, while having a much shorte… ▽ More This paper proposes PuRL - a deep reinforcement learning (RL) based algorithm for pruning neural networks. Unlike current RL based model compression approaches where feedback is given only at the end of each episode to the agent, PuRL provides rewards at every pruning step. This enables PuRL to achieve sparsity and accuracy comparable to current state-of-the-art methods, while having a much shorter training cycle. PuRL achieves more than 80% sparsity on the ResNet-50 model while retaining a Top-1 accuracy of 75.37% on the ImageNet dataset. Through our experiments we show that PuRL is also able to sparsify already efficient architectures like MobileNet-V2. In addition to performance characterisation experiments, we also provide a discussion and analysis of the various RL design choices that went into the tuning of the Markov Decision Process underlying PuRL. Lastly, we point out that PuRL is simple to use and can be easily adapted for various architectures. △ Less

Submitted 9 July, 2020; originally announced July 2020.

Comments: Accepted at the ICML 2020 Workshop on Automated Machine Learning (AutoML 2020)

Localized Adversarial Training for Increased Accuracy and Robustness in Image Classification

Authors: Eitan Rothberg, Tingting Chen, Luo Jie, Hao Ji

Abstract: Today's state-of-the-art image classifiers fail to correctly classify carefully manipulated adversarial images. In this work, we develop a new, localized adversarial attack that generates adversarial examples by imperceptibly altering the backgrounds of normal images. We first use this attack to highlight the unnecessary sensitivity of neural networks to changes in the background of an image, then… ▽ More Today's state-of-the-art image classifiers fail to correctly classify carefully manipulated adversarial images. In this work, we develop a new, localized adversarial attack that generates adversarial examples by imperceptibly altering the backgrounds of normal images. We first use this attack to highlight the unnecessary sensitivity of neural networks to changes in the background of an image, then use it as part of a new training technique: localized adversarial training. By including locally adversarial images in the training set, we are able to create a classifier that suffers less loss than a non-adversarially trained counterpart model on both natural and adversarial inputs. The evaluation of our localized adversarial training algorithm on MNIST and CIFAR-10 datasets shows decreased accuracy loss on natural images, and increased robustness against adversarial inputs. △ Less

Submitted 10 September, 2019; originally announced September 2019.

Comments: 4 pages (excluding references). Presented at AdvML: 1st Workshop on Adversarial Learning Methods for Machine Learning and Data Mining at KDD '19