-
Hybrid of DiffStride and Spectral Pooling in Convolutional Neural Networks
Authors:
Sulthan Rafif,
Mochamad Arfan Ravy Wahyu Pratama,
Mohammad Faris Azhar,
Ahmad Mustafidul Ibad,
Lailil Muflikhah,
Novanto Yudistira
Abstract:
Stride determines the distance between adjacent filter positions as the filter moves across the input. A fixed stride causes important information contained in the image can not be captured, so that important information is not classified. Therefore, in previous research, the DiffStride Method was applied, namely the Strided Convolution Method with which it can learn its own stride value. Severe Q…
▽ More
Stride determines the distance between adjacent filter positions as the filter moves across the input. A fixed stride causes important information contained in the image can not be captured, so that important information is not classified. Therefore, in previous research, the DiffStride Method was applied, namely the Strided Convolution Method with which it can learn its own stride value. Severe Quantization and a constraining lower bound on preserved information are arises with Max Pooling Downsampling Method. Spectral Pooling reduce the constraint lower bound on preserved information by cutting off the representation in the frequency domain. In this research a CNN Model is proposed with the Downsampling Learnable Stride Technique performed by Backpropagation combined with the Spectral Pooling Technique. Diffstride and Spectral Pooling techniques are expected to maintain most of the information contained in the image. In this study, we compare the Hybrid Method, which is a combined implementation of Spectral Pooling and DiffStride against the Baseline Method, which is the DiffStride implementation on ResNet 18. The accuracy result of the DiffStride combination with Spectral Pooling improves over DiffStride which is baseline method by 0.0094. This shows that the Hybrid Method can maintain most of the information by cutting of the representation in the frequency domain and determine the stride of the learning result through Backpropagation.
△ Less
Submitted 17 January, 2024;
originally announced January 2024.
-
Code Llama: Open Foundation Models for Code
Authors:
Baptiste Rozière,
Jonas Gehring,
Fabian Gloeckle,
Sten Sootla,
Itai Gat,
Xiaoqing Ellen Tan,
Yossi Adi,
**gyu Liu,
Romain Sauvestre,
Tal Remez,
Jérémy Rapin,
Artyom Kozhevnikov,
Ivan Evtimov,
Joanna Bitton,
Manish Bhatt,
Cristian Canton Ferrer,
Aaron Grattafiori,
Wenhan Xiong,
Alexandre Défossez,
Jade Copet,
Faisal Azhar,
Hugo Touvron,
Louis Martin,
Nicolas Usunier,
Thomas Scialom
, et al. (1 additional authors not shown)
Abstract:
We release Code Llama, a family of large language models for code based on Llama 2 providing state-of-the-art performance among open models, infilling capabilities, support for large input contexts, and zero-shot instruction following ability for programming tasks. We provide multiple flavors to cover a wide range of applications: foundation models (Code Llama), Python specializations (Code Llama…
▽ More
We release Code Llama, a family of large language models for code based on Llama 2 providing state-of-the-art performance among open models, infilling capabilities, support for large input contexts, and zero-shot instruction following ability for programming tasks. We provide multiple flavors to cover a wide range of applications: foundation models (Code Llama), Python specializations (Code Llama - Python), and instruction-following models (Code Llama - Instruct) with 7B, 13B, 34B and 70B parameters each. All models are trained on sequences of 16k tokens and show improvements on inputs with up to 100k tokens. 7B, 13B and 70B Code Llama and Code Llama - Instruct variants support infilling based on surrounding content. Code Llama reaches state-of-the-art performance among open models on several code benchmarks, with scores of up to 67% and 65% on HumanEval and MBPP, respectively. Notably, Code Llama - Python 7B outperforms Llama 2 70B on HumanEval and MBPP, and all our models outperform every other publicly available model on MultiPL-E. We release Code Llama under a permissive license that allows for both research and commercial use.
△ Less
Submitted 31 January, 2024; v1 submitted 24 August, 2023;
originally announced August 2023.
-
LLaMA: Open and Efficient Foundation Language Models
Authors:
Hugo Touvron,
Thibaut Lavril,
Gautier Izacard,
Xavier Martinet,
Marie-Anne Lachaux,
Timothée Lacroix,
Baptiste Rozière,
Naman Goyal,
Eric Hambro,
Faisal Azhar,
Aurelien Rodriguez,
Armand Joulin,
Edouard Grave,
Guillaume Lample
Abstract:
We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets. In particular, LLaMA-13B outperforms GPT-3 (175B) on most benchmarks, and LLaMA-65B is co…
▽ More
We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets. In particular, LLaMA-13B outperforms GPT-3 (175B) on most benchmarks, and LLaMA-65B is competitive with the best models, Chinchilla-70B and PaLM-540B. We release all our models to the research community.
△ Less
Submitted 27 February, 2023;
originally announced February 2023.
-
Forecasting User Interests Through Topic Tag Predictions in Online Health Communities
Authors:
Amogh Subbakrishna Adishesha,
Lily Jakielaszek,
Fariha Azhar,
Peixuan Zhang,
Vasant Honavar,
Fenglong Ma,
Chandra Belani,
Prasenjit Mitra,
Sharon Xiaolei Huang
Abstract:
The increasing reliance on online communities for healthcare information by patients and caregivers has led to the increase in the spread of misinformation, or subjective, anecdotal and inaccurate or non-specific recommendations, which, if acted on, could cause serious harm to the patients. Hence, there is an urgent need to connect users with accurate and tailored health information in a timely ma…
▽ More
The increasing reliance on online communities for healthcare information by patients and caregivers has led to the increase in the spread of misinformation, or subjective, anecdotal and inaccurate or non-specific recommendations, which, if acted on, could cause serious harm to the patients. Hence, there is an urgent need to connect users with accurate and tailored health information in a timely manner to prevent such harm. This paper proposes an innovative approach to suggesting reliable information to participants in online communities as they move through different stages in their disease or treatment. We hypothesize that patients with similar histories of disease progression or course of treatment would have similar information needs at comparable stages. Specifically, we pose the problem of predicting topic tags or keywords that describe the future information needs of users based on their profiles, traces of their online interactions within the community (past posts, replies) and the profiles and traces of online interactions of other users with similar profiles and similar traces of past interaction with the target users. The result is a variant of the collaborative information filtering or recommendation system tailored to the needs of users of online health communities. We report results of our experiments on an expert curated data set which demonstrate the superiority of the proposed approach over the state of the art baselines with respect to accurate and timely prediction of topic tags (and hence information sources of interest).
△ Less
Submitted 4 November, 2022;
originally announced November 2022.
