-
HyperCLOVA X Technical Report
Authors:
Kang Min Yoo,
Jaegeun Han,
Sookyo In,
Heewon Jeon,
Jisu Jeong,
Jaewook Kang,
Hyunwook Kim,
Kyung-Min Kim,
Munhyong Kim,
Sungju Kim,
Donghyun Kwak,
Hanock Kwak,
Se Jung Kwon,
Bado Lee,
Dongsoo Lee,
Gichang Lee,
Jooho Lee,
Baeseong Park,
Seong** Shin,
Joonsang Yu,
Seolki Baek,
Sumin Byeon,
Eungsup Cho,
Dooseok Choe,
Jeesung Han
, et al. (371 additional authors not shown)
Abstract:
We introduce HyperCLOVA X, a family of large language models (LLMs) tailored to the Korean language and culture, along with competitive capabilities in English, math, and coding. HyperCLOVA X was trained on a balanced mix of Korean, English, and code data, followed by instruction-tuning with high-quality human-annotated datasets while abiding by strict safety guidelines reflecting our commitment t…
▽ More
We introduce HyperCLOVA X, a family of large language models (LLMs) tailored to the Korean language and culture, along with competitive capabilities in English, math, and coding. HyperCLOVA X was trained on a balanced mix of Korean, English, and code data, followed by instruction-tuning with high-quality human-annotated datasets while abiding by strict safety guidelines reflecting our commitment to responsible AI. The model is evaluated across various benchmarks, including comprehensive reasoning, knowledge, commonsense, factuality, coding, math, chatting, instruction-following, and harmlessness, in both Korean and English. HyperCLOVA X exhibits strong reasoning capabilities in Korean backed by a deep understanding of the language and cultural nuances. Further analysis of the inherent bilingual nature and its extension to multilingualism highlights the model's cross-lingual proficiency and strong generalization ability to untargeted languages, including machine translation between several language pairs and cross-lingual inference tasks. We believe that HyperCLOVA X can provide helpful guidance for regions or countries in develo** their sovereign LLMs.
△ Less
Submitted 13 April, 2024; v1 submitted 2 April, 2024;
originally announced April 2024.
-
Gradient Scaling on Deep Spiking Neural Networks with Spike-Dependent Local Information
Authors:
Seongsik Park,
Jeonghee Jo,
Jongkil Park,
Yeonjoo Jeong,
Jaewook Kim,
Suyoun Lee,
Joon Young Kwak,
Inho Kim,
Jong-Keuk Park,
Kyeong Seok Lee,
Gye Weon Hwang,
Hyun Jae Jang
Abstract:
Deep spiking neural networks (SNNs) are promising neural networks for their model capacity from deep neural network architecture and energy efficiency from SNNs' operations. To train deep SNNs, recently, spatio-temporal backpropagation (STBP) with surrogate gradient was proposed. Although deep SNNs have been successfully trained with STBP, they cannot fully utilize spike information. In this work,…
▽ More
Deep spiking neural networks (SNNs) are promising neural networks for their model capacity from deep neural network architecture and energy efficiency from SNNs' operations. To train deep SNNs, recently, spatio-temporal backpropagation (STBP) with surrogate gradient was proposed. Although deep SNNs have been successfully trained with STBP, they cannot fully utilize spike information. In this work, we proposed gradient scaling with local spike information, which is the relation between pre- and post-synaptic spikes. Considering the causality between spikes, we could enhance the training performance of deep SNNs. According to our experiments, we could achieve higher accuracy with lower spikes by adopting the gradient scaling on image classification tasks, such as CIFAR10 and CIFAR100.
△ Less
Submitted 1 August, 2023;
originally announced August 2023.
-
Systemic Risk in Market Microstructure of Crude Oil and Gasoline Futures Prices: A Hawkes Flocking Model Approach
Authors:
Hyun ** Jang,
Kiseop Lee,
Kyungsub Lee
Abstract:
We propose the Hawkes flocking model that assesses systemic risk in high-frequency processes at the two perspectives -- endogeneity and interactivity. We examine the futures markets of WTI crude oil and gasoline for the past decade, and perform a comparative analysis with conditional value-at-risk as a benchmark measure. In terms of high-frequency structure, we derive the empirical findings. The e…
▽ More
We propose the Hawkes flocking model that assesses systemic risk in high-frequency processes at the two perspectives -- endogeneity and interactivity. We examine the futures markets of WTI crude oil and gasoline for the past decade, and perform a comparative analysis with conditional value-at-risk as a benchmark measure. In terms of high-frequency structure, we derive the empirical findings. The endogenous systemic risk in WTI was significantly higher than that in gasoline, and the level at which gasoline affects WTI was constantly higher than in the opposite case. Moreover, although the relative influence's degree was asymmetric, its difference has gradually reduced.
△ Less
Submitted 7 December, 2020;
originally announced December 2020.
-
Optimal Investment, Heterogeneous Consumption and Best Time for Retirement
Authors:
Hyun ** Jang,
Zuo Quan Xu,
Harry Zheng
Abstract:
This paper studies an optimal investment and consumption problem with heterogeneous consumption of basic and luxury goods, together with the choice of time for retirement. The utility for luxury goods is not necessarily a concave function. The optimal heterogeneous consumption strategies for a class of non-homothetic utility maximizer are shown to consume only basic goods when the wealth is small,…
▽ More
This paper studies an optimal investment and consumption problem with heterogeneous consumption of basic and luxury goods, together with the choice of time for retirement. The utility for luxury goods is not necessarily a concave function. The optimal heterogeneous consumption strategies for a class of non-homothetic utility maximizer are shown to consume only basic goods when the wealth is small, to consume basic goods and make savings when the wealth is intermediate, and to consume almost all in luxury goods when the wealth is large. The optimal retirement policy is shown to be both universal, in the sense that all individuals should retire at the same level of marginal utility that is determined only by income, labor cost, discount factor as well as market parameters, and not universal, in the sense that all individuals can achieve the same marginal utility with different utility and wealth. It is also shown that individuals prefer to retire as time goes by if the marginal labor cost increases faster than that of income. The main tools used in analyzing the problem are from PDE and stochastic control theory including variational inequality and dual transformation. We finally conduct the simulation analysis for the featured model parameters to investigate practical and economic implications by providing their figures.
△ Less
Submitted 8 June, 2022; v1 submitted 1 August, 2020;
originally announced August 2020.
-
Orthogonality Constrained Multi-Head Attention For Keyword Spotting
Authors:
Mingu Lee,
**kyu Lee,
Hye ** Jang,
Byeonggeun Kim,
Wonil Chang,
Kyuwoong Hwang
Abstract:
Multi-head attention mechanism is capable of learning various representations from sequential data while paying attention to different subsequences, e.g., word-pieces or syllables in a spoken word. From the subsequences, it retrieves richer information than a single-head attention which only summarizes the whole sequence into one context vector. However, a naive use of the multi-head attention doe…
▽ More
Multi-head attention mechanism is capable of learning various representations from sequential data while paying attention to different subsequences, e.g., word-pieces or syllables in a spoken word. From the subsequences, it retrieves richer information than a single-head attention which only summarizes the whole sequence into one context vector. However, a naive use of the multi-head attention does not guarantee such richness as the attention heads may have positional and representational redundancy. In this paper, we propose a regularization technique for multi-head attention mechanism in an end-to-end neural keyword spotting system. Augmenting regularization terms which penalize positional and contextual non-orthogonality between the attention heads encourages to output different representations from separate subsequences, which in turn enables leveraging structured information without explicit sequence models such as hidden Markov models. In addition, intra-head contextual non-orthogonality regularization encourages each attention head to have similar representations across keyword examples, which helps classification by reducing feature variability. The experimental results demonstrate that the proposed regularization technique significantly improves the keyword spotting performance for the keyword "Hey Snapdragon".
△ Less
Submitted 10 October, 2019;
originally announced October 2019.
-
Measurement Error Effects of Beam Parameters Determined by Beam Profiles
Authors:
Ji-Ho Jang,
Hyo Jae Jang,
Dong-O Jeon
Abstract:
A conventional method to determine beam parameters is using the profile measurements and converting them into the values of twiss parameters and beam emittance at a specified position. The beam information can be used to improve transverse beam matching between two different beam lines or accelerating structures. This work is related with the measurement error effects of the beam parameters and th…
▽ More
A conventional method to determine beam parameters is using the profile measurements and converting them into the values of twiss parameters and beam emittance at a specified position. The beam information can be used to improve transverse beam matching between two different beam lines or accelerating structures. This work is related with the measurement error effects of the beam parameters and the optimal number of profile monitors in a section between MEBT (medium energy beam transport) and QWR (quarter wave resonator) of RAON linear accelerator.
△ Less
Submitted 12 July, 2015;
originally announced July 2015.
-
2-torsion in the grope and solvable filtrations of knots
Authors:
Hye ** Jang
Abstract:
We study knots of order 2 in the grope filtration $\{\G_h\}$ and the solvable filtration $\{\F_h\}$ of the knot concordance group. We show that, for any integer $n\ge4$, there are knots generating a $\Z_2^\infty$ subgroup of $\G_n/\G_{n.5}$. Considering the solvable filtration, our knots generate a $\Z_2^\infty$ subgroup of $\F_n/\F_{n.5}$ $(n\ge2)$ distinct from the subgroup generated by the prev…
▽ More
We study knots of order 2 in the grope filtration $\{\G_h\}$ and the solvable filtration $\{\F_h\}$ of the knot concordance group. We show that, for any integer $n\ge4$, there are knots generating a $\Z_2^\infty$ subgroup of $\G_n/\G_{n.5}$. Considering the solvable filtration, our knots generate a $\Z_2^\infty$ subgroup of $\F_n/\F_{n.5}$ $(n\ge2)$ distinct from the subgroup generated by the previously known 2-torsion knots of Cochran, Harvey, and Leidy. We also present a result on the 2-torsion part in the Cochran, Harvey, and Leidy's primary decomposition of the solvable filtration.
△ Less
Submitted 16 February, 2015;
originally announced February 2015.
-
Crossing number of an alternating knot and canonical genus of its Whitehead double
Authors:
Hee Jeong Jang,
Sang Youl Lee
Abstract:
A conjecture proposed by J. Tripp in 2002 states that the crossing number of any knot coincides with the canonical genus of its Whitehead double. In the meantime, it has been established that this conjecture is true for a large class of alternating knots including $(2, n)$ torus knots, $2$-bridge knots, algebraic alternating knots, and alternating pretzel knots. In this paper, we prove that the co…
▽ More
A conjecture proposed by J. Tripp in 2002 states that the crossing number of any knot coincides with the canonical genus of its Whitehead double. In the meantime, it has been established that this conjecture is true for a large class of alternating knots including $(2, n)$ torus knots, $2$-bridge knots, algebraic alternating knots, and alternating pretzel knots. In this paper, we prove that the conjecture is not true for any alternating $3$-braid knot which is the connected sum of two torus knots of type $(2, m)$ and $(2, n)$. This results in a new modified conjecture that the crossing number of any prime knot coincides with the canonical genus of its Whitehead double. We also give a new large class of prime alternating knots satisfying the conjecture, including all prime alternating $3$-braid knots.
△ Less
Submitted 21 July, 2015; v1 submitted 1 September, 2014;
originally announced September 2014.
-
Smoothly slice boundary links whose derivative links have nonvanishing Milnor invariants
Authors:
Hye ** Jang,
Min Hoon Kim,
Mark Powell
Abstract:
We give an example of a 3-component smoothly slice boundary link, each of whose components has a genus one Seifert surface, such that any metaboliser of the boundary link Seifert form is represented by 3 curves on the Seifert surfaces that form a link with nonvanishing Milnor triple linking number. We also give a generalisation to m-component links and higher Milnor invariants. We prove that our e…
▽ More
We give an example of a 3-component smoothly slice boundary link, each of whose components has a genus one Seifert surface, such that any metaboliser of the boundary link Seifert form is represented by 3 curves on the Seifert surfaces that form a link with nonvanishing Milnor triple linking number. We also give a generalisation to m-component links and higher Milnor invariants. We prove that our examples are ribbon and that all ribbon links are boundary slice.
△ Less
Submitted 1 July, 2015; v1 submitted 29 August, 2013;
originally announced August 2013.
-
Inclusive J/psi production in pp collisions at sqrt(s) = 2.76 TeV
Authors:
ALICE Collaboration,
B. Abelev,
J. Adam,
D. Adamova,
A. M. Adare,
M. M. Aggarwal,
G. Aglieri Rinella,
A. G. Agocs,
A. Agostinelli,
S. Aguilar Salazar,
Z. Ahammed,
A. Ahmad Masoodi,
N. Ahmad,
S. U. Ahn,
A. Akindinov,
D. Aleksandrov,
B. Alessandro,
R. Alfaro Molina,
A. Alici,
A. Alkin,
E. Almaraz Avina,
J. Alme,
T. Alt,
V. Altini,
S. Altinpinar
, et al. (948 additional authors not shown)
Abstract:
The ALICE Collaboration has measured inclusive J/psi production in pp collisions at a center of mass energy sqrt(s)=2.76 TeV at the LHC. The results presented in this Letter refer to the rapidity ranges |y|<0.9 and 2.5<y<4 and have been obtained by measuring the electron and muon pair decay channels, respectively. The integrated luminosities for the two channels are L^e_int=1.1 nb^-1 and L^mu_int=…
▽ More
The ALICE Collaboration has measured inclusive J/psi production in pp collisions at a center of mass energy sqrt(s)=2.76 TeV at the LHC. The results presented in this Letter refer to the rapidity ranges |y|<0.9 and 2.5<y<4 and have been obtained by measuring the electron and muon pair decay channels, respectively. The integrated luminosities for the two channels are L^e_int=1.1 nb^-1 and L^mu_int=19.9 nb^-1, and the corresponding signal statistics are N_J/psi^e+e-=59 +/- 14 and N_J/psi^mu+mu-=1364 +/- 53. We present dsigma_J/psi/dy for the two rapidity regions under study and, for the forward-y range, d^2sigma_J/psi/dydp_t in the transverse momentum domain 0<p_t<8 GeV/c. The results are compared with previously published results at sqrt(s)=7 TeV and with theoretical calculations.
△ Less
Submitted 6 November, 2012; v1 submitted 16 March, 2012;
originally announced March 2012.
-
On fat Hoffman graphs with smallest eigenvalue at least -3
Authors:
Hye ** Jang,
Jack Koolen,
Akihiro Munemasa,
Tetsuji Taniguchi
Abstract:
We investigate fat Hoffman graphs with smallest eigenvalue at least -3, using their special graphs. We show that the special graph S(H) of an indecomposable fat Hoffman graph H is represented by the standard lattice or an irreducible root lattice. Moreover, we show that if the special graph admits an integral representation, that is, the lattice spanned by it is not an exceptional root lattice, th…
▽ More
We investigate fat Hoffman graphs with smallest eigenvalue at least -3, using their special graphs. We show that the special graph S(H) of an indecomposable fat Hoffman graph H is represented by the standard lattice or an irreducible root lattice. Moreover, we show that if the special graph admits an integral representation, that is, the lattice spanned by it is not an exceptional root lattice, then the special graph S(H) is isomorphic to one of the Dynkin graphs A_n, D_n, or extended Dynkin graphs A_n or D_n.
△ Less
Submitted 28 September, 2012; v1 submitted 31 October, 2011;
originally announced October 2011.
-
The canonical genus for Whitehead doubles of a family of alternating knots
Authors:
Hee Jeong Jang,
Sang Youl Lee
Abstract:
For any given integer $r \geq 1$ and a quasitoric braid $β_r=(σ_r^{-ε} σ_{r-1}^ε...$ $ σ_{1}^{(-1)^{r}ε})^3$ with $ε=\pm 1$, we prove that the maximum degree in $z$ of the HOMFLYPT polynomial $P_{W_2(\hatβ_r)}(v,z)$ of the doubled link $W_2(\hatβ_r)$ of the closure $\hatβ_r$ is equal to $6r-1$. As an application, we give a family $\mathcal K^3$ of alternating knots, including $(2,n)$ torus knots,…
▽ More
For any given integer $r \geq 1$ and a quasitoric braid $β_r=(σ_r^{-ε} σ_{r-1}^ε...$ $ σ_{1}^{(-1)^{r}ε})^3$ with $ε=\pm 1$, we prove that the maximum degree in $z$ of the HOMFLYPT polynomial $P_{W_2(\hatβ_r)}(v,z)$ of the doubled link $W_2(\hatβ_r)$ of the closure $\hatβ_r$ is equal to $6r-1$. As an application, we give a family $\mathcal K^3$ of alternating knots, including $(2,n)$ torus knots, 2-bridge knots and alternating pretzel knots as its subfamilies, such that the minimal crossing number of any alternating knot in $\mathcal K^3$ coincides with the canonical genus of its Whitehead double. Consequently, we give a new family $\mathcal K^3$ of alternating knots for which Tripp's conjecture holds.
△ Less
Submitted 7 June, 2011; v1 submitted 7 June, 2011;
originally announced June 2011.
