What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers
Authors:
Boseop Kim,
HyoungSeok Kim,
Sang-Woo Lee,
Gichang Lee,
Donghyun Kwak,
Dong Hyeon Jeon,
Sunghyun Park,
Sungju Kim,
Seonhoon Kim,
Dongpil Seo,
Heungsub Lee,
Minyoung Jeong,
Sungjae Lee,
Minsub Kim,
Suk Hyun Ko,
Seokhun Kim,
Taeyong Park,
**uk Kim,
Soyoung Kang,
Na-Hyeon Ryu,
Kang Min Yoo,
Minsuk Chang,
Soobin Suh,
Sookyo In,
**seong Park
, et al. (12 additional authors not shown)
Abstract:
GPT-3 shows remarkable in-context learning ability of large-scale language models (LMs) trained on hundreds of billion scale data. Here we address some remaining issues less reported by the GPT-3 paper, such as a non-English LM, the performances of different sized models, and the effect of recently introduced prompt optimization on in-context learning. To achieve this, we introduce HyperCLOVA, a K…
▽ More
GPT-3 shows remarkable in-context learning ability of large-scale language models (LMs) trained on hundreds of billion scale data. Here we address some remaining issues less reported by the GPT-3 paper, such as a non-English LM, the performances of different sized models, and the effect of recently introduced prompt optimization on in-context learning. To achieve this, we introduce HyperCLOVA, a Korean variant of 82B GPT-3 trained on a Korean-centric corpus of 560B tokens. Enhanced by our Korean-specific tokenization, HyperCLOVA with our training configuration shows state-of-the-art in-context zero-shot and few-shot learning performances on various downstream tasks in Korean. Also, we show the performance benefits of prompt-based learning and demonstrate how it can be integrated into the prompt engineering pipeline. Then we discuss the possibility of materializing the No Code AI paradigm by providing AI prototy** capabilities to non-experts of ML by introducing HyperCLOVA studio, an interactive prompt engineering interface. Lastly, we demonstrate the potential of our methods with three successful in-house applications.
△ Less
Submitted 28 November, 2021; v1 submitted 9 September, 2021;
originally announced September 2021.
Non-Hermitian CT-Symmetric Spectral Protection of Nonlinear Defect Modes
Authors:
Do Hyeok Jeon,
Mattis Reisner,
Fabrice Mortessagne,
Tsampikos Kottos,
Ulrich Kuhl
Abstract:
We investigate, using a microwave platform consisting of a non-Hermitian Su-Schrieffer-Heeger array of coupled dielectric resonators, the interplay of a lossy nonlinearity and CT-symmetry in the formation of defect modes. The measurements agree with the theory which predicts that, up to moderate pum**, the defect mode is an eigenstate of the CT-symmetric operator and retains its frequency at the…
▽ More
We investigate, using a microwave platform consisting of a non-Hermitian Su-Schrieffer-Heeger array of coupled dielectric resonators, the interplay of a lossy nonlinearity and CT-symmetry in the formation of defect modes. The measurements agree with the theory which predicts that, up to moderate pum**, the defect mode is an eigenstate of the CT-symmetric operator and retains its frequency at the center of the gap. At higher pum** values, the system undergoes a self-induced explicit \CT-symmetry violation which removes the spectral topological protection and alters the shape of the defect mode.
△ Less
Submitted 16 February, 2020;
originally announced February 2020.
Self-Shielded Topological Receiver Protectors
Authors:
Mattis Reisner,
Do Hyeok Jeon,
Carsten Schindler,
Henning Schomerus,
Fabrice Mortessagne,
Ulrich Kuhl,
Tsampikos Kottos
Abstract:
Receiver protectors (RPs) shield sensitive electronics from high-power incoming signals that might damage them. Typical RP schemes range from simple fusing and PIN diodes, to superconducting circuits and plasma cells - each having a variety of drawbacks ranging from unacceptable system downtime and self-destruction to significant insertion losses and power consumption. Here, we theoretically propo…
▽ More
Receiver protectors (RPs) shield sensitive electronics from high-power incoming signals that might damage them. Typical RP schemes range from simple fusing and PIN diodes, to superconducting circuits and plasma cells - each having a variety of drawbacks ranging from unacceptable system downtime and self-destruction to significant insertion losses and power consumption. Here, we theoretically propose and experimentally demonstrate a unique self-shielding RP based on a coupled-resonator-microwave-waveguide (CRMW) with a topological defect being inductively coupled to a diode. This RP utilizes a charge-conjugation (C) symmetric resonant defect mode that is robust against disorder and demonstrates high transmittance at low incident powers. When incident power exceeds a critical value, a self-induced resonant trap** effect occurs leading to a dramatic suppression of transmittance and a simultaneous increase of the reflectance close to unity. The proposed RP device is self-protected from overheating and electrical breakdown and can be utilized in radars, reflection altimeters, and a broad range of communication systems.
△ Less
Submitted 5 April, 2020; v1 submitted 8 September, 2019;
originally announced October 2019.
