-
HyperPIE: Hyperparameter Information Extraction from Scientific Publications
Authors:
Tarek Saier,
Mayumi Ohta,
Takuto Asakura,
Michael Färber
Abstract:
Automatic extraction of information from publications is key to making scientific knowledge machine readable at a large scale. The extracted information can, for example, facilitate academic search, decision making, and knowledge graph construction. An important type of information not covered by existing approaches is hyperparameters. In this paper, we formalize and tackle hyperparameter informat…
▽ More
Automatic extraction of information from publications is key to making scientific knowledge machine readable at a large scale. The extracted information can, for example, facilitate academic search, decision making, and knowledge graph construction. An important type of information not covered by existing approaches is hyperparameters. In this paper, we formalize and tackle hyperparameter information extraction (HyperPIE) as an entity recognition and relation extraction task. We create a labeled data set covering publications from a variety of computer science disciplines. Using this data set, we train and evaluate BERT-based fine-tuned models as well as five large language models: GPT-3.5, GALACTICA, Falcon, Vicuna, and WizardLM. For fine-tuned models, we develop a relation extraction approach that achieves an improvement of 29% F1 over a state-of-the-art baseline. For large language models, we develop an approach leveraging YAML output for structured data extraction, which achieves an average improvement of 5.5% F1 in entity recognition over using JSON. With our best performing model we extract hyperparameter information from a large number of unannotated papers, and analyze patterns across disciplines. All our data and source code is publicly available at https://github.com/IllDepence/hyperpie
△ Less
Submitted 10 January, 2024; v1 submitted 17 December, 2023;
originally announced December 2023.
-
Metamaterial Thermoelectric Conversion
Authors:
Takuya Asakura,
Toshinari Odaka,
Ryosuke Nakayama,
Sohei Saito,
Shohei Katsumata,
Takuo Tanaka,
Wakana Kubo
Abstract:
We propose a thermoelectric device that can produce a thermal gradient even in a uniform-temperature environment. We introduced a metamaterial absorber (MA), which comprised a transparent calcium fluoride layer sandwiched between a silver mirror layer and silver microdisk arrays, at one end of a thermoelectric device made of bismuth antimony telluride. The heating efficiencies of the MA and opposi…
▽ More
We propose a thermoelectric device that can produce a thermal gradient even in a uniform-temperature environment. We introduced a metamaterial absorber (MA), which comprised a transparent calcium fluoride layer sandwiched between a silver mirror layer and silver microdisk arrays, at one end of a thermoelectric device made of bismuth antimony telluride. The heating efficiencies of the MA and opposite electrodes of the device were unbalanced; consequently, the Seebeck effect was induced, resulting in electricity generation. We fabricated the MA on a copper electrode loaded on a thermoelectric device, and the device was placed in a uniform-temperature environment at 364 K. The thermal gradient across the device was experimentally measured and was found to be 0.14 K.
△ Less
Submitted 14 May, 2022; v1 submitted 27 April, 2022;
originally announced April 2022.
-
A Quantitative Evaluation of Natural Language Question Interpretation for Question Answering Systems
Authors:
Takuto Asakura,
**-Dong Kim,
Yasunori Yamamoto,
Yuka Tateisi,
Toshihisa Takagi
Abstract:
Systematic benchmark evaluation plays an important role in the process of improving technologies for Question Answering (QA) systems. While currently there are a number of existing evaluation methods for natural language (NL) QA systems, most of them consider only the final answers, limiting their utility within a black box style evaluation. Herein, we propose a subdivided evaluation approach to e…
▽ More
Systematic benchmark evaluation plays an important role in the process of improving technologies for Question Answering (QA) systems. While currently there are a number of existing evaluation methods for natural language (NL) QA systems, most of them consider only the final answers, limiting their utility within a black box style evaluation. Herein, we propose a subdivided evaluation approach to enable finer-grained evaluation of QA systems, and present an evaluation tool which targets the NL question (NLQ) interpretation step, an initial step of a QA pipeline. The results of experiments using two public benchmark datasets suggest that we can get a deeper insight about the performance of a QA system using the proposed approach, which should provide a better guidance for improving the systems, than using black box style approaches.
△ Less
Submitted 20 September, 2018;
originally announced September 2018.
