-
KInIT at SemEval-2024 Task 8: Fine-tuned LLMs for Multilingual Machine-Generated Text Detection
Authors:
Michal Spiegel,
Dominik Macko
Abstract:
SemEval-2024 Task 8 is focused on multigenerator, multidomain, and multilingual black-box machine-generated text detection. Such a detection is important for preventing a potential misuse of large language models (LLMs), the newest of which are very capable in generating multilingual human-like texts. We have coped with this task in multiple ways, utilizing language identification and parameter-ef…
▽ More
SemEval-2024 Task 8 is focused on multigenerator, multidomain, and multilingual black-box machine-generated text detection. Such a detection is important for preventing a potential misuse of large language models (LLMs), the newest of which are very capable in generating multilingual human-like texts. We have coped with this task in multiple ways, utilizing language identification and parameter-efficient fine-tuning of smaller LLMs for text classification. We have further used the per-language classification-threshold calibration to uniquely combine fine-tuned models predictions with statistical detection metrics to improve generalization of the system detection performance. Our submitted method achieved competitive results, ranking at the fourth place, just under 1 percentage point behind the winner.
△ Less
Submitted 17 June, 2024; v1 submitted 21 February, 2024;
originally announced February 2024.
-
IMGTB: A Framework for Machine-Generated Text Detection Benchmarking
Authors:
Michal Spiegel,
Dominik Macko
Abstract:
In the era of large language models generating high quality texts, it is a necessity to develop methods for detection of machine-generated text to avoid harmful use or simply due to annotation purposes. It is, however, also important to properly evaluate and compare such developed methods. Recently, a few benchmarks have been proposed for this purpose; however, integration of newest detection meth…
▽ More
In the era of large language models generating high quality texts, it is a necessity to develop methods for detection of machine-generated text to avoid harmful use or simply due to annotation purposes. It is, however, also important to properly evaluate and compare such developed methods. Recently, a few benchmarks have been proposed for this purpose; however, integration of newest detection methods is rather challenging, since new methods appear each month and provide slightly different evaluation pipelines. In this paper, we present the IMGTB framework, which simplifies the benchmarking of machine-generated text detection methods by easy integration of custom (new) methods and evaluation datasets. Its configurability and flexibility makes research and development of new detection methods easier, especially their comparison to the existing state-of-the-art detectors. The default set of analyses, metrics and visualizations offered by the tool follows the established practices of machine-generated text detection benchmarking found in state-of-the-art literature.
△ Less
Submitted 21 November, 2023;
originally announced November 2023.
-
Post-CCSD(T) corrections to bond distances and vibrational frequencies: the power of $Λ$
Authors:
Maciej Spiegel,
Emmanouil Semidalas,
Jan M. L. Martin,
Megan R. Bentley,
John F. Stanton
Abstract:
The importance of post-CCSD(T) corrections as high as CCSDTQ56 for ground-state spectroscopic constants ($D_e$, $ω_e$, $ω_ex_e$, and $α_e$) has been surveyed for a sample of two dozen mostly heavy-atom diatomics spanning a broad range of static correlation strength. While CCSD(T) is known to be an unusually felicitous `Pauling point' between accuracy and computational cost, performance leaves some…
▽ More
The importance of post-CCSD(T) corrections as high as CCSDTQ56 for ground-state spectroscopic constants ($D_e$, $ω_e$, $ω_ex_e$, and $α_e$) has been surveyed for a sample of two dozen mostly heavy-atom diatomics spanning a broad range of static correlation strength. While CCSD(T) is known to be an unusually felicitous `Pauling point' between accuracy and computational cost, performance leaves something to be desired for molecules with strong static correlation. We find CCSDT(Q)$_Λ$ to be the next `sweet spot' up, of comparable or superior quality to the much more expensive CCSDTQ. A similar comparison applies to CCSDTQ(5)$_Λ$ vs. CCSDTQ5, while CCSDTQ5(6)$_Λ$ is essentially indistinguishable from CCSDTQ56. A composite of CCSD(T)-X2C/ACV5Z-X2C with [CCSDT(Q)$_Λ$ -- CCSD(T)]/cc-pVTZ or even cc-pVDZ basis sets appears highly effective for computational vibrational spectroscopy. Unlike CCSDT(Q) which breaks down for the ozone vibrational frequencies, CCSDT(Q)$_Λ$ handles them gracefully.
△ Less
Submitted 22 August, 2023; v1 submitted 27 July, 2023;
originally announced July 2023.
-
How Does Magnetic Reconnection Drive the Early Stage Evolution of Coronal Mass Ejections?
Authors:
Chunming Zhu,
Jiong Qiu,
Paulett Liewer,
Angelos Vourlidas,
Michael Spiegel,
Qiang Hu
Abstract:
Theoretically, CME kinematics are related to magnetic reconnection processes in the solar corona. However, the current quantitative understanding of this relationship is based on the analysis of only a handful of events. Here we report a statistical study of 60 CME-flare events from August 2010 to December 2013. We investigate kinematic properties of CMEs and magnetic reconnection in the low coron…
▽ More
Theoretically, CME kinematics are related to magnetic reconnection processes in the solar corona. However, the current quantitative understanding of this relationship is based on the analysis of only a handful of events. Here we report a statistical study of 60 CME-flare events from August 2010 to December 2013. We investigate kinematic properties of CMEs and magnetic reconnection in the low corona during the early phase of the eruptions, by combining limb observations from STEREO with simultaneous on-disk views from SDO. For a subset of 42 events with reconnection rate evaluated by the magnetic fluxes swept by the flare ribbons on the solar disk observed from SDO, we find a strong correlation between the peak CME acceleration and the peak reconnection rate. Also, the maximum velocities of relatively fast CMEs (> 600 km/s) are positively correlated with the reconnection flux, but no such correlation is found for slow CMEs. A time-lagged correlation analysis suggests that the distribution of the time lag of CME acceleration relative to reconnection rate exhibits three peaks, approximately 10 minutes apart, and on average, acceleration-lead events have smaller reconnection rates. We further compare the CME total mechanical energy with the estimated energy in the current sheet. The comparison suggests that, for small-flare events, reconnection in the current sheet alone is insufficient to fuel CMEs. Results from this study suggest that flare reconnection may dominate the acceleration of fast CMEs, but for events of slow CMEs and weak reconnection, other mechanisms may be more important.
△ Less
Submitted 24 March, 2020;
originally announced March 2020.
-
Scalar curvature rigidity for locally conformally flat manifolds with boundary
Authors:
Fabian Michael Spiegel
Abstract:
Inspired by the work of F. Hang and X. Wang and partial results by S. Raulot, we prove a scalar curvature rigitidy result for locally conformally flat manifolds with boundary in the spirit of the well-known Min-Oo conjecture.
Inspired by the work of F. Hang and X. Wang and partial results by S. Raulot, we prove a scalar curvature rigitidy result for locally conformally flat manifolds with boundary in the spirit of the well-known Min-Oo conjecture.
△ Less
Submitted 12 February, 2016; v1 submitted 19 November, 2015;
originally announced November 2015.
