-
ChatGPT Code Detection: Techniques for Uncovering the Source of Code
Authors:
Marc Oedingen,
Raphael C. Engelhardt,
Robin Denz,
Maximilian Hammer,
Wolfgang Konen
Abstract:
In recent times, large language models (LLMs) have made significant strides in generating computer code, blurring the lines between code created by humans and code produced by artificial intelligence (AI). As these technologies evolve rapidly, it is crucial to explore how they influence code generation, especially given the risk of misuse in areas like higher education. This paper explores this is…
▽ More
In recent times, large language models (LLMs) have made significant strides in generating computer code, blurring the lines between code created by humans and code produced by artificial intelligence (AI). As these technologies evolve rapidly, it is crucial to explore how they influence code generation, especially given the risk of misuse in areas like higher education. This paper explores this issue by using advanced classification techniques to differentiate between code written by humans and that generated by ChatGPT, a type of LLM. We employ a new approach that combines powerful embedding features (black-box) with supervised learning algorithms - including Deep Neural Networks, Random Forests, and Extreme Gradient Boosting - to achieve this differentiation with an impressive accuracy of 98%. For the successful combinations, we also examine their model calibration, showing that some of the models are extremely well calibrated. Additionally, we present white-box features and an interpretable Bayes classifier to elucidate critical differences between the code sources, enhancing the explainability and transparency of our approach. Both approaches work well but provide at most 85-88% accuracy. We also show that untrained humans solve the same task not better than random guessing. This study is crucial in understanding and mitigating the potential risks associated with using AI in code generation, particularly in the context of higher education, software development, and competitive programming.
△ Less
Submitted 3 July, 2024; v1 submitted 24 May, 2024;
originally announced May 2024.
-
adjustedCurves: Estimating Confounder-Adjusted Survival Curves in R
Authors:
Robin Denz,
Nina Timmesfeld
Abstract:
Kaplan-Meier curves stratified by treatment allocation are the most popular way to depict causal effects in studies with right-censored time-to-event endpoints. If the treatment is randomly assigned and the sample size of the study is adequate, this method produces unbiased estimates of the population-averaged counterfactual survival curves. However, in the presence of confounding, this is no long…
▽ More
Kaplan-Meier curves stratified by treatment allocation are the most popular way to depict causal effects in studies with right-censored time-to-event endpoints. If the treatment is randomly assigned and the sample size of the study is adequate, this method produces unbiased estimates of the population-averaged counterfactual survival curves. However, in the presence of confounding, this is no longer the case. Instead, specific methods that allow adjustment for confounding must be used. We present the adjustedCurves R package, which can be used to estimate and plot these confounder-adjusted survival curves using a variety of methods from the literature. It provides a convenient wrapper around existing R packages on the topic and adds additional methods and functionality on top of it, uniting the sometimes vastly different methods under one consistent framework. Among the additional features are the estimation of confidence intervals, confounder-adjusted restricted mean survival times and confounder-adjusted survival time quantiles. After giving a brief overview of the implemented methods, we illustrate the package using publicly available data from an observational study including 2982 breast cancer.
△ Less
Submitted 23 February, 2024;
originally announced February 2024.
-
Impact of Record-Linkage Errors in Covid-19 Vaccine-Safety Analyses using German Health-Care Data: A Simulation Study
Authors:
Robin Denz,
Katharina Meiszl,
Peter Ihle,
Doris Oberle,
Ursula Drechsel-Bäuerle,
Katrin Scholz,
Ingo Meyer,
Nina Timmesfeld
Abstract:
With unprecedented speed, 192,248,678 doses of Covid-19 vaccines were administered in Germany by July 11, 2023 to combat the pandemic. Limitations of clinical trials imply that the safety profile of these vaccines is not fully known before marketing. However, routine health-care data can help address these issues. Despite the high proportion of insured people, the analysis of vaccination-related d…
▽ More
With unprecedented speed, 192,248,678 doses of Covid-19 vaccines were administered in Germany by July 11, 2023 to combat the pandemic. Limitations of clinical trials imply that the safety profile of these vaccines is not fully known before marketing. However, routine health-care data can help address these issues. Despite the high proportion of insured people, the analysis of vaccination-related data is challenging in Germany. Generally, the Covid-19 vaccination status and other health-care data are stored in separate databases, without persistent and database-independent person identifiers. Error-prone record-linkage techniques must be used to merge these databases. Our aim was to quantify the impact of record-linkage errors on the power and bias of different analysis methods designed to assess Covid-19 vaccine safety when using German health-care data with a Monte-Carlo simulation study. We used a discrete-time simulation and empirical data to generate realistic data with varying amounts of record-linkage errors. Afterwards, we analysed this data using a Cox model and the self-controlled case series (SCCS) method. Realistic proportions of random linkage errors only had little effect on the power of either method. The SCCS method produced unbiased results even with a high percentage of linkage errors, while the Cox model underestimated the true effect.
△ Less
Submitted 23 October, 2023;
originally announced October 2023.
-
Visualizing the (Causal) Effect of a Continuous Variable on a Time-To-Event Outcome
Authors:
Robin Denz,
Nina Timmesfeld
Abstract:
Visualization is a key aspect of communicating the results of any study aiming to estimate causal effects. In studies with time-to-event outcomes, the most popular visualization approach is depicting survival curves stratified by the variable of interest. This approach cannot be used when the variable of interest is continuous. Simple workarounds, such as categorizing the continuous covariate and…
▽ More
Visualization is a key aspect of communicating the results of any study aiming to estimate causal effects. In studies with time-to-event outcomes, the most popular visualization approach is depicting survival curves stratified by the variable of interest. This approach cannot be used when the variable of interest is continuous. Simple workarounds, such as categorizing the continuous covariate and plotting survival curves for each category, can result in misleading depictions of the main effects. Instead, we propose a new graphic, the survival area plot, to directly depict the survival probability over time and as a function of a continuous covariate simultaneously. This plot utilizes g-computation based on a suitable time-to-event model to obtain the relevant estimates. Through the use of g-computation, those estimates can be adjusted for confounding without additional effort, allowing a causal interpretation under the standard causal identifiability assumptions. If those assumptions are not met, the proposed plot may still be used to depict noncausal associations. We illustrate and compare the proposed graphics to simpler alternatives using data from a large German observational study investigating the effect of the Ankle Brachial Index on survival. To facilitate the usage of these plots, we additionally developed the contsurvplot R-package which includes all methods discussed in this paper.
△ Less
Submitted 6 March, 2023; v1 submitted 9 August, 2022;
originally announced August 2022.
-
A Comparison of Different Methods to Adjust Survival Curves for Confounders
Authors:
Robin Denz,
Renate Klaaßen-Mielke,
Nina Timmesfeld
Abstract:
Treatment specific survival curves are an important tool to illustrate the treatment effect in studies with time-to-event outcomes. In non-randomized studies, unadjusted estimates can lead to biased depictions due to confounding. Multiple methods to adjust survival curves for confounders exist. However, it is currently unclear which method is the most appropriate in which situation. Our goal is to…
▽ More
Treatment specific survival curves are an important tool to illustrate the treatment effect in studies with time-to-event outcomes. In non-randomized studies, unadjusted estimates can lead to biased depictions due to confounding. Multiple methods to adjust survival curves for confounders exist. However, it is currently unclear which method is the most appropriate in which situation. Our goal is to compare forms of Inverse Probability of Treatment Weighting, the G-Formula, Propensity Score Matching, Empirical Likelihood Estimation and augmented estimators as well as their pseudo-values based counterparts in different scenarios with a focus on their bias and goodness-of-fit. We provide a short review of all methods and illustrate their usage by contrasting the survival of smokers and non-smokers, using data from the German Epidemiological Trial on Ankle-Brachial-Index. Subsequently, we compare the methods using a Monte-Carlo simulation. We consider scenarios in which correctly or incorrectly specified models for describing the treatment assignment and the time-to-event outcome are used with varying sample sizes. The bias and goodness-of-fit is determined by taking the entire survival curve into account. When used properly, all methods showed no systematic bias in medium to large samples. Cox regression based methods, however, showed systematic bias in small samples. The goodness-of-fit varied greatly between different methods and scenarios. Methods utilizing an outcome model were more efficient than other techniques, while augmented estimators using an additional treatment assignment model were unbiased when either model was correct with a goodness-of-fit comparable to other methods. These doubly-robust methods have important advantages in every considered scenario.
△ Less
Submitted 15 November, 2022; v1 submitted 18 March, 2022;
originally announced March 2022.
-
Fast Failures in the LHC and the future High Luminosity LHC*
Authors:
B. Lindstrom,
P. Bélanger,
L. Bortot,
R. Denz,
M. Mentink,
E. Ravaioli,
F. Rodriguez Mateos,
R. Schmidt,
J. Uythoven,
M. Valette,
A. Verweij,
C. Wiesner,
D. Wollmann,
M. Zerlauth
Abstract:
An energy of $362\:\text{MJ}$ is stored in each of the two LHC proton beams for nominal beam parameters. This will be further increased to about $700\:\text{MJ}$ in the future High Luminosity LHC (HL-LHC) and uncontrolled beam losses represent a significant hazard for the integrity and safe operation of the machine. In this paper, a number of failure mechanisms that can lead to a fast increase of…
▽ More
An energy of $362\:\text{MJ}$ is stored in each of the two LHC proton beams for nominal beam parameters. This will be further increased to about $700\:\text{MJ}$ in the future High Luminosity LHC (HL-LHC) and uncontrolled beam losses represent a significant hazard for the integrity and safe operation of the machine. In this paper, a number of failure mechanisms that can lead to a fast increase of beam losses are analyzed. Most critical are failures in the magnet protection system, namely the quench heaters and a novel protection system called Coupling-Loss Induced Quench (CLIQ). An important outcome is that magnet protection has to be evaluated for its impact on the beam and designed accordingly. In particular, CLIQ, which is to protect the new HL-LHC triplet magnets, constitutes the fastest known failure in the LHC if triggered spuriously. A schematic change of CLIQ to mitigate the hazard is presented.
A loss of the Beam-Beam Kick due to the extraction of one beam is another source of beam losses with a fast onset. A significantly stronger impact is expected in the upcoming LHC Run III and HL-LHC as compared to the current LHC, mainly due to the increased bunch intensity. Its criticality and mitigation methods are discussed.
It is shown that symmetric quenches in the superconducting magnets for the final focusing triplet can have a significant impact on the beam on short timescales. The impact on the beam due to failures of the Beam-Beam Compensating Wires as well as coherent excitations by the transverse beam damper are also discussed.
△ Less
Submitted 12 May, 2020; v1 submitted 9 May, 2020;
originally announced May 2020.
-
Machine Protection, Interlocks and Availability
Authors:
A. Apollonio,
T. Baer,
K. Dahlerup-Petersen,
R. Denz,
I. Romera Ramirez,
R. Schmidt,
A. Siemko,
J. Wenninger,
D. Wollmann,
M. Zerlauth
Abstract:
Chapter 7 in High-Luminosity Large Hadron Collider (HL-LHC) : Preliminary Design Report. The Large Hadron Collider (LHC) is one of the largest scientific instruments ever built. Since opening up a new energy frontier for exploration in 2010, it has gathered a global user community of about 7,000 scientists working in fundamental particle physics and the physics of hadronic matter at extreme temper…
▽ More
Chapter 7 in High-Luminosity Large Hadron Collider (HL-LHC) : Preliminary Design Report. The Large Hadron Collider (LHC) is one of the largest scientific instruments ever built. Since opening up a new energy frontier for exploration in 2010, it has gathered a global user community of about 7,000 scientists working in fundamental particle physics and the physics of hadronic matter at extreme temperature and density. To sustain and extend its discovery potential, the LHC will need a major upgrade in the 2020s. This will increase its luminosity (rate of collisions) by a factor of five beyond the original design value and the integrated luminosity (total collisions created) by a factor ten. The LHC is already a highly complex and exquisitely optimised machine so this upgrade must be carefully conceived and will require about ten years to implement. The new configuration, known as High Luminosity LHC (HL-LHC), will rely on a number of key innovations that push accelerator technology beyond its present limits. Among these are cutting-edge 11-12 tesla superconducting magnets, compact superconducting cavities for beam rotation with ultra-precise phase control, new technology and physical processes for beam collimation and 300 metre-long high-power superconducting links with negligible energy dissipation. The present document describes the technologies and components that will be used to realise the project and is intended to serve as the basis for the detailed engineering design of HL-LHC.
△ Less
Submitted 26 May, 2017;
originally announced May 2017.
-
Fibre Monitoring System for the Beam Permit Loops at the LHC and Future Evolution of the Beam Interlock System
Authors:
Carlos García-Argos,
Reiner Denz,
Stéphane Gabourin,
Christophe Martin,
Bruno Puccio,
Andrzej P. Siemko
Abstract:
The optical fibres that transmit the beam permit loop signals at the CERN accelerator complex are deployed along radiation areas. This may result in increased attenuation of the fibres, which reduces the power margin of the links. In addition, other events may cause the links to not function properly and result in false dumps, reducing the availability of the accelerator chain and affecting physic…
▽ More
The optical fibres that transmit the beam permit loop signals at the CERN accelerator complex are deployed along radiation areas. This may result in increased attenuation of the fibres, which reduces the power margin of the links. In addition, other events may cause the links to not function properly and result in false dumps, reducing the availability of the accelerator chain and affecting physics data taking. In order to evaluate the state of the fibres, an out-of-band fibre monitoring system is proposed, working in parallel to the actual beam permit loops. The future beam interlock system to be deployed during LHC long shutdown 2 will implement online, real-time monitoring of the fibres, a feature the current system lacks. Commercial off-the-shelf components to implement the optical transceivers are proposed whenever possible instead of ad-hoc designs.
△ Less
Submitted 29 September, 2015;
originally announced September 2015.
