-
Benchmarking the Abilities of Large Language Models for RDF Knowledge Graph Creation and Comprehension: How Well Do LLMs Speak Turtle?
Authors:
Johannes Frey,
Lars-Peter Meyer,
Natanael Arndt,
Felix Brei,
Kirill Bulert
Abstract:
Large Language Models (LLMs) are advancing at a rapid pace, with significant improvements at natural language processing and coding tasks. Yet, their ability to work with formal languages representing data, specifically within the realm of knowledge graph engineering, remains under-investigated. To evaluate the proficiency of various LLMs, we created a set of five tasks that probe their ability to…
▽ More
Large Language Models (LLMs) are advancing at a rapid pace, with significant improvements at natural language processing and coding tasks. Yet, their ability to work with formal languages representing data, specifically within the realm of knowledge graph engineering, remains under-investigated. To evaluate the proficiency of various LLMs, we created a set of five tasks that probe their ability to parse, understand, analyze, and create knowledge graphs serialized in Turtle syntax. These tasks, each embodying distinct degrees of complexity and being able to scale with the size of the problem, have been integrated into our automated evaluation system, the LLM-KG-Bench. The evaluation encompassed four commercially available LLMs - GPT-3.5, GPT-4, Claude 1.3, and Claude 2.0, as well as two freely accessible offline models, GPT4All Vicuna and GPT4All Falcon 13B. This analysis offers an in-depth understanding of the strengths and shortcomings of LLMs in relation to their application within RDF knowledge graph engineering workflows utilizing Turtle representation. While our findings show that the latest commercial models outperform their forerunners in terms of proficiency with the Turtle language, they also reveal an apparent weakness. These models fall short when it comes to adhering strictly to the output formatting constraints, a crucial requirement in this context.
△ Less
Submitted 29 September, 2023;
originally announced September 2023.
-
Superconductivity in Compositionally-Complex Cuprates with the YBa$_2$Cu$_3$O$_{7-x}$ Structure
Authors:
Aditya Raghavan,
Nathan Arndt,
Nayelie Morales-Colón,
Eli Wennen,
Megan Wolfe,
Carolina Oliveira Gandin,
Kade Nelson,
Robert Nowak,
Sam Dillon,
Keon Sahebkar,
Ryan F. Need
Abstract:
High-temperature superconductivity is reported in a series of compositionally-complex cuprates with varying degrees of size and spin disorder. Three compositions of Y-site alloyed YBa$_2$Cu$_3$O$_{7-x}$, i.e., (5Y)BCO, were prepared using solid-state methods with different sets of rare earth ions on the Y-site. Synchrotron X-ray diffraction and energy-dispersive X-ray spectroscopy confirm these sa…
▽ More
High-temperature superconductivity is reported in a series of compositionally-complex cuprates with varying degrees of size and spin disorder. Three compositions of Y-site alloyed YBa$_2$Cu$_3$O$_{7-x}$, i.e., (5Y)BCO, were prepared using solid-state methods with different sets of rare earth ions on the Y-site. Synchrotron X-ray diffraction and energy-dispersive X-ray spectroscopy confirm these samples have high phase-purity and homogeneous mixing of the Y-site elements. The superconducting phase transition was probed using electrical resistivity and AC magnetometry measurements, which reveal the transition temperature, T$_C$, is greater than 91 K for all series when near optimal oxygen do**. Importantly, these T$_C$ values are only $\approx$1$\%$ suppressed relative to pure YBCO (T$_C$ = 93 K). This result highlights the robustness of pairing in the YBCO structure to specific types of disorder. In addition, the chemical flexibility of compositionally-complex cuprates allows spin and lattice disorder to be decoupled to a degree not previously possible in high-temperature superconductors. This feature makes compositionally-complex cuprates a uniquely well-suited materials platform for studying proposed pairing interactions in cuprates.
△ Less
Submitted 21 September, 2023;
originally announced September 2023.
-
Jekyll RDF: Template-Based Linked Data Publication with Minimized Effort and Maximum Scalability
Authors:
Natanael Arndt,
Sebastian Zänker,
Gezim Sejdiu,
Sebastian Tramp
Abstract:
Over the last decades the Web has evolved from a human-human communication network to a network of complex human-machine interactions. An increasing amount of data is available as Linked Data which allows machines to "understand" the data, but RDF is not meant to be understood by humans. With Jekyll RDF we present a method to close the gap between structured data and human accessible exploration i…
▽ More
Over the last decades the Web has evolved from a human-human communication network to a network of complex human-machine interactions. An increasing amount of data is available as Linked Data which allows machines to "understand" the data, but RDF is not meant to be understood by humans. With Jekyll RDF we present a method to close the gap between structured data and human accessible exploration interfaces by publishing RDF datasets as customizable static HTML sites. It consists of an RDF resource map** system to serve the resources under their respective IRI, a template map** based on schema classes, and a markup language to define templates to render customized resource pages. Using the template system, it is possible to create domain specific browsing interfaces for RDF data next to the Linked Data resources. This enables content management and knowledge management systems to serve datasets in a highly customizable, low effort, and scalable way to be consumed by machines as well as humans.
△ Less
Submitted 10 December, 2021;
originally announced January 2022.
-
Relaxation timescales and electron-phonon coupling in optically-pumped YBa$_2$Cu$_3$O$_{6+x}$ revealed by time-resolved Raman scattering
Authors:
N. Pellatz,
S. Roy,
J-W. Lee,
J. L. Schad,
H. Kandel,
N. Arndt,
C. B. Eom,
A. F. Kemper,
D. Reznik
Abstract:
Time resolved measurements provide a new way to disentangle complex interactions in quantum materials due to their different timescales. We used pump-probe Raman scattering to investigate the apical oxygen vibration in YBa$_2$Cu$_3$O$_{6+x}$ under nonequilibrium conditions. Time-dependence of the phonon population demonstrated strong electron-phonon coupling. Most importantly, the phonon shifts to…
▽ More
Time resolved measurements provide a new way to disentangle complex interactions in quantum materials due to their different timescales. We used pump-probe Raman scattering to investigate the apical oxygen vibration in YBa$_2$Cu$_3$O$_{6+x}$ under nonequilibrium conditions. Time-dependence of the phonon population demonstrated strong electron-phonon coupling. Most importantly, the phonon shifts to a higher energy due to transient smearing of the Fermi surface in a remarkable agreement with diagrammatic theory. We also discuss new insights into photoinduced superconductivity reported at lower do** that follow from these results.
△ Less
Submitted 2 December, 2021; v1 submitted 29 October, 2020;
originally announced October 2020.
-
Decentralized Evolution and Consolidation of RDF Graphs
Authors:
Natanael Arndt,
Michael Martin
Abstract:
The World Wide Web and the Semantic Web are designed as a network of distributed services and datasets. In this network and its genesis, collaboration played and still plays a crucial role. But currently we only have central collaboration solutions for RDF data, such as SPARQL endpoints and wiki systems, while decentralized solutions can enable applications for many more use-cases. Inspired by a s…
▽ More
The World Wide Web and the Semantic Web are designed as a network of distributed services and datasets. In this network and its genesis, collaboration played and still plays a crucial role. But currently we only have central collaboration solutions for RDF data, such as SPARQL endpoints and wiki systems, while decentralized solutions can enable applications for many more use-cases. Inspired by a successful distributed source code management methodology in software engineering a framework to support distributed evolution is proposed. The system is based on Git and provides distributed collaboration on RDF graphs. This paper covers the formal expression of the evolution and consolidation of distributed datasets, the synchronization, as well as other supporting operations.
△ Less
Submitted 27 February, 2019;
originally announced February 2019.
-
Decentralized Collaborative Knowledge Management using Git
Authors:
Natanael Arndt,
Patrick Naumann,
Norman Radtke,
Michael Martin,
Edgard Marx
Abstract:
The World Wide Web and the Semantic Web are designed as a network of distributed services and datasets. The distributed character of the Web brings manifold collaborative possibilities to interchange data. The commonly adopted collaborative solutions for RDF data are centralized (e.g. SPARQL endpoints and wiki systems). But to support distributed collaboration, a system is needed, that supports di…
▽ More
The World Wide Web and the Semantic Web are designed as a network of distributed services and datasets. The distributed character of the Web brings manifold collaborative possibilities to interchange data. The commonly adopted collaborative solutions for RDF data are centralized (e.g. SPARQL endpoints and wiki systems). But to support distributed collaboration, a system is needed, that supports divergence of datasets, brings the possibility to conflate diverged states, and allows distributed datasets to be synchronized. In this paper, we present Quit Store, it was inspired by and it builds upon the successful Git system. The approach is based on a formal expression of evolution and consolidation of distributed datasets. During the collaborative curation process, the system automatically versions the RDF dataset and tracks provenance information. It also provides support to branch, merge, and synchronize distributed RDF datasets. The merging process is guarded by specific merge strategies for RDF data. Finally, we use our reference implementation to show overall good performance and demonstrate the practical usability of the system.
△ Less
Submitted 16 October, 2018; v1 submitted 9 May, 2018;
originally announced May 2018.
