-
Word-specific tonal realizations in Mandarin
Authors:
Yu-Ying Chuang,
Melanie J. Bell,
Yu-Hsiang Tseng,
R. Harald Baayen
Abstract:
The pitch contours of Mandarin two-character words are generally understood as being shaped by the underlying tones of the constituent single-character words, in interaction with articulatory constraints imposed by factors such as speech rate, co-articulation with adjacent tones, segmental make-up, and predictability. This study shows that tonal realization is also partially determined by words' m…
▽ More
The pitch contours of Mandarin two-character words are generally understood as being shaped by the underlying tones of the constituent single-character words, in interaction with articulatory constraints imposed by factors such as speech rate, co-articulation with adjacent tones, segmental make-up, and predictability. This study shows that tonal realization is also partially determined by words' meanings. We first show, on the basis of a Taiwan corpus of spontaneous conversations, using the generalized additive regression model, and focusing on the rise-fall tone pattern, that after controlling for effects of speaker and context, word type is a stronger predictor of pitch realization than all the previously established word-form related predictors combined. Importantly, the addition of information about meaning in context improves prediction accuracy even further. We then proceed to show, using computational modeling with context-specific word embeddings, that token-specific pitch contours predict word type with 50% accuracy on held-out data, and that context-sensitive, token-specific embeddings can predict the shape of pitch contours with 30% accuracy. These accuracies, which are an order of magnitude above chance level, suggest that the relation between words' pitch contours and their meanings are sufficiently strong to be functional for language users. The theoretical implications of these empirical findings are discussed.
△ Less
Submitted 11 May, 2024;
originally announced May 2024.
-
Frequency effects in Linear Discriminative Learning
Authors:
Maria Heitmeier,
Yu-Ying Chuang,
Seth D. Axen,
R. Harald Baayen
Abstract:
Word frequency is a strong predictor in most lexical processing tasks. Thus, any model of word recognition needs to account for how word frequency effects arise. The Discriminative Lexicon Model (DLM; Baayen et al., 2018a, 2019) models lexical processing with linear map**s between words' forms and their meanings. So far, the map**s can either be obtained incrementally via error-driven learning…
▽ More
Word frequency is a strong predictor in most lexical processing tasks. Thus, any model of word recognition needs to account for how word frequency effects arise. The Discriminative Lexicon Model (DLM; Baayen et al., 2018a, 2019) models lexical processing with linear map**s between words' forms and their meanings. So far, the map**s can either be obtained incrementally via error-driven learning, a computationally expensive process able to capture frequency effects, or in an efficient, but frequency-agnostic solution modelling the theoretical endstate of learning (EL) where all words are learned optimally. In this study we show how an efficient, yet frequency-informed map** between form and meaning can be obtained (Frequency-informed learning; FIL). We find that FIL well approximates an incremental solution while being computationally much cheaper. FIL shows a relatively low type- and high token-accuracy, demonstrating that the model is able to process most word tokens encountered by speakers in daily life correctly. We use FIL to model reaction times in the Dutch Lexicon Project (Keuleers et al., 2010) and find that FIL predicts well the S-shaped relationship between frequency and the mean of reaction times but underestimates the variance of reaction times for low frequency words. FIL is also better able to account for priming effects in an auditory lexical decision task in Mandarin Chinese (Lee, 2007), compared to EL. Finally, we used ordered data from CHILDES (Brown, 1973; Demuth et al., 2006) to compare map**s obtained with FIL and incremental learning. The map**s are highly correlated, but with FIL some nuances based on word ordering effects are lost. Our results show how frequency effects in a learning model can be simulated efficiently, and raise questions about how to best account for low-frequency words in cognitive models.
△ Less
Submitted 18 March, 2024; v1 submitted 19 June, 2023;
originally announced June 2023.
-
Visual Grounding of Inter-lingual Word-Embeddings
Authors:
Wafaa Mohammed,
Hassan Shahmohammadi,
Hendrik P. A. Lensch,
R. Harald Baayen
Abstract:
Visual grounding of Language aims at enriching textual representations of language with multiple sources of visual knowledge such as images and videos. Although visual grounding is an area of intense research, inter-lingual aspects of visual grounding have not received much attention. The present study investigates the inter-lingual visual grounding of word embeddings. We propose an implicit align…
▽ More
Visual grounding of Language aims at enriching textual representations of language with multiple sources of visual knowledge such as images and videos. Although visual grounding is an area of intense research, inter-lingual aspects of visual grounding have not received much attention. The present study investigates the inter-lingual visual grounding of word embeddings. We propose an implicit alignment technique between the two spaces of vision and language in which inter-lingual textual information interacts in order to enrich pre-trained textual word embeddings. We focus on three languages in our experiments, namely, English, Arabic, and German. We obtained visually grounded vector representations for these languages and studied whether visual grounding on one or multiple languages improved the performance of embeddings on word similarity and categorization benchmarks. Our experiments suggest that inter-lingual knowledge improves the performance of grounded embeddings in similar languages such as German and English. However, inter-lingual grounding of German or English with Arabic led to a slight degradation in performance on word similarity benchmarks. On the other hand, we observed an opposite trend on categorization benchmarks where Arabic had the most improvement on English. In the discussion section, several reasons for those findings are laid out. We hope that our experiments provide a baseline for further research on inter-lingual visual grounding.
△ Less
Submitted 21 November, 2022; v1 submitted 8 September, 2022;
originally announced September 2022.
-
Making sense of spoken plurals
Authors:
Elnaz Shafaei-Bajestan,
Peter Uhrig,
R. Harald Baayen
Abstract:
Distributional semantics offers new ways to study the semantics of morphology. This study focuses on the semantics of noun singulars and their plural inflectional variants in English. Our goal is to compare two models for the conceptualization of plurality. One model (FRACSS) proposes that all singular-plural pairs should be taken into account when predicting plural semantics from singular semanti…
▽ More
Distributional semantics offers new ways to study the semantics of morphology. This study focuses on the semantics of noun singulars and their plural inflectional variants in English. Our goal is to compare two models for the conceptualization of plurality. One model (FRACSS) proposes that all singular-plural pairs should be taken into account when predicting plural semantics from singular semantics. The other model (CCA) argues that conceptualization for plurality depends primarily on the semantic class of the base word. We compare the two models on the basis of how well the speech signal of plural tokens in a large corpus of spoken American English aligns with the semantic vectors predicted by the two models. Two measures are employed: the performance of a form-to-meaning map** and the correlations between form distances and meaning distances. Results converge on a superior alignment for CCA. Our results suggest that usage-based approaches to pluralization in which a given word's own semantic neighborhood is given priority outperform theories according to which pluralization is conceptualized as a process building on high-level abstraction. We see that what has often been conceived of as a highly abstract concept, [+plural], is better captured via a family of mid-level partial generalizations.
△ Less
Submitted 30 January, 2023; v1 submitted 5 July, 2022;
originally announced July 2022.
-
How trial-to-trial learning shapes map**s in the mental lexicon: Modelling Lexical Decision with Linear Discriminative Learning
Authors:
Maria Heitmeier,
Yu-Ying Chuang,
R. Harald Baayen
Abstract:
Trial-to-trial effects have been found in a number of studies, indicating that processing a stimulus influences responses in subsequent trials. A special case are priming effects which have been modelled successfully with error-driven learning (Marsolek, 2008), implying that participants are continuously learning during experiments. This study investigates whether trial-to-trial learning can be de…
▽ More
Trial-to-trial effects have been found in a number of studies, indicating that processing a stimulus influences responses in subsequent trials. A special case are priming effects which have been modelled successfully with error-driven learning (Marsolek, 2008), implying that participants are continuously learning during experiments. This study investigates whether trial-to-trial learning can be detected in an unprimed lexical decision experiment. We used the Discriminative Lexicon Model (DLM; Baayen et al., 2019), a model of the mental lexicon with meaning representations from distributional semantics, which models error-driven incremental learning with the Widrow-Hoff rule. We used data from the British Lexicon Project (BLP; Keuleers et al., 2012) and simulated the lexical decision experiment with the DLM on a trial-by-trial basis for each subject individually. Then, reaction times were predicted with Generalised Additive Models (GAMs), using measures derived from the DLM simulations as predictors. We extracted measures from two simulations per subject (one with learning updates between trials and one without), and used them as input to two GAMs. Learning-based models showed better model fit than the non-learning ones for the majority of subjects. Our measures also provide insights into lexical processing and individual differences. This demonstrates the potential of the DLM to model behavioural data and leads to the conclusion that trial-to-trial learning can indeed be detected in unprimed lexical decision. Our results support the possibility that our lexical knowledge is subject to continuous changes.
△ Less
Submitted 4 September, 2023; v1 submitted 1 July, 2022;
originally announced July 2022.
-
How direct is the link between words and images?
Authors:
Hassan Shahmohammadi,
Maria Heitmeier,
Elnaz Shafaei-Bajestan,
Hendrik P. A. Lensch,
Harald Baayen
Abstract:
Current word embedding models despite their success, still suffer from their lack of grounding in the real world. In this line of research, Gunther et al. 2022 proposed a behavioral experiment to investigate the relationship between words and images. In their setup, participants were presented with a target noun and a pair of images, one chosen by their model and another chosen randomly. Participa…
▽ More
Current word embedding models despite their success, still suffer from their lack of grounding in the real world. In this line of research, Gunther et al. 2022 proposed a behavioral experiment to investigate the relationship between words and images. In their setup, participants were presented with a target noun and a pair of images, one chosen by their model and another chosen randomly. Participants were asked to select the image that best matched the target noun. In most cases, participants preferred the image selected by the model. Gunther et al., therefore, concluded the possibility of a direct link between words and embodied experience. We took their experiment as a point of departure and addressed the following questions. 1. Apart from utilizing visually embodied simulation of given images, what other strategies might subjects have used to solve this task? To what extent does this setup rely on visual information from images? Can it be solved using purely textual representations? 2. Do current visually grounded embeddings explain subjects' selection behavior better than textual embeddings? 3. Does visual grounding improve the semantic representations of both concrete and abstract words? To address these questions, we designed novel experiments by using pre-trained textual and visually grounded word embeddings. Our experiments reveal that subjects' selection behavior is explained to a large extent based on purely text-based embeddings and word-based similarities, suggesting a minor involvement of active embodied experiences. Visually grounded embeddings offered modest advantages over textual embeddings only in certain cases. These findings indicate that the experiment by Gunther et al. may not be well suited for tap** into the perceptual experience of participants, and therefore the extent to which it measures visually grounded knowledge is unclear.
△ Less
Submitted 31 October, 2023; v1 submitted 30 June, 2022;
originally announced June 2022.
-
Language with Vision: a Study on Grounded Word and Sentence Embeddings
Authors:
Hassan Shahmohammadi,
Maria Heitmeier,
Elnaz Shafaei-Bajestan,
Hendrik P. A. Lensch,
Harald Baayen
Abstract:
Grounding language in vision is an active field of research seeking to construct cognitively plausible word and sentence representations by incorporating perceptual knowledge from vision into text-based representations. Despite many attempts at language grounding, achieving an optimal equilibrium between textual representations of the language and our embodied experiences remains an open field. So…
▽ More
Grounding language in vision is an active field of research seeking to construct cognitively plausible word and sentence representations by incorporating perceptual knowledge from vision into text-based representations. Despite many attempts at language grounding, achieving an optimal equilibrium between textual representations of the language and our embodied experiences remains an open field. Some common concerns are the following. Is visual grounding advantageous for abstract words, or is its effectiveness restricted to concrete words? What is the optimal way of bridging the gap between text and vision? To what extent is perceptual knowledge from images advantageous for acquiring high-quality embeddings? Leveraging the current advances in machine learning and natural language processing, the present study addresses these questions by proposing a simple yet very effective computational grounding model for pre-trained word embeddings. Our model effectively balances the interplay between language and vision by aligning textual embeddings with visual information while simultaneously preserving the distributional statistics that characterize word usage in text corpora. By applying a learned alignment, we are able to indirectly ground unseen words including abstract words. A series of evaluations on a range of behavioural datasets shows that visual grounding is beneficial not only for concrete words but also for abstract words, lending support to the indirect theory of abstract concepts. Moreover, our approach offers advantages for contextualized embeddings, such as those generated by BERT, but only when trained on corpora of modest, cognitively plausible sizes. Code and grounded embeddings for English are available at https://github.com/Hazel1994/Visually_Grounded_Word_Embeddings_2.
△ Less
Submitted 31 October, 2023; v1 submitted 17 June, 2022;
originally announced June 2022.
-
Semantic properties of English nominal pluralization: Insights from word embeddings
Authors:
Elnaz Shafaei-Bajestan,
Masoumeh Moradipour-Tari,
Peter Uhrig,
R. Harald Baayen
Abstract:
Semantic differentiation of nominal pluralization is grammaticalized in many languages. For example, plural markers may only be relevant for human nouns. English does not appear to make such distinctions. Using distributional semantics, we show that English nominal pluralization exhibits semantic clusters. For instance, pluralization of fruit words is more similar to one another and less similar t…
▽ More
Semantic differentiation of nominal pluralization is grammaticalized in many languages. For example, plural markers may only be relevant for human nouns. English does not appear to make such distinctions. Using distributional semantics, we show that English nominal pluralization exhibits semantic clusters. For instance, pluralization of fruit words is more similar to one another and less similar to pluralization of other semantic classes. Therefore, reduction of the meaning shift in plural formation to the addition of an abstract plural meaning is too simplistic. A semantically informed method, called CosClassAvg, is introduced that outperforms pluralization methods in distributional semantics which assume plural formation amounts to the addition of a fixed plural vector. In comparison with our approach, a method from compositional distributional semantics, called FRACSS, predicted plural vectors that were more similar to the corpus-extracted plural vectors in terms of direction but not vector length. A modeling study reveals that the observed difference between the two predicted semantic spaces by CosClassAvg and FRACSS carries over to how well a computational model of the listener can understand previously unencountered plural forms. Map**s from word forms, represented with triphone vectors, to predicted semantic vectors are more productive when CosClassAvg-generated semantic vectors are employed as gold standard vectors instead of FRACSS-generated vectors.
△ Less
Submitted 29 March, 2022;
originally announced March 2022.
-
Vector Space Morphology with Linear Discriminative Learning
Authors:
Yu-Ying Chuang,
Mihi Kang,
Xuefeng Luo,
R. Harald Baayen
Abstract:
This paper presents three case studies of modeling aspects of lexical processing with Linear Discriminative Learning (LDL), the computational engine of the Discriminative Lexicon model (Baayen et al., 2019). With numeric representations of word forms and meanings, LDL learns to map one vector space onto the other, without being informed about any morphological structure or inflectional classes. Th…
▽ More
This paper presents three case studies of modeling aspects of lexical processing with Linear Discriminative Learning (LDL), the computational engine of the Discriminative Lexicon model (Baayen et al., 2019). With numeric representations of word forms and meanings, LDL learns to map one vector space onto the other, without being informed about any morphological structure or inflectional classes. The modeling results demonstrated that LDL not only performs well for understanding and producing morphologically complex words, but also generates quantitative measures that are predictive for human behavioral data. LDL models are straightforward to implement with the JudiLing package (Luo et al., 2021). Worked examples are provided for three modeling challenges: producing and understanding Korean verb inflection, predicting primed Dutch lexical decision latencies, and predicting the acoustic duration of Mandarin words.
△ Less
Submitted 8 July, 2021;
originally announced July 2021.
-
Modeling morphology with Linear Discriminative Learning: considerations and design choices
Authors:
Maria Heitmeier,
Yu-Ying Chuang,
R. Harald Baayen
Abstract:
This study addresses a series of methodological questions that arise when modeling inflectional morphology with Linear Discriminative Learning. Taking the semi-productive German noun system as example, we illustrate how decisions made about the representation of form and meaning influence model performance. We clarify that for modeling frequency effects in learning, it is essential to make use of…
▽ More
This study addresses a series of methodological questions that arise when modeling inflectional morphology with Linear Discriminative Learning. Taking the semi-productive German noun system as example, we illustrate how decisions made about the representation of form and meaning influence model performance. We clarify that for modeling frequency effects in learning, it is essential to make use of incremental learning rather than the endstate of learning. We also discuss how the model can be set up to approximate the learning of inflected words in context. In addition, we illustrate how in this approach the wug task can be modeled in considerable detail. In general, the model provides an excellent memory for known words, but appropriately shows more limited performance for unseen data, in line with the semi-productivity of German noun inflection and generalization performance of native German speakers.
△ Less
Submitted 18 November, 2021; v1 submitted 15 June, 2021;
originally announced June 2021.
-
A note on the modeling of the effects of experimental time in psycholinguistic experiments
Authors:
R. Harald Baayen,
Matteo Fasiolo,
Simon Wood,
Yu-Ying Chuang
Abstract:
Thul et al. (2020) called attention to problems that arise when chronometric experiments implementing specific factorial designs are analysed with the generalized additive mixed model (GAMM), using factor smooths to capture trial-to-trial dependencies. From a series of simulations incorporating such dependencies, they conclude that GAMMs are inappropriate for between-subject designs. They argue th…
▽ More
Thul et al. (2020) called attention to problems that arise when chronometric experiments implementing specific factorial designs are analysed with the generalized additive mixed model (GAMM), using factor smooths to capture trial-to-trial dependencies. From a series of simulations incorporating such dependencies, they conclude that GAMMs are inappropriate for between-subject designs. They argue that in addition GAMMs come with too many modeling possibilities, and advise using the linear mixed model (LMM) instead. We address the questions raised by Thul et al. (2020), who clearly demonstrated that problems can indeed arise when using factor smooths in combination with factorial designs. We show that the problem does not arise when using by-smooths. Furthermore, we have traced a bug in the implementation of factor smooths in the mgcv package, which will have been removed from version 1.8-36 onwards. To illustrate that GAMMs now produce correct estimates, we report simulation studies implementing different by-subject longitudinal effects. The maximal LMM emerges as slightly conservative compared to GAMMs, and GAMMs provide estimated coefficients that can be less variable across simulation runs. We also discuss two datasets where time-varying effects interact with numerical predictors in a theoretically informative way. Furthermore, we argue that the wide range of tools that GAMMs make available to researcher across all domains of scientific inquiry do not come with uncontrolled researcher degrees of freedom once confronted with a specific psycholinguistic datasets. We also introduce a distinction between replicable and non-replicable non-linear effects. We conclude that GAMMs are an excellent and reliable tool for understanding experimental data, including chronometric data with time-varying effects.
△ Less
Submitted 17 November, 2021; v1 submitted 28 May, 2021;
originally announced May 2021.
-
Learning Zero-Shot Multifaceted Visually Grounded Word Embeddings via Multi-Task Training
Authors:
Hassan Shahmohammadi,
Hendrik P. A. Lensch,
R. Harald Baayen
Abstract:
Language grounding aims at linking the symbolic representation of language (e.g., words) into the rich perceptual knowledge of the outside world. The general approach is to embed both textual and visual information into a common space -the grounded space-confined by an explicit relationship between both modalities. We argue that this approach sacrifices the abstract knowledge obtained from linguis…
▽ More
Language grounding aims at linking the symbolic representation of language (e.g., words) into the rich perceptual knowledge of the outside world. The general approach is to embed both textual and visual information into a common space -the grounded space-confined by an explicit relationship between both modalities. We argue that this approach sacrifices the abstract knowledge obtained from linguistic co-occurrence statistics in the process of acquiring perceptual information. The focus of this paper is to solve this issue by implicitly grounding the word embeddings. Rather than learning two map**s into a joint space, our approach integrates modalities by determining a reversible grounded map** between the textual and the grounded space by means of multi-task learning. Evaluations on intrinsic and extrinsic tasks show that our embeddings are highly beneficial for both abstract and concrete words. They are strongly correlated with human judgments and outperform previous works on a wide range of benchmarks. Our grounded embeddings are publicly available here.
△ Less
Submitted 13 September, 2021; v1 submitted 15 April, 2021;
originally announced April 2021.
-
Global optimality in model predictive control via hidden invariant convexity
Authors:
Jorn H. Baayen,
Krzysztof Postek
Abstract:
Non-convex optimal control problems occurring in, e.g., water or power systems, typically involve a large number of variables related through nonlinear equality constraints. The ideal goal is to find a globally optimal solution, and numerical experience indicates that algorithms aiming for Karush-Kuhn-Tucker points often find (near-)optimal solutions. In our paper, we provide a theoretical underpi…
▽ More
Non-convex optimal control problems occurring in, e.g., water or power systems, typically involve a large number of variables related through nonlinear equality constraints. The ideal goal is to find a globally optimal solution, and numerical experience indicates that algorithms aiming for Karush-Kuhn-Tucker points often find (near-)optimal solutions. In our paper, we provide a theoretical underpinning for this phenomenon, showing that on a broad class of problems the objective can be shown to be an invariantly convex function (invex function) of the control decision variables when state variables are eliminated using implicit function theory. In this way, near-global optimality can be demonstrated, where the exact nature of the global optimality guarantee depends on the position of the solution within the feasible set. In a numerical example, we show how high-quality solutions are obtained with local search for a river control problem where invexity holds.
△ Less
Submitted 7 September, 2020; v1 submitted 14 July, 2020;
originally announced July 2020.
-
Learning Precise Spike Timings with Eligibility Traces
Authors:
Manuel Traub,
Martin V. Butz,
R. Harald Baayen,
Sebastian Otte
Abstract:
Recent research in the field of spiking neural networks (SNNs) has shown that recurrent variants of SNNs, namely long short-term SNNs (LSNNs), can be trained via error gradients just as effective as LSTMs. The underlying learning method (e-prop) is based on a formalization of eligibility traces applied to leaky integrate and fire (LIF) neurons. Here, we show that the proposed approach cannot fully…
▽ More
Recent research in the field of spiking neural networks (SNNs) has shown that recurrent variants of SNNs, namely long short-term SNNs (LSNNs), can be trained via error gradients just as effective as LSTMs. The underlying learning method (e-prop) is based on a formalization of eligibility traces applied to leaky integrate and fire (LIF) neurons. Here, we show that the proposed approach cannot fully unfold spike timing dependent plasticity (STDP). As a consequence, this limits in principle the inherent advantage of SNNs, that is, the potential to develop codes that rely on precise relative spike timings. We show that STDP-aware synaptic gradients naturally emerge within the eligibility equations of e-prop when derived for a slightly more complex spiking neuron model, here at the example of the Izhikevich model. We also present a simple extension of the LIF model that provides similar gradients. In a simple experiment we demonstrate that the STDP-aware LIF neurons can learn precise spike timings from an e-prop-based gradient signal.
△ Less
Submitted 8 May, 2020;
originally announced June 2020.
-
A continuation approach to the optimization of hydropower operations
Authors:
Jorn H. Baayen,
Julia Rauw,
Teresa Piovesan
Abstract:
The instantaneous power generation from a hydroelectric turbine is proportional to the product of head difference and turbine flow. The equation relating power to hydraulic variables is therefore nonlinear. Hence, optimization problems subject to this relation, such as release schedule optimization, are nonconvex and may admit multiple local isolated minima. This renders such problems problematic…
▽ More
The instantaneous power generation from a hydroelectric turbine is proportional to the product of head difference and turbine flow. The equation relating power to hydraulic variables is therefore nonlinear. Hence, optimization problems subject to this relation, such as release schedule optimization, are nonconvex and may admit multiple local isolated minima. This renders such problems problematic for use in operational model predictive control. This paper shows that release schedule optimization problems subject to the nonlinear turbine generation equation may be set up using a continuation approach to be both zero-convex and path stable. In this way such optimization problems become suitable for decision support systems based on model predictive control. An example problem is studied, and it is shown that significant productivity gains may be realized using the presented methodology.
△ Less
Submitted 2 May, 2018;
originally announced May 2018.
-
Autocorrelated errors in experimental data in the language sciences: Some solutions offered by Generalized Additive Mixed Models
Authors:
R. Harald Baayen,
Jacolien van Rij,
Cecile de Cat,
Simon N. Wood
Abstract:
A problem that tends to be ignored in the statistical analysis of experimental data in the language sciences is that responses often constitute time series, which raises the problem of autocorrelated errors. If the errors indeed show autocorrelational structure, evaluation of the significance of predictors in the model becomes problematic due to potential anti-conservatism of p-values. This paper…
▽ More
A problem that tends to be ignored in the statistical analysis of experimental data in the language sciences is that responses often constitute time series, which raises the problem of autocorrelated errors. If the errors indeed show autocorrelational structure, evaluation of the significance of predictors in the model becomes problematic due to potential anti-conservatism of p-values. This paper illustrates two tools offered by Generalized Additive Mixed Models (GAMMs) (Lin and Zhang, 1999; Wood, 2006, 2011, 2013) for dealing with autocorrelated errors, as implemented in the current version of the fourth author's mgcv package (1.8.9): the possibility to specify an ar(1) error model for Gaussian models, and the possibility of using factor smooths for random-effect factors such as subject and item. These factor smooths are set up to have the same smoothing parameters, and are penalized to yield the non-linear equivalent of random intercepts and random slopes in the classical linear framework. Three case studies illustrate these issues.
△ Less
Submitted 8 January, 2016;
originally announced January 2016.
-
The cave of Shadows. Addressing the human factor with generalized additive mixed models
Authors:
Harald Baayen,
Shravan Vasishth,
Douglas Bates,
Reinhold Kliegl
Abstract:
Generalized additive mixed models are introduced as an extension of the generalized linear mixed model which makes it possible to deal with temporal autocorrelational structure in experimental data. This autocorrelational structure is likely to be a consequence of learning, fatigue, or the ebb and flow of attention within an experiment (the `human factor'). Unlike molecules or plots of barley, sub…
▽ More
Generalized additive mixed models are introduced as an extension of the generalized linear mixed model which makes it possible to deal with temporal autocorrelational structure in experimental data. This autocorrelational structure is likely to be a consequence of learning, fatigue, or the ebb and flow of attention within an experiment (the `human factor'). Unlike molecules or plots of barley, subjects in psycholinguistic experiments are intelligent beings that depend for their survival on constant adaptation to their environment, including the environment of an experiment. Three data sets illustrate that the human factor may interact with predictors of interest, both factorial and metric. We also show that, especially within the framework of the generalized additive model, in the nonlinear world, fitting maximally complex models that take every possible contingency into account is ill-advised as a modeling strategy. Alternative modeling strategies are discussed for both confirmatory and exploratory data analysis.
△ Less
Submitted 14 November, 2016; v1 submitted 10 November, 2015;
originally announced November 2015.
-
Balancing Type I Error and Power in Linear Mixed Models
Authors:
Hannes Matuschek,
Reinhold Kliegl,
Shravan Vasishth,
Harald Baayen,
Douglas Bates
Abstract:
Linear mixed-effects models have increasingly replaced mixed-model analyses of variance for statistical inference in factorial psycholinguistic experiments. Although LMMs have many advantages over ANOVA, like ANOVAs, setting them up for data analysis also requires some care. One simple option, when numerically possible, is to fit the full variance-covariance structure of random effects (the maxima…
▽ More
Linear mixed-effects models have increasingly replaced mixed-model analyses of variance for statistical inference in factorial psycholinguistic experiments. Although LMMs have many advantages over ANOVA, like ANOVAs, setting them up for data analysis also requires some care. One simple option, when numerically possible, is to fit the full variance-covariance structure of random effects (the maximal model; Barr et al. 2013), presumably to keep Type I error down to the nominal alpha in the presence of random effects. Although it is true that fitting a model with only random intercepts may lead to higher Type I error, fitting a maximal model also has a cost: it can lead to a significant loss of power. We demonstrate this with simulations and suggest that for typical psychological and psycholinguistic data, higher power is achieved without inflating Type I error rate if a model selection criterion is used to select a random effect structure that is supported by the data.
△ Less
Submitted 2 January, 2017; v1 submitted 5 November, 2015;
originally announced November 2015.
-
Parsimonious Mixed Models
Authors:
Douglas Bates,
Reinhold Kliegl,
Shravan Vasishth,
Harald Baayen
Abstract:
The analysis of experimental data with mixed-effects models requires decisions about the specification of the appropriate random-effects structure. Recently, Barr, Levy, Scheepers, and Tily, 2013 recommended fitting `maximal' models with all possible random effect components included. Estimation of maximal models, however, may not converge. We show that failure to converge typically is not due to…
▽ More
The analysis of experimental data with mixed-effects models requires decisions about the specification of the appropriate random-effects structure. Recently, Barr, Levy, Scheepers, and Tily, 2013 recommended fitting `maximal' models with all possible random effect components included. Estimation of maximal models, however, may not converge. We show that failure to converge typically is not due to a suboptimal estimation algorithm, but is a consequence of attempting to fit a model that is too complex to be properly supported by the data, irrespective of whether estimation is based on maximum likelihood or on Bayesian hierarchical modeling with uninformative or weakly informative priors. Importantly, even under convergence, overparameterization may lead to uninterpretable models. We provide diagnostic tools for detecting overparameterization and guiding model simplification.
△ Less
Submitted 26 May, 2018; v1 submitted 16 June, 2015;
originally announced June 2015.
-
Trajectory tracking control of kites with system delay
Authors:
Jorn H. Baayen
Abstract:
A previously published algorithm for trajectory tracking control of tethered wings, i.e. kites, is updated in light of recent experimental evidence. The algorithm is, furthermore, analyzed in the framework of delay differential equations. It is shown how the presence of system delay influences the stability of the control system, and a methodology is derived for gain selection using the Lambert W…
▽ More
A previously published algorithm for trajectory tracking control of tethered wings, i.e. kites, is updated in light of recent experimental evidence. The algorithm is, furthermore, analyzed in the framework of delay differential equations. It is shown how the presence of system delay influences the stability of the control system, and a methodology is derived for gain selection using the Lambert W function. The validity of the methodology is demonstrated with simulation results. The analysis sheds light on previously poorly understood stability problems.
△ Less
Submitted 27 December, 2012;
originally announced December 2012.
-
Vortexje - An Open-Source Panel Method for Co-Simulation
Authors:
Jorn H. Baayen
Abstract:
This paper discusses the use of the 3-dimensional panel method for dynamical system simulation. Specifically, the advantages and disadvantages of model exchange versus co-simulation of the aerodynamics and the dynamical system model are discussed. Based on a trade-off analysis, a set of recommendations for a panel method implementation and for a co-simulation environment is proposed. These recomme…
▽ More
This paper discusses the use of the 3-dimensional panel method for dynamical system simulation. Specifically, the advantages and disadvantages of model exchange versus co-simulation of the aerodynamics and the dynamical system model are discussed. Based on a trade-off analysis, a set of recommendations for a panel method implementation and for a co-simulation environment is proposed. These recommendations are implemented in a C++ library, offered on-line under an open source license. This code is validated against XFLR5, and its suitability for co-simulation is demonstrated with an example of a tethered wing, i.e, a kite. The panel method implementation and the co-simulation environment are shown to be able to solve this stiff problem in a stable fashion.
△ Less
Submitted 9 March, 2013; v1 submitted 25 October, 2012;
originally announced October 2012.
-
Tracking control with adaption of kites
Authors:
Jorn H. Baayen,
Wubbo J. Ockels
Abstract:
A novel tracking paradigm for flying geometric trajectories using tethered kites is presented. It is shown how the differential-geometric notion of turning angle can be used as a one-dimensional representation of the kite trajectory, and how this leads to a single-input single-output (SISO) tracking problem. Based on this principle a Lyapunov-based nonlinear adaptive controller is developed that o…
▽ More
A novel tracking paradigm for flying geometric trajectories using tethered kites is presented. It is shown how the differential-geometric notion of turning angle can be used as a one-dimensional representation of the kite trajectory, and how this leads to a single-input single-output (SISO) tracking problem. Based on this principle a Lyapunov-based nonlinear adaptive controller is developed that only needs control derivatives of the kite aerodynamic model. The resulting controller is validated using simulations with a point-mass kite model.
△ Less
Submitted 15 July, 2011; v1 submitted 3 November, 2010;
originally announced November 2010.
-
Estimating Lexical Priors for Low-Frequency Syncretic Forms
Authors:
Harald Baayen,
Richard Sproat
Abstract:
Given a previously unseen form that is morphologically n-ways ambiguous, what is the best estimator for the lexical prior probabilities for the various functions of the form? We argue that the best estimator is provided by computing the relative frequencies of the various functions among the hapax legomena --- the forms that occur exactly once in a corpus. This result has important implications…
▽ More
Given a previously unseen form that is morphologically n-ways ambiguous, what is the best estimator for the lexical prior probabilities for the various functions of the form? We argue that the best estimator is provided by computing the relative frequencies of the various functions among the hapax legomena --- the forms that occur exactly once in a corpus. This result has important implications for the development of stochastic morphological taggers, especially when some initial hand-tagging of a corpus is required: For predicting lexical priors for very low-frequency morphologically ambiguous types (most of which would not occur in any given corpus) one should concentrate on tagging a good representative sample of the hapax legomena, rather than extensively tagging words of all frequency ranges.
△ Less
Submitted 24 April, 1995;
originally announced April 1995.
