-
Materials In Paintings (MIP): An interdisciplinary dataset for perception, art history, and computer vision
Authors:
Mitchell J. P. van Zuijlen,
Hubert Lin,
Kavita Bala,
Sylvia C. Pont,
Maarten W. A. Wijntjes
Abstract:
A painter is free to modify how components of a natural scene are depicted, which can lead to a perceptually convincing image of the distal world. This signals a major difference between photos and paintings: paintings are explicitly created for human perception. Studying these painterly depictions could be beneficial to a multidisciplinary audience. In this paper, we capture and explore the paint…
▽ More
A painter is free to modify how components of a natural scene are depicted, which can lead to a perceptually convincing image of the distal world. This signals a major difference between photos and paintings: paintings are explicitly created for human perception. Studying these painterly depictions could be beneficial to a multidisciplinary audience. In this paper, we capture and explore the painterly depictions of materials to enable the study of depiction and perception of materials through the artists' eye. We annotated a dataset of 19k paintings with 200k+ bounding boxes from which polygon segments were automatically extracted. Each bounding box was assigned a coarse label (e.g., fabric) and a fine-grained label (e.g., velvety, silky). We demonstrate the cross-disciplinary utility of our dataset by presenting novel findings across art history, human perception, and computer vision. Our experiments include analyzing the distribution of materials depicted in paintings, showing how painters create convincing depictions using a stylized approach, and demonstrating how paintings can be used to build more robust computer vision models. We conclude that our dataset of painterly material depictions is a rich source for gaining insights into the depiction and perception of materials across multiple disciplines. The MIP dataset is freely accessible at https://materialsinpaintings.tudelft.nl
△ Less
Submitted 10 December, 2020; v1 submitted 5 December, 2020;
originally announced December 2020.
-
What Can Style Transfer and Paintings Do For Model Robustness?
Authors:
Hubert Lin,
Mitchell van Zuijlen,
Sylvia C. Pont,
Maarten W. A. Wijntjes,
Kavita Bala
Abstract:
A common strategy for improving model robustness is through data augmentations. Data augmentations encourage models to learn desired invariances, such as invariance to horizontal flip** or small changes in color. Recent work has shown that arbitrary style transfer can be used as a form of data augmentation to encourage invariance to textures by creating painting-like images from photographs. How…
▽ More
A common strategy for improving model robustness is through data augmentations. Data augmentations encourage models to learn desired invariances, such as invariance to horizontal flip** or small changes in color. Recent work has shown that arbitrary style transfer can be used as a form of data augmentation to encourage invariance to textures by creating painting-like images from photographs. However, a stylized photograph is not quite the same as an artist-created painting. Artists depict perceptually meaningful cues in paintings so that humans can recognize salient components in scenes, an emphasis which is not enforced in style transfer. Therefore, we study how style transfer and paintings differ in their impact on model robustness. First, we investigate the role of paintings as style images for stylization-based data augmentation. We find that style transfer functions well even without paintings as style images. Second, we show that learning from paintings as a form of perceptual data augmentation can improve model robustness. Finally, we investigate the invariances learned from stylization and from paintings, and show that models learn different invariances from these differing forms of data. Our results provide insights into how stylization improves model robustness, and provide evidence that artist-created paintings can be a valuable source of data for model robustness.
△ Less
Submitted 27 May, 2021; v1 submitted 29 November, 2020;
originally announced November 2020.
-
Insights From A Large-Scale Database of Material Depictions In Paintings
Authors:
Hubert Lin,
Mitchell Van Zuijlen,
Maarten W. A. Wijntjes,
Sylvia C. Pont,
Kavita Bala
Abstract:
Deep learning has paved the way for strong recognition systems which are often both trained on and applied to natural images. In this paper, we examine the give-and-take relationship between such visual recognition systems and the rich information available in the fine arts. First, we find that visual recognition systems designed for natural images can work surprisingly well on paintings. In parti…
▽ More
Deep learning has paved the way for strong recognition systems which are often both trained on and applied to natural images. In this paper, we examine the give-and-take relationship between such visual recognition systems and the rich information available in the fine arts. First, we find that visual recognition systems designed for natural images can work surprisingly well on paintings. In particular, we find that interactive segmentation tools can be used to cleanly annotate polygonal segments within paintings, a task which is time consuming to undertake by hand. We also find that FasterRCNN, a model which has been designed for object recognition in natural scenes, can be quickly repurposed for detection of materials in paintings. Second, we show that learning from paintings can be beneficial for neural networks that are intended to be used on natural images. We find that training on paintings instead of natural images can improve the quality of learned features and we further find that a large number of paintings can be a valuable source of test data for evaluating domain adaptation algorithms. Our experiments are based on a novel large-scale annotated database of material depictions in paintings which we detail in a separate manuscript.
△ Less
Submitted 24 November, 2020;
originally announced November 2020.
-
Sketch-and-test: picture-centered research with p5.js assisted crowdsourcing
Authors:
Maarten W. A. Wijntjes,
Mitchell van Zuijlen
Abstract:
Relating human judgements to pictures is central to a wide variety of scientific disciplines. Pictures are used to evoke and study faculties of the human mind, while human input is used to label, understand and model pictorial representations. Human input is often collected through online crowdsourcing experiments. This paper discusses the usage of crowdsourcing in two major branches of picture-ce…
▽ More
Relating human judgements to pictures is central to a wide variety of scientific disciplines. Pictures are used to evoke and study faculties of the human mind, while human input is used to label, understand and model pictorial representations. Human input is often collected through online crowdsourcing experiments. This paper discusses the usage of crowdsourcing in two major branches of picture-centered research, human and computer vision, and identifies novel directions such as art history and design. We demonstrate that a wide variety of experiments can be conducted by using p5.js, a library originally intended to facilitate visual creation. We report five complementary experimental paradigms to illustrated the accessibility and versatility of p5.js: Change blindness, BubbleView, 3D shape perception, Composition, and Perspective reconstruction. Results reveal that literature findings can be reproduced and novel insights can easily be achieved with the p5.js library. The creative freedom of p5.js combined with low threshold access to crowdsourcing seems like a powerful combination for all picture-centred research areas: perception, design, art history, communication, and beyond.
△ Less
Submitted 8 March, 2021; v1 submitted 17 April, 2020;
originally announced April 2020.
-
Tomaszewski's problem on randomly signed sums, revisited
Authors:
Ravi B. Boppana,
Harrie Hendriks,
Martien C. A. van Zuijlen
Abstract:
Let $v_1$, $v_2$, ..., $v_n$ be real numbers whose squares add up to 1. Consider the $2^n$ signed sums of the form $S = \sum \pm v_i$. Boppana and Holzman (2017) proved that at least 13/32 of these sums satisfy $|S| \le 1$. Here we improve their bound to $0.427685$.
Let $v_1$, $v_2$, ..., $v_n$ be real numbers whose squares add up to 1. Consider the $2^n$ signed sums of the form $S = \sum \pm v_i$. Boppana and Holzman (2017) proved that at least 13/32 of these sums satisfy $|S| \le 1$. Here we improve their bound to $0.427685$.
△ Less
Submitted 3 April, 2020; v1 submitted 13 March, 2020;
originally announced March 2020.
-
An improvement of the Boppana-Holzman bound for Rademacher random variables
Authors:
Harrie Hendriks,
Martien C. A. van Zuijlen
Abstract:
Let $v_1,v_2,...,v_n$ be real numbers whose squares add up to $1$. Consider the $2^n$ signed sums of the form $S=\sum_{i=1}^n \pm v_i.$ Holzman and Kleitman (1992) proved that at least $\frac38=0.375$ of these sums satisfy $|S|\leq 1.$ By using bounds for appropriate moments of $S,$ Boppana and Holzman (2017) were able to improve the bound to $\frac{13}{32}=0.40625$ and even a bit better to…
▽ More
Let $v_1,v_2,...,v_n$ be real numbers whose squares add up to $1$. Consider the $2^n$ signed sums of the form $S=\sum_{i=1}^n \pm v_i.$ Holzman and Kleitman (1992) proved that at least $\frac38=0.375$ of these sums satisfy $|S|\leq 1.$ By using bounds for appropriate moments of $S,$ Boppana and Holzman (2017) were able to improve the bound to $\frac{13}{32}=0.40625$ and even a bit better to $\frac{13}{32}+9\times10^{-6}.$ By following their approach, but using a key result of Bentkus and Dzindzalieta (2015), we will drastically improve (by more than 5\%) the latter barrier $\frac{13}{32}$ to $\frac{1}{2}-\frac{Φ(-2)}{4Φ(-\sqrt{2})}\approx 0.42768.$
△ Less
Submitted 28 January, 2021; v1 submitted 5 March, 2020;
originally announced March 2020.
-
Linear combinations of Rademacher random variables
Authors:
Harrie Hendriks,
Martien C. A. van Zuijlen
Abstract:
For a fixed unit vector $a=(a_1,a_2,\ldots,a_n)\in S^{n-1}$, we consider the $2^n$ sign vectors $\varepsilon=(\varepsilon^1,\varepsilon^2,\ldots,\varepsilon^n)\in \{+1,-1\}^n$ and the corresponding scalar products $\varepsilon\cdot a=\sum_{i=1}^n \varepsilon^ia_i$. In this paper we will solve for $n=1,2,\ldots,9$ an old conjecture stating that of the $2^n$ sums of the form $\sum\pm a_i$ it is impo…
▽ More
For a fixed unit vector $a=(a_1,a_2,\ldots,a_n)\in S^{n-1}$, we consider the $2^n$ sign vectors $\varepsilon=(\varepsilon^1,\varepsilon^2,\ldots,\varepsilon^n)\in \{+1,-1\}^n$ and the corresponding scalar products $\varepsilon\cdot a=\sum_{i=1}^n \varepsilon^ia_i$. In this paper we will solve for $n=1,2,\ldots,9$ an old conjecture stating that of the $2^n$ sums of the form $\sum\pm a_i$ it is impossible that there are more with $|\sum_{i=1}^n \pm a_i|>1$ than there are with $|\sum_{i=1}^n \pm a_i|\leq1$. Although the problem has been solved completely in case the $a_i$'s are equal, the more general problem with possible non-equal $a_i$'s remains open for values of $n\geq 10$. The present method can also be used for $n\geq 10$, but unfortunately the technical difficulties seem to grow exponentially with $n$ and no "induction type of argument" has been found. The conjecture has an appealing reformulation in probability theory and in geometry. In probability theory the results lead to upper bounds which are much better than for instance Chebyshevnequalities.
△ Less
Submitted 21 March, 2017;
originally announced March 2017.
-
Joint ML estimation of all parameters in a discrete time random field HJM type interest rate model
Authors:
József Gáll,
Gyula Pap,
Martien van Zuijlen
Abstract:
We consider discrete time Heath-Jarrow-Morton type interest rate models, where the interest rate curves are driven by a geometric spatial autoregression field. Strong consistency and asymptotic normality of the maximum likelihood estimators of the parameters are proved for stable no-arbitrage models containing a general stochastic discounting factor, where explicit form of the ML estimators is not…
▽ More
We consider discrete time Heath-Jarrow-Morton type interest rate models, where the interest rate curves are driven by a geometric spatial autoregression field. Strong consistency and asymptotic normality of the maximum likelihood estimators of the parameters are proved for stable no-arbitrage models containing a general stochastic discounting factor, where explicit form of the ML estimators is not available given a non-i.i.d. sample. The results form the basis of further statistical problems in such models.
△ Less
Submitted 14 January, 2014;
originally announced January 2014.
-
Sharp upper bounds for the deviations from the mean of the sum of independent Rademacher random variables
Authors:
Harrie Hendriks,
Martien C. A. van Zuijlen
Abstract:
For a fixed unit vector a=(a_1,a_2,...,a_n) in S^{n-1}, i.e. sum_{i=1}^n a_i^2=1, we consider the 2^n sign vectors epsilon=(epsilon_1,epsilon_2,...,epsilon_n) in {-1,1}^n and the corresponding scalar products a.epsilon=sum_{i=1}^n a_i epsilon_i. Holtzman and Kleitman formulated the following conjecture. It states that among the 2^n sums of the form sum +/- a_i there are not more with |sum_{i=1}^n…
▽ More
For a fixed unit vector a=(a_1,a_2,...,a_n) in S^{n-1}, i.e. sum_{i=1}^n a_i^2=1, we consider the 2^n sign vectors epsilon=(epsilon_1,epsilon_2,...,epsilon_n) in {-1,1}^n and the corresponding scalar products a.epsilon=sum_{i=1}^n a_i epsilon_i. Holtzman and Kleitman formulated the following conjecture. It states that among the 2^n sums of the form sum +/- a_i there are not more with |sum_{i=1}^n +/- a_i|>1 than there are with |sum_{i=1}^n +/- a_i| <= 1. The result is of interest in itself, but has also an appealing reformulation in probability theory and in geometry. In this paper we will solve an extension of this problem in the uniform case where all the a's are equal. More precisely, for S_n being a sum of n independent Rademacher random variables, we will give, for several values of xi, precise lower bounds for the probabilities P_n:=P{-xi sqrt{n} <= S_n <= xi sqrt{n}}. There is an obvious relationship with the binomial distribution with parameters n and p=1/2.
The obtained lower bounds are sharp and much better than for instance the bound that can be obtained from application of the Chebishev inequality. In case xi=1 Van Zuijlen solved this problem. We remark that our bound will have nice applications in probability theory and especially in random walk theory.
△ Less
Submitted 3 October, 2012;
originally announced October 2012.
-
On a conjecture concerning the sum of independent Rademacher random variables
Authors:
Martien C. A. van Zuijlen
Abstract:
It is shown that at least 50% of the probability mass of a sum of independent Rademacher random variables is within one standard deviation from its mean. This lower bound is sharp, it is much better than for instance the bound that can be obtained from application of the Chebishev inequality and the bound will have nice applications in finite sampling theory and in random walk theory. This old con…
▽ More
It is shown that at least 50% of the probability mass of a sum of independent Rademacher random variables is within one standard deviation from its mean. This lower bound is sharp, it is much better than for instance the bound that can be obtained from application of the Chebishev inequality and the bound will have nice applications in finite sampling theory and in random walk theory. This old conjecture is of interest in itself, but has also an appealing reformulation in probability theory and in geometry.
△ Less
Submitted 21 December, 2011;
originally announced December 2011.
