Skip to main content

Showing 1–26 of 26 results for author: Sannai, A

.
  1. arXiv:2402.02362  [pdf, other

    cs.LG cs.AI hep-th physics.comp-ph

    Unification of Symmetries Inside Neural Networks: Transformer, Feedforward and Neural ODE

    Authors: Koji Hashimoto, Yuji Hirono, Akiyoshi Sannai

    Abstract: Understanding the inner workings of neural networks, including transformers, remains one of the most challenging puzzles in machine learning. This study introduces a novel approach by applying the principles of gauge symmetries, a key concept in physics, to neural network architectures. By regarding model functions as physical observables, we find that parametric redundancies of various machine le… ▽ More

    Submitted 4 February, 2024; originally announced February 2024.

    Comments: 11 pages, 3 figures

    Report number: KUNS-2992

  2. arXiv:2402.01454  [pdf, other

    cs.LG cs.AI stat.ME stat.ML

    Integrating Large Language Models in Causal Discovery: A Statistical Causal Approach

    Authors: Masayuki Takayama, Tadahisa Okuda, Thong Pham, Tatsuyoshi Ikenoue, Shingo Fukuma, Shohei Shimizu, Akiyoshi Sannai

    Abstract: In practical statistical causal discovery (SCD), embedding domain expert knowledge as constraints into the algorithm is significant for creating consistent meaningful causal models, despite the challenges in systematic acquisition of the background knowledge. To overcome these challenges, this paper proposes a novel methodology for causal inference, in which SCD methods and knowledge based causal… ▽ More

    Submitted 21 May, 2024; v1 submitted 2 February, 2024; originally announced February 2024.

  3. arXiv:2401.17780  [pdf, other

    cs.LG

    A Policy Gradient Primal-Dual Algorithm for Constrained MDPs with Uniform PAC Guarantees

    Authors: Toshinori Kitamura, Tadashi Kozuno, Masahiro Kato, Yuki Ichihara, Soichiro Nishimori, Akiyoshi Sannai, Sho Sonoda, Wataru Kumagai, Yutaka Matsuo

    Abstract: We study a primal-dual (PD) reinforcement learning (RL) algorithm for online constrained Markov decision processes (CMDPs). Despite its widespread practical use, the existing theoretical literature on PD-RL algorithms for this problem only provides sublinear regret guarantees and fails to ensure convergence to optimal policies. In this paper, we introduce a novel policy gradient PD algorithm with… ▽ More

    Submitted 1 July, 2024; v1 submitted 31 January, 2024; originally announced January 2024.

  4. arXiv:2309.13078  [pdf, other

    cs.AI cs.LG cs.PL

    LPML: LLM-Prompting Markup Language for Mathematical Reasoning

    Authors: Ryutaro Yamauchi, Sho Sonoda, Akiyoshi Sannai, Wataru Kumagai

    Abstract: In utilizing large language models (LLMs) for mathematical reasoning, addressing the errors in the reasoning and calculation present in the generated text by LLMs is a crucial challenge. In this paper, we propose a novel framework that integrates the Chain-of-Thought (CoT) method with an external tool (Python REPL). We discovered that by prompting LLMs to generate structured text in XML-like marku… ▽ More

    Submitted 11 October, 2023; v1 submitted 20 September, 2023; originally announced September 2023.

  5. arXiv:2205.11099  [pdf, other

    math.OC cs.LG

    Bézier Flow: a Surface-wise Gradient Descent Method for Multi-objective Optimization

    Authors: Akiyoshi Sannai, Yasunari Hikima, Ken Kobayashi, Akinori Tanaka, Naoki Hamada

    Abstract: In this paper, we propose a strategy to construct a multi-objective optimization algorithm from a single-objective optimization algorithm by using the Bézier simplex model. Also, we extend the stability of optimization algorithms in the sense of Probability Approximately Correct (PAC) learning and define the PAC stability. We prove that it leads to an upper bound on the generalization with high pr… ▽ More

    Submitted 23 May, 2022; originally announced May 2022.

  6. arXiv:2110.08092  [pdf, other

    cs.LG

    Equivariant and Invariant Reynolds Networks

    Authors: Akiyoshi Sannai, Makoto Kawano, Wataru Kumagai

    Abstract: Invariant and equivariant networks are useful in learning data with symmetry, including images, sets, point clouds, and graphs. In this paper, we consider invariant and equivariant networks for symmetries of finite groups. Invariant and equivariant networks have been constructed by various researchers using Reynolds operators. However, Reynolds operators are computationally expensive when the orde… ▽ More

    Submitted 15 October, 2021; originally announced October 2021.

    Comments: 15 pages, 4 figures

  7. arXiv:2104.04679  [pdf, other

    cs.LG stat.ML

    Approximate Bayesian Computation of Bézier Simplices

    Authors: Akinori Tanaka, Akiyoshi Sannai, Ken Kobayashi, Naoki Hamada

    Abstract: Bézier simplex fitting algorithms have been recently proposed to approximate the Pareto set/front of multi-objective continuous optimization problems. These new methods have shown to be successful at approximating various shapes of Pareto sets/fronts when sample points exactly lie on the Pareto set/front. However, if the sample points scatter away from the Pareto set/front, those methods often lik… ▽ More

    Submitted 12 April, 2021; v1 submitted 10 April, 2021; originally announced April 2021.

    Report number: RIKEN-iTHEMS-Report-21

  8. arXiv:2102.08759  [pdf, other

    cs.LG stat.ML

    Group Equivariant Conditional Neural Processes

    Authors: Makoto Kawano, Wataru Kumagai, Akiyoshi Sannai, Yusuke Iwasawa, Yutaka Matsuo

    Abstract: We present the group equivariant conditional neural process (EquivCNP), a meta-learning method with permutation invariance in a data set as in conventional conditional neural processes (CNPs), and it also has transformation equivariance in data space. Incorporating group equivariance, such as rotation and scaling equivariance, provides a way to consider the symmetry of real-world data. We give a d… ▽ More

    Submitted 17 February, 2021; originally announced February 2021.

  9. arXiv:2012.13882  [pdf, ps, other

    stat.ML cs.LG

    Universal Approximation Theorem for Equivariant Maps by Group CNNs

    Authors: Wataru Kumagai, Akiyoshi Sannai

    Abstract: Group symmetry is inherent in a wide variety of data distributions. Data processing that preserves symmetry is described as an equivariant map and often effective in achieving high performance. Convolutional neural networks (CNNs) have been known as models with equivariance and shown to approximate equivariant maps for some specific groups. However, universal approximation theorems for CNNs have b… ▽ More

    Submitted 27 December, 2020; originally announced December 2020.

  10. arXiv:2010.12125  [pdf, other

    cs.LG stat.ML

    On the Number of Linear Functions Composing Deep Neural Network: Towards a Refined Definition of Neural Networks Complexity

    Authors: Yuuki Takai, Akiyoshi Sannai, Matthieu Cordonnier

    Abstract: The classical approach to measure the expressive power of deep neural networks with piecewise linear activations is based on counting their maximum number of linear regions. This complexity measure is quite relevant to understand general properties of the expressivity of neural networks such as the benefit of depth over width. Nevertheless, it appears limited when it comes to comparing the express… ▽ More

    Submitted 25 February, 2021; v1 submitted 22 October, 2020; originally announced October 2020.

    Comments: 16 pages

  11. arXiv:1910.06552  [pdf, other

    stat.ML cs.LG

    Improved Generalization Bounds of Group Invariant / Equivariant Deep Networks via Quotient Feature Spaces

    Authors: Akiyoshi Sannai, Masaaki Imaizumi, Makoto Kawano

    Abstract: Numerous invariant (or equivariant) neural networks have succeeded in handling invariant data such as point clouds and graphs. However, a generalization theory for the neural networks has not been well developed, because several essential factors for the theory, such as network size and margin distribution, are not deeply connected to the invariance and equivariance. In this study, we develop a no… ▽ More

    Submitted 19 June, 2021; v1 submitted 15 October, 2019; originally announced October 2019.

    Comments: Old title: "Improved Generalization Bound of Permutation Invariant Deep Neural Networks"

  12. arXiv:1906.06924  [pdf, other

    cs.LG stat.ML

    Asymptotic Risk of Bezier Simplex Fitting

    Authors: Akinori Tanaka, Akiyoshi Sannai, Ken Kobayashi, Naoki Hamada

    Abstract: The Bezier simplex fitting is a novel data modeling technique which exploits geometric structures of data to approximate the Pareto front of multi-objective optimization problems. There are two fitting methods based on different sampling strategies. The inductive skeleton fitting employs a stratified subsampling from each skeleton of a simplex, whereas the all-at-once fitting uses a non-stratified… ▽ More

    Submitted 17 June, 2019; originally announced June 2019.

  13. arXiv:1903.01939  [pdf, ps, other

    cs.LG stat.ML

    Universal approximations of permutation invariant/equivariant functions by deep neural networks

    Authors: Akiyoshi Sannai, Yuuki Takai, Matthieu Cordonnier

    Abstract: In this paper, we develop a theory about the relationship between $G$-invariant/equivariant functions and deep neural networks for finite group $G$. Especially, for a given $G$-invariant/equivariant function, we construct its universal approximator by deep neural network whose layers equip $G$-actions and each affine transformations are $G$-equivariant/invariant. Due to representation theory, we c… ▽ More

    Submitted 26 September, 2019; v1 submitted 5 March, 2019; originally announced March 2019.

  14. arXiv:1812.05222  [pdf, other

    math.OC

    Bezier Simplex Fitting: Describing Pareto Fronts of Simplicial Problems with Small Samples in Multi-objective Optimization

    Authors: Ken Kobayashi, Naoki Hamada, Akiyoshi Sannai, Akinori Tanaka, Kenichi Bannai, Masashi Sugiyama

    Abstract: Multi-objective optimization problems require simultaneously optimizing two or more objective functions. Many studies have reported that the solution set of an M-objective optimization problem often forms an (M-1)-dimensional topological simplex (a curved line for M=2, a curved triangle for M=3, a curved tetrahedron for M=4, etc.). Since the dimensionality of the solution set increases as the numb… ▽ More

    Submitted 12 December, 2018; originally announced December 2018.

    Comments: To appear in AAAI 2019

  15. arXiv:1805.07337  [pdf, other

    stat.ML cs.CR cs.LG

    Reconstruction of training samples from loss functions

    Authors: Akiyoshi Sannai

    Abstract: This paper presents a new mathematical framework to analyze the loss functions of deep neural networks with ReLU functions. Furthermore, as as application of this theory, we prove that the loss functions can reconstruct the inputs of the training samples up to scalar multiplication (as vectors) and can provide the number of layers and nodes of the deep neural network. Namely, if we have all input… ▽ More

    Submitted 18 May, 2018; originally announced May 2018.

    Comments: 11 pages, 3 figures

  16. Infinitely generated symbolic Rees algebras over finite fields

    Authors: Akiyoshi Sannai, Hiromu Tanaka

    Abstract: For the polynomial ring over an arbitrary field with twelve variables, there exists a prime ideal whose symbolic Rees algebra is not finitely generated.

    Submitted 13 June, 2019; v1 submitted 27 March, 2017; originally announced March 2017.

    Comments: 16 pages, v2: minor revisions, v3: minor revisions

    Journal ref: Alg. Number Th. 13 (2019) 1879-1891

  17. arXiv:1702.04209  [pdf, ps, other

    math.AG

    A characterization of ordinary abelian varieties by the Frobenius push-forward of the structure sheaf II

    Authors: Sho Ejiri, Akiyoshi Sannai

    Abstract: In this paper, we prove that a smooth projective variety $X$ of characteristic $p>0$ is an ordinary abelian variety if and only if $K_X$ is pseudo-effective and $F^e_*\mathcal O_X$ splits into a direct sum of line bundles for an integer $e$ with $p^e>2$.

    Submitted 29 August, 2017; v1 submitted 14 February, 2017; originally announced February 2017.

    Comments: 10 pages, v2: Abstract and Sections 3-5 shortened, Introduction revised, the statements of Theorem 1.3, Proposition 3.2 and Lemma 5.3 simplified, Corollary 1.4 moved to Remark 5.6, typos corrected

  18. arXiv:1411.5294  [pdf, ps, other

    math.AG

    A characterization of ordinary abelian varieties by the Frobenius push-forward of the structure sheaf

    Authors: Akiyoshi Sannai, Hiromu Tanaka

    Abstract: For an ordinary abelian variety $X$, $F^e_*\mathcal{O}_X$ is decomposed into line bundles for every positive integer $e$. Conversely, if a smooth projective variety $X$ satisfies this property and its Kodaira dimension is non-negative, then $X$ is an ordinary abelian variety.

    Submitted 12 January, 2016; v1 submitted 19 November, 2014; originally announced November 2014.

    Comments: 22 pages; v2:we fixed the proofs of 4.10 and 5.1; v3:we shortened Subsection 2.1 and added Section 6; v4:we gave many changes, v5:the final version

  19. arXiv:1304.3784  [pdf, ps, other

    math.AG math.AC math.AT math.CT math.KT

    Homotopy invariance of higher K-theory for abelian categories

    Authors: Satoshi Mochizuki, Akiyoshi Sannai

    Abstract: The main theorem in this paper is that the base change functor from a noetherian abelian category to its noetherian polynomial category induces an isomorphism on K-theory. The main theorem implies the well-known fact that A^1-homotopy invariance of K'-theory for noetherian schemes.

    Submitted 13 December, 2014; v1 submitted 13 April, 2013; originally announced April 2013.

    Comments: arXiv admin note: substantial text overlap with arXiv:1104.4240

  20. arXiv:1301.2381  [pdf, ps, other

    math.AC math.AG

    Dual F-signature

    Authors: Akiyoshi Sannai

    Abstract: We define the dual F-signature of modules, which is equivalent to the F-signature if the module is the base ring. By using this invariant, We give characterizations of regular, F-regular, F-rational, and Gorenstein singularities.

    Submitted 29 June, 2013; v1 submitted 10 January, 2013; originally announced January 2013.

    Comments: Typos corrected and other minor changes. To appear in International Mathematics Research Notices

  21. arXiv:1201.1133  [pdf, ps, other

    math.AG math.AC

    Characterization of varieties of Fano type via singularities of Cox rings

    Authors: Yoshinori Gongyo, Shinnosuke Okawa, Akiyoshi Sannai, Shunsuke Takagi

    Abstract: We show that every Mori dream space of globally $F$-regular type is of Fano type. As an application, we give a characterization of varieties of Fano type in terms of the singularities of their Cox rings.

    Submitted 5 January, 2012; originally announced January 2012.

    Comments: 22 pages

    MSC Class: 14J45 (Primary) 13A35; 14B05; 14E30 (Secondary)

  22. arXiv:1109.5321  [pdf, ps, other

    math.AG math.AC

    Jet schemes of homogeneous hypersurfaces

    Authors: Shihoko Ishii, Akiyoshi Sannai, Kei-ichi Watanabe

    Abstract: This paper studies the singularities of jet schemes of homogeneous hypersurfaces of general type. We obtain the condition of the degree and the dimension for the singularities of the jet schemes to be of dense $F$-regular type. This provides us with examples of singular varieties whose $m$-jet schemes have rational singularities for every $m$.

    Submitted 24 September, 2011; originally announced September 2011.

    Comments: 11 pages, to appear in the Proceedings of fifth Franco-Japanese Conference of Singularities

    MSC Class: 14B05; 14E18

  23. arXiv:1104.4242  [pdf, ps, other

    math.AC

    Generalized Koszul resolutions

    Authors: Satoshi Mochizuki, Akiyoshi Sannai

    Abstract: The main objective of this paper is to generalize a notion of Koszul resolutions and charcterizing modules which admits such a resolution. We turn out that for a noetherian ring $A$ and a coherent $A$ module $M$, $M$ has a two dimensional generalized Koszul resolution if and only if $M$ is a pure weight two module in the sense of \cite{HM09}.

    Submitted 21 April, 2011; originally announced April 2011.

    Comments: 14 pages

  24. arXiv:1104.4240  [pdf, ps, other

    math.AC math.KT

    Higher K-theory of polynomial categories

    Authors: Satoshi Mochizuki, Akiyoshi Sannai

    Abstract: The main theorem in this paper is that the base change functor from an abelian category $\cA$ to its polynomial category in the sense of Schlichting $-\otimes_{\cA}\bbZ[t]:\cA \to \cA[t]$ induces an isomorphism on their $K$-theories if $\cA$ is noetherian and has enough projective objects. The main theorem implies the well-known fact that $\mathbb{A}^1$-homotopy invariance of $K'$-theory for noeth… ▽ More

    Submitted 21 April, 2011; originally announced April 2011.

    Comments: 13 pages

  25. arXiv:1104.4236  [pdf, ps, other

    math.AC

    F-signature of graded Gorenstein rings

    Authors: Akiyoshi Sannai, Kei-ichi Watanabe

    Abstract: For a commutative ring $R$, the $F$-signature was defined by Huneke and Leuschke \cite{H-L}. It is an invariant that measures the order of the rank of the free direct summand of $R^{(e)}$. Here, $R^{(e)}$ is $R$ itself, regarded as an $R$-module through $e$-times Frobenius action $F^e$.In this paper, we show a connection of the F-signature of a graded ring with other invariants. More precisely, fo… ▽ More

    Submitted 21 April, 2011; originally announced April 2011.

    Comments: 8 pages

  26. arXiv:1104.0413  [pdf, ps, other

    math.AC

    Galois extensions, plus closure, and maps on local cohomology

    Authors: Akiyoshi Sannai, Anurag K. Singh

    Abstract: Given a local domain $(R,m)$ of prime characteristic that is a homomorphic image of a Gorenstein ring, Huneke and Lyubeznik proved that there exists a module-finite extension domain $S$ such that the induced map on local cohomology modules $H^i_m(R)\to H^i_m(S)$ is zero for each $i<\dim R$. We prove that the extension $S$ may be chosen to be generically Galois, and analyze the Galois groups that a… ▽ More

    Submitted 3 April, 2011; originally announced April 2011.

    MSC Class: Primary 13D45; Secondary 13A35; 14B15; 14F17

    Journal ref: Advances in Mathematics 229 (2012) 1847-1861