Skip to main content

Showing 1–23 of 23 results for author: Barbero, F

Searching in archive cs. Search in all archives.
.
  1. arXiv:2406.04267  [pdf, other

    cs.CL cs.LG

    Transformers need glasses! Information over-squashing in language tasks

    Authors: Federico Barbero, Andrea Banino, Steven Kapturowski, Dharshan Kumaran, João G. M. Araújo, Alex Vitvitskyi, Razvan Pascanu, Petar Veličković

    Abstract: We study how information propagates in decoder-only Transformers, which are the architectural backbone of most existing frontier large language models (LLMs). We rely on a theoretical signal propagation analysis -- specifically, we analyse the representations of the last token in the final layer of the Transformer, as this is the representation used for next-token prediction. Our analysis reveals… ▽ More

    Submitted 6 June, 2024; originally announced June 2024.

  2. arXiv:2405.15540  [pdf, other

    cs.LG

    Bundle Neural Networks for message diffusion on graphs

    Authors: Jacob Bamberger, Federico Barbero, Xiaowen Dong, Michael Bronstein

    Abstract: The dominant paradigm for learning on graph-structured data is message passing. Despite being a strong inductive bias, the local message passing mechanism suffers from pathological issues such as over-smoothing, over-squashing, and limited node-level expressivity. To address these limitations we propose Bundle Neural Networks (BuNN), a new type of GNN that operates via message diffusion over flat… ▽ More

    Submitted 24 May, 2024; originally announced May 2024.

  3. arXiv:2312.07223  [pdf, ps, other

    cs.LO math.LO math.ST

    On the logic of interventionist counterfactuals under indeterministic causal laws

    Authors: Fausto Barbero

    Abstract: We investigate the generalization of causal models to the case of indeterministic causal laws that was suggested in Halpern (2000). We give an overview of what differences in modeling are enforced by this more general perspective, and propose an implementation of generalized models in the style of the causal team semantics of Barbero & Sandu (2020). In these models, the laws are not represented by… ▽ More

    Submitted 15 April, 2024; v1 submitted 12 December, 2023; originally announced December 2023.

    Comments: 24 pages. Extended version of the paper published in the conference proceedings of FoIKS 2024. Substantial changes in the basic definitions

    MSC Class: 62D20 (Primary); 03A10; 03B70 (Secondary) ACM Class: F.4.3; I.2.4

  4. arXiv:2310.01668  [pdf, other

    cs.LG

    Locality-Aware Graph-Rewiring in GNNs

    Authors: Federico Barbero, Ameya Velingker, Amin Saberi, Michael Bronstein, Francesco Di Giovanni

    Abstract: Graph Neural Networks (GNNs) are popular models for machine learning on graphs that typically follow the message-passing paradigm, whereby the feature of a node is updated recursively upon aggregating information over its neighbors. While exchanging messages over the input graph endows GNNs with a strong inductive bias, it can also make GNNs susceptible to over-squashing, thereby preventing them f… ▽ More

    Submitted 4 May, 2024; v1 submitted 2 October, 2023; originally announced October 2023.

  5. arXiv:2309.12764  [pdf, other

    cs.SI

    Multi-Modal Embeddings for Isolating Cross-Platform Coordinated Information Campaigns on Social Media

    Authors: Fabio Barbero, Sander op den Camp, Kristian van Kuijk, Carlos Soto García-Delgado, Gerasimos Spanakis, Adriana Iamnitchi

    Abstract: Coordinated multi-platform information operations are implemented in a variety of contexts on social media, including state-run disinformation campaigns, marketing strategies, and social activism. Characterized by the promotion of messages via multi-platform coordination, in which multiple user accounts, within a short time, post content advancing a shared informational agenda on multiple platform… ▽ More

    Submitted 22 September, 2023; originally announced September 2023.

    Comments: To appear in the 5th Multidisciplinary International Symposium on Disinformation in Open Online Media (MISDOOM 2023)

    ACM Class: H.3.5; H.3.1

  6. arXiv:2305.02613  [pdf, ps, other

    cs.LO

    Multiteam semantics for interventionist counterfactuals: probabilities and causation

    Authors: Fausto Barbero, Gabriel Sandu

    Abstract: In [4], we introduced an extension of team semantics (causal teams) which assigns an interpretation to interventionist counterfactuals and causal notions based on them (as e.g. in Pearl's and Woodward's manipulationist approaches to causation). We now present a further extension of this framework (causal multiteams) which allows us to talk about probabilistic causal statements. We analyze the expr… ▽ More

    Submitted 22 May, 2023; v1 submitted 4 May, 2023; originally announced May 2023.

    Comments: Minor corrections

    MSC Class: 03B48 ACM Class: F.4.3; G.3

  7. arXiv:2304.02964  [pdf, ps, other

    cs.LO

    Strongly complete axiomatization for a logic with probabilistic interventionist counterfactuals

    Authors: Fausto Barbero, Jonni Virtema

    Abstract: Causal multiteam semantics is a framework where probabilistic notions and causal inference can be studied in a unified setting. We study a logic (PCO) that features marginal probabilities and interventionist counterfactuals, and allows expressing conditional probability statements, do expressions and other mixtures of causal and probabilistic reasoning. Our main contribution is a strongly complete… ▽ More

    Submitted 6 April, 2023; originally announced April 2023.

    Comments: 15 pages

    MSC Class: 03B48; 03F99 ACM Class: F.4; G.3

  8. arXiv:2303.11993  [pdf, ps, other

    cs.LO math.LO

    Expressivity Landscape for Logics with Probabilistic Interventionist Counterfactuals

    Authors: Fausto Barbero, Jonni Virtema

    Abstract: Causal multiteam semantics is a framework where probabilistic dependencies arising from data and causation between variables can be together formalized and studied logically. We consider several logics in the setting of causal multiteam semantics that can express probability comparisons concerning formulae and constants, and encompass interventionist counterfactuals and selective implications that… ▽ More

    Submitted 21 March, 2023; originally announced March 2023.

  9. arXiv:2302.02941  [pdf, other

    cs.LG cs.AI cs.DM stat.ML

    On Over-Squashing in Message Passing Neural Networks: The Impact of Width, Depth, and Topology

    Authors: Francesco Di Giovanni, Lorenzo Giusti, Federico Barbero, Giulia Luise, Pietro Lio', Michael Bronstein

    Abstract: Message Passing Neural Networks (MPNNs) are instances of Graph Neural Networks that leverage the graph to send messages over the edges. This inductive bias leads to a phenomenon known as over-squashing, where a node feature is insensitive to information contained at distant nodes. Despite recent methods introduced to mitigate this issue, an understanding of the causes for over-squashing and of pos… ▽ More

    Submitted 24 May, 2023; v1 submitted 6 February, 2023; originally announced February 2023.

    Comments: Accepted at ICML 2023; 21 pages

  10. arXiv:2211.16199  [pdf, other

    cs.LG

    Latent Graph Inference using Product Manifolds

    Authors: Haitz Sáez de Ocáriz Borde, Anees Kazi, Federico Barbero, Pietro Liò

    Abstract: Graph Neural Networks usually rely on the assumption that the graph topology is available to the network as well as optimal for the downstream task. Latent graph inference allows models to dynamically learn the intrinsic graph structure of problems where the connectivity patterns of data may not be directly accessible. In this work, we generalize the discrete Differentiable Graph Module (dDGM) for… ▽ More

    Submitted 27 June, 2023; v1 submitted 26 November, 2022; originally announced November 2022.

  11. arXiv:2209.13410  [pdf, other

    cs.LG q-bio.BM q-bio.QM

    Graph Neural Network Expressivity and Meta-Learning for Molecular Property Regression

    Authors: Haitz Sáez de Ocáriz Borde, Federico Barbero

    Abstract: We demonstrate the applicability of model-agnostic algorithms for meta-learning, specifically Reptile, to GNN models in molecular regression tasks. Using meta-learning we are able to learn new chemical prediction tasks with only a few model updates, as compared to using randomly initialized GNNs which require learning each regression task from scratch. We experimentally show that GNN layer express… ▽ More

    Submitted 24 November, 2022; v1 submitted 24 September, 2022; originally announced September 2022.

  12. arXiv:2206.08702  [pdf, other

    cs.LG math.AT math.DG

    Sheaf Neural Networks with Connection Laplacians

    Authors: Federico Barbero, Cristian Bodnar, Haitz Sáez de Ocáriz Borde, Michael Bronstein, Petar Veličković, Pietro Liò

    Abstract: A Sheaf Neural Network (SNN) is a type of Graph Neural Network (GNN) that operates on a sheaf, an object that equips a graph with vector spaces over its nodes and edges and linear maps between these spaces. SNNs have been shown to have useful theoretical properties that help tackle issues arising from heterophily and over-smoothing. One complication intrinsic to these models is finding a good shea… ▽ More

    Submitted 17 June, 2022; originally announced June 2022.

    Comments: Presented at the ICML 2022 Workshop on Topology, Algebra, and Geometry in Machine Learning

  13. arXiv:2111.12978  [pdf, ps, other

    cs.AI cs.LO

    Observing Interventions: A logic for thinking about experiments

    Authors: Fausto Barbero, Katrin Schulz, Fernando R. Velázquez-Quesada, Kaibo Xie

    Abstract: This paper makes a first step towards a logic of learning from experiments. For this, we investigate formal frameworks for modeling the interaction of causal and (qualitative) epistemic reasoning. Crucial for our approach is the idea that the notion of an intervention can be used as a formal expression of a (real or hypothetical) experiment. In a first step we extend the well-known causal models w… ▽ More

    Submitted 1 December, 2021; v1 submitted 25 November, 2021; originally announced November 2021.

    Comments: This is the extended version of a paper that will appear in a special issue of the Journal of Logic and Computation dedicated to the 3rd DaL{í} Workshop on Dynamic Logic: New Trends and Applications. Different from the journal version, here the reader can find the full technical appendix

  14. arXiv:2010.16217  [pdf, ps, other

    cs.AI cs.LO

    Thinking About Causation: A Causal Language with Epistemic Operators

    Authors: Fausto Barbero, Katrin Schulz, Sonja Smets, Fernando R. Velázquez-Quesada, Kaibo Xie

    Abstract: This paper proposes a formal framework for modeling the interaction of causal and (qualitative) epistemic reasoning. To this purpose, we extend the notion of a causal model with a representation of the epistemic state of an agent. On the side of the object language, we add operators to express knowledge and the act of observing new information. We provide a sound and complete axiomatization of the… ▽ More

    Submitted 30 October, 2020; originally announced October 2020.

    Comments: This is the long version of a paper that is to be published in the post-proceedings of the 3rd Dali Workshop on Dynamic Logic: New Trends and Applications. The post-proceedings will be published by Springer as a Lecture Notes in Computer Science volume

  15. arXiv:2010.03856  [pdf, other

    cs.CR

    Transcending Transcend: Revisiting Malware Classification in the Presence of Concept Drift

    Authors: Federico Barbero, Feargus Pendlebury, Fabio Pierazzi, Lorenzo Cavallaro

    Abstract: Machine learning for malware classification shows encouraging results, but real deployments suffer from performance degradation as malware authors adapt their techniques to evade detection. This phenomenon, known as concept drift, occurs as new malware examples evolve and become less and less like the original training examples. One promising method to cope with concept drift is classification wit… ▽ More

    Submitted 8 January, 2024; v1 submitted 8 October, 2020; originally announced October 2020.

    Comments: Version accepted at IEEE Symposium on Security & Privacy (Oakland), 2022. Errata Corrige to the published version: https://s2lab.cs.ucl.ac.uk/downloads/transcending-errata_corrige.pdf. Project Website: https://s2lab.cs.ucl.ac.uk/projects/transcend/

  16. Interventionist Counterfactuals on Causal Teams

    Authors: Fausto Barbero, Gabriel Sandu

    Abstract: We introduce an extension of team semantics which provides a framework for the logic of manipulationist theories of causation based on structural equation models, such as Woodward's and Pearl's; our causal teams incorporate (partial or total) information about functional dependencies that are invariant under interventions. We give a unified treatment of observational and causal aspects of causal m… ▽ More

    Submitted 2 January, 2019; originally announced January 2019.

    Comments: In Proceedings CREST 2018, arXiv:1901.00073

    Journal ref: EPTCS 286, 2019, pp. 16-30

  17. arXiv:1810.03868  [pdf, other

    cs.CC

    On the Distance Identifying Set meta-problem and applications to the complexity of identifying problems on graphs

    Authors: Florian Barbero, Lucas Isenmann, Jocelyn Thiebaut

    Abstract: Numerous problems consisting in identifying vertices in graphs using distances are useful in domains such as network verification and graph isomorphism. Unifying them into a meta-problem may be of main interest. We introduce here a promising solution named Distance Identifying Set. The model contains Identifying Code (IC), Locating Dominating Set (LD) and their generalizations $r$-IC and $r$-LD wh… ▽ More

    Submitted 9 October, 2018; originally announced October 2018.

  18. arXiv:1803.02643  [pdf, ps, other

    cs.DM

    Quasiperiods of biinfinite Sturmian words

    Authors: Florian Barbero, Guilhem Gamard, Anaël Grandjean

    Abstract: We study the notion of quasiperiodicity, in the sense of "coverability", for biinfinite words. All previous work about quasiperiodicity focused on right infinite words, but the passage to the biinfinite case could help to prove stronger results about quasiperiods of Sturmian words. We demonstrate this by showing that all biinfinite Sturmian words have infinitely many quasiperiods, which is not qui… ▽ More

    Submitted 7 March, 2018; originally announced March 2018.

  19. arXiv:1707.03563  [pdf, other

    cs.DM math.CO

    Strong immersion is a well-quasi-ordering for semi-complete digraphs

    Authors: Florian Barbero, Christophe Paul, Michal Pilipczuk

    Abstract: We prove that the strong immersion order is a well-quasi-ordering on the class of semi-complete digraphs, thereby strengthening a result of Chudnovsky and Seymour that this holds for the class of tournaments.

    Submitted 12 July, 2017; originally announced July 2017.

  20. arXiv:1706.00617  [pdf, ps, other

    cs.DS cs.CC cs.DM

    Exploring the complexity of layout parameters in tournaments and semi-complete digraphs

    Authors: Florian Barbero, Christophe Paul, Michał Pilipczuk

    Abstract: A simple digraph is semi-complete if for any two of its vertices $u$ and $v$, at least one of the arcs $(u,v)$ and $(v,u)$ is present. We study the complexity of computing two layout parameters of semi-complete digraphs: cutwidth and optimal linear arrangement (OLA). We prove that: (1) Both parameters are $\mathsf{NP}$-hard to compute and the known exact and parameterized algorithms for them have… ▽ More

    Submitted 2 June, 2017; originally announced June 2017.

  21. arXiv:1510.03564  [pdf, ps, other

    cs.DS cs.CC

    Linear-Vertex Kernel for the Problem of Packing $r$-Stars into a Graph without Long Induced Paths

    Authors: Florian Barbero, Gregory Gutin, Mark Jones, Bin Sheng, Anders Yeo

    Abstract: Let integers $r\ge 2$ and $d\ge 3$ be fixed. Let ${\cal G}_d$ be the set of graphs with no induced path on $d$ vertices. We study the problem of packing $k$ vertex-disjoint copies of $K_{1,r}$ ($k\ge 2$) into a graph $G$ from parameterized preprocessing, i.e., kernelization, point of view. We show that every graph $G\in {\cal G}_d$ can be reduced, in polynomial time, to a graph $G'\in {\cal G}_d$… ▽ More

    Submitted 13 October, 2015; originally announced October 2015.

  22. arXiv:1412.3023  [pdf, ps, other

    cs.DS cs.DM

    Parameterized and Approximation Algorithms for the Load Coloring Problem

    Authors: F. Barbero, G. Gutin, M. Jones, B. Sheng

    Abstract: Let $c, k$ be two positive integers and let $G=(V,E)$ be a graph. The $(c,k)$-Load Coloring Problem (denoted $(c,k)$-LCP) asks whether there is a $c$-coloring $\varphi: V \rightarrow [c]$ such that for every $i \in [c]$, there are at least $k$ edges with both endvertices colored $i$. Gutin and Jones (IPL 2014) studied this problem with $c=2$. They showed $(2,k)$-LCP to be fixed parameter tractable… ▽ More

    Submitted 18 December, 2014; v1 submitted 9 December, 2014; originally announced December 2014.

  23. arXiv:1205.2554  [pdf, ps, other

    math.LO cs.LO

    On existential declarations of independence in IF Logic

    Authors: Fausto Barbero

    Abstract: We analyze the behaviour of declarations of independence between existential quantifiers in quantifier prefixes of IF sentences; we give a syntactical criterion for deciding whether a sentence beginning with such prefix exists such that its truth values may be affected by removal of the declaration of independence. We extend the result also to equilibrium semantics values for undetermined IF sente… ▽ More

    Submitted 11 May, 2012; originally announced May 2012.