-
CompanyName2Vec: Company Entity Matching Based on Job Ads
Authors:
Ran Ziv,
Ilan Gronau,
Michael Fire
Abstract:
Entity Matching is an essential part of all real-world systems that take in structured and unstructured data coming from different sources. Typically no common key is available for connecting records. Massive data cleaning and integration processes require completion before any data analytics, or further processing can be performed. Although record linkage is frequently regarded as a somewhat tedi…
▽ More
Entity Matching is an essential part of all real-world systems that take in structured and unstructured data coming from different sources. Typically no common key is available for connecting records. Massive data cleaning and integration processes require completion before any data analytics, or further processing can be performed. Although record linkage is frequently regarded as a somewhat tedious but necessary step, it reveals valuable insights, supports data visualization, and guides further analytic approaches to the data. Here, we focus on organization entity matching. We introduce CompanyName2Vec, a novel algorithm to solve company entity matching (CEM) using a neural network model to learn company name semantics from a job ad corpus, without relying on any information on the matched company besides its name. Based on a real-world data, we show that CompanyName2Vec outperforms other evaluated methods and solves the CEM challenge with an average success rate of 89.3%.
△ Less
Submitted 12 January, 2022;
originally announced January 2022.
-
Eliminating unwanted patterns with minimal interference
Authors:
Zehavit Leibovich,
Ilan Gronau
Abstract:
Artificial synthesis of DNA molecules is an essential part of the study of biological mechanisms. The design of a synthetic DNA molecule usually involves many objectives. One of the important objectives is to eliminate short sequence patterns that correspond to binding sites of restriction enzymes or transcription factors. While many design tools address this problem, no adequate formal solution e…
▽ More
Artificial synthesis of DNA molecules is an essential part of the study of biological mechanisms. The design of a synthetic DNA molecule usually involves many objectives. One of the important objectives is to eliminate short sequence patterns that correspond to binding sites of restriction enzymes or transcription factors. While many design tools address this problem, no adequate formal solution exists for the pattern elimination problem. In this work, we present a formal description of the elimination problem and suggest efficient algorithms that eliminate unwanted patterns and allow optimization of other objectives with minimal interference to the desired DNA functionality. Our approach is flexible, efficient, and straightforward, and therefore can be easily incorporated in existing DNA design tools, making them considerably more powerful.
△ Less
Submitted 3 August, 2021;
originally announced August 2021.
-
Genome-wide inference of ancestral recombination graphs
Authors:
Matthew D. Rasmussen,
Melissa J. Hubisz,
Ilan Gronau,
Adam Siepel
Abstract:
The complex correlation structure of a collection of orthologous DNA sequences is uniquely captured by the "ancestral recombination graph" (ARG), a complete record of coalescence and recombination events in the history of the sample. However, existing methods for ARG inference are computationally intensive, highly approximate, or limited to small numbers of sequences, and, as a consequence, explic…
▽ More
The complex correlation structure of a collection of orthologous DNA sequences is uniquely captured by the "ancestral recombination graph" (ARG), a complete record of coalescence and recombination events in the history of the sample. However, existing methods for ARG inference are computationally intensive, highly approximate, or limited to small numbers of sequences, and, as a consequence, explicit ARG inference is rarely used in applied population genomics. Here, we introduce a new algorithm for ARG inference that is efficient enough to apply to dozens of complete mammalian genomes. The key idea of our approach is to sample an ARG of n chromosomes conditional on an ARG of n-1 chromosomes, an operation we call "threading." Using techniques based on hidden Markov models, we can perform this threading operation exactly, up to the assumptions of the sequentially Markov coalescent and a discretization of time. An extension allows for threading of subtrees instead of individual sequences. Repeated application of these threading operations results in highly efficient Markov chain Monte Carlo samplers for ARGs. We have implemented these methods in a computer program called ARGweaver. Experiments with simulated data indicate that ARGweaver converges rapidly to the true posterior distribution and is effective in recovering various features of the ARG for dozens of sequences generated under realistic parameters for human populations. In applications of ARGweaver to 54 human genome sequences from Complete Genomics, we find clear signatures of natural selection, including regions of unusually ancient ancestry associated with balancing selection and reductions in allele age in sites under directional selection. Preliminary results also indicate that our methods can be used to gain insight into complex features of human population structure, even with a noninformative prior distribution.
△ Less
Submitted 3 December, 2013; v1 submitted 21 June, 2013;
originally announced June 2013.
-
Genome Sequencing Highlights Genes Under Selection and the Dynamic Early History of Dogs
Authors:
Adam H. Freedman,
Rena M. Schweizer,
Ilan Gronau,
Eunjung Han,
Diego Ortega-Del Vecchyo,
Pedro M. Silva,
Marco Galaverni,
Zhenxin Fan,
Peter Marx,
Belen Lorente-Galdos,
Holly Beale,
Oscar Ramirez,
Farhad Hormozdiari,
Can Alkan,
Carles Vilà,
Kevin Squire,
Eli Geffen,
Josip Kusak,
Adam R. Boyko,
Heidi G. Parker,
Clarence Lee,
Vasisht Tadigotla,
Adam Siepel,
Carlos D. Bustamante,
Timothy T. Harkins
, et al. (5 additional authors not shown)
Abstract:
To identify genetic changes underlying dog domestication and reconstruct their early evolutionary history, we analyzed novel high-quality genome sequences of three gray wolves, one from each of three putative centers of dog domestication, two ancient dog lineages (Basenji and Dingo) and a golden jackal as an outgroup. We find dogs and wolves diverged through a dynamic process involving population…
▽ More
To identify genetic changes underlying dog domestication and reconstruct their early evolutionary history, we analyzed novel high-quality genome sequences of three gray wolves, one from each of three putative centers of dog domestication, two ancient dog lineages (Basenji and Dingo) and a golden jackal as an outgroup. We find dogs and wolves diverged through a dynamic process involving population bottlenecks in both lineages and post-divergence gene flow, which confounds previous inferences of dog origins. In dogs, the domestication bottleneck was severe involving a 17 to 49-fold reduction in population size, a much stronger bottleneck than estimated previously from less intensive sequencing efforts. A sharp bottleneck in wolves occurred soon after their divergence from dogs, implying that the pool of diversity from which dogs arose was far larger than represented by modern wolf populations. Conditional on mutation rate, we narrow the plausible range for the date of initial dog domestication to an interval from 11 to 16 thousand years ago. This period predates the rise of agriculture, implying that the earliest dogs arose alongside hunter-gathers rather than agriculturists. Regarding the geographic origin of dogs, we find that surprisingly, none of the extant wolf lineages from putative domestication centers are more closely related to dogs, and the sampled wolves instead form a sister monophyletic clade. This result, in combination with our finding of dog-wolf admixture during the process of domestication, suggests a re-evaluation of past hypotheses of dog origin is necessary. Finally, we also detect signatures of selection, including evidence for selection on genes implicated in morphology, metabolism, and neural development. Uniquely, we find support for selective sweeps at regulatory sites suggesting gene regulatory changes played a critical role in dog domestication.
△ Less
Submitted 4 June, 2013; v1 submitted 31 May, 2013;
originally announced May 2013.
-
Inference of Natural Selection from Interspersed Genomic Elements Based on Polymorphism and Divergence
Authors:
Ilan Gronau,
Leonardo Arbiza,
Jaaved Mohammed,
Adam Siepel
Abstract:
Complete genome sequences contain valuable information about natural selection, but extracting this information for short, widely scattered noncoding elements remains a challenging problem. Here we introduce a new computational method for addressing this problem called Inference of Natural Selection from Interspersed Genomically coHerent elemenTs (INSIGHT). INSIGHT uses a generative probabilistic…
▽ More
Complete genome sequences contain valuable information about natural selection, but extracting this information for short, widely scattered noncoding elements remains a challenging problem. Here we introduce a new computational method for addressing this problem called Inference of Natural Selection from Interspersed Genomically coHerent elemenTs (INSIGHT). INSIGHT uses a generative probabilistic model to contrast patterns of polymorphism and divergence in the elements of interest with those in flanking neutral sites, pooling weak information from many short elements in a manner that accounts for variation among loci in mutation rates and genealogical backgrounds. The method is able to disentangle the contributions of weak negative, strong negative, and positive selection based on their distinct effects on patterns of polymorphism and divergence. Information about divergence is obtained from multiple outgroup genomes using a full phylogenetic model. The model is efficiently fitted to genome-wide data by decomposing the maximum likelihood estimation procedure into three straightforward stages. The key selection-related parameters are estimated by expectation maximization. Using simulations, we show that INSIGHT can accurately estimate several parameters of interest even in complex demographic scenarios. We apply our methods to noncoding RNAs, promoter regions, and transcription factor binding sites in the human genome, and find clear evidence of natural selection. We also present a detailed analysis of particular nucleotide positions within GATA2 binding sites and primary micro-RNA transcripts.
△ Less
Submitted 23 April, 2013; v1 submitted 28 September, 2011;
originally announced September 2011.
