-
Denoising as well as the best of any two denoisers
Authors:
Erik Ordentlich
Abstract:
Given two arbitrary sequences of denoisers for block lengths tending to infinity we ask if it is possible to construct a third sequence of denoisers with an asymptotically vanishing (in block length) excess expected loss relative to the best expected loss of the two given denoisers for all clean channel input sequences. As in the setting of DUDE [1], which solves this problem when the given denois…
▽ More
Given two arbitrary sequences of denoisers for block lengths tending to infinity we ask if it is possible to construct a third sequence of denoisers with an asymptotically vanishing (in block length) excess expected loss relative to the best expected loss of the two given denoisers for all clean channel input sequences. As in the setting of DUDE [1], which solves this problem when the given denoisers are sliding block denoisers, the construction is allowed to depend on the two given denoisers and the channel transition probabilities. We show that under certain restrictions on the two given denoisers the problem can be solved using a straightforward application of a known loss estimation paradigm. We then show by way of a counter-example that the loss estimation approach fails in the general case. Finally, we show that for the binary symmetric channel, combining the loss estimation with a randomization step leads to a solution to the stated problem under no restrictions on the given denoisers.
△ Less
Submitted 12 July, 2020;
originally announced July 2020.
-
Learning to Bid Optimally and Efficiently in Adversarial First-price Auctions
Authors:
Yanjun Han,
Zhengyuan Zhou,
Aaron Flores,
Erik Ordentlich,
Tsachy Weissman
Abstract:
First-price auctions have very recently swept the online advertising industry, replacing second-price auctions as the predominant auction mechanism on many platforms. This shift has brought forth important challenges for a bidder: how should one bid in a first-price auction, where unlike in second-price auctions, it is no longer optimal to bid one's private value truthfully and hard to know the ot…
▽ More
First-price auctions have very recently swept the online advertising industry, replacing second-price auctions as the predominant auction mechanism on many platforms. This shift has brought forth important challenges for a bidder: how should one bid in a first-price auction, where unlike in second-price auctions, it is no longer optimal to bid one's private value truthfully and hard to know the others' bidding behaviors? In this paper, we take an online learning angle and address the fundamental problem of learning to bid in repeated first-price auctions, where both the bidder's private valuations and other bidders' bids can be arbitrary. We develop the first minimax optimal online bidding algorithm that achieves an $\widetilde{O}(\sqrt{T})$ regret when competing with the set of all Lipschitz bidding policies, a strong oracle that contains a rich set of bidding strategies. This novel algorithm is built on the insight that the presence of a good expert can be leveraged to improve performance, as well as an original hierarchical expert-chaining structure, both of which could be of independent interest in online learning. Further, by exploiting the product structure that exists in the problem, we modify this algorithm--in its vanilla form statistically optimal but computationally infeasible--to a computationally efficient and space efficient algorithm that also retains the same $\widetilde{O}(\sqrt{T})$ minimax optimal regret guarantee. Additionally, through an impossibility result, we highlight that one is unlikely to compete this favorably with a stronger oracle (than the considered Lipschitz bidding policies). Finally, we test our algorithm on three real-world first-price auction datasets obtained from Verizon Media and demonstrate our algorithm's superior performance compared to several existing bidding algorithms.
△ Less
Submitted 9 July, 2020;
originally announced July 2020.
-
Scalable Semantic Matching of Queries to Ads in Sponsored Search Advertising
Authors:
Mihajlo Grbovic,
Nemanja Djuric,
Vladan Radosavljevic,
Fabrizio Silvestri,
Ricardo Baeza-Yates,
Andrew Feng,
Erik Ordentlich,
Lee Yang,
Gavin Owens
Abstract:
Sponsored search represents a major source of revenue for web search engines. This popular advertising model brings a unique possibility for advertisers to target users' immediate intent communicated through a search query, usually by displaying their ads alongside organic search results for queries deemed relevant to their products or services. However, due to a large number of unique queries it…
▽ More
Sponsored search represents a major source of revenue for web search engines. This popular advertising model brings a unique possibility for advertisers to target users' immediate intent communicated through a search query, usually by displaying their ads alongside organic search results for queries deemed relevant to their products or services. However, due to a large number of unique queries it is challenging for advertisers to identify all such relevant queries. For this reason search engines often provide a service of advanced matching, which automatically finds additional relevant queries for advertisers to bid on. We present a novel advanced matching approach based on the idea of semantic embeddings of queries and ads. The embeddings were learned using a large data set of user search sessions, consisting of search queries, clicked ads and search links, while utilizing contextual information such as dwell time and skipped ads. To address the large-scale nature of our problem, both in terms of data and vocabulary size, we propose a novel distributed algorithm for training of the embeddings. Finally, we present an approach for overcoming a cold-start problem associated with new ads and queries. We report results of editorial evaluation and online tests on actual search traffic. The results show that our approach significantly outperforms baselines in terms of relevance, coverage, and incremental revenue. Lastly, we open-source learned query embeddings to be used by researchers in computational advertising and related fields.
△ Less
Submitted 6 July, 2016;
originally announced July 2016.
-
Network-Efficient Distributed Word2vec Training System for Large Vocabularies
Authors:
Erik Ordentlich,
Lee Yang,
Andy Feng,
Peter Cnudde,
Mihajlo Grbovic,
Nemanja Djuric,
Vladan Radosavljevic,
Gavin Owens
Abstract:
Word2vec is a popular family of algorithms for unsupervised training of dense vector representations of words on large text corpuses. The resulting vectors have been shown to capture semantic relationships among their corresponding words, and have shown promise in reducing a number of natural language processing (NLP) tasks to mathematical operations on these vectors. While heretofore applications…
▽ More
Word2vec is a popular family of algorithms for unsupervised training of dense vector representations of words on large text corpuses. The resulting vectors have been shown to capture semantic relationships among their corresponding words, and have shown promise in reducing a number of natural language processing (NLP) tasks to mathematical operations on these vectors. While heretofore applications of word2vec have centered around vocabularies with a few million words, wherein the vocabulary is the set of words for which vectors are simultaneously trained, novel applications are emerging in areas outside of NLP with vocabularies comprising several 100 million words. Existing word2vec training systems are impractical for training such large vocabularies as they either require that the vectors of all vocabulary words be stored in the memory of a single server or suffer unacceptable training latency due to massive network data transfer. In this paper, we present a novel distributed, parallel training system that enables unprecedented practical training of vectors for vocabularies with several 100 million words on a shared cluster of commodity servers, using far less network traffic than the existing solutions. We evaluate the proposed system on a benchmark dataset, showing that the quality of vectors does not degrade relative to non-distributed training. Finally, for several quarters, the system has been deployed for the purpose of matching queries to ads in Gemini, the sponsored search advertising platform at Yahoo, resulting in significant improvement of business metrics.
△ Less
Submitted 27 June, 2016;
originally announced June 2016.
-
On the Degrees-of-Freedom of the K-User Gaussian Interference Channel
Authors:
Raul Etkin,
Erik Ordentlich
Abstract:
The degrees-of-freedom of a K-user Gaussian interference channel (GIFC) has been defined to be the multiple of (1/2)log_2(P) at which the maximum sum of achievable rates grows with increasing P. In this paper, we establish that the degrees-of-freedom of three or more user, real, scalar GIFCs, viewed as a function of the channel coefficients, is discontinuous at points where all of the coefficien…
▽ More
The degrees-of-freedom of a K-user Gaussian interference channel (GIFC) has been defined to be the multiple of (1/2)log_2(P) at which the maximum sum of achievable rates grows with increasing P. In this paper, we establish that the degrees-of-freedom of three or more user, real, scalar GIFCs, viewed as a function of the channel coefficients, is discontinuous at points where all of the coefficients are non-zero rational numbers. More specifically, for all K>2, we find a class of K-user GIFCs that is dense in the GIFC parameter space for which K/2 degrees-of-freedom are exactly achievable, and we show that the degrees-of-freedom for any GIFC with non-zero rational coefficients is strictly smaller than K/2. These results are proved using new connections with number theory and additive combinatorics.
△ Less
Submitted 12 January, 2009;
originally announced January 2009.
-
Error Exponents of Optimum Decoding for the Interference Channel
Authors:
Raul Etkin,
Neri Merhav,
Erik Ordentlich
Abstract:
Exponential error bounds for the finite-alphabet interference channel (IFC) with two transmitter-receiver pairs, are investigated under the random coding regime. Our focus is on optimum decoding, as opposed to heuristic decoding rules that have been used in previous works, like joint typicality decoding, decoding based on interference cancellation, and decoding that considers the interference as…
▽ More
Exponential error bounds for the finite-alphabet interference channel (IFC) with two transmitter-receiver pairs, are investigated under the random coding regime. Our focus is on optimum decoding, as opposed to heuristic decoding rules that have been used in previous works, like joint typicality decoding, decoding based on interference cancellation, and decoding that considers the interference as additional noise. Indeed, the fact that the actual interfering signal is a codeword and not an i.i.d. noise process complicates the application of conventional techniques to the performance analysis of the optimum decoder. Using analytical tools rooted in statistical physics, we derive a single letter expression for error exponents achievable under optimum decoding and demonstrate strict improvement over error exponents obtainable using suboptimal decoding rules, but which are amenable to more conventional analysis.
△ Less
Submitted 10 October, 2008;
originally announced October 2008.
