Showing 1–23 of 23 results for author: Rosenberg, C

Performance Analysis and Comparison of Full-Fledged 5G Standalone Experimental TDD Testbeds in Single & Multi-UE Scenarios

Authors: Maryam Amini, Catherine Rosenberg

Abstract: Open-source software and Commercial Off-The-Shelf hardware are finally paving their way into the 5G world, resulting in a proliferation of experimental 5G testbeds. Surprisingly, very few studies have been published on the comparative analysis of testbeds with different hardware and software elements. In this paper, we first introduce a precise nomenclature to characterize a 5G-standalone single… ▽ More Open-source software and Commercial Off-The-Shelf hardware are finally paving their way into the 5G world, resulting in a proliferation of experimental 5G testbeds. Surprisingly, very few studies have been published on the comparative analysis of testbeds with different hardware and software elements. In this paper, we first introduce a precise nomenclature to characterize a 5G-standalone single-cell testbed based on its constituent elements and main configuration parameters. We then build 30 distinct such testbeds and systematically analyze their performance with an emphasis on element interoperability (by considering different combinations of hardware and software elements from different sources), the number and type of User Equipment (UE) as well as the Radio Access Network hardware and software elements to address the following questions: 1) How is the performance (in terms of bit rate and latency) impacted by different elements? 2) How does the number of UEs affect these results? 3) What is the impact of the user(s)' location(s) on the performance? 4) What is the impact of the UE type on these results? 5) How far does each testbed provide coverage? 6) And finally, what is the effect of the computing resources available to each open-source software? This study focuses on TDD testbeds. △ Less

Submitted 2 July, 2024; originally announced July 2024.

OmniSearchSage: Multi-Task Multi-Entity Embeddings for Pinterest Search

Authors: Prabhat Agarwal, Minhazul Islam Sk, Nikil Pancha, Kurchi Subhra Hazra, Jia**g Xu, Chuck Rosenberg

Abstract: In this paper, we present OmniSearchSage, a versatile and scalable system for understanding search queries, pins, and products for Pinterest search. We jointly learn a unified query embedding coupled with pin and product embeddings, leading to an improvement of $>8\%$ relevance, $>7\%$ engagement, and $>5\%$ ads CTR in Pinterest's production search system. The main contributors to these gains are… ▽ More In this paper, we present OmniSearchSage, a versatile and scalable system for understanding search queries, pins, and products for Pinterest search. We jointly learn a unified query embedding coupled with pin and product embeddings, leading to an improvement of $>8\%$ relevance, $>7\%$ engagement, and $>5\%$ ads CTR in Pinterest's production search system. The main contributors to these gains are improved content understanding, better multi-task learning, and real-time serving. We enrich our entity representations using diverse text derived from image captions from a generative LLM, historical engagement, and user-curated boards. Our multitask learning setup produces a single search query embedding in the same space as pin and product embeddings and compatible with pre-existing pin and product embeddings. We show the value of each feature through ablation studies, and show the effectiveness of a unified model compared to standalone counterparts. Finally, we share how these embeddings have been deployed across the Pinterest search stack, from retrieval to ranking, scaling to serve $300k$ requests per second at low latency. Our implementation of this work is available at https://github.com/pinterest/atg-research/tree/main/omnisearchsage. △ Less

Submitted 24 April, 2024; originally announced April 2024.

Comments: 8 pages, 5 figures, to be published as an oral paper in TheWebConf Industry Track 2024

ACM Class: H.3.3

Planning and Operation of Millimeter-wave Downlink Systems with Hybrid Beamforming

Authors: Yuan Quan, Shahram Shahsavari, Catherine Rosenberg

Abstract: This paper investigates downlink radio resource management (RRM) in millimeter-wave systems with codebook-based hybrid beamforming in a single cell. We consider a practical but often overlooked multi-channel scenario where the base station is equipped with fewer radio frequency chains than there are user equipment (UEs) in the cell. In this case, analog beam selection is important because not all… ▽ More This paper investigates downlink radio resource management (RRM) in millimeter-wave systems with codebook-based hybrid beamforming in a single cell. We consider a practical but often overlooked multi-channel scenario where the base station is equipped with fewer radio frequency chains than there are user equipment (UEs) in the cell. In this case, analog beam selection is important because not all beams preferred by UEs can be selected simultaneously, and since the beam selection cannot vary across subchannels in a time slot, this creates a coupling between subchannels within a time slot. None of the solutions proposed in the literature deal with this important constraint. The paper begins with an offline study that analyzes the impact of different RRM procedures and system parameters on performance. An offline joint RRM optimization problem is formulated and solved that includes beam set selection, UE set selection, power distribution, modulation and coding scheme selection, and digital beamforming as a part of hybrid beamforming. The evaluation results of the offline study provide valuable insights that shows the importance of not neglecting the constraint and guide the design of low-complexity and high-performance online downlink RRM schemes in the second part of the paper. The proposed online RRM algorithms perform close to the performance targets obtained from the offline study while offering acceptable runtime. △ Less

Submitted 18 April, 2024; originally announced April 2024.

Rethinking Personalized Ranking at Pinterest: An End-to-End Approach

Authors: Jia**g Xu, Andrew Zhai, Charles Rosenberg

Abstract: In this work, we present our journey to revolutionize the personalized recommendation engine through end-to-end learning from raw user actions. We encode user's long-term interest in Pinner- Former, a user embedding optimized for long-term future actions via a new dense all-action loss, and capture user's short-term intention by directly learning from the real-time action sequences. We conducted b… ▽ More In this work, we present our journey to revolutionize the personalized recommendation engine through end-to-end learning from raw user actions. We encode user's long-term interest in Pinner- Former, a user embedding optimized for long-term future actions via a new dense all-action loss, and capture user's short-term intention by directly learning from the real-time action sequences. We conducted both offline and online experiments to validate the performance of the new model architecture, and also address the challenge of serving such a complex model using mixed CPU/GPU setup in production. The proposed system has been deployed in production at Pinterest and has delivered significant online gains across organic and Ads applications. △ Less

Submitted 17 September, 2022; originally announced September 2022.

Modeling User Behavior With Interaction Networks for Spam Detection

Authors: Prabhat Agarwal, Manisha Srivastava, Vishwakarma Singh, Charles Rosenberg

Abstract: Spam is a serious problem plaguing web-scale digital platforms which facilitate user content creation and distribution. It compromises platform's integrity, performance of services like recommendation and search, and overall business. Spammers engage in a variety of abusive and evasive behavior which are distinct from non-spammers. Users' complex behavior can be well represented by a heterogeneous… ▽ More Spam is a serious problem plaguing web-scale digital platforms which facilitate user content creation and distribution. It compromises platform's integrity, performance of services like recommendation and search, and overall business. Spammers engage in a variety of abusive and evasive behavior which are distinct from non-spammers. Users' complex behavior can be well represented by a heterogeneous graph rich with node and edge attributes. Learning to identify spammers in such a graph for a web-scale platform is challenging because of its structural complexity and size. In this paper, we propose SEINE (Spam DEtection using Interaction NEtworks), a spam detection model over a novel graph framework. Our graph simultaneously captures rich users' details and behavior and enables learning on a billion-scale graph. Our model considers neighborhood along with edge types and attributes, allowing it to capture a wide range of spammers. SEINE, trained on a real dataset of tens of millions of nodes and billions of edges, achieves a high performance of 80% recall with 1% false positive rate. SEINE achieves comparable performance to the state-of-the-art techniques on a public dataset while being pragmatic to be used in a large-scale production system. △ Less

Submitted 21 July, 2022; originally announced July 2022.

Comments: 6 pages, 2 figures, accepted to SIGIR 2022

ACM Class: I.2.6; H.3.5

Journal ref: In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (2022), pp. 2437-2442

ItemSage: Learning Product Embeddings for Shop** Recommendations at Pinterest

Authors: Paul Baltescu, Haoyu Chen, Nikil Pancha, Andrew Zhai, Jure Leskovec, Charles Rosenberg

Abstract: Learned embeddings for products are an important building block for web-scale e-commerce recommendation systems. At Pinterest, we build a single set of product embeddings called ItemSage to provide relevant recommendations in all shop** use cases including user, image and search based recommendations. This approach has led to significant improvements in engagement and conversion metrics, while r… ▽ More Learned embeddings for products are an important building block for web-scale e-commerce recommendation systems. At Pinterest, we build a single set of product embeddings called ItemSage to provide relevant recommendations in all shop** use cases including user, image and search based recommendations. This approach has led to significant improvements in engagement and conversion metrics, while reducing both infrastructure and maintenance cost. While most prior work focuses on building product embeddings from features coming from a single modality, we introduce a transformer-based architecture capable of aggregating information from both text and image modalities and show that it significantly outperforms single modality baselines. We also utilize multi-task learning to make ItemSage optimized for several engagement types, leading to a candidate generation system that is efficient for all of the engagement objectives of the end-to-end recommendation system. Extensive offline experiments are conducted to illustrate the effectiveness of our approach and results from online A/B experiments show substantial gains in key business metrics (up to +7% gross merchandise value/user and +11% click volume). △ Less

Submitted 23 May, 2022; originally announced May 2022.

Comments: 9 pages, 5 figures

Journal ref: KDD 2022

MultiBiSage: A Web-Scale Recommendation System Using Multiple Bipartite Graphs at Pinterest

Authors: Saket Gurukar, Nikil Pancha, Andrew Zhai, Eric Kim, Samson Hu, Srinivasan Parthasarathy, Charles Rosenberg, Jure Leskovec

Abstract: Graph Convolutional Networks (GCN) can efficiently integrate graph structure and node features to learn high-quality node embeddings. These embeddings can then be used for several tasks such as recommendation and search. At Pinterest, we have developed and deployed PinSage, a data-efficient GCN that learns pin embeddings from the Pin-Board graph. The Pin-Board graph contains pin and board entities… ▽ More Graph Convolutional Networks (GCN) can efficiently integrate graph structure and node features to learn high-quality node embeddings. These embeddings can then be used for several tasks such as recommendation and search. At Pinterest, we have developed and deployed PinSage, a data-efficient GCN that learns pin embeddings from the Pin-Board graph. The Pin-Board graph contains pin and board entities and the graph captures the pin belongs to a board interaction. However, there exist several entities at Pinterest such as users, idea pins, creators, and there exist heterogeneous interactions among these entities such as add-to-cart, follow, long-click. In this work, we show that training deep learning models on graphs that captures these diverse interactions would result in learning higher-quality pin embeddings than training PinSage on only the Pin-Board graph. To that end, we model the diverse entities and their diverse interactions through multiple bipartite graphs and propose a novel data-efficient MultiBiSage model. MultiBiSage can capture the graph structure of multiple bipartite graphs to learn high-quality pin embeddings. We take this pragmatic approach as it allows us to utilize the existing infrastructure developed at Pinterest -- such as Pixie system that can perform optimized random-walks on billion node graphs, along with existing training and deployment workflows. We train MultiBiSage on six bipartite graphs including our Pin-Board graph. Our offline metrics show that MultiBiSage significantly outperforms the deployed latest version of PinSage on multiple user engagement metrics. △ Less

Submitted 21 May, 2022; originally announced May 2022.

PinnerFormer: Sequence Modeling for User Representation at Pinterest

Authors: Nikil Pancha, Andrew Zhai, Jure Leskovec, Charles Rosenberg

Abstract: Sequential models have become increasingly popular in powering personalized recommendation systems over the past several years. These approaches traditionally model a user's actions on a website as a sequence to predict the user's next action. While theoretically simplistic, these models are quite challenging to deploy in production, commonly requiring streaming infrastructure to reflect the lates… ▽ More Sequential models have become increasingly popular in powering personalized recommendation systems over the past several years. These approaches traditionally model a user's actions on a website as a sequence to predict the user's next action. While theoretically simplistic, these models are quite challenging to deploy in production, commonly requiring streaming infrastructure to reflect the latest user activity and potentially managing mutable data for encoding a user's hidden state. Here we introduce PinnerFormer, a user representation trained to predict a user's future long-term engagement using a sequential model of a user's recent actions. Unlike prior approaches, we adapt our modeling to a batch infrastructure via our new dense all-action loss, modeling long-term future actions instead of next action prediction. We show that by doing so, we significantly close the gap between batch user embeddings that are generated once a day and realtime user embeddings generated whenever a user takes an action. We describe our design decisions via extensive offline experimentation and ablations and validate the efficacy of our approach in A/B experiments showing substantial improvements in Pinterest's user retention and engagement when comparing PinnerFormer against our previous user representation. PinnerFormer is deployed in production as of Fall 2021. △ Less

Submitted 9 May, 2022; originally announced May 2022.

Comments: 11 pages, 6 figures, submitted to KDD '22

Planning 5G Networks for Rural Fixed Wireless Access

Authors: Andrew Lappalainen, Yuhao Zhang, Catherine Rosenberg

Abstract: We study the planning of a rural 5G multi-user massive MIMO fixed wireless access system to offer fixed broadband service to homes. Specifically, we aim to determine the user limit, i.e., the maximum number of homes that can simultaneously receive target minimum bit rates (MBRs) on the downlink (DL) and on the uplink (UL) given a set of network resources and a cell radius. To compute that limit, w… ▽ More We study the planning of a rural 5G multi-user massive MIMO fixed wireless access system to offer fixed broadband service to homes. Specifically, we aim to determine the user limit, i.e., the maximum number of homes that can simultaneously receive target minimum bit rates (MBRs) on the downlink (DL) and on the uplink (UL) given a set of network resources and a cell radius. To compute that limit, we must understand how resources should be shared between the DL and UL and how user and stream selection, precoding and combining, and power distribution should be performed. We use block diagonalization and propose a static grou** strategy that organizes homes into fixed groups (of possibly different sizes) in the DL and UL; then we develop a simple approach to compute the user limit that we validate numerically. We study the impact of group size and show that smaller groups yield larger user limits in a 3.5 GHz band. We show how the user limit at different cell radii is impacted by the system bandwidth, the number of antennas at the base station and homes, the transmit power, and the DL and UL MBRs. Lastly, we offer insights into how the network could be operated. △ Less

Submitted 17 August, 2022; v1 submitted 4 October, 2021; originally announced October 2021.

Comments: 16 pages, 9 figures, 5 tables

Journal ref: IEEE Transactions on Network and Service Management, vol. 20, no. 1, pp. 441-455, Mar. 2023

Multi-User Scheduling in Hybrid Millimeter Wave Massive MIMO Systems

Authors: Seyedeh Maryam Hosseini, Shahram Shahsavari, Catherine Rosenberg

Abstract: While mmWave bands provide a large bandwidth for mobile broadband services, they suffer from severe path loss and shadowing. Multiple-antenna techniques such as beamforming (BF) can be applied to compensate the signal attenuation. We consider a special case of hybrid BF called per-stream hybrid BF (PSHBF) which is easier to implement than the general hybrid BF because it circumvents the need for j… ▽ More While mmWave bands provide a large bandwidth for mobile broadband services, they suffer from severe path loss and shadowing. Multiple-antenna techniques such as beamforming (BF) can be applied to compensate the signal attenuation. We consider a special case of hybrid BF called per-stream hybrid BF (PSHBF) which is easier to implement than the general hybrid BF because it circumvents the need for joint analog-digital beamformer optimization. Employing BF at the base station enables the transmission of multiple data streams to several users in the same resource block. In this paper, we provide an offline study of proportional fair multi-user scheduling in a mmWave system with PSHBF to understand the impact of various system parameters on the performance. We formulate multi-user scheduling as an optimization problem. To tackle the non-convexity, we provide a feasible solution and show through numerical examples that the performance of the provided solution is very close to an upper-bound. Using this framework, we provide extensive numerical investigations revealing several engineering insights. △ Less

Submitted 12 September, 2021; originally announced September 2021.

A General Framework for Airplane Air-to-Ground Communications in mmWave and Microwave Bands

Authors: Ararat Shaverdian, Shahram Shahsavari, Catherine Rosenberg

Abstract: Airplane sensors and on-board equipment collect an increasingly large amount of maintenance data during flights that are used for airplane maintenance. We propose to download part of the data during airplane's descent via a cellular base station (BS) located at the airport. We formulate and solve an offline optimization problem to quantify how much data can be offloaded in a non-dedicated band whi… ▽ More Airplane sensors and on-board equipment collect an increasingly large amount of maintenance data during flights that are used for airplane maintenance. We propose to download part of the data during airplane's descent via a cellular base station (BS) located at the airport. We formulate and solve an offline optimization problem to quantify how much data can be offloaded in a non-dedicated band while ensuring that the interference at the terrestrial BSs in the vicinity of the airport remains below a maximum allowable threshold. Our problem allows for adaptive tuning of transmit power, number of frequency channels to be used, and beamforming according to the position of the plane on the descent path. Our results show that during the last 5 minutes of descent, in the microwave band the plane can offload up to 5 GB of maintenance data in a 20~MHz band, while in the mmWave band the plane can offload up to 24 times more data in a 1~GHz band. Beamforming, power and bandwidth tuning are all crucial in maintaining a good performance in the mmWave band while in the microwave band, dynamic tuning of bandwidth does not improve the performance much. △ Less

Submitted 9 November, 2020; originally announced November 2020.

Comments: 38 pages, 7 figures

Improving Problem Identification via Automated Log Clustering using Dimensionality Reduction

Authors: Carl Martin Rosenberg, Leon Moonen

Abstract: Goal: We consider the problem of automatically grou** logs of runs that failed for the same underlying reasons, so that they can be treated more effectively, and investigate the following questions: (1) Does an approach developed to identify problems in system logs generalize to identifying problems in continuous deployment logs? (2) How does dimensionality reduction affect the quality of automa… ▽ More Goal: We consider the problem of automatically grou** logs of runs that failed for the same underlying reasons, so that they can be treated more effectively, and investigate the following questions: (1) Does an approach developed to identify problems in system logs generalize to identifying problems in continuous deployment logs? (2) How does dimensionality reduction affect the quality of automated log clustering? (3) How does the criterion used for merging clusters in the clustering algorithm affect clustering quality? Method: We replicate and extend earlier work on clustering system log files to assess its generalization to continuous deployment logs. We consider the optional inclusion of one of these dimensionality reduction techniques: Principal Component Analysis (PCA), Latent Semantic Indexing (LSI), and Non-negative Matrix Factorization (NMF). Moreover, we consider three alternative cluster merge criteria (Single Linkage, Average Linkage, and Weighted Linkage), in addition to the Complete Linkage criterion used in earlier work. We empirically evaluate the 16 resulting configurations on continuous deployment logs provided by our industrial collaborator. Results: Our study shows that (1) identifying problems in continuous deployment logs via clustering is feasible, (2) including NMF significantly improves overall accuracy and robustness, and (3) Complete Linkage performs best of all merge criteria analyzed. Conclusions: We conclude that problem identification via automated log clustering is improved by including dimensionality reduction, as it decreases the pipeline's sensitivity to parameter choice, thereby increasing its robustness for handling different inputs. △ Less

Submitted 7 September, 2020; originally announced September 2020.

Journal ref: Published in ESEM'18, Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, October 2018, Article: 16, pp. 1-10,

Spectrum-Based Log Diagnosis

Authors: Carl Martin Rosenberg, Leon Moonen

Abstract: We present and evaluate Spectrum-Based Log Diagnosis (SBLD), a method to help developers quickly diagnose problems found in complex integration and deployment runs. Inspired by Spectrum-Based Fault Localization, SBLD leverages the differences in event occurrences between logs for failing and passing runs, to highlight events that are stronger associated with failing runs. Using data provided by… ▽ More We present and evaluate Spectrum-Based Log Diagnosis (SBLD), a method to help developers quickly diagnose problems found in complex integration and deployment runs. Inspired by Spectrum-Based Fault Localization, SBLD leverages the differences in event occurrences between logs for failing and passing runs, to highlight events that are stronger associated with failing runs. Using data provided by our industrial partner, we empirically investigate the following questions: (i) How well does SBLD reduce the effort needed to identify all failure-relevant events in the log for a failing run? (ii) How is the performance of SBLD affected by available data? (iii) How does SBLD compare to searching for simple textual patterns that often occur in failure-relevant events? We answer (i) and (ii) using summary statistics and heatmap visualizations, and for (iii) we compare three configurations of SBLD (with resp. minimum, median and maximum data) against a textual search using Wilcoxon signed-rank tests and the Vargha-Delaney measure of stochastic superiority. Our evaluation shows that (i) SBLD achieves a significant effort reduction for the dataset used, (ii) SBLD benefits from additional logs for passing runs in general, and it benefits from additional logs for failing runs when there is a proportional amount of logs for passing runs in the data. Finally, (iii) SBLD and textual search are roughly equally effective at effort-reduction, while textual search has a slightly better recall. We investigate the cause, and discuss how it is due to the characteristics of a specific part of our data. We conclude that SBLD shows promise as a method for diagnosing failing runs, that its performance is positively affected by additional data, but that it does not outperform textual search on the dataset considered. Future work includes investigating SBLD's generalizability on additional datasets. △ Less

Submitted 16 August, 2020; originally announced August 2020.

Comments: Published in ESEM'20: ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), October 8-9, 2020, Bari, Italy. ACM, 12 pages

PinnerSage: Multi-Modal User Embedding Framework for Recommendations at Pinterest

Authors: Aditya Pal, Chantat Eksombatchai, Yitong Zhou, Bo Zhao, Charles Rosenberg, Jure Leskovec

Abstract: Latent user representations are widely adopted in the tech industry for powering personalized recommender systems. Most prior work infers a single high dimensional embedding to represent a user, which is a good starting point but falls short in delivering a full understanding of the user's interests. In this work, we introduce PinnerSage, an end-to-end recommender system that represents each user… ▽ More Latent user representations are widely adopted in the tech industry for powering personalized recommender systems. Most prior work infers a single high dimensional embedding to represent a user, which is a good starting point but falls short in delivering a full understanding of the user's interests. In this work, we introduce PinnerSage, an end-to-end recommender system that represents each user via multi-modal embeddings and leverages this rich representation of users to provides high quality personalized recommendations. PinnerSage achieves this by clustering users' actions into conceptually coherent clusters with the help of a hierarchical clustering method (Ward) and summarizes the clusters via representative pins (Medoids) for efficiency and interpretability. PinnerSage is deployed in production at Pinterest and we outline the several design decisions that makes it run seamlessly at a very large scale. We conduct several offline and online A/B experiments to show that our method significantly outperforms single embedding methods. △ Less

Submitted 7 July, 2020; originally announced July 2020.

Comments: 10 pages, 7 figures

Journal ref: KDD 2020

Shop The Look: Building a Large Scale Visual Shop** System at Pinterest

Authors: Raymond Shiau, Hao-Yu Wu, Eric Kim, Yue Li Du, Anqi Guo, Zhiyuan Zhang, Eileen Li, Kunlong Gu, Charles Rosenberg, Andrew Zhai

Abstract: As online content becomes ever more visual, the demand for searching by visual queries grows correspondingly stronger. Shop The Look is an online shop** discovery service at Pinterest, leveraging visual search to enable users to find and buy products within an image. In this work, we provide a holistic view of how we built Shop The Look, a shop** oriented visual search system, along with lesso… ▽ More As online content becomes ever more visual, the demand for searching by visual queries grows correspondingly stronger. Shop The Look is an online shop** discovery service at Pinterest, leveraging visual search to enable users to find and buy products within an image. In this work, we provide a holistic view of how we built Shop The Look, a shop** oriented visual search system, along with lessons learned from addressing shop** needs. We discuss topics including core technology across object detection and visual embeddings, serving infrastructure for realtime inference, and data labeling methodology for training/evaluation data collection and human evaluation. The user-facing impacts of our system design choices are measured through offline evaluations, human relevance judgements, and online A/B experiments. The collective improvements amount to cumulative relative gains of over 160% in end-to-end human relevance judgements and over 80% in engagement. Shop The Look is deployed in production at Pinterest. △ Less

Submitted 18 June, 2020; originally announced June 2020.

Comments: 10 pages, 7 figures, Accepted to KDD'20

ACM Class: I.2.10; I.4.8; I.4.9; I.4.10; I.5.4; K.4.4

Learning a Unified Embedding for Visual Search at Pinterest

Authors: Andrew Zhai, Hao-Yu Wu, Eric Tzeng, Dong Huk Park, Charles Rosenberg

Abstract: At Pinterest, we utilize image embeddings throughout our search and recommendation systems to help our users navigate through visual content by powering experiences like browsing of related content and searching for exact products for shop**. In this work we describe a multi-task deep metric learning system to learn a single unified image embedding which can be used to power our multiple visual… ▽ More At Pinterest, we utilize image embeddings throughout our search and recommendation systems to help our users navigate through visual content by powering experiences like browsing of related content and searching for exact products for shop**. In this work we describe a multi-task deep metric learning system to learn a single unified image embedding which can be used to power our multiple visual search products. The solution we present not only allows us to train for multiple application objectives in a single deep neural network architecture, but takes advantage of correlated information in the combination of all training data from each application to generate a unified embedding that outperforms all specialized embeddings previously deployed for each product. We discuss the challenges of handling images from different domains such as camera photos, high quality web images, and clean product catalog images. We also detail how to jointly train for multiple product objectives and how to leverage both engagement data and human labeled data. In addition, our trained embeddings can also be binarized for efficient storage and retrieval without compromising precision and recall. Through comprehensive evaluations on offline metrics, user studies, and online A/B experiments, we demonstrate that our proposed unified embedding improves both relevance and engagement of our visual search products for both browsing and searching purposes when compared to existing specialized embeddings. Finally, the deployment of the unified embedding at Pinterest has drastically reduced the operation and engineering cost of maintaining multiple embeddings while improving quality. △ Less

Submitted 5 August, 2019; originally announced August 2019.

Comments: in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge and Discovery and Data Mining, 2019

Hierarchical Temporal Convolutional Networks for Dynamic Recommender Systems

Authors: Jiaxuan You, Yichen Wang, Aditya Pal, Pong Eksombatchai, Chuck Rosenberg, Jure Leskovec

Abstract: Recommender systems that can learn from cross-session data to dynamically predict the next item a user will choose are crucial for online platforms. However, existing approaches often use out-of-the-box sequence models which are limited by speed and memory consumption, are often infeasible for production environments, and usually do not incorporate cross-session information, which is crucial for e… ▽ More Recommender systems that can learn from cross-session data to dynamically predict the next item a user will choose are crucial for online platforms. However, existing approaches often use out-of-the-box sequence models which are limited by speed and memory consumption, are often infeasible for production environments, and usually do not incorporate cross-session information, which is crucial for effective recommendations. Here we propose Hierarchical Temporal Convolutional Networks (HierTCN), a hierarchical deep learning architecture that makes dynamic recommendations based on users' sequential multi-session interactions with items. HierTCN is designed for web-scale systems with billions of items and hundreds of millions of users. It consists of two levels of models: The high-level model uses Recurrent Neural Networks (RNN) to aggregate users' evolving long-term interests across different sessions, while the low-level model is implemented with Temporal Convolutional Networks (TCN), utilizing both the long-term interests and the short-term interactions within sessions to predict the next interaction. We conduct extensive experiments on a public XING dataset and a large-scale Pinterest dataset that contains 6 million users with 1.6 billion interactions. We show that HierTCN is 2.5x faster than RNN-based models and uses 90% less data memory compared to TCN-based models. We further develop an effective data caching scheme and a queue-based mini-batch generator, enabling our model to be trained within 24 hours on a single GPU. Our model consistently outperforms state-of-the-art dynamic recommendation methods, with up to 18% improvement in recall and 10% in mean reciprocal rank. △ Less

Submitted 10 April, 2019; v1 submitted 8 April, 2019; originally announced April 2019.

Comments: Accepted by the Web Conference 2019 (WWW 2019) as a full paper

Joint User Association and Resource Allocation in Heterogeneous Cellular Networks: Comparison of Two Modeling Approaches

Authors: Dariush Fooladivanda, Catherine Rosenberg

Abstract: The performance of different combinations of user association (UA) and resource allocation (RA) in heterogeneous cellular networks has been extensively studied using a classic modeling approach based on system snapshots. There have been also many studies focusing on the dynamics of the system using queueing models. These modeling approaches are rarely compared with each other though they each brin… ▽ More The performance of different combinations of user association (UA) and resource allocation (RA) in heterogeneous cellular networks has been extensively studied using a classic modeling approach based on system snapshots. There have been also many studies focusing on the dynamics of the system using queueing models. These modeling approaches are rarely compared with each other though they each bring different insights to the design problem. In this paper, we consider a queueing model-based approach to study the interplay of UA and RA, and compare the results to those obtained using snapshot models. Specifically, we formulate three different joint UA and RA optimization problems corresponding to the following three performance metrics: the maximum achievable arrival rate, the average system delay, and the maximum per-user delay. These problems are non-convex integer programs. We have therefore developed numerical techniques to compute either their exact solutions or tight lower bounds. We obtain results for different combinations of RA and UA schemes, and compare the trends with those obtained via the snapshot approach. The trends on RA are very similar, which we take as a cross-validation of the two modeling approaches for this kind of problem. The trends on user association are somewhat different which indicates a lack of robustness of the results and the need for a careful validation of UA models. △ Less

Submitted 20 March, 2019; originally announced March 2019.

Complete the Look: Scene-based Complementary Product Recommendation

Authors: Wang-Cheng Kang, Eric Kim, Jure Leskovec, Charles Rosenberg, Julian McAuley

Abstract: Modeling fashion compatibility is challenging due to its complexity and subjectivity. Existing work focuses on predicting compatibility between product images (e.g. an image containing a t-shirt and an image containing a pair of jeans). However, these approaches ignore real-world 'scene' images (e.g. selfies); such images are hard to deal with due to their complexity, clutter, variations in lighti… ▽ More Modeling fashion compatibility is challenging due to its complexity and subjectivity. Existing work focuses on predicting compatibility between product images (e.g. an image containing a t-shirt and an image containing a pair of jeans). However, these approaches ignore real-world 'scene' images (e.g. selfies); such images are hard to deal with due to their complexity, clutter, variations in lighting and pose (etc.) but on the other hand could potentially provide key context (e.g. the user's body type, or the season) for making more accurate recommendations. In this work, we propose a new task called 'Complete the Look', which seeks to recommend visually compatible products based on scene images. We design an approach to extract training data for this task, and propose a novel way to learn the scene-product compatibility from fashion or interior design images. Our approach measures compatibility both globally and locally via CNNs and attention mechanisms. Extensive experiments show that our method achieves significant performance gains over alternative systems. Human evaluation and qualitative analysis are also conducted to further understand model behavior. We hope this work could lead to useful applications which link large corpora of real-world scenes with shoppable products. △ Less

Submitted 15 April, 2019; v1 submitted 4 December, 2018; originally announced December 2018.

Comments: Accepted to CVPR'19

Towards a Semantic Perceptual Image Metric

Authors: Troy Chinen, Johannes Ballé, Chunhui Gu, Sung ** Hwang, Sergey Ioffe, Nick Johnston, Thomas Leung, David Minnen, Sean O'Malley, Charles Rosenberg, George Toderici

Abstract: We present a full reference, perceptual image metric based on VGG-16, an artificial neural network trained on object classification. We fit the metric to a new database based on 140k unique images annotated with ground truth by human raters who received minimal instruction. The resulting metric shows competitive performance on TID 2013, a database widely used to assess image quality assessments me… ▽ More We present a full reference, perceptual image metric based on VGG-16, an artificial neural network trained on object classification. We fit the metric to a new database based on 140k unique images annotated with ground truth by human raters who received minimal instruction. The resulting metric shows competitive performance on TID 2013, a database widely used to assess image quality assessments methods. More interestingly, it shows strong responses to objects potentially carrying semantic relevance such as faces and text, which we demonstrate using a visualization technique and ablation experiments. In effect, the metric appears to model a higher influence of semantic context on judgments, which we observe particularly in untrained raters. As the vast majority of users of image processing systems are unfamiliar with Image Quality Assessment (IQA) tasks, these findings may have significant impact on real-world applications of perceptual metrics. △ Less

Submitted 1 August, 2018; originally announced August 2018.

Learning Fine-grained Image Similarity with Deep Ranking

Authors: Jiang Wang, Yang song, Thomas Leung, Chuck Rosenberg, **bin Wang, James Philbin, Bo Chen, Ying Wu

Abstract: Learning fine-grained image similarity is a challenging task. It needs to capture between-class and within-class image differences. This paper proposes a deep ranking model that employs deep learning techniques to learn similarity metric directly from images.It has higher learning capability than models based on hand-crafted features. A novel multiscale network structure has been developed to desc… ▽ More Learning fine-grained image similarity is a challenging task. It needs to capture between-class and within-class image differences. This paper proposes a deep ranking model that employs deep learning techniques to learn similarity metric directly from images.It has higher learning capability than models based on hand-crafted features. A novel multiscale network structure has been developed to describe the images effectively. An efficient triplet sampling algorithm is proposed to learn the model with distributed asynchronized stochastic gradient. Extensive experiments show that the proposed algorithm outperforms models based on hand-crafted visual features and deep classification models. △ Less

Submitted 17 April, 2014; originally announced April 2014.

Comments: CVPR 2014

The Capacity of Ad hoc Networks under Random Packet Losses

Authors: Vivek P. Mhatre, Catherine P. Rosenberg, Ravi R. Mazumdar

Abstract: We consider the problem of determining asymptotic bounds on the capacity of a random ad hoc network. Previous approaches assumed a link layer model in which if a transmitter-receiver pair can communicate with each other, i.e., the Signal to Interference and Noise Ratio (SINR) is above a certain threshold, then every transmitted packet is received error-free by the receiver thereby. Using this mo… ▽ More We consider the problem of determining asymptotic bounds on the capacity of a random ad hoc network. Previous approaches assumed a link layer model in which if a transmitter-receiver pair can communicate with each other, i.e., the Signal to Interference and Noise Ratio (SINR) is above a certain threshold, then every transmitted packet is received error-free by the receiver thereby. Using this model, the per node capacity of the network was shown to be $Θ(\frac{1}{\sqrt{n\log{n}}})$. In reality, for any finite link SINR, there is a non-zero probability of erroneous reception of the packet. We show that in a large network, as the packet travels an asymptotically large number of hops from source to destination, the cumulative impact of packet losses over intermediate links results in a per-node throughput of only $O(\frac{1}{n})$. We then propose a new scheduling scheme to counter this effect. The proposed scheme provides tight guarantees on end-to-end packet loss probability, and improves the per-node throughput to $Ω(\frac{1}{\sqrt{n} ({\log{n}})^{\frac{α{+2}}{2(α-2)}}})$ where $α>2$ is the path loss exponent. △ Less

Submitted 21 November, 2008; originally announced November 2008.

Comments: 12 pages, earlier version in ISIT 2006

The Capacity of Random Ad hoc Networks under a Realistic Link Layer Model

Authors: Vivek P. Mhatre, Catherine P. Rosenberg

Abstract: The problem of determining asymptotic bounds on the capacity of a random ad hoc network is considered. Previous approaches assumed a threshold-based link layer model in which a packet transmission is successful if the SINR at the receiver is greater than a fixed threshold. In reality, the map** from SINR to packet success probability is continuous. Hence, over each hop, for every finite SINR,… ▽ More The problem of determining asymptotic bounds on the capacity of a random ad hoc network is considered. Previous approaches assumed a threshold-based link layer model in which a packet transmission is successful if the SINR at the receiver is greater than a fixed threshold. In reality, the map** from SINR to packet success probability is continuous. Hence, over each hop, for every finite SINR, there is a non-zero probability of packet loss. With this more realistic link model, it is shown that for a broad class of routing and scheduling schemes, a fixed fraction of hops on each route have a fixed non-zero packet loss probability. In a large network, a packet travels an asymptotically large number of hops from source to destination. Consequently, it is shown that the cumulative effect of per-hop packet loss results in a per-node throughput of only O(1/n) (instead of Theta(1/sqrt{n log{n}})) as shown previously for the threshold-based link model). A scheduling scheme is then proposed to counter this effect. The proposed scheme improves the link SINR by using conservative spatial reuse, and improves the per-node throughput to O(1/(K_n sqrt{n log{n}})), where each cell gets a transmission opportunity at least once every K_n slots, and K_n tends to infinity as n tends to infinity. △ Less

Submitted 24 April, 2005; originally announced April 2005.

Comments: 12 pages, 6 figures, submitted to IEEE Transactions on Information Theory, April 21st 2005