-
KF-LAX: Kronecker-factored curvature estimation for control variate optimization in reinforcement learning
Authors:
Mohammad Firouzi
Abstract:
A key challenge for gradient based optimization methods in model-free reinforcement learning is to develop an approach that is sample efficient and has low variance. In this work, we apply Kronecker-factored curvature estimation technique (KFAC) to a recently proposed gradient estimator for control variate optimization, RELAX, to increase the sample efficiency of using this gradient estimation met…
▽ More
A key challenge for gradient based optimization methods in model-free reinforcement learning is to develop an approach that is sample efficient and has low variance. In this work, we apply Kronecker-factored curvature estimation technique (KFAC) to a recently proposed gradient estimator for control variate optimization, RELAX, to increase the sample efficiency of using this gradient estimation method in reinforcement learning. The performance of the proposed method is demonstrated on a synthetic problem and a set of three discrete control task Atari games.
△ Less
Submitted 10 December, 2018;
originally announced December 2018.
-
HellRank: A Hellinger-based Centrality Measure for Bipartite Social Networks
Authors:
Seyed Mohammad Taheri,
Hamidreza Mahyar,
Mohammad Firouzi,
Elahe Ghalebi K.,
Radu Grosu,
Ali Movaghar
Abstract:
Measuring centrality in a social network, especially in bipartite mode, poses several challenges such as requirement of full knowledge of the network topology and lack of properly detection of top-k behavioral representative users. In this paper, to overcome the aforementioned challenging issues, we propose an accurate centrality measure, called HellRank, to identify central nodes in bipartite soc…
▽ More
Measuring centrality in a social network, especially in bipartite mode, poses several challenges such as requirement of full knowledge of the network topology and lack of properly detection of top-k behavioral representative users. In this paper, to overcome the aforementioned challenging issues, we propose an accurate centrality measure, called HellRank, to identify central nodes in bipartite social networks. HellRank is based on the Hellinger distance between two nodes on the same side of a bipartite network. We theoretically analyze the impact of the Hellinger distance on a bipartite network and find an upper and lower bounds for this distance. The computation of HellRank centrality measure can be distributed by letting each node uses only local information on its immediate neighbors and therefore do not need a central entity to have full knowledge of the network topological structure. We experimentally evaluate performance of the HellRank measure in correlation with other centrality measures on real-world networks. The results show partial ranking similarity between the HellRank and the other conventional metrics according to the Kendall and Spearman rank correlation coefficient.
△ Less
Submitted 17 March, 2017; v1 submitted 5 December, 2016;
originally announced December 2016.
-
Extracting Implicit Social Relation for Social Recommendation Techniques in User Rating Prediction
Authors:
Seyed Mohammad Taheri,
Hamidreza Mahyar,
Mohammad Firouzi,
Elahe Ghalebi K.,
Radu Grosu,
Ali Movaghar
Abstract:
Recommendation plays an increasingly important role in our daily lives. Recommender systems automatically suggest items to users that might be interesting for them. Recent studies illustrate that incorporating social trust in Matrix Factorization methods demonstrably improves accuracy of rating prediction. Such approaches mainly use the trust scores explicitly expressed by users. However, it is of…
▽ More
Recommendation plays an increasingly important role in our daily lives. Recommender systems automatically suggest items to users that might be interesting for them. Recent studies illustrate that incorporating social trust in Matrix Factorization methods demonstrably improves accuracy of rating prediction. Such approaches mainly use the trust scores explicitly expressed by users. However, it is often challenging to have users provide explicit trust scores of each other. There exist quite a few works, which propose Trust Metrics to compute and predict trust scores between users based on their interactions. In this paper, first we present how social relation can be extracted from users' ratings to items by describing Hellinger distance between users in recommender systems. Then, we propose to incorporate the predicted trust scores into social matrix factorization models. By analyzing social relation extraction from three well-known real-world datasets, which both: trust and recommendation data available, we conclude that using the implicit social relation in social recommendation techniques has almost the same performance compared to the actual trust scores explicitly expressed by users. Hence, we build our method, called Hell-TrustSVD, on top of the state-of-the-art social recommendation technique to incorporate both the extracted implicit social relations and ratings given by users on the prediction of items for an active user. To the best of our knowledge, this is the first work to extend TrustSVD with extracted social trust information. The experimental results support the idea of employing implicit trust into matrix factorization whenever explicit trust is not available, can perform much better than the state-of-the-art approaches in user rating prediction.
△ Less
Submitted 17 March, 2017; v1 submitted 5 December, 2016;
originally announced December 2016.
