-
Autoencoders
Authors:
Dor Bank,
Noam Koenigstein,
Raja Giryes
Abstract:
An autoencoder is a specific type of a neural network, which is mainly designed to encode the input into a compressed and meaningful representation, and then decode it back such that the reconstructed input is similar as possible to the original one. This chapter surveys the different types of autoencoders that are mainly used today. It also describes various applications and use-cases of autoenco…
▽ More
An autoencoder is a specific type of a neural network, which is mainly designed to encode the input into a compressed and meaningful representation, and then decode it back such that the reconstructed input is similar as possible to the original one. This chapter surveys the different types of autoencoders that are mainly used today. It also describes various applications and use-cases of autoencoders.
△ Less
Submitted 3 April, 2021; v1 submitted 12 March, 2020;
originally announced March 2020.
-
An ETF view of Dropout regularization
Authors:
Dor Bank,
Raja Giryes
Abstract:
Dropout is a popular regularization technique in deep learning. Yet, the reason for its success is still not fully understood. This paper provides a new interpretation of Dropout from a frame theory perspective. By drawing a connection to recent developments in analog channel coding, we suggest that for a certain family of autoencoders with a linear encoder, optimizing the encoder with dropout reg…
▽ More
Dropout is a popular regularization technique in deep learning. Yet, the reason for its success is still not fully understood. This paper provides a new interpretation of Dropout from a frame theory perspective. By drawing a connection to recent developments in analog channel coding, we suggest that for a certain family of autoencoders with a linear encoder, optimizing the encoder with dropout regularization leads to an equiangular tight frame (ETF). Since this optimization is non-convex, we add another regularization that promotes such structures by minimizing the cross-correlation between filters in the network. We demonstrate its applicability in convolutional and fully connected layers in both feed-forward and recurrent networks. All these results suggest that there is indeed a relationship between dropout and ETF structure of the regularized linear operations.
△ Less
Submitted 19 August, 2020; v1 submitted 14 October, 2018;
originally announced October 2018.
-
Online Advance Admission Scheduling for Services with Customer Preferences
Authors:
Xinshang Wang,
Van-Anh Truong,
David Bank
Abstract:
We study web and mobile applications that are used to schedule advance service, from medical appointments to restaurant reservations. We model them as online weighted bipartite matching problems with non-stationary arrivals. We propose new algorithms with performance guarantees for this class of problems. Specifically, we show that the expected performance of our algorithms is bounded below by…
▽ More
We study web and mobile applications that are used to schedule advance service, from medical appointments to restaurant reservations. We model them as online weighted bipartite matching problems with non-stationary arrivals. We propose new algorithms with performance guarantees for this class of problems. Specifically, we show that the expected performance of our algorithms is bounded below by $1-\sqrt{\frac{2}π}\frac{1}{\sqrt{k}}+O(\frac{1}{k})$ times that of an optimal offline algorithm, which knows all future information upfront, where $k$ is the minimum capacity of a resource. This is the tightest known lower bound. This performance analysis holds for any Poisson arrival process. Our algorithms can also be applied to a number of related problems, including display ad allocation problems and revenue management problems for opaque products. We test the empirical performance of our algorithms against several well-known heuristics by using appointment scheduling data from a major academic hospital system in New York City. The results show that the algorithms exhibit the best performance among all the tested policies. In particular, our algorithms are $21\%$ more effective than the actual scheduling strategy used in the hospital system according to our performance metric.
△ Less
Submitted 25 May, 2018;
originally announced May 2018.
-
Reaching Distributed Equilibrium with Limited ID Space
Authors:
Dor Bank,
Moshe Sulamy,
Eyal Waserman
Abstract:
We examine the relation between the size of the id space and the number of rational agents in a network under which equilibrium in distributed algorithms is possible. When the number of agents in the network is not a-priori known, a single agent may duplicate to gain an advantage, pretending to be more than one agent. However, when the id space is limited, each duplication involves a risk of being…
▽ More
We examine the relation between the size of the id space and the number of rational agents in a network under which equilibrium in distributed algorithms is possible. When the number of agents in the network is not a-priori known, a single agent may duplicate to gain an advantage, pretending to be more than one agent. However, when the id space is limited, each duplication involves a risk of being caught. By comparing the risk against the advantage, given an id space of size $L$, we provide a method of calculating the minimal threshold $t$, the required number of agents in the network, such that the algorithm is in equilibrium. That is, it is the minimal value of $t$ such that if agents a-priori know that $n \geq t$ then the algorithm is in equilibrium. We demonstrate this method by applying it to two problems, Leader Election and Knowledge Sharing, as well as providing a constant-time approximation $t \approx \frac{L}{5}$ of the minimal threshold for Leader Election.
△ Less
Submitted 18 April, 2018; v1 submitted 17 April, 2018;
originally announced April 2018.
-
Improved Training for Self-Training by Confidence Assessments
Authors:
Gal Hyams,
Daniel Greenfeld,
Dor Bank
Abstract:
It is well known that for some tasks, labeled data sets may be hard to gather. Therefore, we wished to tackle here the problem of having insufficient training data. We examined learning methods from unlabeled data after an initial training on a limited labeled data set. The suggested approach can be used as an online learning method on the unlabeled test set. In the general classification task, wh…
▽ More
It is well known that for some tasks, labeled data sets may be hard to gather. Therefore, we wished to tackle here the problem of having insufficient training data. We examined learning methods from unlabeled data after an initial training on a limited labeled data set. The suggested approach can be used as an online learning method on the unlabeled test set. In the general classification task, whenever we predict a label with high enough confidence, we treat it as a true label and train the data accordingly. For the semantic segmentation task, a classic example for an expensive data labeling process, we do so pixel-wise. Our suggested approaches were applied on the MNIST data-set as a proof of concept for a vision classification task and on the ADE20K data-set in order to tackle the semi-supervised semantic segmentation problem.
△ Less
Submitted 5 April, 2018; v1 submitted 30 September, 2017;
originally announced October 2017.