-
Fine-Grained Stochastic Architecture Search
Authors:
Shraman Ray Chaudhuri,
Elad Eban,
Hanhan Li,
Max Moroz,
Yair Movshovitz-Attias
Abstract:
State-of-the-art deep networks are often too large to deploy on mobile devices and embedded systems. Mobile neural architecture search (NAS) methods automate the design of small models but state-of-the-art NAS methods are expensive to run. Differentiable neural architecture search (DNAS) methods reduce the search cost but explore a limited subspace of candidate architectures. In this paper, we int…
▽ More
State-of-the-art deep networks are often too large to deploy on mobile devices and embedded systems. Mobile neural architecture search (NAS) methods automate the design of small models but state-of-the-art NAS methods are expensive to run. Differentiable neural architecture search (DNAS) methods reduce the search cost but explore a limited subspace of candidate architectures. In this paper, we introduce Fine-Grained Stochastic Architecture Search (FiGS), a differentiable search method that searches over a much larger set of candidate architectures. FiGS simultaneously selects and modifies operators in the search space by applying a structured sparse regularization penalty based on the Logistic-Sigmoid distribution. We show results across 3 existing search spaces, matching or outperforming the original search algorithms and producing state-of-the-art parameter-efficient models on ImageNet (e.g., 75.4% top-1 with 2.6M params). Using our architectures as backbones for object detection with SSDLite, we achieve significantly higher mAP on COCO (e.g., 25.8 with 3.0M params) than MobileNetV3 and MnasNet.
△ Less
Submitted 16 June, 2020;
originally announced June 2020.
-
Deep Tensor Convolution on Multicores
Authors:
David Budden,
Alexander Matveev,
Shibani Santurkar,
Shraman Ray Chaudhuri,
Nir Shavit
Abstract:
Deep convolutional neural networks (ConvNets) of 3-dimensional kernels allow joint modeling of spatiotemporal features. These networks have improved performance of video and volumetric image analysis, but have been limited in size due to the low memory ceiling of GPU hardware. Existing CPU implementations overcome this constraint but are impractically slow. Here we extend and optimize the faster W…
▽ More
Deep convolutional neural networks (ConvNets) of 3-dimensional kernels allow joint modeling of spatiotemporal features. These networks have improved performance of video and volumetric image analysis, but have been limited in size due to the low memory ceiling of GPU hardware. Existing CPU implementations overcome this constraint but are impractically slow. Here we extend and optimize the faster Winograd-class of convolutional algorithms to the $N$-dimensional case and specifically for CPU hardware. First, we remove the need to manually hand-craft algorithms by exploiting the relaxed constraints and cheap sparse access of CPU memory. Second, we maximize CPU utilization and multicore scalability by transforming data matrices to be cache-aware, integer multiples of AVX vector widths. Treating 2-dimensional ConvNets as a special (and the least beneficial) case of our approach, we demonstrate a 5 to 25-fold improvement in throughput compared to previous state-of-the-art.
△ Less
Submitted 11 June, 2017; v1 submitted 20 November, 2016;
originally announced November 2016.
-
A Novel Session Based Dual Steganographic Technique Using DWT and Spread Spectrum
Authors:
Tanmay Bhattacharya,
Nilanjan Dey,
S. R. Bhadra Chaudhuri
Abstract:
This paper proposed a DWT based Steganographic technique. Cover image is decomposed into four sub bands using DWT. Two secret images are embedded within the HL and HH sub bands respectively. During embedding secret images are dispersed within each band using a pseudo random sequence and a Session key. Secret images are extracted using the session key and the size of the images. In this approach th…
▽ More
This paper proposed a DWT based Steganographic technique. Cover image is decomposed into four sub bands using DWT. Two secret images are embedded within the HL and HH sub bands respectively. During embedding secret images are dispersed within each band using a pseudo random sequence and a Session key. Secret images are extracted using the session key and the size of the images. In this approach the stego image generated is of acceptable level of imperceptibility and distortion compared to the cover image and the overall security is high.
△ Less
Submitted 1 September, 2012;
originally announced September 2012.
-
A Session based Multiple Image Hiding Technique using DWT and DCT
Authors:
Tanmay Bhattacharya,
Nilanjan Dey,
S. R. Bhadra Chaudhuri
Abstract:
This work proposes Steganographic technique for hiding multiple images in a color image based on DWT and DCT. The cover image is decomposed into three separate color planes namely R, G and B. Individual planes are decomposed into subbands using DWT. DCT is applied in HH component of each plane. Secret images are dispersed among the selected DCT coefficients using a pseudo random sequence and a Ses…
▽ More
This work proposes Steganographic technique for hiding multiple images in a color image based on DWT and DCT. The cover image is decomposed into three separate color planes namely R, G and B. Individual planes are decomposed into subbands using DWT. DCT is applied in HH component of each plane. Secret images are dispersed among the selected DCT coefficients using a pseudo random sequence and a Session key. Secret images are extracted using the session key and the size of the images from the planer decomposed stego image. In this approach the stego image generated is of acceptable level of imperceptibility and distortion compared to the cover image and the overall security is high.
△ Less
Submitted 4 August, 2012;
originally announced August 2012.
