-
Neural Video Compression with Temporal Layer-Adaptive Hierarchical B-frame Coding
Authors:
Yeongwoong Kim,
Suyong Bahk,
Seungeon Kim,
Won Hee Lee,
Dokwan Oh,
Hui Yong Kim
Abstract:
Neural video compression (NVC) is a rapidly evolving video coding research area, with some models achieving superior coding efficiency compared to the latest video coding standard Versatile Video Coding (VVC). In conventional video coding standards, the hierarchical B-frame coding, which utilizes a bidirectional prediction structure for higher compression, had been well-studied and exploited. In N…
▽ More
Neural video compression (NVC) is a rapidly evolving video coding research area, with some models achieving superior coding efficiency compared to the latest video coding standard Versatile Video Coding (VVC). In conventional video coding standards, the hierarchical B-frame coding, which utilizes a bidirectional prediction structure for higher compression, had been well-studied and exploited. In NVC, however, limited research has investigated the hierarchical B scheme. In this paper, we propose an NVC model exploiting hierarchical B-frame coding with temporal layer-adaptive optimization. We first extend an existing unidirectional NVC model to a bidirectional model, which achieves -21.13% BD-rate gain over the unidirectional baseline model. However, this model faces challenges when applied to sequences with complex or large motions, leading to performance degradation. To address this, we introduce temporal layer-adaptive optimization, incorporating methods such as temporal layer-adaptive quality scaling (TAQS) and temporal layer-adaptive latent scaling (TALS). The final model with the proposed methods achieves an impressive BD-rate gain of -39.86% against the baseline. It also resolves the challenges in sequences with large or complex motions with up to -49.13% more BD-rate gains than the simple bidirectional extension. This improvement is attributed to the allocation of more bits to lower temporal layers, thereby enhancing overall reconstruction quality with smaller bits. Since our method has little dependency on a specific NVC model architecture, it can serve as a general tool for extending unidirectional NVC models to the ones with hierarchical B-frame coding.
△ Less
Submitted 5 September, 2023; v1 submitted 30 August, 2023;
originally announced August 2023.
-
UpCycling: Semi-supervised 3D Object Detection without Sharing Raw-level Unlabeled Scenes
Authors:
Sunwook Hwang,
Youngseok Kim,
Seongwon Kim,
Saewoong Bahk,
Hyung-Sin Kim
Abstract:
Semi-supervised Learning (SSL) has received increasing attention in autonomous driving to reduce the enormous burden of 3D annotation. In this paper, we propose UpCycling, a novel SSL framework for 3D object detection with zero additional raw-level point cloud: learning from unlabeled de-identified intermediate features (i.e., smashed data) to preserve privacy. Since these intermediate features ar…
▽ More
Semi-supervised Learning (SSL) has received increasing attention in autonomous driving to reduce the enormous burden of 3D annotation. In this paper, we propose UpCycling, a novel SSL framework for 3D object detection with zero additional raw-level point cloud: learning from unlabeled de-identified intermediate features (i.e., smashed data) to preserve privacy. Since these intermediate features are naturally produced by the inference pipeline, no additional computation is required on autonomous vehicles. However, generating effective consistency loss for unlabeled feature-level scene turns out to be a critical challenge. The latest SSL frameworks for 3D object detection that enforce consistency regularization between different augmentations of an unlabeled raw-point scene become detrimental when applied to intermediate features. To solve the problem, we introduce a novel combination of hybrid pseudo labels and feature-level Ground Truth sampling (F-GT), which safely augments unlabeled multi-type 3D scene features and provides high-quality supervision. We implement UpCycling on two representative 3D object detection models: SECOND-IoU and PV-RCNN. Experiments on widely-used datasets (Waymo, KITTI, and Lyft) verify that UpCycling outperforms other augmentation methods applied at the feature level. In addition, while preserving privacy, UpCycling performs better or comparably to the state-of-the-art methods that utilize raw-level unlabeled data in both domain adaptation and partial-label scenarios.
△ Less
Submitted 16 March, 2023; v1 submitted 21 November, 2022;
originally announced November 2022.
-
Bitwidth-Adaptive Quantization-Aware Neural Network Training: A Meta-Learning Approach
Authors:
Jiseok Youn,
Jaehun Song,
Hyung-Sin Kim,
Saewoong Bahk
Abstract:
Deep neural network quantization with adaptive bitwidths has gained increasing attention due to the ease of model deployment on various platforms with different resource budgets. In this paper, we propose a meta-learning approach to achieve this goal. Specifically, we propose MEBQAT, a simple yet effective way of bitwidth-adaptive quantization aware training (QAT) where meta-learning is effectivel…
▽ More
Deep neural network quantization with adaptive bitwidths has gained increasing attention due to the ease of model deployment on various platforms with different resource budgets. In this paper, we propose a meta-learning approach to achieve this goal. Specifically, we propose MEBQAT, a simple yet effective way of bitwidth-adaptive quantization aware training (QAT) where meta-learning is effectively combined with QAT by redefining meta-learning tasks to incorporate bitwidths. After being deployed on a platform, MEBQAT allows the (meta-)trained model to be quantized to any candidate bitwidth then helps to conduct inference without much accuracy drop from quantization. Moreover, with a few-shot learning scenario, MEBQAT can also adapt a model to any bitwidth as well as any unseen target classes by adding conventional optimization or metric-based meta-learning. We design variants of MEBQAT to support both (1) a bitwidth-adaptive quantization scenario and (2) a new few-shot learning scenario where both quantization bitwidths and target classes are jointly adapted. We experimentally demonstrate their validity in multiple QAT schemes. By comparing their performance to (bitwidth-dedicated) QAT, existing bitwidth adaptive QAT and vanilla meta-learning, we find that merging bitwidths into meta-learning tasks achieves a higher level of robustness.
△ Less
Submitted 20 July, 2022;
originally announced July 2022.
-
A Dual-Connection based Handover Scheme for Ultra-Dense Millimeter-Wave Cellular Networks
Authors:
Seongjoon Kang,
Siyoung Choi,
Goodsol Lee,
Saewoong Bahk
Abstract:
Mobile users in an ultra-dense millimeter-wave cellular network experience handover events more frequently than in conventional networks, which results in increased service interruption time and performance degradation due to blockages. Multi-connectivity has been proposed to resolve this, and it also extends the coverage of millimeter-wave communications. In this paper, we propose a dual-connecti…
▽ More
Mobile users in an ultra-dense millimeter-wave cellular network experience handover events more frequently than in conventional networks, which results in increased service interruption time and performance degradation due to blockages. Multi-connectivity has been proposed to resolve this, and it also extends the coverage of millimeter-wave communications. In this paper, we propose a dual-connection based handover scheme for mobile UEs in an environment where they are connected simultaneously with two millimeter-wave cells to overcome frequent handover problems. This scheme allows a mobile UE to choose its serving link between the two mmWave connections according to the measured SINRs and then the corresponding base stations may forward duplicate packets to the UE. We compare our dual-connection based scheme with a conventional single-connection based scheme through ns-3 simulation. The simulation results show that the proposed scheme significantly reduces handover rate and delay. Therefore, we argue that the dual-connection based scheme helps mobile users achieve performance goals they require in ultra-dense cellular environments.
△ Less
Submitted 9 July, 2021;
originally announced July 2021.
-
Full-stack Hybrid Beamforming in mmWave 5G Networks
Authors:
Felipe Gomez-Cuba,
Tommaso Zugno,
Junseok Kim,
Michele Polese,
Saewoong Bahk,
Michele Zorzi
Abstract:
This paper analyzes Hybrid Beamforming (HBF) and Multi-User Multiple-Input Multiple-Output (MU-MIMO) in millimeter wave (mmWave) 5th generation (5G) cellular networks considering the full protocol stack with TCP/IP traffic and MAC scheduling. Prior work on HBF and MU-MIMO has assumed full-buffer transmissions and studied link-level performance. We report non-trivial interactions between the HBF te…
▽ More
This paper analyzes Hybrid Beamforming (HBF) and Multi-User Multiple-Input Multiple-Output (MU-MIMO) in millimeter wave (mmWave) 5th generation (5G) cellular networks considering the full protocol stack with TCP/IP traffic and MAC scheduling. Prior work on HBF and MU-MIMO has assumed full-buffer transmissions and studied link-level performance. We report non-trivial interactions between the HBF technique, the front-loaded channel estimation pilot scheme in NR, and the constraints of MU-MIMO scheduling. We also report that joint multi-user beamforming design is imperative, in the sense that the MU-MIMO system cannot be fully exploited when implemented as a mere collection of single-user analog beams working in parallel. By addressing these issues, throughput can be dramatically increased in mmWave 5G networks by means of Spatial Division Multiple Access (SDMA).
△ Less
Submitted 13 October, 2020;
originally announced October 2020.
-
Hybrid Beamforming in 5G mmWave Networks: a Full-stack Perspective
Authors:
Felipe Gomez-Cuba,
Tommaso Zugno,
Junseok Kim,
Michele Polese,
Saewoong Bahk,
Michele Zorzi
Abstract:
This paper studies the cross-layer challenges and performance of Hybrid Beamforming (HBF) and Multi-User Multiple-Input Multiple-Output (MU-MIMO) in 5G millimeter wave (mmWave) cellular networks with full-stack TCP/IP traffic and MAC scheduling. While previous research on HBF and MU-MIMO has focused on link-level analysis of full-buffer transmissions, this work reveals the interplay between HBF te…
▽ More
This paper studies the cross-layer challenges and performance of Hybrid Beamforming (HBF) and Multi-User Multiple-Input Multiple-Output (MU-MIMO) in 5G millimeter wave (mmWave) cellular networks with full-stack TCP/IP traffic and MAC scheduling. While previous research on HBF and MU-MIMO has focused on link-level analysis of full-buffer transmissions, this work reveals the interplay between HBF techniques and the higher layers of the protocol stack. To this aim, prior work on full stack simulation of mmWave cellular network has been extended by including the modeling of MU-MIMO and HBF. Our results reveal novel relations between the networking layers and the HBF MU-MIMO performance in the physical layer. Particularly, throughput can be increased in 5G networks by means of Spatial Division Multiple Access (SDMA). However, in order to achieve such benefits it is necessary to take into account certain trade-offs and the implementation complexity of a full-stack HBF solution.
△ Less
Submitted 8 October, 2020;
originally announced October 2020.
-
Large-scale Antenna Operation in Heterogeneous Cloud Radio Access Networks: A Partial Centralization Approach
Authors:
Sangkyu Park,
Chan-Byoung Chae,
Saewoong Bahk
Abstract:
To satisfy the ever-increasing capacity demand and quality of service (QoS) requirements of users, 5G cellular systems will take the form of heterogeneous networks (HetNets) that consist of macro cells and small cells. To build and operate such systems, mobile operators have given significant attention to cloud radio access networks (C-RANs) due to their beneficial features of performance optimiza…
▽ More
To satisfy the ever-increasing capacity demand and quality of service (QoS) requirements of users, 5G cellular systems will take the form of heterogeneous networks (HetNets) that consist of macro cells and small cells. To build and operate such systems, mobile operators have given significant attention to cloud radio access networks (C-RANs) due to their beneficial features of performance optimization and cost effectiveness. Along with the architectural enhancement of C-RAN, large-scale antennas (a.k.a. massive MIMO) at cell sites contribute greatly to increased network capacity either with higher spectral efficiency or through permitting many users at once. In this article, we discuss the challenging issues of C-RAN based HetNets (H-CRAN), especially with respect to large-scale antenna operation. We provide an overview of existing C-RAN architectures in terms of large-scale antenna operation and promote a partially centralized approach. This approach reduces, remarkably, fronthaul overheads in CRANs with large-scale antennas. We also provide some insights into its potential and applicability in the fronthaul bandwidthlimited H-CRAN with large-scale antennas.
△ Less
Submitted 17 April, 2015; v1 submitted 12 April, 2015;
originally announced April 2015.