-
Towards AI-Assisted Sustainable Adaptive Video Streaming Systems: Tutorial and Survey
Authors:
Reza Farahani,
Zoha Azimi,
Christian Timmerer,
Radu Prodan
Abstract:
Improvements in networking technologies and the steadily increasing numbers of users, as well as the shift from traditional broadcasting to streaming content over the Internet, have made video applications (e.g., live and Video-on-Demand (VoD)) predominant sources of traffic. Recent advances in Artificial Intelligence (AI) and its widespread application in various academic and industrial fields ha…
▽ More
Improvements in networking technologies and the steadily increasing numbers of users, as well as the shift from traditional broadcasting to streaming content over the Internet, have made video applications (e.g., live and Video-on-Demand (VoD)) predominant sources of traffic. Recent advances in Artificial Intelligence (AI) and its widespread application in various academic and industrial fields have focused on designing and implementing a variety of video compression and content delivery techniques to improve user Quality of Experience (QoE). However, providing high QoE services results in more energy consumption and carbon footprint across the service delivery path, extending from the end user's device through the network and service infrastructure (e.g., cloud providers). Despite the importance of energy efficiency in video streaming, there is a lack of comprehensive surveys covering state-of-the-art AI techniques and their applications throughout the video streaming lifecycle. Existing surveys typically focus on specific parts, such as video encoding, delivery networks, playback, or quality assessment, without providing a holistic view of the entire lifecycle and its impact on energy consumption and QoE. Motivated by this research gap, this survey provides a comprehensive overview of the video streaming lifecycle, content delivery, energy and Video Quality Assessment (VQA) metrics and models, and AI techniques employed in video streaming. In addition, it conducts an in-depth state-of-the-art analysis focused on AI-driven approaches to enhance the energy efficiency of end-to-end aspects of video streaming systems (i.e., encoding, delivery network, playback, and VQA approaches). Finally, it discusses prospective research directions for develo** AI-assisted energy-aware video streaming systems.
△ Less
Submitted 4 June, 2024;
originally announced June 2024.
-
Towards Low-Latency and Energy-Efficient Hybrid P2P-CDN Live Video Streaming
Authors:
Reza Farahani,
Christian Timmerer,
Hermann Hellwagner
Abstract:
Streaming segmented videos over the Hypertext Transfer Protocol (HTTP) is an increasingly popular approach in both live and video-on-demand (VoD) applications. However, designing a scalable and adaptable framework that reduces servers energy consumption and supports low latency and high quality services, particularly for live video streaming scenarios, is still challenging for Over-The-Top (OTT) s…
▽ More
Streaming segmented videos over the Hypertext Transfer Protocol (HTTP) is an increasingly popular approach in both live and video-on-demand (VoD) applications. However, designing a scalable and adaptable framework that reduces servers energy consumption and supports low latency and high quality services, particularly for live video streaming scenarios, is still challenging for Over-The-Top (OTT) service providers. To address such challenges, this paper introduces a new hybrid P2P-CDN framework that leverages new networking and computing paradigms, i.e., Network Function Virtualization (NFV) and edge computing for live video streaming. The proposed framework introduces a multi-layer architecture and a tree of possible actions therein (an action tree), taking into account all available resources from peers, edge, and CDN servers to efficiently distribute video fetching and transcoding tasks across a hybrid P2P-CDN network, consequently enhancing the users latency and video quality. We also discuss our testbed designed to validate the framework and compare it with baseline methods. The experimental results indicate that the proposed framework improves user Quality of Experience (QoE), reduces client serving latency, and improves edge server energy consumption compared to baseline approaches.
△ Less
Submitted 25 March, 2024;
originally announced March 2024.
-
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, SDN, and MEC
Authors:
Reza Farahani
Abstract:
Multimedia applications, mainly video streaming services, are currently the dominant source of network load worldwide. In recent Video-on-Demand (VoD) and live video streaming services, traditional streaming delivery techniques have been replaced by adaptive solutions based on the HTTP protocol. Current trends toward high-resolution (e.g., 8K) and/or low-latency VoD and live video streaming pose n…
▽ More
Multimedia applications, mainly video streaming services, are currently the dominant source of network load worldwide. In recent Video-on-Demand (VoD) and live video streaming services, traditional streaming delivery techniques have been replaced by adaptive solutions based on the HTTP protocol. Current trends toward high-resolution (e.g., 8K) and/or low-latency VoD and live video streaming pose new challenges to end-to-end (E2E) bandwidth demand and have stringent delay requirements. To do this, video providers typically rely on Content Delivery Networks (CDNs) to ensure that they provide scalable video streaming services. To support future streaming scenarios involving millions of users, it is necessary to increase the CDNs' efficiency. It is widely agreed that these requirements may be satisfied by adopting emerging networking techniques to present Network-Assisted Video Streaming (NAVS) methods. Motivated by this, this thesis goes one step beyond traditional pure client-based HAS algorithms by incorporating (an) in-network component(s) with a broader view of the network to present completely transparent NAVS solutions for HAS clients.
△ Less
Submitted 25 March, 2024;
originally announced March 2024.
-
Random Search as a Baseline for Sparse Neural Network Architecture Search
Authors:
Rezsa Farahani
Abstract:
Sparse neural networks have shown similar or better generalization performance than their dense counterparts while having higher parameter efficiency. This has motivated a number of works to learn or search for high performing sparse networks. While reports of task performance or efficiency gains are impressive, standard baselines are lacking leading to poor comparability and unreliable reproducib…
▽ More
Sparse neural networks have shown similar or better generalization performance than their dense counterparts while having higher parameter efficiency. This has motivated a number of works to learn or search for high performing sparse networks. While reports of task performance or efficiency gains are impressive, standard baselines are lacking leading to poor comparability and unreliable reproducibility across methods. In this work, we propose Random Search as a baseline algorithm for finding good sparse configurations and study its performance. We apply Random Search on the node space of an overparameterized network with the goal of finding better initialized sparse sub-networks that are positioned more advantageously in the loss landscape. We record the post-training performances of the found sparse networks and at various levels of sparsity, and compare against both their fully connected parent networks and random sparse configurations at the same sparsity levels. First, we demonstrate performance at different levels of sparsity and highlight that a significant level of performance can still be preserved even when the network is highly sparse. Second, we observe that for this sparse architecture search task, initialized sparse networks found by Random Search neither perform better nor converge more efficiently than their random counterparts. Thus we conclude that Random Search may be viewed as a reasonable neutral baseline for sparsity search methods.
△ Less
Submitted 14 March, 2024; v1 submitted 13 March, 2024;
originally announced March 2024.
-
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Streaming
Authors:
Vignesh V Menon,
Reza Farahani,
Prajit T Rajendran,
Samira Afzal,
Klaus Schoeffmann,
Christian Timmerer
Abstract:
With the emergence of multiple modern video codecs, streaming service providers are forced to encode, store, and transmit bitrate ladders of multiple codecs separately, consequently suffering from additional energy costs for encoding, storage, and transmission. To tackle this issue, we introduce an online energy-efficient Multi-Codec Bitrate ladder Estimation scheme (MCBE) for adaptive video strea…
▽ More
With the emergence of multiple modern video codecs, streaming service providers are forced to encode, store, and transmit bitrate ladders of multiple codecs separately, consequently suffering from additional energy costs for encoding, storage, and transmission. To tackle this issue, we introduce an online energy-efficient Multi-Codec Bitrate ladder Estimation scheme (MCBE) for adaptive video streaming applications. In MCBE, quality representations within the bitrate ladder of new-generation codecs (e.g., High Efficiency Video Coding (HEVC), Alliance for Open Media Video 1 (AV1)) that lie below the predicted rate-distortion curve of the Advanced Video Coding (AVC) codec are removed. Moreover, perceptual redundancy between representations of the bitrate ladders of the considered codecs is also minimized based on a Just Noticeable Difference (JND) threshold. Therefore, random forest-based models predict the VMAF score of bitrate ladder representations of each codec. In a live streaming session where all clients support the decoding of AVC, HEVC, and AV1, MCBE achieves impressive results, reducing cumulative encoding energy by 56.45%, storage energy usage by 94.99%, and transmission energy usage by 77.61% (considering a JND of six VMAF points). These energy reductions are in comparison to a baseline bitrate ladder encoding based on current industry practice.
△ Less
Submitted 14 October, 2023;
originally announced October 2023.
-
Synthetic Query Generation for Privacy-Preserving Deep Retrieval Systems using Differentially Private Language Models
Authors:
Aldo Gael Carranza,
Rezsa Farahani,
Natalia Ponomareva,
Alex Kurakin,
Matthew Jagielski,
Milad Nasr
Abstract:
We address the challenge of ensuring differential privacy (DP) guarantees in training deep retrieval systems. Training these systems often involves the use of contrastive-style losses, which are typically non-per-example decomposable, making them difficult to directly DP-train with since common techniques require per-example gradients. To address this issue, we propose an approach that prioritizes…
▽ More
We address the challenge of ensuring differential privacy (DP) guarantees in training deep retrieval systems. Training these systems often involves the use of contrastive-style losses, which are typically non-per-example decomposable, making them difficult to directly DP-train with since common techniques require per-example gradients. To address this issue, we propose an approach that prioritizes ensuring query privacy prior to training a deep retrieval system. Our method employs DP language models (LMs) to generate private synthetic queries representative of the original data. These synthetic queries can be used in downstream retrieval system training without compromising privacy. Our approach demonstrates a significant enhancement in retrieval quality compared to direct DP-training, all while maintaining query-level privacy guarantees. This work highlights the potential of harnessing LMs to overcome limitations in standard DP-training methods.
△ Less
Submitted 23 May, 2024; v1 submitted 10 May, 2023;
originally announced May 2023.
-
Transcoding Quality Prediction for Adaptive Video Streaming
Authors:
Vignesh V Menon,
Reza Farahani,
Prajit T Rajendran,
Mohammed Ghanbari,
Hermann Hellwagner,
Christian Timmerer
Abstract:
In recent years, video streaming applications have proliferated the demand for Video Quality Assessment VQA). Reduced reference video quality assessment (RR-VQA) is a category of VQA where certain features (e.g., texture, edges) of the original video are provided for quality assessment. It is a popular research area for various applications such as social media, online games, and video streaming.…
▽ More
In recent years, video streaming applications have proliferated the demand for Video Quality Assessment VQA). Reduced reference video quality assessment (RR-VQA) is a category of VQA where certain features (e.g., texture, edges) of the original video are provided for quality assessment. It is a popular research area for various applications such as social media, online games, and video streaming. This paper introduces a reduced reference Transcoding Quality Prediction Model (TQPM) to determine the visual quality score of the video possibly transcoded in multiple stages. The quality is predicted using Discrete Cosine Transform (DCT)-energy-based features of the video (i.e., the video's brightness, spatial texture information, and temporal activity) and the target bitrate representation of each transcoding stage. To do that, the problem is formulated, and a Long Short-Term Memory (LSTM)-based quality prediction model is presented. Experimental results illustrate that, on average, TQPM yields PSNR, SSIM, and VMAF predictions with an R2 score of 0.83, 0.85, and 0.87, respectively, and Mean Absolute Error (MAE) of 1.31 dB, 1.19 dB, and 3.01, respectively, for single-stage transcoding. Furthermore, an R2 score of 0.84, 0.86, and 0.91, respectively, and MAE of 1.32 dB, 1.33 dB, and 3.25, respectively, are observed for a two-stage transcoding scenario. Moreover, the average processing time of TQPM for 4s segments is 0.328s, making it a practical VQA method in online streaming applications.
△ Less
Submitted 20 April, 2023;
originally announced April 2023.
-
Video Quality Assessment with Texture Information Fusion for Streaming Applications
Authors:
Vignesh V Menon,
Prajit T Rajendran,
Reza Farahani,
Klaus Schoeffmann,
Christian Timmerer
Abstract:
The rise in video streaming applications has increased the demand for video quality assessment (VQA). In 2016, Netflix introduced Video Multi-Method Assessment Fusion (VMAF), a full reference VQA metric that strongly correlates with perceptual quality, but its computation is time-intensive. We propose a Discrete Cosine Transform (DCT)-energy-based VQA with texture information fusion (VQ-TIF) model…
▽ More
The rise in video streaming applications has increased the demand for video quality assessment (VQA). In 2016, Netflix introduced Video Multi-Method Assessment Fusion (VMAF), a full reference VQA metric that strongly correlates with perceptual quality, but its computation is time-intensive. We propose a Discrete Cosine Transform (DCT)-energy-based VQA with texture information fusion (VQ-TIF) model for video streaming applications that determines the visual quality of the reconstructed video compared to the original video. VQ-TIF extracts Structural Similarity (SSIM) and spatiotemporal features of the frames from the original and reconstructed videos and fuses them using a long short-term memory (LSTM)-based model to estimate the visual quality. Experimental results show that VQ-TIF estimates the visual quality with a Pearson Correlation Coefficient (PCC) of 0.96 and a Mean Absolute Error (MAE) of 2.71, on average, compared to the ground truth VMAF scores. Additionally, VQ-TIF estimates the visual quality at a rate of 9.14 times faster than the state-of-the-art VMAF implementation, along with an 89.44 % reduction in energy consumption, assuming an Ultra HD (2160p) display resolution.
△ Less
Submitted 24 January, 2024; v1 submitted 28 February, 2023;
originally announced February 2023.
-
Bounds of Zagreb indices and hyper Zagreb indices
Authors:
Shaohui Wang,
Wei Gao,
Muhammad K. Jamil,
Mohammad R. Farahani,
Jia-Bao Liu
Abstract:
The hyper Zagreb index is a kind of extensions of Zagreb index, used for predicting physicochemical properties of organic compounds. Given a graph $G= (V(G), E(G))$, the first hyper-Zagreb index is the sum of the square of edge degree over edge set $E(G)$ and defined as $HM_1(G)=\sum_{e=uv\in E(G)}d(e)^2$, where $d(e)=d(u)+d(v)$ is the edge degree. In this work we define the second hyper-Zagreb in…
▽ More
The hyper Zagreb index is a kind of extensions of Zagreb index, used for predicting physicochemical properties of organic compounds. Given a graph $G= (V(G), E(G))$, the first hyper-Zagreb index is the sum of the square of edge degree over edge set $E(G)$ and defined as $HM_1(G)=\sum_{e=uv\in E(G)}d(e)^2$, where $d(e)=d(u)+d(v)$ is the edge degree. In this work we define the second hyper-Zagreb index on the adjacent edges as $HM_2(G)=\sum_{e\sim f}d(e)d(f)$, where $e\sim f$ represents the adjacent edges of $G$. By inequalities, we explore some upper and lower bounds of these hyper-Zagreb indices, and provide the relation between Zagreb indices and hyper Zagreb indices.
△ Less
Submitted 7 December, 2016;
originally announced December 2016.
-
The Wiener Index and the Hosoya Polynomial of the Jahangir Graphs
Authors:
Shaohui Wang,
Mohammad Reza Farahani,
M. R. Rajesh Kanna,
Muhammad Kamran Jamil,
R. Pradeep Kumar
Abstract:
Let G be a simple connected graph having vertex set V and edge set E. The vertex-set and edge-set of G denoted by V(G) and E(G), respectively. The length of the smallest path between two vertices is called the distance. Mathematical chemistry is the area of research engaged in new application of mathematics in chemistry. In mathematics chemistry, we have many topological indices for any molecular…
▽ More
Let G be a simple connected graph having vertex set V and edge set E. The vertex-set and edge-set of G denoted by V(G) and E(G), respectively. The length of the smallest path between two vertices is called the distance. Mathematical chemistry is the area of research engaged in new application of mathematics in chemistry. In mathematics chemistry, we have many topological indices for any molecular graph, that they are invariant on the graph automorphism. In this research paper, we computing the Wiener index and the Hosoya polynomial of the Jahangir graphs $J_5,m$ for all integer number $m \geq 3$.
△ Less
Submitted 1 July, 2016;
originally announced July 2016.
