Abstract
This paper presents vbench, a publicly available benchmark for cloud video services. We are the first study, to the best of our knowledge, to characterize the emerging video-as-a-service workload. Unlike prior video processing benchmarks, vbench's videos are algorithmically selected to represent a large commercial corpus of millions of videos. Reflecting the complex infrastructure that processes and hosts these videos, vbench includes carefully constructed metrics and baselines. The combination of validated corpus, baselines, and metrics reveal nuanced tradeoffs between speed, quality, and compression. We demonstrate the importance of video selection with a microarchitectural study of cache, branch, and SIMD behavior. vbench reveals trends from the commercial corpus that are not visible in other video corpuses. Our experiments with GPUs under vbench's scoring scenarios reveal that context is critical: GPUs are well suited for live-streaming, while for video-on-demand shift costs from compute to storage and network. Counterintuitively, they are not viable for popular videos, for which highly compressed, high quality copies are required. We instead find that popular videos are currently well-served by the current trajectory of software encoders.
- M. Alvarez, E. Salami, A. Ramirez, and M. Valero. 2007. HD-VideoBench: A Benchmark for Evaluating High Definition Digital Video Applications IEEE 10th International Symposium on Workload Characterization (IISWC). Google Scholar
Digital Library
- Christian Bienia. 2011. Benchmarking Modern Multiprocessors. Ph.D. Dissertation. bibinfoschoolPrinceton University. Google Scholar
Digital Library
- Blender Foundation. 2002. Blender - a 3D modelling and rendering package. deftempurl%http://www.blender.org tempurlGoogle Scholar
- Rajkumar Buyya, Mukaddim Pathan, and Athena Vakali. 2008. Content Delivery Networks (bibinfoedition1st ed.). Springer Publishing Company. Google Scholar
Digital Library
- Meeyoung Cha, Haewoon Kwak, Pablo Rodriguez, Yong-Yeol Ahn, and Sue Moon. 2009. Analyzing the video popularity characteristics of large-scale user generated content systems. IEEE/ACM Transactions on Networking (TON) Vol. 17, 5 (2009). Google Scholar
Digital Library
- Chao Chen, Mohammad Izadi, and Anil Kokaram. 2016. A Perceptual Quality Metric for Videos Distorted by Spatially Correlated Noise Proceedings of the ACM Multimedia Conference. Google Scholar
Digital Library
- Jan De Cock, Aditya Mavlankar, Anush Moorthy, and Anne M. Aaron. 2016. A large-scale video codec comparison of x264, x265 and libvpx for practical VOD applications. In Applications of Digital Image Processing XXXIX.Google Scholar
- Jesus Corbal, Roger Espasa, and Mateo Valero. 1999. MOM: A Matrix SIMD Instruction Set Architecture for Multimedia Applications Proceedings of the ACM/IEEE Conference on Supercomputing (SC). Google Scholar
Digital Library
- Intel Corp.. 2017. Intel Quick Sync Video. https://www.intel.com/content/www/us/en/architecture-and-technology/quick-sync-video/quick-sync-video-general.html. (2017).Google Scholar
- CreativeCommons.org. 2007. Creative Commons Attribution 3.0 License. https://creativecommons.org/licenses/by/3.0/legalcode. (2007).Google Scholar
- Youtube Engineering and Developers Blog. 2016. A look into YouTube's video file anatomy. https://youtube-eng.googleblog.com/2016/04/a-look-into-youtubes-video-file-anatomy.html. (2016).Google Scholar
- Michael Ferdman, Almutaz Adileh, Onur Kocberber, Stavros Volos, Mohammad Alisafaee, Djordje Jevdjic, Cansu Kaynak, Adrian Daniel Popescu, Anastasia Ailamaki, and Babak Falsafi. 2012. Clearing the Clouds: A Study of Emerging Scale-out Workloads on Modern Hardware. In Proceedings of the 17th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). Google Scholar
Digital Library
- Sadjad Fouladi, Riad S. Wahby, Brennan Shacklett, Karthikeyan Vasuki Balasubramaniam, William Zeng, Rahul Bhalerao, Anirudh Sivaraman, George Porter, and Keith Winstein. 2017. Encoding, Fast and Slow: Low-Latency Video Processing Using Thousands of Tiny Threads. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI). Google Scholar
Digital Library
- Mozilla & The Xiph.Org Foundation. 2017. Progress in the Alliance for Open Media. https://people.xiph.org/tterribe/pubs/lca2017/aom.pdf. (2017).Google Scholar
- R. Gonzalez and M. Horowitz. 1996. Energy dissipation in general purpose microprocessors. IEEE Journal of Solid-State Circuits Vol. 31, 9 (1996).Google Scholar
Cross Ref
Index Terms
vbench: Benchmarking Video Transcoding in the Cloud
Recommendations
vbench: Benchmarking Video Transcoding in the Cloud
ASPLOS '18: Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating SystemsThis paper presents vbench, a publicly available benchmark for cloud video services. We are the first study, to the best of our knowledge, to characterize the emerging video-as-a-service workload. Unlike prior video processing benchmarks, vbench's ...
NNBench-X: A Benchmarking Methodology for Neural Network Accelerator Designs
The tremendous impact of deep learning algorithms over a wide range of application domains has encouraged a surge of neural network (NN) accelerator research. Facilitating the NN accelerator design calls for guidance from an evolving benchmark suite ...
The DaCapo benchmarks: java benchmarking development and analysis
OOPSLA '06: Proceedings of the 21st annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applicationsSince benchmarks drive computer science research and industry product development, which ones we use and how we evaluate them are key questions for the community. Despite complex runtime tradeoffs due to dynamic compilation and garbage collection ...







Comments