With the remarkable growth of adaptive streaming media applications, especially the wide usage of dynamic adaptive streaming schemes over HTTP (DASH), it becomes ever more important to understand the perceptual quality-of-experience (QoE) of end users, who may be constantly experiencing adaptations (switchings) of video bitrate, spatial resolution, and frame-rate from one time segment to another in a scale of a few seconds. This is a sophisticated and challenging problem, for which existing visual studies provide very limited guidance. Here we build a new adaptive streaming video database and carry out a series of subjective experiments to understand human QoE behaviors in this multi-dimensional adaptation space. Our study leads to several useful findings. First, our path-analytic results show that quality deviation introduced by quality adaptation is asymmetric with respect to the adaptation direction (positive or negative), and is further influenced by the intensity of quality change (intensity), dimension of adaptation (type), intrinsic video quality (level), content, and the interactions between them. Second, we find that for the same intensity of quality adaptation, a positive adaptation occurred in the low-quality range has more impact on QoE, suggesting an interesting Weber's law effect; while such phenomenon is reversed for a negative adaptation. Third, existing objective video quality assessment models are very limited in predicting time-varying video quality. The video database together with the subjective data will be made available to the public.
@inproceedings{duanmu2017qoe,
author = {Duanmu, Zhengfang and Ma, Kede and Wang, Zhou},
title = {Quality-of-Experience of Adaptive Video Streaming: Exploring the Space of Adaptations},
booktitle = {ACM Multimedia},
pages={1752--1760},
year = {2017}
}
The Waterloo Streaming Quality-of-Experience Database-II consists of 12 RAW HD reference videos covering diverse content. An 8-second video segment is extracted from each source video, which is further partitioned into two non-overlapping 4-second segments, referred to as short segments (SS). We encode SS into 7 representations with H.264 encoder using three compression levels, spatial resolutions, and frame rates. To simulate quality adaptation events in adaptive streaming, we concatenated two consecutive 4-second segments with different representations from the same content into an 8-second long segment (LS). In total, we obtain 168 SS and 588 LS. The study involved 36 subjects. Subjects score the quality of each video sequence according to the eleven-grade 0-10 numerical quality scale. The released data is linearly scaled to 0-100.
We carry out three subjective experiments as illustrated in Figure above. Subjects are invited to rate the quality of SS in Experiment I. The subjective rating of each SS is defined as the intrinsic quality. We perform Experiment II on LS, wherein subjects give two opinions (post-hoc quality) to the first and second 4-second video segments (referred to as SS-I and SS-II, respectively). An audio stimulus is introduced in the middle of each LS, indicating the end of SS-I and the start of SS-II. In Experiment III, subjects are requested to watch the LS but to provide a single score to reflect their overall QoE.
Rights to all videos are retained by the photographers. For researchers and educators who wish to use the videos for non-commercial research and/or educational purposes, we can provide access under the following terms:
- Researcher shall use the Database only for non-commercial research and educational purposes.
- University of Waterloo makes no representations or warranties regarding the Database, including but not limited to warranties of non-infringement or fitness for a particular purpose.
- Researcher accepts full responsibility for his or her use of the Database and shall defend and indemnify University of Waterloo, including their employees, Trustees, officers and agents, against any and all claims arising from Researcher's use of the Database, including but not limited to Researcher's use of any copies of copyrighted images that he or she may create from the Database.
- Researcher may provide research associates and colleagues with access to the Database provided that they first agree to be bound by these terms and conditions.
- University of Waterloo reserves the right to terminate Researcher's access to the Database at any time.
- If Researcher is employed by a for-profit, commercial entity, Researcher's employer shall also be bound by these terms and conditions, and Researcher hereby represents that he or she is fully authorized to enter into this agreement on behalf of such employer.