Budapest University of Technology and Economics, Faculty of Electrical Engineering and Informatics

Az előadók eléréséhez szükséges adatok a tanszék honlapján megtalálhatók:

The goal of the lectures is to enable an in-depth understanding of the complexities of Quality of Experience (QoE), from the perspectives of both the users, the service providers, the device manufacturers and the content providers. By completing the course, the student shall be able to:

• approach the notion of QoE from an engineer's point of view,
• extract relevant information from the available standards and recommendations,
• employ existing objective measurement metrics,
• design novel objective measurement metrics,
• take cognitive bias into account during the configuration of experimental setups, and
• design and carry out QoE studies.

• Week 1 Lecture 1: Introduction
• Fundamental definitions, terminologies, the presence of QoE in the modern world and its importance.
• Week 1 Lecture 2: Introduction (continued)
• User expectations, user satisfaction, usability, and quality models. Recent case studies of success and failure with regards to QoE.
• Week 2 Lecture 3: QoE in its historical context
• The historical overview of QoE, the first services where QoE was relevant, its first appearance as a field of research, the emergence of Key Performance Indicators (KPIs).
• Week 2 Lecture 4: QoE in modern contexts
• The QoE of devices and services in the present day. The perspectives of manufacturers and providers.
• Week 3 Lecture 5: Subjective test methodology
• Assessment tasks, test participant activity (i.e., passives observer or active participant), rating scales, pre-experiment questionnaires, post-experiment questionnaires.
• Week 3 Lecture 6: Subjective test methodology (continued)
• Pre-test training phase, form of consent, viewing conditions, demographic distributions.
• Week 4 Lecture 7: Objective metrics
• Metric classification, general objective metrics (can be applied to any technology).
• Week 4 Lecture 8: Objective metrics (continued)
• Technology-specific objective metrics (can only be applied to specific technologies). Objective metric training and validation.
• Week 5 Lecture 9: Standardization
• Standardization bodies and working groups (IEEE, ISO, ITU, ETSI, VQEG).
• Week 5 Lecture 10: Standardization (continued)
• The most relevant QoE standards in force.
• Week 6 Lecture 11: Designing and carrying out QoE research
• Research questions, test environments and their properties (e.g., lighting conditions, sound insulation etc.), individual and group tests, crowdsourcing, devices, experimental validity.
• Week 6 Lecture 12: Designing and carrying out QoE research (continued)
• Test conditions, source contents, reference quality, test protocol, data collection, participant anonymity, data handling, data processing, experimental errors and unexpected events.
• Week 7 Lecture 13: Cognitive bias
• Cognitive dissonance, confirmation bias, misinformation effect, framing effect, labeling effect, loss aversion, anchoring.
• Week 7 Lecture 14: Cognitive bias (continued)
• Analysis of the most relevant publications on the presence of cognitive bias in the consumer market and in QoE studies.
• Week 8 Lecture 15: Audio QoE
• Telecommunications QoE in general, listening tests, two-way conversations, transmission quality (delay, jitter and packet loss), audio quality of devices, stereo hearing, spatial sound.
• Week 8 Lecture 16: The Human Visual System
• Visual thresholds, Just Noticeable Difference (JND), stereo vision, color vision, perceived contrast, light sensitivity, case study on medical QoE, telemedicine and diagnostics accuracy.
• Week 9 Lecture 17: Visual and audiovisual QoE
• Conventional and novel technologies, QoE over time and adaptation, HVS-based perceptual coding.
• Week 9 Lecture 18: UHD and HDR QoE
• Display capabilities, image resolution, dynamic range, color depth.
• Week 10 Lecture 19: VR and AR QoE
• Near-eye technologies, visual endurance, fatigue, spatial depth, convergence, parallax effect.
• Week 10 Lecture 20: Gaming QoE
• Casual gaming, competitive gaming, online gaming, gaming devices and visualization (e.g., split-screen gaming).
• Week 11 Lecture 21: Light field QoE
• Light field visualization fundamentals, spatial resolution, angular resolution, field of view, horizontal and vertical parallax.
• Week 11 Lecture 22: Light field QoE (continued)
• The state-of-the-art light field displays, services enabled by light field and their requirements, use cases in development, super resolution.
• Week 12 Lecture 23: Case studies
• In-depth analysis of major publications and research efforts within the field of QoE.
• Week 12 Lecture 24: Case studies (continued)
• In-depth analysis of major publications and research efforts within the field of QoE.
• Week 13 Lecture 25: The future of QoE
• The systems and services of the future, new capabilities of existing technologies, content diversity, QoE adaptation over time.
• Week 13 Lecture 26: The future of QoE (continued)
• QoE estimation and modeling based on machine learning.
• Week 14 Lecture 27: Presentations
• The students present their assignments.
• Week 14 Lecture 28: Presentations
• The students present their assignments.

4 hours (2 × 2 hours) of lecture per week.

• 1 mid-term exam one on week 7, covering the topics of weeks 1 to 6
• Written assignment delivered by week 13 and oral presentation (based on the selected topic of the assignment) on week 14, during the scheduled times of the lectures. The assignment can be done either individually or in pairs, but in case of the latter, the individual contributions must be clarified and both students must participate in the presentation.
• 1 written exam at the end of the semester, covering the topics of weeks 1 to 13

The mid-term exam can be retaken on week 15. The deadline for the late submission of the assignment is week 15, and the same week provides the opportunity for late presentations.

Prior to the exams, and also any time during the semester, based on demand.

Scientific literature:

• K. Brunnström, S. A. Beker, K. De Moor, A. Dooms, S. Egger, M.-N. Garcia, T. Hossfeld, S. Jumisko-Pyykkö, C. Keimel, M.-C. Larabi et al., "Qualinet White Paper on Definitions of Quality of Experience," 2013.
• A. Sackl, K. Masuch, S. Egger, and R. Schatz, "Wireless vs. wireline shootout: How user expectations influence Quality of Experience," in Fourth International Workshop on Quality of Multimedia Experience (QoMEX). IEEE, 2012, pp. 148-149.
• S. Y. Rieh and N. J. Belkin, "Understanding judgment of information quality and cognitive authority in the WWW," in Proceedings of the 61^st Annual Meeting of the American Society for Information Science, vol. 35, 1998, pp. 279-289.
• K. Lamm, T. Mandl, C. Womser-Hacker, and W. Greve, "User experiments with search services: Methodological challenges for measuring the perceived quality," in 3^rd Workshop on Perceptual Quality of Systems (PQS), 2010.
• M. Narwaria, M. P. Da Silva, and P. Le Callet, "High dynamic range visual quality of experience measurement: Challenges and perspectives," in Visual Signal Quality Assessment. Springer, 2015, pp. 129-155.
• U. Engelke, D. P. Darcy, G. H. Mulliken, S. Bosse, M. G. Martini, S. Arndt, J.-N. Antons, K. Y. Chan, N. Ramzan, and K. Brunnström, "Psychophysiology-based QoE assessment: a survey," IEEE Journal of Selected Topics in Signal Processing, vol. 11, no. 1, pp. 6-21, 2017.
• N. Staelens, S. Moens, W. Van den Broeck, I. Marien, B. Vermeulen, P. Lambert, R. Van de Walle, and P. Demeester, "Assessing quality of experience of IPTV and video on demand services in real-life environments," IEEE Transactions on broadcasting, vol. 56, no. 4, pp. 458-466, 2010.
• S. Bouchard, S. Dumoulin, J. Talbot, A.-A. Ledoux, J. Phillips, J. Monthuy-Blanc, G. Labonté-Chartrand, G. Robillard, M. Cantamesse, and P. Renaud, "Manipulating subjective realism and its impact on presence: Preliminary results on feasibility and neuroanatomical correlates," Interacting with Computers, vol. 24, no. 4, pp. 227-236, 2012.
• T. Balogh, Z. Nagy, P. T. Kovács, and V. K. Adhikarla, "Natural 3D content on glasses-free light-field 3D cinema," in IS&T/SPIE Electronic Imaging, 2013.
• P. T. Kovács, A. Boev, R. Bregovic, and A. Gotchev, "Quality measurements of 3D light-field displays," in Eighth International Workshop on Video Processing and Quality Metrics for Consumer Electronics (VPQM), 2014.
• P. A. Kara, W. Robitza, N. Pinter, A. Raake, M. G. Martini, A. Simon, "Comparison of HD and UHD video quality with and without the influence of the labeling effect" in Springer Quality and User Experience, vol. 4, no. 4, pp. 1-29, 2019.
• B. Szajna and R. W. Scamell, "The effects of information system user expectations on their performance and perceptions," MIS Quarterly, vol. 17, no. 4, pp. 493-516, 1993.