Exploring the Dynamic Interplay of Cognitive Load and Emotional Arousal by Using Multimodal Measurements: Correlation of Pupil Diameter and Emotional Arousal in Emotionally Engaging Tasks
Main Article Content
Abstract
Multimodal data analysis and validation based on streams from state-of-the-art sensor technology, such as eye-tracking or emotion recognition using the Facial Action Coding System (FACS) with deep learning, allows educational researchers to study multifaceted learning and problem-solving processes and to improve educational experiences. This study aims to investigate the correlation between two continuous sensor streams—pupil diameter as an indicator of cognitive workload and FACS with deep learning as an indicator of emotional arousal (RQ1a)—specifically for epochs of high, medium, and low arousal (RQ1b). Furthermore, the time lag between emotional arousal and pupil diameter data will be analyzed (RQ2). A total of 28 participants worked on three cognitively demanding and emotionally engaging everyday moral dilemmas while eye-tracking and emotion recognition data were collected. The data were preprocessed in Phyton (synchronization, blink control, and downsampling) and analyzed using correlation analysis and Granger causality tests. The results show negative and statistically significant correlations between the data streams for emotional arousal and pupil diameter. However, the correlation is negative and significant only for epochs of high arousal, while positive but nonsignificant relationships were found for epochs of medium or low arousal. The average time lag for the relationship between arousal and pupil diameter was 2.8 ms. In contrast to previous findings without a multimodal approach suggesting a positive correlation between the constructs, the results contribute to the state of research by highlighting the importance of multimodal data validation and convergent validity. Future research should consider emotional regulation strategies and emotional valence.
Article Details
FLR adopts the Attribution-NonCommercial-NoDerivs Creative Common License (BY-NC-ND). That is, Copyright for articles published in this journal is retained by the authors with, however, first publication rights granted to the journal. By virtue of their appearance in this open access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.
References
Abdallah, T. B., Elleuch, I., & Guermazi, R. (2021). Student behavior recognition in classroom using deep transfer learning with VGG-16. Procedia Computer Science, 192, 951–960. https://doi.org/10.1016/j.procs.2021.08.098
Alshanskaia, E. I., Portnova, G. V., Liaukovich, K. & Martynova, O. V. (2024). Pupillometry and autonomic nervous system responses to cognitive load and false feedback: an unsupervised machine learning approach. Frontiers in Neuroscience, 18. https://doi.org/10.3389/fnins.2024.1445697
Al-Elq, A. H. (2010). Simulation-based medical teaching and learning. Journal of Family and Community Medicine, 17(1), 35. https://doi.org/10.4103/1319-1683.68787
Antonenko, P. D., Paas, F., Grabner, R. H. & Van Gog, T. (2010). Using electro-encephalography to measure cognitive load. Educational Psychology Review, 22(4), 425–438. https://doi.org/10.1007/s10648-010-9130-y
Alshehri, M. & Alghowinem, S. (2013). An exploratory study of detecting emotion states using eye-tracking technology. Proceedings of 2013 Science and Information Conference.
Azevedo, R. & Gašević, D. (2019). Analyzing Multimodal Multichannel Data about Self-Regulated Learning with Advanced Learning Technologies: Issues and Challenges. Computers in Human Behavior, 96, 207–210. https://doi.org/10.1016/j.chb.2019.03.025
Azevedo, R., Taub, M., & Mudrick, N. V. (2018). Understanding and reasoning about real-time cognitive, affective, and metacognitive processes to foster self-regulation with advanced learning technologies. In D. H. Schunk & J. A. Greene (Eds.), Handbook of self-regulation of learning and performance (2nd ed., pp. 254–270). Routledge/Taylor & Francis Group. https://doi.org/10.4324/9781315697048-17
Babiker, A., Faye, I., & Malik, A. (2013). Pupillary behavior in positive and negative emotions. 2013 IEEE International Conference on Signal and Image Processing Applications. https://doi.org/10.1109/icsipa.2013.6708037
Barnett, L., Barrett, A. B., & Seth, A. K. (2009). Granger Causality and Transfer Entropy Are Equivalent for Gaussian Variables. Physical Review Letters, 103(23). https://doi.org/10.1103/physrevlett.103.238701
Beatty, J., & Lucero-Wagoner, B. (2000). The pupillar system. In C. JT, T. LG, & B. GG (Eds.), Handbook of Psychophysiology (pp. 142– 162). Cambridge, UK: Cambridge University Press
Bebko, G. M., Franconeri, S. L., Ochsner, K. N., & Chiao, J. Y. (2011). Look before you regulate: Differential perceptual strategies underlying expressive suppression and cognitive reappraisal. Emotion, 11(4), 732–742. https://doi.org/10.1037/a0024009
Bhattacharyya, A., Chatterjee, S., Sen, S., Sinitca, A. M., Kaplun, D. & Sarkar, R. (2021). A deep learning model for classifying human facial expressions from infrared thermal images. Scientific Reports, 11(1). https://doi.org/10.1038/s41598-021-99998-z
Bradley, M. M., Miccoli, L., Escrig, M. A., & Lang, P. J. (2008). The pupil as a measure of emotional arousal and autonomic activation. Psychophysiology, 45(4), 602–607. https://doi.org/10.1111/j.1469-8986.2008.00654.x
Bauman, C. W., McGraw, A. P., Bartels, D. M., & Warren, C. (2014). Revisiting External Validity: Concerns about Trolley Problems and Other Sacrificial Dilemmas in Moral Psychology. Social and Personality Psychology Compass, 8(9), 536–554. https://doi.org/10.1111/spc3.12131
Chanel, C., Wilson, M. D., & Scannella, S. (2019). Online ECG-based Features for Cognitive Load Assessment. Open Archive Toulouse Archive Ouverte (University of Toulouse). https://doi.org/10.1109/smc.2019.8914002
Chernikova, O., Heitzmann, N., Stadler, M., Holzberger, D., Seidel, T. & Fischer, F. (2020). Simulation-Based Learning in Higher Education: A Meta-Analysis. Review of Educational Research, 90(4), 499–541. https://doi.org/10.3102/0034654320933544
Chen, S. and Epps, J. (2013) Automatic Classification of Eye Activity for Cognitive Load Measurement with Emotion Interference. Computer Methods and Programs in Biomedicine, 110, 111-124. https://doi.org/10.1016/j.cmpb.2012.10.021
Child, S., Oakhill, J., & Garnham, A. (2020). Tracking your emotions: An eye-tracking study on reader’s engagement with perspective during text comprehension. Quarterly Journal of Experimental Psychology, https://doi.org/10.1177/1747021820905561
Christensen, J. F. & Gomila, A. (2012). Moral dilemmas in cognitive neuroscience of moral decision-making: A principled review. Neuroscience & Biobehavioral Reviews, 36(4), 1249–1264. https://doi.org/10.1016/j.neubiorev.2012.02.008
Cloude, E. B., Azevedo, R., Winne, P. H., Biswas, G., & Jang, E. E. (2022). System design for using multimodal trace data in modeling self-regulated learning. Frontiers in Education, 7. https://doi.org/10.3389/feduc.2022.928632
de Berker, A. O., Tirole, M., Rutledge, R. B., Cross, G. F., Dolan, R. J., & Bestmann, S. (2016). Acute stress selectively impairs learning to act. Scientific Reports, 6(1). https://doi.org/10.1038/srep29816
Delliaux, S., Delaforge, A., Deharo, J. & Chaumet, G. (2019). Mental workload alters heart rate variability, lowering non-linear dynamics. Frontiers in Physiology, 10. https://doi.org/10.3389/fphys.2019.00565
Cloude, E. B., Azevedo, R., Winne, P. H., Biswas, G. & Jang, E. E. (2022). System design for using multimodal trace data in modeling self-regulated learning. Frontiers in Education, 7. https://doi.org/10.3389/feduc.2022.928632
Dindar, M., Järvelä, S., Ahola, S., Huang, X. & Zhao, G. (2022). Leaders and Followers Identified by Emotional Mimicry During Collaborative Learning: A Facial Expression Recognition Study on Emotional Valence. IEEE Transactions On Affective Computing, 13(3), 1390–1400. https://doi.org/10.1109/taffc.2020.3003243
Dubovi, I. (2022). Cognitive and emotional engagement while learning with VR: The perspective of multimodal methodology. Computers & Education, 183, 104495. https://doi.org/10.1016/j.compedu.2022.104495
Dyer, S. A. & Dyer, J. S. (2001). Cubic-spline interpolation. IEEE Instrumentation & Measurement Magazine, 4(1), 44–46. https://doi.org/10.1109/5289.911175
Elahi, E., & Islam, D. (2019). Galvanic Skin Response signal based Cognitive Load classification using Machine Learning classifier. https://doi.org/10.1109/icecte48615.2019.9303564
Ez-zaouia, M. & Lavoué, E. (2018). EMODA: a Tutor Oriented Multimodal and Contextual Emotional Dashboard. LAK '17: Proceedings of the Seventh International Learning Analytics & Knowledge Conference. https://doi.org/10.1145/3027385.3027434
Fiedler, S., Glöckner, A., Dickert, S. & Nicklisch, A. (2013). Social Value Orientation and information search in social dilemmas: An eye-tracking analysis. Organizational Behavior And Human Decision Processes, 120(2), 272–284. https://doi.org/10.1016/j.obhdp.2012.07.002
Fink, L., Simola, J., Tavano, A., Lange, E. B., Wallot, S. & Laeng, B. (2023). From pre-processing to advanced dynamic modeling of pupil data. Behavior Research Methods. https://doi.org/10.3758/s13428-023-02098-1
Fisher, C.D. & Ashkanasy, N.M. (2000). The emerging role of emotions in work life: an introduction. Journal of Organizational Behavior, 21 (2), 123-129.
Frei-Landau, R. & Levin, O. (2023). Simulation-based learning in teacher Education: Using Maslow’s hierarchy of needs to conceptualize instructors’ needs. Frontiers in Psychology, 14. https://doi.org/10.3389/fpsyg.2023.1149576
Fritz, T., Begel, A., Müller, S., Yiǧit-Elliott, S. & Züger, M. (2014). Using psycho-physiological measures to assess task difficulty in software development. Proceedings of the 36th International Conference on Software Engineering. https://doi.org/10.1145/2568225.2568266
Ghaffari, M. & Fiedler, S. (2018). The Power of Attention: Using Eye Gaze to Predict Other-Regarding and Moral Choices. Psychological Science, 29(11), 1878–1889. https://doi.org/10.1177/0956797618799301
Gagné, R. M., Wager, W. W., Golas, K. & Keller, J. M. (2005). Principles of instructional design. Cengage Learning.
Gao, H. & Ma, B. (2020). A robust improved network for facial expression recognition. Frontiers in signal processing, 4(4). https://doi.org/10.22606/fsp.2020.44001
Garon, M., Lavallée, M. M., Estay, E. V. & Beauchamp, M. H. (2018). Visual encoding of social cues predicts sociomoral reasoning. PLOS ONE, 13(7). https://doi.org/10.1371/journal.pone.0201099
Gleichgerrcht, E. & Young, L. (2013). Low Levels of Empathic Concern Predict Utilitarian Moral Judgment. PLOS ONE, 8(4). https://doi.org/10.1371/journal.pone.0060418
Granger, C. W. J. (1969). Investigating causal relations by econometric models and cross-spectral methods. Econometrica, 37(3), 424. https://doi.org/10.2307/1912791
Greene, J. D., Cushman, F. A., Stewart, L. E., Lowenberg, K., Nystrom, L. E., & Cohen, J. D. (2009). Pushing moral buttons: the interaction between personal force and intention in moral judgment. Cognition, 111(3), 364–371. https://doi.org/10.1016/j.cognition.2009.02.001
Grassmann, M., Vlemincx, E., Von Leupoldt, A., Mittelstädt, J. & Van Den Bergh, O. (2016). Respiratory Changes in Response to Cognitive Load: A Systematic review. Neural Plasticity, 2016, 1–16. https://doi.org/10.1155/2016/8146809
Harley, J. M., Bouchet, F., Hussain, S., Azevedo, R., & Calvo, R. (2014). A multi-componential analysis of emotions during complex learning with an intelligent multi-agent system. Paper to be presented at a symposium on Interdisciplinary Approaches for Analyzing Data from Multiple Affective Channels with Computer-Based Learning Environments at the 2014 annual meeting of the American Educational Research Association, Philadelphia, PA.
Harris, C. R., Millman, K. J., Van Der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., Kern, R., Picus, M., Hoyer, S., Van Kerkwijk, M. H., Brett, M., Haldane, A., Del Río, J. F., Wiebe, M., Peterson, P., . . . Oliphant, T. E. (2020). Array programming with NumPy. Nature, 585(7825), 357–362. https://doi.org/10.1038/s41586-020-2649-2
Hess, E. H., & Polt, J. M. (1960). Pupil size as related to interest value of visual stimuli. Science, 132, 349–350. https://doi.org/10.1126/science.132.3423.34
Hollander, J. & Huette, S. (2022). Extracting blinks from continuous eye-tracking data in a mind wandering paradigm. Consciousness and Cognition, 100, 103303. https://doi.org/10.1016/j.concog.2022.103303
Horvers, A., Tombeng, N., Bosse, T., Lazonder, A. W. & Molenaar, I. (2021). Detecting Emotions through electrodermal Activity in learning Contexts: A Systematic review. Sensors, 21(23), 7869. https://doi.org/10.3390/s21237869
Huette, S. (2016). Blink durations reflect mind wandering during reading. https://pdfs.semanticscholar.org/fba2/314dea3944723ddc0348dc41a6e823dd5410.pdf
Hulshof, C. D. (2005). Log File Analysis. Encyclopedia of Social Measurement, 577–583. https://doi.org/10.1016/b0-12-369398-5/00509-0
Janning, R., Schatten, C. & Schmidt-Thieme, L. (2016). Perceived Task-Difficulty recognition from log-file information for the use in adaptive intelligent tutoring systems. International Journal of Artificial Intelligence in Education, 26(3), 855–876. https://doi.org/10.1007/s40593-016-0097-9
Jerčić, P., Sennersten, C. & Lindley, C. A. (2018). Modeling cognitive load and physiological arousal through pupil diameter and heart rate. Multimedia Tools and Applications, 79(5–6), 3145–3159. https://doi.org/10.1007/s11042-018-6518-z
Kahane, G. (2015). Sidetracked by trolleys: Why sacrificial moral dilemmas tell us little (or nothing) about utilitarian judgment. Social Neuroscience, 10(5), 551–560. https://doi.org/10.1080/17470919.2015.1023400
Karch, J. M., Valles, J. C. G. & Sevian, H. (2019). Looking into the Black Box: Using Gaze and Pupillometric Data to Probe How Cognitive Load Changes with Mental Tasks. Journal Of Chemical Education, 96(5), 830–840. https://doi.org/10.1021/acs.jchemed.9b00014
Kinner, V. L., Kuchinke, L., Dierolf, A. M., Merz, C. J., Otto, T. & Wolf, O. T. (2017). What our eyes tell us about feelings: Tracking pupillary responses during emotion regulation processes. Psychophysiology, 54(4), 508–518. https://doi.org/10.1111/psyp.12816
Kulke, L., Feyerabend, D. & Schacht, A. (2020). A Comparison of the Affectiva iMotions Facial Expression Analysis Software With EMG for Identifying Facial Expressions of Emotion. Frontiers in Psychology, 11. https://doi.org/10.3389/fpsyg.2020.00329
Kret, M. E. & Sjak-Shie, E. E. (2018). Preprocessing pupil size data: guidelines and code. Behavior Research Methods, 51(3), 1336–1342. https://doi.org/10.3758/s13428-018-1075-y
Landmann, E. (2023). I can see how you feel—Methodological considerations and handling of Noldus’s FaceReader software for emotion measurement. Technological Forecasting And Social Change, 197, 122889. https://doi.org/10.1016/j.techfore.2023.122889
LeDoux, J. E. (2021). As soon as there was life, there was danger: the deep history of survival behaviours and the shallower history of consciousness. Philosophical Transactions of the Royal Society B, 377(1844). https://doi.org/10.1098/rstb.2021.0292
Liu, Z. (2004). Measuring the degree of synchronization from time series data. Europhysics Letters (EPL), 68(1), 19–25. https://doi.org/10.1209/epl/i2004-10173-x
Löwenstein, O. (1920). Experimentelle Beiträge zur Lehre von den katatonischen Pupillenveränderungen. European Neurology, 47(4), 194–215. https://doi.org/10.1159/000190690
Loijens & Krips (n.d.), Face reader – Methodology Note. Wageningen, The Netherlands: Noldus Information Technology.
Lu, J., Hallinger, P. & Showanasai, P. (2014). Simulation-based learning in management education. Journal of Management Development, 33(3), 218–244. https://doi.org/10.1108/jmd-11-2011-0115
Mallick, R., Slayback, D., Touryan, J., Ries, A. J. & Lance, B. J. (2016). The use of eye metrics to index cognitive workload in video games. 2016 IEEE Conference On Eye Tracking And Visualization (ETVIS), 60–64. https://doi.org/10.1109/etvis.2016.7851168
Malmberg, J., Haataja, E., Seppänen, T. & Järvelä, S. (2019). Are we together or not? The temporal interplay of monitoring, physiological arousal and physiological synchrony during a collaborative exam. International Journal of Computer-Supported Collaborative Learning, 14(4), 467–490. https://doi.org/10.1007/s11412-019-09311-4
Martin, A. J., Ginns, P., Burns, E., Kennett, R., Munro‐Smith, V., Collie, R. J. & Pearson, J. (2021). Assessing instructional cognitive load in the context of students’ psychological challenge and threat orientations: A Multi-Level Latent Profile Analysis of students and Classrooms. Frontiers in Psychology, 12. https://doi.org/10.3389/fpsyg.2021.656994
Mathôt, S. (2018). Pupillometry: Psychology, Physiology, and Function. Journal Of Cognition, 1(1). https://doi.org/10.5334/joc.18
Mauss, I. B. & Robinson, M. D. (2009). Measures of Emotion: A review. Cognition & Emotion, 23(2), 209–237. https://doi.org/10.1080/02699930802204677
Mayer, C., Rausch, A. & Seifried, J. (2023). Analysing domain-specific problem-solving processes within authentic computer-based learning and training environments by using eye-tracking: a scoping review. Empirical Research in Vocational Education And Training, 15(1). https://doi.org/10.1186/s40461-023-00140-2
McGaugh, J. L. (2004). The amygdala modulates the consulidation of memories of emotionally arousing experiences. Annual Review Of Neuroscience, 27(1), 1–28. https://doi.org/10.1146/annurev.neuro.27.070203.144157
McKinney, W. (2010). Data structures for statistical computing in Python. Proceedings of the Python in Science Conferences. https://doi.org/10.25080/majora-92bf1922-00a
Mehrotra, S. & Gunalakshmi, K. (2020). Impact of intrinsic cognitive load and extraneous cognitive load over emotions. International journal of scientific and research publications, 10(06), 318–328. https://doi.org/10.29322/ijsrp.10.06.2020.p10237
Mu, S., Cui, M. & Huang, X. (2020). Multimodal Data Fusion in Learning Analytics: A Systematic review. Sensors, 20(23), 6856. https://doi.org/10.3390/s20236856
Noldus (2021). FaceReader™ 8: Tool for automatic analysis of facial expressions. Wageningen, The Netherlands: Noldus Information Technology.
Noroozi, O., Alikhani, I., Järvelä, S., Kirschner, P. A., Juuso, I. & Seppänen, T. (2019). Multimodal data to design visual learning analytics for understanding regulation of learning. Computers in Human Behavior, 100, 298–304. https://doi.org/10.1016/j.chb.2018.12.019
Noroozi, O., Pijeira‐Díaz, H. J., Sobocinski, M., Dindar, M., Järvelä, S. & Kirschner, P. A. (2020). Multimodal data indicators for capturing cognitive, motivational, and emotional learning processes: A systematic literature review. Education And Information Technologies, 25(6), 5499–5547. https://doi.org/10.1007/s10639-020-10229-w
Oliva, M., & Anikin, A. (2018). Pupil dilation reflects the time course of emotion recognition in human vocalizations. Scientific Reports, 8(1), 1–10. https://doi.org/10.1038/s41598-018-23265-x
Olszanowski, M., Pochwatko, G., Kuklinski, K., Scibor-Rylski, M., & Ohme, R. (2008, June). Warsaw set of emotional facial expression pictures validation study. Opatija, Croatia: EAESP General Meeting
Paas, F. & Ayres, P. (2014). Cognitive Load Theory: A Broader view on the role of memory in learning and education. Educational Psychology Review, 26(2), 191–195. https://doi.org/10.1007/s10648-014-9263-5
Paas, F., Van Merriënboer, J. J. G. & Adam, J. J. (1994). Measurement of cognitive load in instructional research. Perceptual and Motor Skills, 79(1), 419–430. https://doi.org/10.2466/pms.1994.79.1.419
Pärnamets, P., Johansson, P., Häll, L., Balkenius, C., Spivey, M. J. & Richardson, D. C. (2015). Biasing moral decisions by exploiting the dynamics of eye gaze. Proceedings Of The National Academy Of Sciences Of The United States Of America, 112(13), 4170–4175. https://doi.org/10.1073/pnas.1415250112
Partala, T. & Surakka, V. (2003). Pupil size variation as an indication of affective processing. International Journal Of Human-Computer Studies, 59(1–2), 185–198. https://doi.org/10.1016/s1071-5819(03)00017-x
Paulus, P. C., Dabas, A., Felber, A. & Benoit, R. G. (2022). Simulation-based learning influences real-life attitudes. Cognition, 227, 105202. https://doi.org/10.1016/j.cognition.2022.105202
Pekrun, R. (1992). The impact of emotions on learning and achievement: Towards a theory of cognitive/motivational mediators. Applied Psychology: An International Review, 41(4), 359–376.
Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review, 18(4), 315–341.
Podder, T., Bhattacharya, D., Majumder, P. & Balas, V. E. (2023). A feature boosted deep learning method for automatic facial expression recognition. PeerJ, 9, e1216. https://doi.org/10.7717/peerj-cs.1216
Rodemer, M., Karch, J. M. & Bernholt, S. (2023). Pupil dilation as cognitive load measure in instructional videos on complex chemical representations. Frontiers in Education, 8. https://doi.org/10.3389/feduc.2023.1062053
Sailer, M., Bauer, E., Hofmann, R., Kiesewetter, J., Glas, J., Gurevych, I. & Fischer, F. (2023). Adaptive feedback from artificial neural networks facilitates pre-service teachers’ diagnostic reasoning in simulation-based learning. Learning and Instruction, 83, 101620. https://doi.org/10.1016/j.learninstruc.2022.101620
Schmidt-Weigand, F., Kohnert, A. & Glowalla, U. (2010). A closer look at split visual attention in system- and self-paced instruction in multimedia learning. Learning and Instruction, 20(2), 100–110. https://doi.org/10.1016/j.learninstruc.2009.02.011
Skiendziel, T., Rösch, A. G. & Schultheiss, O. C. (2019). Assessing the convergent validity between the automated emotion recognition software Noldus FaceReader 7 and Facial Action Coding System Scoring. PLoS ONE, 14(10), e0223905. https://doi.org/10.1371/journal.pone.0223905
Schmidt-Weigand, F. & Scheiter, K. (2011). Cognitive Load Questionnaire. PsycTESTS Dataset. https://doi.org/10.1037/t12856-000
Samuelsen, J., Chen, E. & Wasson, B. (2019). Integrating multiple data sources for learning analytics – review of literature. Research and Practice in Technology Enhanced Learning, 14(11). https://doi.org/10.1186/s41039-019-0105-4
Schnaubert, L. & Schneider, S. (2022). Analysing the relationship between mental load or mental effort and metacomprehension under different conditions of multimedia design. Frontiers in Education, 6. https://doi.org/10.3389/feduc.2021.648319
Sherwood, J. J. (1965). A relation between arousal and performance. American Journal of Psychology, 78(3), 461. https://doi.org/10.2307/1420581
Shojaie, A. & Fox, E. B. (2022). Granger Causality: A Review and Recent Advances. Annual Review Of Statistics And Its Application, 9(1), 289–319. https://doi.org/10.1146/annurev-statistics-040120-010930
Šimić, G., Tkalčić, M., Vukić, V., Mulc, D., Španić, E., Šagud, M., Olucha-Bordonau, F. E., Vukšić, M. & Hof, P. R. (2021). Understanding Emotions: Origins and Roles of the Amygdala. Biomolecules, 11(6), 823. https://doi.org/10.3390/biom11060823
Song, Z. (2021). Facial Expression Emotion Recognition model integrating philosophy and machine learning theory. Frontiers in Psychology, 12. https://doi.org/10.3389/fpsyg.2021.759485
Sörqvist, P., Dahlström, Ö., Karlsson, T. & Rönnberg, J. (2016). Concentration: the neural underpinnings of how cognitive load shields against distraction. Frontiers in Human Neuroscience, 10. https://doi.org/10.3389/fnhum.2016.00221
Su, Y., Lin, Y. & Liu, T. (2022). Applying machine learning technologies to explore students’ learning features and performance prediction. Frontiers in Neuroscience, 16. https://doi.org/10.3389/fnins.2022.1018005
Stanners, R. F., Coulter, I. M., Sweet, A. W. & Murphy, P. (1979). The pupillary response as an indicator of arousal and cognition. Motivation And Emotion, 3(4), 319–340. https://doi.org/10.1007/bf00994048
Sweller, J., Van Merriënboer, J. J. G. & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251–296. https://doi.org/10.1023/a:1022193728205
Thompson, R. J. & Berenbaum, H. (2006). Shame Reactions to Everyday Dilemmas are Associated with Depressive Disorder. Cognitive Therapy And Research, 30(4), 415–425. https://doi.org/10.1007/s10608-006-9056-3
Tobii Pro AB (2014). Tobii Pro Lab User Manual [Apparatus and software]. Tobii Pro AB, Danderyd, Sweden.
Voigt, P., & von dem Bussche, A. (2017). The EU General Data Protection Regulation (GDPR). Springer International Publishing. https://doi.org/10.1007/978-3-319-57959-7
Tyng, C. M., Amin, H. U., Saad, M. N. M. & Malik, A. S. (2017). The influences of emotion on learning and memory. Frontiers in Psychology, 8. https://doi.org/10.3389/fpsyg.2017.01454
Valdesolo, P. & DeSteno, D. (2006). Manipulations of Emotional Context Shape Moral Judgment. Psychological Science, 17(6), 476–477. https://doi.org/10.1111/j.1467-9280.2006.01731.x
Van Acker, B. B., Parmentier, D., Vlerick, P. & Saldien, J. (2018). Understanding mental workload: from a clarifying concept analysis toward an implementable framework. Cognition, Technology & Work, 20(3), 351–365. https://doi.org/10.1007/s10111-018-0481-3
van der Schalk, J., Hawk, S. T., Fischer, A. H., & Doosje, B. (2011). Moving faces, looking places: Validation of the Amsterdam Dynamic Facial Expression Set (ADFES). Emotion, 11, 907–920. http://dx.doi.org/10.1037/a0023853
Van Der Wel, P. & Van Steenbergen, H. (2018). Pupil dilation as an index of effort in Cognitive control Tasks: a review. Psychonomic Bulletin & Review, 25(6), 2005–2015. https://doi.org/10.3758/s13423-018-1432-y
Vanneste, P., Raes, A., Morton, J., Bombeke, K., Van Acker, B. B., Larmuseau, C., Depaepe, F. & Van Den Noortgate, W. (2020). Towards measuring cognitive load through multimodal physiological data. Cognition, Technology & Work, 23(3), 567–585. https://doi.org/10.1007/s10111-020-00641-0
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., Van Der Walt, S. J., Brett, M., Wilson, J., Millman, K. J., Mayorov, N., Nelson, A., Jones, E. D., Kern, R., Larson, E. B., . . . Vázquez-Baeza, Y. (2020). SCIPY 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 17(3), 261–272. https://doi.org/10.1038/s41592-019-0686-2
Yerkes, R. M. & Dodson, J. (1908). The relation of strength of stimulus to rapidity of habit-formation. The Journal of Comparative Neurology, 18(5), 459–482. https://doi.org/10.1002/cne.920180503
Yoo, G., Kim, H. & Hong, S. (2023). Prediction of cognitive load from electroencephalography signals using Long Short-Term Memory Network. Bioengineering, 10(3), 361. https://doi.org/10.3390/bioengineering10030361
Young, J. Q., Van Merriënboer, J., Durning, S. & Cate, O. T. (2014). Cognitive Load Theory: Implications for Medical Education: AMEE Guide No. 86. Medical Teacher, 36(5), 371–384. https://doi.org/10.3109/0142159x.2014.889290
Zandi, B., Lode, M., Herzog, A. G., Sakas, G. & Khanh, T. Q. (2021). PupilEXT: flexible Open-Source platform for High-Resolution Pupillometry in vision Research. Frontiers in Neuroscience, 15. https://doi.org/10.3389/fnins.2021.676220
Zimmerman, P., Bolhuis, J., Willemsen, A., Meyer, E. S. & Noldus, L. (2009). The Observer XT: a tool for the integration and synchronization of multimodal signals. Behavior Research Methods, 41(3), 731–735. https://doi.org/10.3758/brm.41.3.731