Project overview
WSI Pilot Project
Digitally mediated communication – via tools like Teams, Skype, Zoom – has, since COVID, become ubiquitous. Since lockdown, organisations have started to translate these “work from home” practices into policies to enable greater use of these tools. As research over the past several years notes, however, this significant socio-technical transformation in work, has also created new kinds of costs. These include longer hours working; “zoom fatigue” and most particularly, increased experiences of isolation [1], known effects on sense of social cohesion [2] and, for organisations, impact on team trust and efficiency [2].
Research looking at the differences between in person and virtually mediated visual communication has for decades looked at environmental factors such as what one can see or not of a person’s body language; how the audio affects perception; lack of breaks between meetings etc. What has not been considered are the more fundamental physiological/neurological differences between physical vs digital co-presence. Without such knowledge, we can only guess at best ways to mitigate negative effects of these approaches. With models of how these different modes affect us, fundamentally, we cannot design nor evaluate how well our tools are addressing these basic effects.
Our mission in this interdisciplinary pilot is to interrogate comparatively a range of physiological responses between virtually mediated vs in-person communication.
Our proposal is situated as fundamental, interdisciplinary work involving biosignal engineering (CT), psychiatry (NH) and human-systems interaction (mc). Our collaboration will bring together biomedical data science and signal processing with psychologically informed experimental design and interaction research to understand specific neurophysiological and cognitive attributes of face-to-face vs virtual communication, and how to translate them to improve the ‘humanness’ of online communication. The work will contribute to related, further interdisciplinary activities across campus and in related funding applications.
Our work is also situated within socio-technical systems (STS) research. STS research explores the complex interaction between social and technical aspects within a system, particularly within organisations. Central to STS theory, and integral to our project, is the notion that the design and performance of any system, such as online communication tools, can only be understood and enhanced when both ‘social’ and ‘technical’ aspects are considered together.
In a very brief preliminary pilot in our lab informing this proposal, we have already seen fundamental physiological differences between in-person and virtual interactions: increase in heart rate, heart rate variability, and increased head movements occur during in-person interactions. These differences confirm that this is a pathway worth further exploration for theorising the roles of these effects, and what happens in their absence for cognitive processes like attention, connection, memory.
What our pilot proposes is (1) to explore what are the ‘special’ features of in-person interaction vs virtually mediated interaction, and (2) to develop a preliminary model of these differences in order to propose (3) next-steps towards applying these models to interaction designs to improve the “humanness” of these tools – what we call “synthesizing co-presence."
Our approach involves both the design and validation of the study tasks for comparison between in person and mediated conditions. It also includes comprehensive analysis of data across various modalities, including Electroencephalogram (EEG), Galvanic Skin Response (GSR), Electrocardiogram (ECG), eye trackers, and breathing trackers1. Recent research has demonstrated that simply viewing faces in-person, as opposed to online, can induce significant differences in EEG, functional Near-Infrared Spectroscopy (fNIRS) activity, and pupil size [3]. Finally, it will develop and deliver an analysis of the data towards models for future interaction implementation and assessment. That is: while our work will offer fundamental insights about our human responses to these types of mediated contacts, our focus, as per STS research, is to move to translate these findings into actionable applications.
Digitally mediated communication – via tools like Teams, Skype, Zoom – has, since COVID, become ubiquitous. Since lockdown, organisations have started to translate these “work from home” practices into policies to enable greater use of these tools. As research over the past several years notes, however, this significant socio-technical transformation in work, has also created new kinds of costs. These include longer hours working; “zoom fatigue” and most particularly, increased experiences of isolation [1], known effects on sense of social cohesion [2] and, for organisations, impact on team trust and efficiency [2].
Research looking at the differences between in person and virtually mediated visual communication has for decades looked at environmental factors such as what one can see or not of a person’s body language; how the audio affects perception; lack of breaks between meetings etc. What has not been considered are the more fundamental physiological/neurological differences between physical vs digital co-presence. Without such knowledge, we can only guess at best ways to mitigate negative effects of these approaches. With models of how these different modes affect us, fundamentally, we cannot design nor evaluate how well our tools are addressing these basic effects.
Our mission in this interdisciplinary pilot is to interrogate comparatively a range of physiological responses between virtually mediated vs in-person communication.
Our proposal is situated as fundamental, interdisciplinary work involving biosignal engineering (CT), psychiatry (NH) and human-systems interaction (mc). Our collaboration will bring together biomedical data science and signal processing with psychologically informed experimental design and interaction research to understand specific neurophysiological and cognitive attributes of face-to-face vs virtual communication, and how to translate them to improve the ‘humanness’ of online communication. The work will contribute to related, further interdisciplinary activities across campus and in related funding applications.
Our work is also situated within socio-technical systems (STS) research. STS research explores the complex interaction between social and technical aspects within a system, particularly within organisations. Central to STS theory, and integral to our project, is the notion that the design and performance of any system, such as online communication tools, can only be understood and enhanced when both ‘social’ and ‘technical’ aspects are considered together.
In a very brief preliminary pilot in our lab informing this proposal, we have already seen fundamental physiological differences between in-person and virtual interactions: increase in heart rate, heart rate variability, and increased head movements occur during in-person interactions. These differences confirm that this is a pathway worth further exploration for theorising the roles of these effects, and what happens in their absence for cognitive processes like attention, connection, memory.
What our pilot proposes is (1) to explore what are the ‘special’ features of in-person interaction vs virtually mediated interaction, and (2) to develop a preliminary model of these differences in order to propose (3) next-steps towards applying these models to interaction designs to improve the “humanness” of these tools – what we call “synthesizing co-presence."
Our approach involves both the design and validation of the study tasks for comparison between in person and mediated conditions. It also includes comprehensive analysis of data across various modalities, including Electroencephalogram (EEG), Galvanic Skin Response (GSR), Electrocardiogram (ECG), eye trackers, and breathing trackers1. Recent research has demonstrated that simply viewing faces in-person, as opposed to online, can induce significant differences in EEG, functional Near-Infrared Spectroscopy (fNIRS) activity, and pupil size [3]. Finally, it will develop and deliver an analysis of the data towards models for future interaction implementation and assessment. That is: while our work will offer fundamental insights about our human responses to these types of mediated contacts, our focus, as per STS research, is to move to translate these findings into actionable applications.
