PhyCS 2015 Abstracts


Area 1 - Methodologies and Methods

Full Papers
Paper Nr: 14
Title:

Selection of the Most Relevant Physiological Features for Classifying Emotion

Authors:

C. Godin, F. Prost-Boucle, A. Campagne, S. Charbonnier, S. Bonnet and A. Vidal

Abstract: With the development of wearable physiological sensors, emotion estimation becomes a hot topic in the literature. Databases of physiological signals recorded during emotional stimulation are acquired and machine learning algorithms are used. Yet, which are the most relevant signals to detect emotions is still a question to be answered. In order to better understand the contribution of each signal, and thus sensor, to the emotion estimation problem, several feature selection algorithms were implemented on two databases freely available to the research community (DEAP and MANHOB-HCI). Both databases manipulate emotions by showing participants short videos (video clips or part of movies respectively). Features extracted from Galvanic Skin response were found to be relevant for arousal estimation in both databases. Other relevant features were eye closing rate for arousal, variance of zygomatic EMG for valence (those features being only available for DEAP). The hearth rate variability power in three frequency bands also appeared to be very relevant, but only for MANHOB-HCI database where heat rate was measured using ECG (whereas DEAP used PPG). This suggests that PPG is not accurate enough to estimate HRV precisely. Finally we showed on DEAP database that emotion classifiers need just a few well selected features to obtain similar performances to literature classifiers using more features.

Area 2 - Devices

Full Papers
Paper Nr: 17
Title:

Towards Voluntary Pupil Control - Training Affective Strategies?

Authors:

Jan Ehlers, Nikola Bubalo, Markus Loose and Anke Huckauf

Abstract: During the past years, increasing attention is being paid to operationalize pupil dynamics for affective classification (Jacobs, 1996). Thereby it is generally assumed that pupil size displays a genuine impression of user’s cognitive state but defies any voluntary control (Loewenfeld, 1993). Based on Ekman (2008) we applied graphical feedback on pupil diameter changes to utilize mechanisms of operant conditioning to gradually enable voluntary control over pupil size. Participants underwent a training program to exert control by utilizing affective associations to expand pupil size and relaxation strategies to reduce it. As a result, more than half of the participants demonstrated to be able to increase pupil sizes relative to baseline recordings. Training effects did not show up. Furthermore, controlling influence diminishes after about ten seconds. Intentional increase of sympathetic activity seems to be subject to habituation processes that allow central inhibition of parasympathetic pathways only over a short period. Beside strategy-based factors, physiological mechanisms like baseline pupil activity may determine inter-individual differences in exerting voluntary control. In summary it can be noted that pupil-based communication in HCI extends affective monitoring and may constitute an active input channel to reliably interfere by means of simple cognitive strategies.

Area 3 - Methodologies and Methods

Full Papers
Paper Nr: 20
Title:

Emotion Recognition based on Heart Rate and Skin Conductance

Authors:

Mickael Ménard, Paul Richard, Hamza Hamdi, Bruno Daucé and Takehiko Yamaguchi

Abstract: Information on a customer’s emotional states concerning a product or an advertisement is a very important aspect of marketing research. Most studies aimed at identifying emotions through speech or facial expressions. However, these two vary greatly with people’s talking habits, which cause the data lacking continuous availability. Furthermore, bio-signal data is also required in order to fully assess a user’s emotional state in some cases. We focused on recognising the six basic primary emotions proposed by Ekman using biofeedback sensors, which measure heart rate and skin conductance. Participants were shown a series of 12 video-based stimuli that have been validated by a subjective rating protocol. Experiment results showed that the collected signals allow us to identify user's emotional state with a good ratio. In addition, a partial correlation between objective and subjective data has been observed.

Area 4 - Human Factors

Full Papers
Paper Nr: 22
Title:

Quantifying Negative Affect - Usability Testing to Observe the Effect of Negative Emotions on User Productivity Through the Use of Biosignals and OCC Theory

Authors:

Gloria Washington

Abstract: Humans sometimes experience negative emotions caused by electronic devices that impede their task(s). User experience researchers have examined technology-caused negative affect by collecting task performance metrics, user feedback, and/or human physiological data like skin temperature or blood pressure for more insight. Much research has been done to determine the amount of negative affect produced by the humans during these events. However, these methods usually require the user to self-report their negative feelings through Likert scales, pressure-sensitive devices or other manual methods. Task performance measures have also been used in lieu of asking a user what they feel. In this research, we adapt OCC Theory for use with physiological data for quantifying negative affect in human-computer interactions, along with asking a person how they feel about an application. In addition, we observe how negative affect amounts impact task performance measures in a usability study by adding random system delays into an application to induce negative feelings. Results from this work showed productivity does not always degrade when negative feelings are experienced by a user. In addition, some types of negative affect may have the opposite effect and allow a user to increase their performance under the right conditions.

Area 5 - Methodologies and Methods

Short Papers
Paper Nr: 2
Title:

Implementation of a Motor Imagery based BCI System using Python Programming Language

Authors:

Luz Maria Alonso-Valerdi and Francisco Sepulveda

Abstract: At present, there is a wide variety of free open-source brain-computer interface (BCI) software. Even though the available software is very complete, it often runs under a Matlab environment. Matlab is a high performance language for scientific computing, but its limitations concerning the license cost, the restricted access to the algorithm code, and the portability difficulties complicates its use. Therefore, we proposed to implement a motor imagery (MI) based BCI system using Python programming language. This system was called miBCI software, was designed to discriminate up to three control tasks and was structured on the basis of online and offline data analyses. The functionality and efficiency of the software were firstly assessed in a pilot study, and then, its applicability and utility were demonstrated in two subsequent studies associated with the external and internal influences on MI-related control tasks. Results of the pilot study and preliminary outcomes of the subsequent studies are herein presented. This work contributes by promoting the utilization of tools which facilitate the advance of BCI research. The advantage of using Python instead of Matlab, which is the widely used programming language at the moment, is the opportunity to develop BCI software in a public and collaborative way, without property license restrictions.

Area 6 - Human Factors

Short Papers
Paper Nr: 7
Title:

Comparing the Sensor Glove and Questionnaire as Measures of Computer Anxiety

Authors:

Tlholohelo Nkalai, Lizette de Wet and Robert Schall

Abstract: Contradictory findings are reported in the literature concerning computer anxiety and how it affects the performance of individuals executing computer-related tasks. The discrepancies in the findings could be caused by the sole use of computer anxiety questionnaires. The aims of the present study were to establish whether using a sensor glove provided complementary information to an existing computer anxiety questionnaire; and to compare the computer anxiety of participants using a sensor glove and an anxiety questionnaire with relation to performance. The study results suggest that the sensor glove and the anxiety questionnaire provided different information concerning participants’ anxiety before and after performing tasks on the computer. A negative correlation between computer anxiety and performance was found using both the sensor glove measurements and the computer anxiety scores. It is concluded that the sensor glove possibly measures a different variable from the anxiety questionnaire and further research is necessary in that regard. Additionally, it is concluded that the higher an individual’s levels of anxiety, the poorer he/she performed on the assessment.

Paper Nr: 11
Title:

Influence of Workload on Auditory Evoked Potentials in a Single-stimulus Paradigm

Authors:

R. N. Roy, A. Breust, S. Bonnet, J. Porcherot, S. Charbonnier, C. Godin and A. Campagne

Abstract: Mental workload can be assessed via neurophysiological markers. Temporal features such as event related potentials (ERPs) are one of those which are very often described in the literature. However, most of the studies that evaluate their sensitivity to workload use secondary tasks. Yet potentials elicited by ignored stimuli could provide mental state monitoring systems with less intrusive probing methods. For instance, auditory probing systems could be used in adaptive driving or e-learning applications. This study evaluates how workload influences auditory evoked potentials (AEPs) elicited by a single-stimulus paradigm when probes are to be ignored. Ten participants performed a Sternberg memory task on a touchpad with three levels of difficulty plus a view-only condition. In addition, they performed two ecological tasks of their choice, one deemed easy (e.g. reading novels), and the other difficult (e.g. programming). AEPs were elicited thanks to pure tones presented during the memory task retention period, and during the whole extent of the external tasks. Performance and AEPs were recorded and analyzed. Participants’ accuracy decreased linearly with increasing workload, whereas the difference in amplitude between the P3 and its adjacent components, N2 and SW, increased. This reveals the relevance of this triphasic sequence for mental workload assessment.

Area 7 - Methodologies and Methods

Short Papers
Paper Nr: 16
Title:

Baran: An Interaction-centred User Monitoring Framework

Authors:

Mohammad Hashemi and John Herbert

Abstract: User Quality of Experience (QoE) is a subjective entity and difficult to measure. One important aspect of it, User Experience (UX), corresponds to the sensory and emotional state of a user. For a user interacting through a User Interface (UI), precise information on how they are using the UI can contribute to understanding their UX, and thereby understanding their QoE. As well as a user’s use of the UI such as clicking, scrolling, touching, or selecting, other real-time digital information about the user such as from smart phone sensors (e.g. accelerometer, light level) and physiological sensors (e.g. heart rate, ECG, EEG) could contribute to understanding UX. Baran is a framework that is designed to capture, record, manage and analyse the User Digital Imprint (UDI) which, is the data structure containing all user context information. Baran simplifies the process of collecting experimental information in Human and Computer Interaction (HCI) studies, by recording comprehensive real-time data for any UI experiment, and making the data available as a standard UDI data structure. This paper presents an overview of the Baran framework, and provides an example of its use to record user interaction and perform some basic analysis of the interaction.

Posters
Paper Nr: 15
Title:

Improving Physiological Signal Classification Using Logarithmic Quantization and a Progressive Calibration Technique

Authors:

Nick Merrill, Thomas Maillart, Benjamin Johnson and John Chuang

Abstract: This paper exhibits two methods for decreasing the time associated with training a machine learning classifier on biometric signals. Using electroencephalography (EEG) data obtained from a consumer-grade headset with a single electrode, we show that these methods produce significant gains in the computational performance and calibration time of a simple brain-computer interface (BCI) without significantly decreasing accuracy. We discuss the relevance of reduced feature vector size to the design of physiological computing applications.

Area 8 - Human Factors

Posters
Paper Nr: 26
Title:

Evaluating EEG Measures as a Workload Assessment in an Operational Video Game Setup

Authors:

Lucille Lecoutre, Sami Lini, Christophe Bey, Quentin Lebour and Pierre-Alexandre Favier

Abstract: We tested the electroencephalography (EEG) B-Alert X10 system (Advance Brain Monitoring, Inc.) mental workload metrics. When we evaluate a human-systems interfaces (HSI), we need to assess the operator’s state during a task in order evaluate the systems efficiency at helping the operator. Physiological metrics are of good help when it comes to evaluate the operator’s mental workload, and EEG is a promising tool. The B-Alert system includes an internal signal processing algorithm computing a mental workload index. We set up a simple experiment on a video game in order to evaluate the reliability of this index. Participants were asked to play a video game with different levels of goal (easy vs hard) as we measured subjective, behavioral and physiological indices (B-Alert mental workload index, pupillometry) of mental workload. Our results indicate that, although most of the measure point toward the same direction, the B-Alert metrics fails to give a clear indication of the mental workload state of the participants. The use of the B-Alert workload index alone is not precise enough to assess an operator mental workload condition with certainty. Further evaluations of this measure need to be done.

Paper Nr: 27
Title:

Impressions of Size-Changing in a Companion Robot

Authors:

Martin Cooney and Stefan M. Karlsson

Abstract: Physiological data such as head movements can be used to intuitively control a companion robot to perform useful tasks. We believe that some tasks such as reaching for high objects or getting out of a person’s way could be accomplished via size changes, but such motions should not seem threatening or bothersome. To gain insight into how size changes are perceived, the Think Aloud Method was used to gather typical impressions of a new robotic prototype which can expand in height or width based on a user’s head movements. The results indicate promise for such systems, also highlighting some potential pitfalls.

Area 9 - Methodologies and Methods

Posters
Paper Nr: 28
Title:

Slow Trends - A Problem in Analysing Pupil Dynamics

Authors:

Christoph Strauch, Juliane Georgi, Anke Huckauf and Jan Ehlers

Abstract: As of recently, research efforts are intensified to operationalize pupil dynamics for cognitive and affective classification in human-machine interaction. However, signal analysis of pupil diameter changes is problematic since the respective dynamics consist of three essential components that have to be disentangled: Very slow diameter changes, slow and high frequencies. The current paper discusses the amount of slow trends in pupillary signal courses and the effects on functional parameters of pupil dilations. Thereby we confront our data with linear detrending approaches and reveal various forms of trend progressions that differ over time and cannot be fixed with conventional linear procedures.

Paper Nr: 29
Title:

Is Human Visual Activity in Simple Human-Computer Interaction Search Tasks a Lévy Flight?

Authors:

Jerzy Grobelny, Rafal Michalski and Rafał Weron

Abstract: The paper tries to answer the question regarding the nature of the statistical distribution of data gathered by eye tracking software. The experimental data regarding typical search tasks performed while using web sites were formally analysed and discussed. Results show some resemblance of the obtained experimental distributions of distance travelled to heavy tailed power-law type distributions characteristic of Lévy flights. However, the similarity is not as strong as it has been suggested by previous studies. The results of this paper may be used in further attempts of modelling human visual processing in the context of simple human-computer interfaces.

Paper Nr: 30
Title:

Addressing Subject-dependency for Affective Signal Processing - Modeling Subjects’ Idiosyncracies

Authors:

François Courtemanche, Emma Campbell, Pierre-Majorique Léger and Franco Lepore

Abstract: Most works on Affective Signal Processing (ASP) focus on user-dependent emotion recognition models which are personalized to a specific subject. As these types of approach have good accuracy rates, they cannot easily be reused with other subjects for industrial or research purposes. On the other hand, the reported accuracy rates of user-independent models are substantially lower. This performance decrease is mostly due to the greater variance in the physiological training data set drawn from multiple users. In this paper, we propose an approach to address this problem and enhance the performance of user-independent models by explicitly modeling subjects’ idiosyncrasies. As a first exemplification, we describe how personality traits can be used to improve the accuracy of user-independent emotion recognition models. We also present the experiment that will be carried on to validate the proposed approach.

Area 10 - Applications

Full Papers
Paper Nr: 4
Title:

Heart Rate Monitoring as an Easy Way to Increase Engagement in Human-Agent Interaction

Authors:

Jérémy Frey

Abstract: Physiological sensors are gaining the attention of manufacturers and users. As denoted by devices such as smartwatches or the newly released Kinect 2 – which can covertly measure heartbeats – or by the popularity of smartphone apps that track heart rate during fitness activities. Soon, physiological monitoring could become widely accessible and transparent to users. We demonstrate how one could take advantage of this situation to increase users’ engagement and enhance user experience in human-agent interaction. We created an experimental protocol involving embodied agents – “virtual avatars”. Those agents were displayed alongside a beating heart. We compared a condition in which this feedback was simply duplicating the heart rates of users to another condition in which it was set to an average heart rate. Results suggest a superior social presence of agents when they display feedback similar to users’ internal state. This physiological “similarity-attraction” effect may lead, with little effort, to a better acceptance of agents and robots by the general public.

Paper Nr: 31
Title:

Physiology-based Affect Recognition During Driving in Virtual Environment for Autism Intervention

Authors:

Dayi Bian, Joshua Wade, Amy Swanson, Zachary Warren and Nilanjan Sarkar

Abstract: Independent driving is believed to be an important factor of quality of life for individual with autism spectrum disorder (ASD). In recent years, several computer technologies, particularly Virtual Reality (VR), have been explored to improve driving skills in this population. In this work a VR-based driving environment was developed for skill training for teenagers with ASD. Eight channels of physiological signals were recorded in real time for affect recognition during driving. A large set of physiological features were investigated to determine their correlation with four categories of affective states: engagement, enjoyment, frustration and boredom, of teenagers with ASD. In order to have reliable reference points to link the physiological data with the affective states, the subjective reports from a therapist were recorded and analyzed. Six well-known classifiers were used to develop physiology-based affect recognition models, which yielded reliable predictions. These models could potentially be used in future physiology-based adaptive driving skill training system such that the system could adapt based on individual affective states.

Short Papers
Paper Nr: 6
Title:

Physiological Measurement on Students’ Engagement in a Distributed Learning Environment

Authors:

Chen Wang and Pablo Cesar

Abstract: Measuring students’ engagement in a distributed learning environment is a challenge. In particular, a teacher gives a lecture at one location, while at the same time the remote students watch the lecture through a display screen. In such situation, it is difficult for the teacher to know the reaction at the remote location. In this paper, we conducted a field study to measure students’ engagement by using galvanic skin response (GSR) sensors, where students simultaneously watched the lecture at the two locations. Our results showed the students’ GSR response was aligned with the surveys, which means that during a distributed learning environment, GSR sensors can be used as an indicator on students’ engagement. Furthermore, our user studies resulted in non-engaging student learning experiences that would be difficult obtained at a lab condition. Based on the findings, we found that the patterns of GSR readings were rather different when compared to the previous relevant studies, where users were engaged. In addition, we noticed that the density of GSR response at the remote location was higher when compared to the one at the lecture room. We believe that our studies are beneficial on physiological computing, as we first presented the patterns of GSR sensors on non-engaging user experiences. Moreover, as an alternative method, GSR sensors can be easily implemented in a distributed learning environment to provide feedback to teachers.

Posters
Paper Nr: 21
Title:

Physiological Computing Gaming - Use of Electrocardiogram as an Input for Video Gaming

Authors:

Adam Chęć, Dominika Olczak, Tiago Fernandes and Hugo Ferreira

Abstract: There are several ways of creating a human-computer interaction (HCI). One of those is physiological computing (PC) i.e. the use of body signals as a real-time input to control a user interface. In this paper one describes the development of a new solution in which electrocardiography (ECG) signals are used as input for video gaming. The solution includes: a tailored belt with conductive textiles as ECG electrodes; a specialized data acquisition board (Bitalino); signal processing algorithms implemented in Python for signal filtering, QRS complex detection and heart rate calculation; and use of Unity 3D, a game development engine, in which the heart rate is used as an input of a proof-of-concept PC video game – FlappyHeartPC. With this project we conclude that nowadays it is possible to build tools that can make the bridge between the machine and the human body in order to respond to innovations required in the gaming business.

Paper Nr: 24
Title:

Physiological-based Dynamic Difficulty Adaptation in a Theragame for Children with Cerebral Palsy

Authors:

Adrien Verhulst, Takehiko Yamaguchi and Paul Richard

Abstract: The purpose of this research is to provide a physiological-based Dynamic Difficulty Adaptation (DDA) for rehabilitation of children with Cerebral Palsy (CP). In this paper, we present all the steps of the DDA development by going through (1) the acquisition of physiological signals, (2) the extraction of the physiological signals’ features, (3) the training of a learning classifier of physiological signals' features, and (4) the implementation of the DDA in a game-based rehabilitation system. As a result, we successfully implement a physiological-based DDA based on the user affective state (anxiety and boredom).