PhyCS 2016 Abstracts


Area 1 - Devices

Full Papers
Paper Nr: 7
Title:

NeuRow: An Immersive VR Environment for Motor-Imagery Training with the Use of Brain-Computer Interfaces and Vibrotactile Feedback

Authors:

Athanasios Vourvopoulos, André Ferreira and Sergi Bermudez i Badia

Abstract: Motor-Imagery offers a solid foundation for the development of Brain-Computer Interfaces (BCIs), capable of direct brain-to-computer communication but also effective in alleviating neurological impairments. The fusion of BCIs with Virtual Reality (VR) allowed the enhancement of the field of virtual rehabilitation by including patients with low-level of motor control with limited access to treatment. BCI-VR technology has pushed research towards finding new solutions for better and reliable BCI control. Based on our previous work, we have developed NeuRow, a novel multiplatform prototype that makes use of multimodal feedback in an immersive VR environment delivered through a state-of-the-art Head Mounted Display (HMD). In this article we present the system design and development, including important features for creating a closed neurofeedback loop in an implicit manner, and preliminary data on user performance and user acceptance of the system.

Short Papers
Paper Nr: 16
Title:

Comparison of an Open-hardware Electroencephalography Amplifier with Medical Grade Device in Brain-computer Interface Applications

Authors:

Jérémy Frey

Abstract: Brain-computer interfaces (BCI) are promising communication devices between humans and machines. BCI based on non-invasive neuroimaging techniques such as electroencephalography (EEG) have many applications, however the dissemination of the technology is limited, in part because of the price of the hardware. In this paper we compare side by side two EEG amplifiers, the consumer grade OpenBCI and the medical grade g.tec g.USBamp. For this purpose, we employed an original montage, based on the simultaneous recording of the same set of electrodes. Two set of recordings were performed. During the first experiment a simple adapter with a direct connection between the amplifiers and the electrodes was used. Then, in a second experiment, we attempted to discard any possible interference that one amplifier could cause to the other by adding “ideal” diodes to the adapter. Both spectral and temporal features were tested – the former with a workload monitoring task, the latter with an visual P300 speller task. Overall, the results suggest that the OpenBCI board – or a similar solution based on the Texas Instrument ADS1299 chip – could be an effective alternative to traditional EEG devices. Even though a medical grade equipment still outperforms the OpenBCI, the latter gives very close EEG readings, resulting in practice in a classification accuracy that may be suitable for popularizing BCI uses.

Paper Nr: 21
Title:

Detecting Thermal Emotional Profile

Authors:

Yang Fu and Claude Frasson

Abstract: Human can react emotionally to specific situations provoking some physiological changes that can be detected using a variety of devices, facial expression, electrodermal activity, and EEG systems are among the efficient devices which can assess the emotional reactions. However, emotions can trigger some small changes in blood flow with an impact on skin temperature. In the present research we use EEG and a thermal camera to determine the emotional profile of a user submitted to a set of emotional pictures. Six experiments were performed to study the thermal reactions to emotions, and in each experiment, 80 selected standard stimuli pictures of 20 various emotional profiles from IAPS (a database of emotional images) were displayed to participants every three seconds. An infrared camera and EEG were used to capture both thermal pictures of participants and their electrical brain activities. We used several area of the face to train a classifier for emotion recognition using Machine Learning models. Results indicate that some specific areas are more significant than others to show a change in temperature. These changes are also slower than with the EEG signal. Two methods were used to train the HMM, one is training classifier per the participant self data (participant-independent), another is training classifier based on all participants` thermal data (participant-dependent). The result showed the later method brings more accuracy emotion recognition.

Area 2 - Human Factors

Full Papers
Paper Nr: 5
Title:

Evaluating Body Tracking Interaction in Floor Projection Displays with an Elderly Population

Authors:

Afonso Gonçalves and Mónica Cameirão

Abstract: The recent development of affordable full body tracking sensors has made this technology accessible to millions of users and gives the opportunity to develop new natural user interfaces. In this paper we focused on developing 2 natural user interfaces that could easily be used by an elderly population for interaction with a floor projection display. One interface uses feet positions to control a cursor and feet distance to activate interaction. In the second interface, the cursor is controlled by ray casting the forearm into the projection and interaction is activated by hand pose. The interfaces were tested by 19 elderly participants in a point-and-click and a drag-and-drop task using a between-subjects experimental design. The usability and perceived workload for each interface was assessed as well as performance indicators. Results show a clear preference by the participants for the feet controlled interface and also marginal better performance for this method.

Paper Nr: 6
Title:

How Are We Connected? - Measuring Audience Galvanic Skin Response of Connected Performances

Authors:

Chen Wang, Xintong Zhu, Erik Geelhoed, Ian Biscoe, Thomas Röggla and Pablo Cesar

Abstract: Accurately measuring the audience response during a performance is a difficult task. This is particularly the case for connected performances. In this paper, we staged a connected performance in which a remote audience enjoyed the performance in real-time. Both objective (galvanic skin response and behaviours) and subjective (interviews) responses from the live and remote audience members were recorded. To capture galvanic skin response, a group of self-built sensors was used to record the electrical conductance of the skin. The results of the measurements showed that both the live and the remote audience members had a similar response to the connected performance even though more vivid artistic artefacts had a stronger effect on the live audience. Some technical issues also influenced the experience of the remote audience. In conclusion we found that the remoteness had little influence on the connected performance.

Short Papers
Paper Nr: 8
Title:

Effect of a Real-Time Psychophysiological Feedback, Its Display Format and Reliability on Cognitive Workload and Performance

Authors:

Sami Lini, Lise Hannotte and Margot Beugniot

Abstract: For a long time, literature has identified some psychophysiological metrics that proved reliable to assess cognitive states in controlled conditions. Smaller, more reliable and more affordable sensors made the industrial community plan to design systems that would adapt themselves to the ability of their users to operate them. Thus an important human factors question must be asked: what is the impact of such a feedback on users’ performance and cognitive workload? Does the display format of this feedback have an influence over subjects? What if the feedback provides erroneous data? We designed a protocol to compare the influence of providing a cognitive load assessment gauge versus raw data versus no feedback in a Multiple Objects Tracking task. Reliability of this feedback was also evaluated. Performance in a dual task paradigm, pupil dilation and questionnaire were used to assess cognitive load. Trials duration and learning effect were used as control results. Raw feedback showed a negative effect while low reliability showed inconsistent results.

Paper Nr: 9
Title:

Measuring the Effect of Classification Accuracy on User Experience in a Physiological Game

Authors:

Gregor Geršak, Sean M. McCrea and Domen Novak

Abstract: Physiological games use classification algorithms to extract information about the player from physiological measurements and adapt game difficulty accordingly. However, little is known about how the classification accuracy affects the overall user experience and how to measure this effect. Following up on a previous study, we artificially predefined classification accuracy in a game of Snake where difficulty increases or decreases after each round. The game was played in a laboratory setting by 110 participants at different classification accuracies. The participants reported their satisfaction with the difficulty adaptation algorithm as well as their in-game fun, with 85 participants using electronic questionnaires and 25 using paper questionnaires. We observed that the classification accuracy must be at least 80% for the physiological game to be accepted by users and that there are notable differences between different methods of measuring the effect of classification accuracy. The results also show that laboratory settings are more effective than online settings, and paper questionnaires exhibit higher correlations between classification accuracy and user experience than electronic questionnaires. Implications for the design and evaluation of physiological games are presented.

Paper Nr: 22
Title:

Effects of Hunger on Sympathetic Activation and Attentional Processes for Physiological Computing

Authors:

Ferdinand Pittino, Sandra Mai, Anke Huckauf and Olga Pollatos

Abstract: Assessing users’ states becomes increasingly important also for technical systems. In the present study, we assessed the influence of hunger on processing food versus household items by monitoring eye movements in a picture categorization task. As indicator for sympathetic activation, pupil dilation was additionally assessed in hungry and satiated participants. Food and household items were presented in the left and right visual field and the task of the participants was to indicate whether the pictures in both visual fields represented the same (household vs. food) or different categories (household and food). Although behavioural data did not differ between hungry and satiated participants, more thorough investigations of gaze behaviour showed that hungry participants were more impaired in processing household items than the satiated ones. In addition, mean pupil dilation differed between hungry and satiated participants. Pupil size was shown to correlate with hunger ratings suggesting that gaze-based measures can indeed serve as diagnostic tool for sensing user states.

Area 3 - Methodologies and Methods

Full Papers
Paper Nr: 2
Title:

A Systematic Assessment of Operational Metrics for Modeling Operator Functional State

Authors:

Jean-François Gagnon, Olivier Gagnon, Daniel Lafond, Mark Parent and Sébastien Tremblay

Abstract: This paper addresses critical issues and reports key findings with regards to the development of participant-generic operator functional state (OFS) models in the context of cognitive work. Conceptually, this research is concerned with the nature of the relationship between the physiological state of individuals and human performance. Participants were physiologically monitored (cardiac, respiratory, and eye activity) during the execution a set of two cognitive tasks – n-back and visual search – for which there were two levels of difficulty. Levels of difficulty were associated with levels of mental workload. Performance on the tasks was also monitored and linked with OFS. Modeling of the relationship between physiological state and OFS involved systematic manipulation of three parameters: (1) size of smoothing window for performance, (2) performance decrement threshold for labelling functional and sub-functional states, and (3) the mode of classification being either prospective or descriptive. Modeling was performed using two types of classifiers. Results show that (1) models that use bio-behavioral data were capable of classifying performance on new participant data above chance, (2) levels of mental workload were better classified than OFS, (3) size of smoothing window had a significant impact on classifier performance, and (4) size of smoothing window, threshold values, and classifier type had a significant impact on sensitivity and specificity. Implications for the use of OFS models in operational contexts are discussed.

Paper Nr: 10
Title:

Automatic Detection and Recognition of Human Movement Patterns in Manipulation Tasks

Authors:

Lisa Gutzeit and Elsa Andrea Kirchner

Abstract: Understanding human behavior is an active research area which plays an important role in robotic learning and human-computer interaction. The identification and recognition of behaviors is important in learning from demonstration scenarios to determine behavior sequences that should be learned by the system. Furthermore, behaviors need to be identified which are already available to the system and therefore do not need to be learned. Beside this, the determination of the current state of a human is needed in interaction tasks in order that a system can react to the human in an appropriate way. In this paper, characteristic movement patterns in human manipulation behavior are identified by decomposing the movement into its elementary building blocks using a fully automatic segmentation algorithm. Afterwards, the identified movement segments are assigned to known behaviors using k-Nearest Neighbor classification. The proposed approach is applied to pick-and-place and ball-throwing movements recorded by using a motion tracking system. It is shown that the proposed classification method outperforms the widely used Hidden Markov Model-based approaches in case of a small number of labeled training examples which considerably minimizes manual efforts.

Short Papers
Paper Nr: 17
Title:

Feature and Sensor Selection for Detection of Driver Stress

Authors:

Simon Ollander, Christelle Godin, Sylvie Charbonnier and Aurélie Campagne

Abstract: This study presents a real-life application-based feature and sensor relevance analysis for detecting stress in drivers. Using the MIT Database for Stress Recognition in Automobile Drivers, the relevance of various physiological sensor signals and features for distinguishing the driver’s state have been analyzed. Features related to heart rate, skin conductivity, electromuscular activity, and respiration have been compared using filter and wrapper selection methods. For distinguishing rest from activity, relevant sensors have been found to be heart rate, skin conductivity, and respiration (giving up to 94.6 ± 1.9 % accuracy). For distinguishing low stress from high stress, relevant sensors have been found to be heart rate and respiration (giving up to 78.1±4.1 % accuracy). In both cases, a multi-user model that requires only a calibration from the user in rest, without prior knowledge of the user’s individual stress dynamics, resulted in a different optimal sensor and feature configuration, giving 87.3±2.8 % and 72.1±4.3 % accuracy respectively.

Paper Nr: 18
Title:

The Challenge of Brain Complexity - A Brief Discussion about a Fractal Intermittency-based Approach

Authors:

Paolo Paradisi, Marco Righi and Umberto Barcaro

Abstract: In the last years, the complexity paradigm is gaining momentum in many research fields where large multidimensional datasets are made available by the advancements in instrumental technology. A complex system is a multi-component system with a large number of units characterized by cooperative behavior and, consequently, emergence of well-defined self-organized structures, such as communities in a complex network. The self-organizing behavior of the brain neural network is probably the most important prototype of complexity and is studied by means of physiological signals such as the ElectroEncephaloGram (EEG). Physiological signals are typically intermittent, i.e., display non-smooth rapid variations or crucial events (e.g., cusps or abrupt jumps) that occur randomly in time, or whose frequency changes randomly. In this work, we introduce a complexity-based approach to the analysis and modeling of physiological data that is focused on the characterization of intermittent events. Recent findings about self-similar or fractal intermittency in human EEG are reviewed. The definition of brain event is a crucial aspect of this approach that is discussed in the last part of the paper, where we also propose and discuss a first version of a general-purpose event detection algorithm for EEG signals.

Paper Nr: 19
Title:

Physiology-based Recognition of Facial Micro-expressions using EEG and Identification of the Relevant Sensors by Emotion

Authors:

Mohamed S. Benlamine, Maher Chaouachi, Claude Frasson and Aude Dufresne

Abstract: In this paper, we present a novel work about predicting the facial expressions from physiological signals of the brain. The main contributions of this paper are twofold. a) Investigation of the predictability of facial micro-expressions from EEG. b) Identification of the relevant features to the prediction. To reach our objectives, an experiment was conducted and we have proceeded in three steps: i) We recorded facial expressions and the corresponding EEG signals of participant while he/she is looking at pictures stimuli from the IAPS (International Affective Picture System). ii) We fed machine learning algorithms with time-domain and frequency-domain features of one second EEG signals with also the corresponding facial expression data as ground truth in the training phase. iii) Using the trained classifiers, we predict facial emotional reactions without the need to a camera. Our method leads us to very promising results since we have reached high accuracy. It also provides an additional important result by locating which electrodes can be used to characterize specific emotion. This system will be particularly useful to evaluate emotional reactions in virtual reality environments where the user is wearing VR headset that hides the face and makes the traditional webcam facial expression detectors obsolete.

Posters
Paper Nr: 20
Title:

Relationship between Depression Level and Bio-signals by Emotional Stimuli

Authors:

Eun-Hye Jang, Ah Young Kim, Sang-Hyeob Kim and Han-Young Yu

Abstract: Recent studies in mental/physical health monitoring have noted to improve health and wellbeing with the help of Information and Communication Technology (ICT) and in particular, application of biosensors has mainly done because signal acquisition by non-invasive sensors is relatively simple as well as bio-signal is less sensitive to social/cultural difference. Prior to developing a depression monitoring system based on non-invasive bio-signals, we examined a relationship of depressive level and changes of biological features during exposure of emotional stimuli. Ninety-six subjects’ depressive level was measured by a self-rating depression scale (SDS). Electrocardiogram (ECG) and photoplethysmograph (PPG) were recorded during six baseline and emotional states (interest, joy, neutral, pain, sadness and surprise) and heart rate (HR) and pulse transit time (PTT) were extracted. Pearson’s correlation was conducted to examine the relation of depressive level and biological features. The results showed that relation of depressive level and HR is positive in emotional states and there is a negative correlation between depressive level and PTT. We identified that they are meaningful biological features related to depression.

Area 4 - Applications

Short Papers
Paper Nr: 4
Title:

Multi-factor Authentication for Improved Efficiency in ECG: Based Login

Authors:

Pedro Neves, Luís Nunes and André Lourenço

Abstract: Electrocardiogram (ECG) based biometrics have proven to be a reliable source of identification. ECG can now be measured off-the-person, requiring nothing more than dry electrodes or conductive fabrics to acquire a usable ECG signal. However, identification still has a relatively poor performance when using large user databases. In this paper we suggest using ECG authentication associated with a smartphone security token in order to improve performance and decrease the time required for the recognition. This paper reposts the implementation of this technique in a user authentication scenario for a Windows login using normal Bluetooth (BT) and Bluetooth Low Energy (BLE). This paper also uses Intel Edison’s mobility features to create a more versatile environment. Results proved our solution to be feasible and present improvements in authentication times when compared to a simple ECG identification.

Paper Nr: 12
Title:

Space Connection - A Multiplayer Collaborative Biofeedback Game to Promote Empathy in Teenagers: A Feasibility Study

Authors:

J. E. Muñoz, A. Gonçalves, T. Vieira, D. Cró, Y. Chisik and S. Bermúdez i Badia

Abstract: Biofeedback videogames are physiologically driven games that offer opportunities to individually improve emotional self-regulation and produce mental and physical health benefits. To investigate the feasibility of a novel collaborative multiplayer methodology, we created Space Connection, a videogame to promote empathy in teenagers. Space Connection depicts a futuristic adventure aboard a spaceship in which players have to jointly use their powers to solve a set of physics-based puzzles. The game relies on the use of physiological self-regulation to activate the playing partner powers. Using a low-cost brain computer interface and a respiration rate sensor we provided players with two game powers, namely telekinesis and time-manipulation which are mapped to changes in attention and relaxation. In this paper we describe the game mechanics in three different scenarios: i) the cryogenic room, ii) the space ship corridor and iii) the cargo hold. Finally, we performed a feasibility study with 10 users (aged 22.2 ± 5.6) to evaluate the game experience. Results revealed high scores in enjoyment and empathy but low scores on interface control. Our preliminary data supports the use of novel biofeedback strategies combined with videogames to promote positive emotions and incentive collaboration and teamwork.

Paper Nr: 15
Title:

Detecting and Capitalizing on Physiological Dimensions of Psychiatric Illness

Authors:

Mark Matthews, Saeed Abdullah, Geri Gay and Tanzeem Choudhury

Abstract: Serious mental illnesses, including bipolar disorders (BD), account for a large share of the worldwide healthcare burden—estimated at $62.7B in the U.S. alone. Bipolar disorders represent a family of common, lifelong illnesses associated with poor functional and clinical outcomes, high suicide rates, and huge societal costs. Interpersonal and Social Rhythm Therapy (IPSRT), a validated treatment for BD, helps patients lead lives characterized by greater stability of daily rhythms, using a 5 item paper-and-pencil self-monitoring instrument called the Social Rhythm Metric (SRM). IPSRT has been shown to improve patient outcomes, yet many patients struggle to monitor their daily routine or even access the treatment. In this paper we describe how biological characteristics of bipolar disorder can be taken into consideration when developing systems to detect and stabilize mood episodes. We describe the co-design of MoodRhythm, a smartphone and web app, with patients and therapists. It is designed to support patients in tracking their health passively and actively over a long period of time. MoodRhythm uses the phone’s onboard sensors to automatically track sleep and social activity patterns. We report results of a small clinical pilot with experienced IPSRT clinicians and patients with bipolar disorder and finish by describing the role physiological computing could have not just in monitoring psychiatric illnesses according to existing broad categories of diagnosis but in helping radically tailor diagnoses to each individual patient and develop interventions that take advantage of idiosyncratic characteristics of each person’s illness in order to increase patient engagement in and adherence to treatment.