协议进行数据采集和分析应用到自动人脸表情分析技术和时序分析感官评价
概要
一种用于捕获和统计分析人口的情绪反应饮料和流质食物中使用自动面部表情分析软件感官评价实验室协议描述。
Abstract
We demonstrate a method for capturing emotional response to beverages and liquefied foods in a sensory evaluation laboratory using automated facial expression analysis (AFEA) software. Additionally, we demonstrate a method for extracting relevant emotional data output and plotting the emotional response of a population over a specified time frame. By time pairing each participant’s treatment response to a control stimulus (baseline), the overall emotional response over time and across multiple participants can be quantified. AFEA is a prospective analytical tool for assessing unbiased response to food and beverages. At present, most research has mainly focused on beverages. Methodologies and analyses have not yet been standardized for the application of AFEA to beverages and foods; however, a consistent standard methodology is needed. Optimizing video capture procedures and resulting video quality aids in a successful collection of emotional response to foods. Furthermore, the methodology of data analysis is novel for extracting the pertinent data relevant to the emotional response. The combinations of video capture optimization and data analysis will aid in standardizing the protocol for automated facial expression analysis and interpretation of emotional response data.
Introduction
自动面部表情分析（AFEA）是表征情绪反应饮料和食品的前瞻性分析工具。情感分析，可以添加一个额外的维度现有感官科学方法，食品评价做法，和享乐规模等级通常在研究和工业环境中使用的两种。情感分析可以提供显露对食品和饮料更准确的响应的附加指标。享乐得分可能包括参与者的偏见引起的故障记录反应1。
AFEA的研究已经在许多研究中的应用，包括电脑游戏，用户行为，教育/教学，并在同情和欺骗心理学研究中使用。大多数食品相关的研究集中在表征情绪反应的食品质量和与食品的人的行为。随着越来越见解食品行为的近期走势来看，文献报道越来越多使用的AFEA用于表征与食品，饮料，和增味剂1-12相关联的人的情绪反应。
AFEA从面部动作编码系统（FACS）的。面部动作编码系统（FACS）歧视在5点强度等级13特点是动作单元（AU）的面部动作。流式细胞仪方法需要训练有素的审评专家，手工编码，显著评估时间，并提供有限的数据分析选项。 AFEA是作为一个快速评价方法来确定的情绪。 AFEA软件依赖于面部肌肉运动，面部数据库和算法来刻画情绪反应14-18。在这项研究中所使用的AFEA软件平均同时在华沙集情感的面部表情图片（WSEFEP）和阿姆斯特丹动态面部表情集（ADFES），这是接近0.70的标准协议达成的“0.67协议FACS指数为手动编码“19 </sup>。包括在分析通用的情绪是快乐（正），悲（负），恶心（负），惊讶（正或负），愤怒（负），害怕（负）和中性每对0一单独的刻度为1（ 0 =不表达; 1 =完全表达）20。此外，心理学文献包括高兴，惊讶，和愤怒“办法”情绪（对刺激）和悲伤，恐惧，厌恶和作为“撤军”的情绪（远离厌恶刺激）21。
目前AFEA软件表征与食物有关的情绪的一个限制是从咀嚼和吞咽以及其他严重电机的动作，比如极端的头部运动相关的面部动作的干扰。该软件的目标更小的面部肌肉运动，有关位置和运动的程度，基于关于面部16,17超过500肌肉分。咀嚼运动干扰表情分类。此限制通货膨胀可以利用液化食品来解决。然而，其他的方法的挑战也能降低视频灵敏度和AFEA分析包括数据收集环境，技术，研究人员的指令，参与者的行为，和参加者的属性。
一个标准的方法学还没有得到开发和验证为最佳的视频捕获和使用AFEA情绪响应于食物和饮料在感官评价实验室环境数据分析。许多方面可以影响视频采集环境，包括照明，照明，参加者的方向，与会者行为，参加者的高度，以及，照相机高度，照相机钓鱼和设备设置遮蔽所致。此外，数据分析方法是不一致的，缺乏评估的情绪反应的标准方法。在这里，我们将展示我们的标准作业程序，用于捕获情绪数据和处理数据到使用饮料有意义的结果（调味乳，进行评价无味牛奶和未调味的水）。据我们所知，只有一个同行审查的出版物，从我们的实验室群，已用于对情感分析的8个数据解释的时间序列;然而，该方法已被更新为我们提出的方法。我们的目标是建立一个完善和一致的方法，以帮助重复性感官评价实验室环境。为了演示，该研究模型的目的是评估如果AFEA可以补充的调味奶，无味牛奶和水无味传统享乐接受评估。这个视频协议的目的是帮助建立AFEA方法，规范的感官评定实验室视频拍摄标准（感觉展位设置），并说明了人口的时空情绪数据分析的方法。
Protocol
Representative Results
Discussion
在有关食品和饮料文学AFEA应用是非常有限的1-11。食品中的应用是新的，从而为建立方法和数据解释的机会。 Arnade（2013年）7间发现个人的情绪反应巧克力牛奶和曲线分析和方差分析下，使用面积奶白色高个体变异。然而，即使参与者可变性，产生参加一个幸福的响应，而较长的悲伤和厌恶有更短的响应时间7。在另一项研究中使用的高和低浓度基本的味道，Arnade（2013年<su…
開示
The authors have nothing to disclose.
Acknowledgements
This project was funded, in part, by ConAgra Foods (Omaha, NE, USA), the Virginia Agricultural Experiment Station, the Hatch Program of the National Institute of Food and Agriculture, U.S. Department of Agriculture, and the Virginia Tech Water INTERface Interdisciplinary Graduate Education Program.
Materials
| 2% Reduced Fat Milk | Kroger Brand, Cincinnati, OH or DZA Brands, LLC, Salisbury, NC | na | for solutions |
| Drinking Water | Kroger Brand, Cincinnati, OH | na | for solutions |
| Imitation Clear Vanilla Flavor | Kroger Brand, Cincinnati, OH | na | for solutions |
| Iodized Salt | Kroger Brand, Cincinnati, OH | na | for solutions |
| FaceReader 6 | Noldus Information Technology, Wageningen, The Netherlands | na | For Facial Analysis |
| Sensory Information Management System (SIMS) 2000 | Sensory Computer Systems, Berkeley Heights, NJ | Version 6 | For Sensory Data Capture |
| Rhapsody | Acuity Brands Lighting, Inc., Conyers, GA | For Environment Illumination | |
| R Version | R Core Team 2015 | 3.1.1 | For Statistical Analysis |
| Microsoft Office | Microsoft | na | For Statistical Analysis |
| JMP | Statistical Analysis Software (SAS) Version 9.2, SAS Institute, Cary, NC | na | For Statistical Analysis |
| Media Recorder 2.5 | Noldus Information Technology, Wageningen, The Netherlands | na | For capturing participants sensory evaluation |
| Axis M1054 Camera | Axis Communications, Lund, Sweden | na | |
| Beverage | na | Beverage or soft food for evaluation |
参考文献
- De Wijk, R. A., Kooijman, V., Verhoeven, R. H. G., Holthuysen, N. T. E., De Graaf, C. Autonomic nervous system responses on and facial expressions to the sight, smell, and taste of liked and disliked foods. Food Qual Prefer. 26 (2), 196-203 (2012).
- De Wijk, R. A., He, W., Mensink, M. G. J., Verhoeven, R. H. G., De Graaf, C. ANS responses and facial expression differentiate between the taste of commercial breakfast drinks. PLoS ONE. 9 (4), 1-9 (2014).
- He, W., Boesveldt, S., De Graaf, C., De Wijk, R. A. Behavioural and physiological responses to two food odours. Appetite. 59 (2), 628 (2012).
- He, W., Boesveldt, S., De Graaf, C., De Wijk, R. A. Dynamics of autonomic nervous system responses and facial expressions to odors. Front Psychol. 5 (110), 1-8 (2014).
- Danner, L., Sidorkina, L., Joechl, M., Duerrschmid, K. Make a face! Implicit and explicit measurement of facial expressions elicited by orange juices using face reading technology. Food Qual Prefer. 32 (2014), 167-172 (2013).
- Danner, L., Haindl, S., Joechl, M., Duerrschmid, K. Facial expression and autonomous nervous system responses elicited by tasting different juices. Food Res Int. 64 (2014), 81-90 (2014).
- Arnade, E. A. . Measuring consumer emotional response to tastes and foods through facial expression analysis [thesis]. , 1-187 (2013).
- Leitch, K. A., Duncan, S. E., O’Keefe, S., Rudd, R., Gallagher, D. L. Characterizing consumer emotional response to sweeteners using an emotion terminology questionnaire and facial expression analysis. Food Res Int. 76, 283-292 (2015).
- Crist, C. A., et al. Application of emotional facial analysis technology to consumer acceptability using a basic tastes model. , (2014).
- Garcia-Burgos, D., Zamora, M. C. Facial affective reactions to bitter-tasting foods and body mass index in adults. Appetite. 71 (2013), 178-186 (2013).
- Garcia-Burgos, D., Zamora, M. C. Exploring the hedonic and incentive properties in preferences for bitter foods via self-reports, facial expressions and instrumental behaviours. Food Qual Prefer. 39 (2015), 73-81 (2015).
- Lewinski, P., Fransen, M. L., Tan, E. S. H. Predicting advertising effectiveness by facial expressions in response to amusing persuasive stimuli. J. Neurosci. Psychol. Econ. 7 (1), 1-14 (2014).
- Ekman, P., Friesen, W. V. Facial action coding system: A technique for the measurement of facial movement. , (1978).
- Viola, P., Jones, M. Rapid object detection using a boosted cascade of simple features. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern. 1, 511-518 (2001).
- Sung, K. K., Poggio, T. Example-based learning for view-based human face detection. IEEE Trans. Pattern Anal. Mach. Intell. 20 (1), 39-51 (1998).
- . . FaceReader 5™ Technical Specifications. , (2014).
- Cootes, T., Taylor, C. . Statistical models of appearance for computer vision: Technical report. , (2000).
- Bishop, C. M. . Neural networks for pattern recognition. , (1995).
- Lewinski, P., den Uyl, T. M., Butler, C. Automated facial coding: validation of basic emotions and FACS AUs in FaceReader. J. Neurosci. Psychol. Econ. 7 (4), 227-236 (2014).
- Noldus Information Technology. . FaceReader Reference Manual Version 6. , (2014).
- Alves, N. T., Fukusima, S. S., Aznar-Casanova, J. A. Models of brain asymmetry in emotional processing. Psychol Neurosci. 1 (1), 63-66 (2008).
- Costello, M., Clark, S., Clark, S., Costello, M., Drake, M., Bodyfelt, F. Preparation of samples for instructing students and staff in dairy products evaluation (Appendix F). The sensory evaluation of dairy foods. , 551-560 (2009).
- Porcherot, C., et al. How do you feel when you smell this? Optimization of a verbal measurement of odor-elicited emotions. Food Qual Prefer. 21, 938-947 (2010).
- Warrenburg, S. Effects of fragrance on emotions: Moods and physiology. Chem. Senses. 30, i248-i249 (2005).
- Bredie, W. L. P., Tan, H. S. G., Wendin, K. A comparative study on facially expressed emotions in response to basic tastes. Chem. Percept. 7 (1), 1-9 (2014).
- Wendin, K., Allesen-Holm, B. H., Bredie, L. P. Do facial reactions add new dimensions to measuring sensory responses to basic tastes?. Food Qual Prefer. 22, 346-354 (2011).
- Rosenstein, D., Oster, H. Differential facial responses to four basic tastes in newborns. Child Dev. 59 (6), 1555-1568 (1988).
- Rosenstein, D., Oster, H., P, E. k. m. a. n., E, R. o. s. e. n. b. e. r. g. Differential facial responses to four basic tastes in newborns. What the face reveals: Basic and applied studies of spontaneous expression using the facial action coding system (FACS). , 302-327 (1997).
- Rozin, P., Fallon, A. E. A perspective on disgust. Psychol. Rev. 94 (1), 23-41 (1987).
- Delarue, J., Blumenthal, D. Temporal aspects of consumer preferences. Curr. Opin. Food Sci. 3, 41-46 (2015).
- Sudre, J., Pineau, N., Loret, C., Marin, N. Comparison of methods to monitor liking of food during consumption. Food Qual Prefer. 24 (1), 179-189 (2012).
- Weiland, R., Ellgring, H., Macht, M. Gustofacial and olfactofacial responses in human adults. Chem. Senses. 35 (9), 841-853 (2010).
- Ekman, P., Cole, J. Universal and cultural differences in facial expressions of emotion. Nebraska symposium on motivation. , 207-283 (1971).
- Griemel, E., Macht, M., Krumhuber, E., Ellgring, H. Facial and affective reactions to tastes and their modulation by sadness and joy. Physiol Behav. 89 (2), 261-269 (2006).
