Use of Galvanic Skin Responses, Salivary Biomarkers, and Self-reports to Assess Undergraduate Student Performance During a Laboratory Exam Activity
Özet
Research on undergraduate students’ academic achievement emotions have primarily relied on self-reports in laboratory settings. Studies rarely include bio-physiological measures to these self-reports. This protocol will describe a methodology that integrates self-reports with and bio-physiological measures to assess student response and performance during a laboratory examination activity.
Abstract
Typically, self-reports are used in educational research to assess student response and performance to a classroom activity. Yet, addition of biological and physiological measures such as salivary biomarkers and galvanic skin responses are rarely included, limiting the wealth of information that can be obtained to better understand student performance. A laboratory protocol to study undergraduate students' responses to classroom events (e.g., exams) is presented. Participants were asked to complete a representative exam for their degree. Before and after the laboratory exam session, students completed an academic achievement emotions self-report and an interview that paralleled these questions when participants wore a galvanic skin sensor and salivary biomarkers were collected. Data collected from the three methods resulted in greater depth of information about students' performance when compared to the self-report. The work can expand educational research capabilities through more comprehensive methods for obtaining nearer to real-time student responses to an examination activity.
Introduction
In the area of academic achievement emotions, studies indicate that understanding of students' motivations could predict students' performance, achievement, and career plans1. Students' abilities to 'emotionally respond' to challenging course tasks2 are pivotal to students' professional development. Yet, nearer to real-time responses related to academic achievement emotions are under-explored3-6. This paper presents a protocol to study ways to explore nearer real-time responses from students (e.g., physiological responses) when presented with representative classroom situations (e.g., test taking) using salivary biomarkers, galvanic skin responses, and self-reported surveys and interviews. While the work will not seek to establish connections between these salivary biomarkers, galvanic skin responses and self-reports, future work will aim to further explore the underlying mechanisms that associate each response.
Self-reporting of academic achievement emotions in the classroom can be used to assess affective, cognitive, motivational, physiological, and behavioral components that represent the human mind. Due to its cheap cost, easy dissemination and traceability, self-report surveys are highly used in classroom settings7. However, these have some disadvantages. For example, self-reports are limited to the representations of the conscious mind8, which can change the manner that individuals represent themselves. Also, language and semantics in self-reports may be understood differently between cultures and individuals7; its meanings can change over time or represent something different in light of the situation that the participant is involved in8. Moreover, self-reporting in academic settings can be multifaceted, idiosyncratic and dependent on memory, social desirability, and individual beliefs7, 9-11. For example, participants' beliefs about professors' expectations and motives can affect how students respond and perform during classroom activities7, 9-11. As such, complementary methods based on nearer to real-time responses are needed to reduce sampling biases and subjectivity when using self-reports. This work will supplement self-reports with salivary biomarkers and galvanic skin responses to better understand the nearer real-time responses of students to classroom activities.
Sampling of salivary biomarkers has become popular in understanding physiological foundations of individuals' responses to various stressors that can impact cognitive abilities9. Psychological development of cognition is affected by hormones in many species, including human beings7,12. During sensitive periods of development, hormones are capable of making changes in the organization of the brain, which can have long-lasting effects on behavior8. Different aspects of cognition, for example, can be affected by hormones during different times of an individual's development. Spatial ability, which studies have shown can include gender differences13-17, is moderately enhanced by androgens (e.g., dehydroepiandrosterone-DHEA, testosterone) in prenatal development and then again throughout adulthood18. On the contrary, verbal abilities have been linked to enhancement of oestrogens (e.g., estradiol) and progesterones18. Physiological biomarkers of stress such as cortisol, are found in the hypothalamic-pituitary-adrenal axis in humans12-16, 19-21. When a situation is perceived as uncontrollable, cortisol levels elevate19 and can result in differential responses in individuals. Recent work has begun to use hormones to study academic achievement emotions, although to this point it is very limited20,22.
Research in understanding psychophysiological responses that measure emotion via physiological arousal in education has used galvanic skin responses (GSR). GSR is a measure of microscopic amounts of sweat secreted from the skin and is related to the autonomic nervous system (ANS). When a person becomes nervous or anxious about a task, palms become sweaty. Therefore, emotional regulation and cognitive processes, among other brain functions, can influence the control of sweating. More activation of the system (i.e., high stress, cognitive load or strong emotional responses) results in more sweat secretion than low activation states (i.e., boredom, low cognitive load). As sweat secretion fluctuates, the electrical conductivity of the skin changes. Thus, GSR is widely considered as a proxy for quantifying stress level or cognitive load. GSR is typically measured by bands containing electrodes that come into contact with hands, wrist, or feet and is recently being used in classroom settings22,23 due to its low cost and feasibility compared to available neuroimaging techniques7. The combination of galvanic skin responses with salivary biomarkers will allow for a more comprehensive assessment of student responses to classroom activities nearer to real-time.
The proposed protocol discussed here will serve to combine educational and physiological techniques to establish a methodology to help educational researchers understand student performance and response to classroom activities (e.g., exams). While the work will not focus on understanding fundamental connections between emotions and physiological and biological constructs, this protocol is a starting point to help researchers move in that direction. This protocol will cover methods to measure salivary biomarkers and galvanic skin responses during an exam activity and compare it against the information obtained from self-reports and interviews. For this work, an engineering exam and students were selected due to the difficult and complex nature of the discipline1,6 and concepts, which may ignite both cognitive and emotional responses in the participants.
Protocol
Procedures have been approved by the Institutional Review Board (IRB) at Utah State University for studies on human subjects. Care should be taken that IRB procedures are approved by the host institution and considerations regarding the protection of human subjects should take place prior, during, and after performance of any aspect of this protocol. As per IRB regulations, involvement of external parties or companies in the data collection and analysis processes must follow proper protocols to de-identify participant information and protect the confidential nature of the data.
1. Selection of Participants and Activities to Test
- Select student participants at the middle of the semester or 3 months after a desired course content was presented by the professor to diminish any short-term memory recollections from the participants.
- Exclude any participants from a laboratory study that parallels this work if the participant has a: (a) history of pre-determined condition (e.g. arrhythmia) or metabolic disorder; (b) left-handedness or ambidextrous ability (this can interfere with physiological sampling); (c) psychological history (current or past) or history of behavioral or emotional disorder (this can skew the collection from the self-reports); (d) medical history for heart, metabolic, or cognitive disorders; (e) physical disability that would prevent participation in the laboratory sessions of the study; and (f) traumatic brain injury. Female participants wishing to be part of the hormonal study have additional restrictions explained in step 2.5.
- Recruit according to established Institutional Review Board (IRB) protocols by including detailed procedures and restrictions about protection of confidentiality.
NOTE: If a company will have access to the data, (in this case salivary samples and galvanic skin response data were collected from third party companies) ensure that an agreement is in place for proper handling of data (e.g., time for refrigeration), sharing (e.g., security of server), and include a timeline to destroy the data (e.g., one year after publication of data). Include these guidelines in your IRB application and follow proper guidelines as required by the IRB organization within your institution. - Select course activities that represent the complexity of knowledge and tasks that students need to perform in.
NOTE: For this protocol, freshmen engineering students were selected due to the difficult and complex nature of the course content. The activities that were deemed representative were a combination of two types of engineering problems from two commonly used engineering exams: the Mental Cutting Plane Test (MCT) and the Purdue Spatial Visualization Test (PSVT-R). MCT looks at two-dimensional cut sections that corresponds to a three-dimensional object24 while PSVT-R looks for three-dimensional rotation of an object25. Each were used to test for spatial performance and verbal memory for this student population26, 27.
2. Prior to the Laboratory Session
- Host an orientation for the volunteer participants at least 1 – 2 weeks prior to the laboratory session. Clarify all terms from the IRB informed consent. Perform a demonstration of the use and fitting of the galvanic skin sensor and proper data collection procedures for salivary hormone collection, according to the manufacturer protocols28. Explain the risks and benefits of the study.
- Establish a calendar that participants can enroll in. If participants are enrolling in the salivary biomarker piece of the study, be sure to establish times during the morning before breakfast as cortisol levels are lowest at this time of day and include any disclosures for enrolling in the allotted times (see step 2.5 for more information).
NOTE: Ensure that all participants provide at least three available time slots for the study. Accommodate participants in a randomized fashion to minimize biases and provide more statistically valid results. - Pre-fit the galvanic wrist sensor for each volunteer by ensuring that the sensor straps secure the participant's wrist during the orientation session. Enter the participant study ID code and the unique wrist sensor bar code provided by the supplier to ensure consistency for follow-up laboratory sessions if needed.
- Send a reminder to the participants of the procedures and restrictions for data collection and pre-conditions prior to the day of sampling, according to manufacturer protocols28.
NOTE: Examples of procedures and restrictions can include: (a) no food, dietary or energetic products, or sugary drinks during diurnal collections or at least 1 hr prior to the study; (b) no water consumption 1 hr prior to data collection; (c) no teeth brushing or mouth rinses; (d) no chemical products (e.g., hand lotions, creams, chapsticks, perfumes, dyes) in the required contact areas (e.g., wrists, face, lips); (f) no hair products or styling; and (g) no physical activities (e.g., brisk walking, cycling, running, weights) 24 hr before the study. - For female participants, collect salivary samples between 10 – 12 days after the first day of the participant's menstrual period to minimize hormonal fluctuations during salivary collection28.
NOTE: During the stated time period, if female participants are taking any form of hormone-based treatment such as any birth control pills, intrauterine devices, creams or sprays (oral, topical, or vaginal), sublingual and troches, patches, films, injections, allow 12 – 36 hr to pass from the time of ingestion/application to allow the treatment effects to level off. The timeline required to collect salivary samples will vary by the biomarker and the specifications of the company's salivary kit. Galvanic skin response collection is done according to Section 3.2 for female participants. Male participant salivary collection is done according to Section 3.1.5.1 and galvanic skin response according to Section 3.2.
3. Day of Laboratory Session
NOTE: The following procedures are presented in recommended order of data collection under the assumption that 1 – 2 researchers are involved. However, some of these procedures could be run in parallel if more than 2 researchers are collaborating in the study.
- Salivary Biomarkers
- Provide participants with a checklist to outline physical activities exerted during the 24 hr period prior to the study (e.g., physical activity, food and drink consumption, use of lotions or hormone based products) as described in the manufacturer protocol28(see step 2.4 and 2.5).
NOTE: If a participant has performed any or all of the activities from the checklist, re-assign the participant to meet at another day. If water consumption was the only restriction not met, ask the participant to stay an extended time in the study to allow an appropriate time, according to manufacturer guidelines, to pass (~ 1 hr) before salivary collection occurs. - Ask the participants to fill out the health information checklist provided by the saliva assay kit28. Ensure that all salivary vials are pre-labelled and time stamped before and after the laboratory sessions.
NOTE: Salivary cortisol has a reactivity of approximately 20 min19,20 and as such, spit collection procedures should be done within this timeframe. Follow spit collection procedures according to step 3.1.5 and manufacturer guidelines28. - Keep a copy of the health information checklist and shipping information for future record-keeping, tracking, and further analysis. Be sure to follow IRB protocols and procedures during storage of checklist items. Complete a timeline of the laboratory activities, personnel, and collection vials for each participant.
- Ensure that all participants and researchers handling salivary samples are wearing aseptic gloves at all times to minimize cross-contamination.
- When collecting spit, instruct the participant to pool saliva in the tip of the participant's lips before spitting into the vial to minimize bubble formation that may affect accuracy of results, according to manufacturer instructions28.
- Instruct participants to not touch the tip of the vial with the participant's lips or fingers as this can introduce contamination to the sample (spit vial method). To encourage salivary production, recommend to the participant to smell a citrus fruit such as a lemon or head tilt forward to speed up formation of saliva.
- Store all collected salivary samples -20 oC for up to 7 days if analysis of these samples will not occur immediately. Longer storage (up to 30 days) will require samples to be stored at -80 oC.
- To ship the frozen samples to a third party company, have a sealed foam box containing dry ice ready and an airtight and sealed copy of the health information sheet pre-filled by the participant. Make sure that participants place their unique study ID code in the identifier section of the sheet to prevent private information to be sent to the company.
- When ready, ship the vials to the company for analysis of biomarker for the Salivary Profile 1: Testosterone, Progesterone, DHEA, Cortisol, and Estradiol. Analysis of biomarkers was carried out by the manufacturer according to established protocols28. Place samples in an immunoassay plate reader and read the sample optical density at a wavelength of 450 nm.
NOTE: All biomarkers require a solid-phase, competitive enzyme immunoassay. A fixed amount of the conjugated biomarker competes for the binding sites with an antibody that corresponds to each biomarker. After incubation, unbound components are washed away and an enzyme substrate solution is added, forming a color representative to the biomarker. For example, progesterone begins with a blue color and upon reacting with the enzyme becomes yellow.
- Provide participants with a checklist to outline physical activities exerted during the 24 hr period prior to the study (e.g., physical activity, food and drink consumption, use of lotions or hormone based products) as described in the manufacturer protocol28(see step 2.4 and 2.5).
- Galvanic Skin Response
- Clean the galvanic skin sensors with prepackaged sterile 70% alcohol wipes or a sterile gauze containing a small amount of 70% isopropanol to remove residues from prior participants. Clean the device before and after every data collection session or participant, whichever comes first.
- One hour before the study session, remind participants to clean and dry their hands.
NOTE: Collection of salivary samples in combination with the galvanic skin response will require that collections time points are considered. For example, cortisol salivary collection should be collected within 20 min (step 3.1.2) while galvanic skin sensors need to be fitted and calibrated with 5 – 10 min as described in step 3.2.6. - Place a new or clean sports wrist band in the non-dominant writing wrist of the participant before installing the galvanic skin response sensor. Allow the participant to wear the wrist band for 1 hr prior to collecting data.
NOTE: The sports wrist band should be pre-labelled according to the unique wrist ID to avoid cross-contamination between participants - After 1 hr, remove the sport wrist band and place the galvanic skin sensor. Ensure that the galvanic skin sensor touches the median nerve of the wrist to ensure proper collection of pulse and heart rates. Unite the bands together in the sensor to ensure a tight fit on the wrist.
- Press the galvanic skin sensor indicator light. Wait until the light turns from red to green to white; a white color indicates that data collection has started.
- Ask the participant to wear the galvanic wrist sensor for a set period of time (5 – 10 min) with no physical movement from the non-dominating wrist to calibrate and collect baseline tonic data.
NOTE: A baseline data will be found when the galvanic skin response data levels off and no spikes or noise is longer visible. Check the manufacturer software using the assigned account by the manufacturer. Open a session and verify that data is being collected29. - Start a slide presentation containing representative and time stamped problem sets for each of the exams activities described in section 1.5. Ask the participant to record the times after completing each slide to allow for timestamps between the collected GSR data and the events to be compared.
- Periodically check that the galvanic skin sensor lights are white and verify that data is being collected.
NOTE: Sampling rate of the sensors presented in this work was based on manufacturer specifications and capabilities (rate of 8 Hz with a low electrical current of 1100 mAh).Galvanic response through electrodermal activity (EDA) was reported in microSiemens (µS) as tonic (baseline responses to an exam event or self-report/interview question) and phasic (acute, immediate response to an exam event or self-report/interview question, above a certain µS threshold). - Have handy a replacement battery and sensor, in case data collection problems occur during the session. At the end of the session, press the indicator light until it is turned off. Carefully remove the wrist sensor and wipe it with alcohol.
- Retrieve the data from the manufacturer software29. Log in to the account and click on Sessions. Select the date of the study and click on the Download tab. Download the data in .csv file for easy use.
- Track timelines of the events in the study. Download the data in the wrist sensor software session. Download data as a UNIX epoch time, which is the number of seconds that has elapsed since January 1, 1970 (not counting leap seconds). Convert the UNIX time to a GMT time zone if timelines need to be tracked in the data using an open source timestamp converter30. To convert this time, simply convert the time in the data sheet using this conversion (1 year = 365.24 days = 31556926 sec).
- Retrieve electrodermal data in wrist sensor software session. Click View on any of the desired sessions. Present the data as real-time or select by a set period of time by the researcher.
- Self-report Survey and Emotional Inquiries
- Before and after the slides, provide a pre- and post- survey.During the second collection point (end of the exam), collect the final salivary sample and then provide the self-report while collecting EDA data. Record all time points.
NOTE: This study used a modified Topics in Emotion scale developed by Broughton and colleagues31. - While wearing the galvanic skin sensor, ask the participants questions that paralleled the self-report.
NOTE: Keep in mind that timelines need to be considered if you are collecting salivary samples as indicated in step 3.1.2 and 3.3.3. - As possible, record the date and time for cross-checking of GSR data and its alignment with the timing of the self-reported responses in a laboratory notebook. In addition, include instructions in the slide show presentation explaining to the participants to annotate the time after completing the required instructions after each slide to enable cross-comparison of timed events with the GSR data.
- Ensure that the computer based software has consistent instructions and that each slide is clearly labelled. If keystroke setups are needed for the study, indicate in the slide instructions what keystroke should be pressed in each slide.
- Before and after the slides, provide a pre- and post- survey.During the second collection point (end of the exam), collect the final salivary sample and then provide the self-report while collecting EDA data. Record all time points.
- Wrap-Up Activities
- After collecting all the information, remove the wrist sensor and turn off the sensor data in reverse order to step 3.2.5. Guide the participants to a designated room offering food, beverage, drinks, and provide receipts for a future monetary compensation (if it was stated in the IRB consent form) for participation in the study.
- Follow-up the activity with an emailed notification of gratitude for participation and include available timeslots for discussion of the results with the participants.
Representative Results
This section illustrates representative examples of results that can be obtained from each measure, including the self-report. The intent of the figures is to present the utility of adding measures such as salivary biomarkers (Figure 1) and galvanic skin responses (Figure 2) to self-reports (Figure 3) in order to gain a greater spectrum of information from a classroom event (e.g., exams). For triangulation of self-reports with a specified measure (e.g., galvanic skin response), additional techniques such as interviews can provide a useful comparison method (Figure 3). The results in Figure 1 show that participants' biological responses differed by gender when comparing hormonal levels between the beginning and end of the exam. Hormones such as estradiol levels increased in males while progesterone levels increased in females (p <0.05). High estradiol levels has been tied to brain activation of verbal performance tasks32 while progesterone is related to spatial abilities such as mental rotation of an object12-16,.Spatial activities were upregulated in females through increased testosterone (p <0.05) whereas DHEA showed no significant differences due to gender (p = 0.39). DHEA and testosterone has been linked to increased visual-spatial performance in adults12-16, 34. Cortisol levels for females and males did not change pre- and post- the laboratory session (p = 0.41) possibly due to exceeding short half-life of cortisol (~45 min) during the second salivary collection19,20,32.These results show that student performance is differential, in this case by gender, and that hormonal biomarkers can be a useful tool in identifying these differences.
Measurement of emotion via physiological arousal using galvanic skin response, demonstrated differential responses unrelated to the type of exam (p >0.05) but showed instances of cognitive engagement (sustained tonic levels) during the test-taking experience for all participants as seen by the GSR tonic peaks compared to the baseline (initial rest phase). A representative GSR data set is included in Figure 2. A sustained physiological arousal was found during the final data collection session where self-report and interview responses were collected. Self-report responses completed by the participants as they wore the galvanic skin sensor (Figure 3A)31,36 suggests no perceived differences in emotions by the participant to the exam (p = 0.055). However, when students were ask to response to the interview questions that paralleled the self-report, physiological arousals were found as seen by the increased tonic responses (Figure 3B). This result suggest that emotional activation may require cognitive and verbalized recollection of events.
The data demonstrates that verbal and spatial activation is differential in participants despite obtaining equal scores in their exams (data not shown). Physiological arousals were dependent on the mental recollection by the participants when performing a sequentially harder exam problem (progression from MCT to PSVT-R). Furthermore, use of written self-reports did not demonstrate a significant physiological response or a self-reported emotionally significant difference by the participants. When asked to verbalize the self-reported items, emotional arousals were found in the participants. Together, the data points to the different mental resources used by students when performing an exam. It highlights the importance of allowing students to cognitively recollect their thoughts as they sequentially perform increasingly difficult problems in and exam. Finally, the study results highlight how self-report alone are not sufficient to fully represent the spectrum of responses from students. Thus, inclusion of biological and physiological measures to self-reports and interviews can assist educational researchers to acquire a more robust data set that can help explain the complex performance of students in an educational setting.
Figure 1. Hormonal Activities for Female and Male Engineering Student Participants Before and After a Laboratory Study Session. Percent change of hormone levels before and after the laboratory session36 indicates that females increased in both progesterone and testosterone and decreased in estradiol levels when compared to males before and after the exam (paired T-test p <0.05 for estradiol; paired T-test p <0.05 for progesterone; paired T-test p <0.05 for testosterone). Cortisol and DHEA, measures of stress19,20 and spatial ability12-16,33, did not show significant changes (p = 0.39 and p = 0.42, respectively), despite obtaining equal scores in their exams (data not shown). Please click here to view a larger version of this figure.
Figure 2. Galvanic Skin Response of an Engineering Student Performance during a Laboratory Study Session. Galvanic skin tonic response in µS collected for an engineering student undergoing a series of representative activities in a laboratory study session. The initial time represents a resting phase to establish a baseline. Following this, the student completed representative engineering problems for the exam. Physiological arousals were dependent on the mental recollection by the participants when performing a sequentially harder exam problem (progression from MCT to PSVT-R). Afterwards, the student completed a written self-reflective 10-item survey, which showed no effect followed by an interview (containing parallel questions to the self-report) where an emotional arousal was seen. Please click here to view a larger version of this figure.
Figure 3. Representative Side-by-side Comparison of Self-reported Responses from Emotion Survey and Interview. Comparison of galvanic skin response of engineering student self-reported responses (A) and interview (B). Panel A shows descriptive statistics (n = 7) in which a paired sample T-test analysis showed no statistically significant differences (p = 0.055) between the pre- and post- self-report surveys. Panel B demonstrates GSR increase in arousal during the second self-report (step 13 – 14) as well as during the interviews (15a – 16). Arousal fluctuations were greater during the interview, compared to the surveys. Please click here to view a larger version of this figure.
Discussion
This protocol describes the integration of self-report surveys and inquiries, salivary biomarkers, and galvanic skin responses to study individual differences in classroom activities during a laboratory session. This protocol has many advantages for researchers seeking to identify academic achievement emotions, emotional regulation, and affective responses to different activities in an instructional setting, especially during assessment periods (e.g., exams). Whereas traditionally, self-reports and academic grades have been used to understand how students develop competencies in the classroom and/or engagement to a course, our methods can more comprehensively represent nearer to real-time responses of students during academic activities.
For success of this protocol, it is crucial that salivary biomarkers and galvanic skin responses are collected diurnally, that the participants are aware of the behavioral and biological restrictions of this study (e.g., health conditions), and that continual follow-up with the participants is conducted. Also, care should be taken in the handling of the samples as this should be done in aseptic conditions. Some limitations of the study include the allowable time frame (e.g., 10 – 12 days after the menstrual period) for salivary data collection for females as well as difficulty in precisely timing the events with the real-time galvanic skin response collection (e.g., salivary cortisol may require a 20 min collection time while galvanic skin sensor calibration requires 5 min). As such, data collection procedures should consider proper time stamping of sections and problems that participants complete during the laboratory session to ensure proper statistical data analysis and triangulation. Finally, due to the delicate nature of the self-reported emotions, emotional responses, and biological and physiological data is attained and assessed, protocols should follow the Institutional Review Board for Human Subjects policies and procedures.
With technological developments in non-intrusive wearable technologies and biological markers, methodologies can be combined to triangulate complex undergraduate students' experiences and performance to academic tasks. This method expands the potential of self-reported surveys and physiological information that has normally not been combined to understand nearer to real-time emotion and cognitive responses to different classroom activities.
One direction for this protocol is to incorporate the methods in a larger setting (e.g., classroom) in real-life contexts. For this, additional considerations on time, coordination on the use of resources and software, management of hormone kit and GSR wrist sensors usage will need to be considered as well as important mechanisms between the constructs.
Açıklamalar
The authors have nothing to disclose.
Acknowledgements
Maria Manuela Valladares is supported by a Utah State University Research Catalyst SEED Grant attained from Idalis Villanueva.
Materials
| Salivary Kit | ZRT Laboratories | Female/Male Saliva Profile I | An account needs to be created with the company before sending samples; diurnal samples are recommended for this protocol |
| Galvanic Skin Response Wrist Sensor and Band | Empatica, Inc. | E3P.v4 | M/L fits for the majority of the population although you may need to purchase a smaller size for thin wrists |
| E-Prime Software | Psychology Software Tools | E-Prime 2.0 Professional | Consider time stamping separately from software |
Referanslar
- Jones, B. D., Paretti, M. C., Hein, S. F., Knott, T. W. An analysis of motivation constructs with first-year engineering students: Relationships among expectancies, values, achievement, and career plans. J. Eng. Ed. 99 (4), 319-336 (2010).
- Pekrun, R. The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Ed. Psych. Rev. 18, 315-341 (2006).
- Bandura, A., Millard, M., Peloso, E. A., Ortman, N. Effects of peer education training on peer educators: leadership, self-esteem, health knowledge, and health behaviors. J. Coll. Stud. Dev. 41 (5), 471-478 (2000).
- Christenson, S. L., Havsy, L. H., Zins, J. E., Weissberg, R. P., Wng, M. C., Walberg, H. J. Family-school-peer relationships: Significance for social, emotional, and academic learning. Building academic success on social and emotional learning: What does the research. , 59-79 (2004).
- Struthers, C. W., Perry, R. P., Menec, V. H. An examination of the relationship among academic stress, coping, motivation, and performance in college. Res. Higher Ed. 41 (5), 581-592 (2000).
- Hsieh, P., Sullivan, J. R., Sass, D. A., Guerra, N. S. Undergraduate engineering students’ beliefs, coping strategies, and academic performance: An evaluation of theoretical models. J. Exp. Ed. 80 (2), 196-218 (2012).
- Pekrun, R., Bühner, M., Pekrun, R., Linnenbrink-Garcia, L. Chapter 28, Self-report measures of academic emotions. International Handbook of Emotions in Education. , 561-566 (2014).
- Frenzel, A. C., Pekrun, R., Dicke, A. L., Goetz, T. Beyond quantitative decline: Conceptual shifts in adolescents’ development of interest in mathematics. Dev. Psych. 48, 1069-1082 (2012).
- Benson, L., Kirn, A., Faber, C. J. CAREER: Student Motivation and Learning in Engineering. 121st ASEE Conference and Exposition, Paper ID 9338. , (2014).
- Matusovich, H., Streveler, R. A., Miller, R. L. Why Do Students Choose Engineering? A Qualitative, Longitudinal Investigation of Students’ Motivational Values. J. Eng. Ed. 99 (4), 289-303 (2013).
- Stump, G. S., Husman, J., Corby, M. Engineering Students’ Intelligence Beliefs and Learning. J. Eng. Ed. 103 (3), 369-387 (2014).
- Berenbaum, S. A. Gonadal hormones and sex differences in behavior. Dev. Neuropsych. 14, 175-441 (1998).
- Halpern, D. F., LaMay, M. L. The Smarter Sex: A Critical Review of Sex Differences in Intelligence. Educational Psychology Review. 12 (2), 229-246 (2000).
- Schöning, S., et al. Functional anatomy of visuo-spatial working memory during mental rotation is influenced by sex, menstrual cycle, and sex steroid hormones. Neuropsychologia. 45 (4), 3203-3214 (2007).
- Hausman, M., Slabberkoorn, D., Van Goozen, S. H. M., Cohen-Kettenis, C. Sex Hormones Affect Spatial Abilities during the Menstrual Cycle. Behav. Neurosci. 114 (6), 1245-1250 (2000).
- Kimura, D. Sex Differences in the Brain: Men and women display patterns of behavioral and cognitive differences that reflect varying hormonal influences on brain development. Sci. Am. , 26-31 (2002).
- Valla, J., Ceci, S. J. Can Sex Differences in Science Be Tied to the Long Reach of Prenatal Hormones? Brain Organization Theory, Digit Ratio (2D/4D), and Sex Differences in Preferences and Cognition. Persp. Psy. Sci. 6 (2), 134-136 (2011).
- Berenbaum, S. A., Moffat, S., Wisniewski, A., Resnick, S., de Haan, M., Johnson, M. H. Neuroendocrinology: Cognitive effects of sex hormones. The Cognitive Neuroscience of Development: Studies in Developmental Psychology. , 207-210 (2003).
- Lundberg, U., Frankenhaeuser, M. Pituitary-adrenal and sympathetic-adrenal correlates of distress and effort. J. Psychosom. Res. 24 (3-4), 125-130 (1980).
- Rappolt-Schlichtmann, G., Wtamura, S. Inter- and transdisciplinary work: Connecting research on hormones to problems of educational practice. Mind, Brain, Ed. 4, 157-210 (2010).
- Jamieson, J. P., Nock, M. K., Mendes, W. B. Mind over matter: reappraising arousal cardiovascular and cognitive responses to stress. J. Exp. Psychol. 141, 417-422 (2012).
- Kreibig, S. D., Gendolla, G. H. E., Pekrun, R., Linnenbrink-Garcia, L. Chapter 30, Autonomic Nervous System Measurement of Emotion in Education and Achievement Settings. International Handbook of Emotions in Education. , 625-642 (2014).
- Villanueva, I., Raikes, A., Ruben, N., Schaefer, S., Gunther, J. The use of physiological tools to identify changes in affective responses for graduate students recently admitted into a scientific discipline. IEEE FIE Proceeding. , (2014).
- CEEB. . Special aptitude test in spatial relations. , (1939).
- Guay, R. . Purdue spatial visualization test. , (1976).
- Sorby, S. A., Baartmans, B. J. The development and assessment of a course for enhancing the 3-D spatial visualization skills of first year engineering students. J. Eng. Ed. 89 (3), 301-307 (2000).
- Bishop, J. E. Developing Students’ Spatial Ability. Sci. Teach. 45 (8), 20-23 (1978).
- . Monitoring the Autonomic Nervous System, Empatica Available from: https://www.empatica.com/science#electro-dermal-activity (2014)
- . . Unix Timestamp Conversion Tools. , (2014).
- Broughton, S. H., Sinatra, G. M., Nussbaum, E. M. “Pluto has been a planet my whole life!” Emotions, attitudes, and conceptual change in elementary students learning about Pluto’s reclassification. Res. Sci. Ed. 43 (2), 529-550 (2013).
- Wolkowitz, O. M., Rothschild, A. J. . Psychoneuroendocrinology: The Scientific Basis of Clinical Practice. , (2003).
- Craig, M. C., et al. Physiological variation in estradiol and brain function: a functional magnetic resonance imaging study of verbal memory across the follicular phase of the menstrual cycle. Horm. Beh. 53 (4), 503-508 (2008).
- Stangl, B., Hirshman, E., Verbalis, J. Administration of dehydroepiandrosterone (DHEA) enhances visual-spatial performance in post-menopausal women. Beh. Neurosci. 125 (5), 742-752 (2011).
- Hromatko, I., Tadinac, M. Testosterone levels influence spatial ability: further evidence for curvilinear relationship. Rev. of Psychol. 13 (1), 27-34 (2007).
- Villanueva, I., Goodridge, W. H., Wan, N. J. A., Valladares, M. M., Robinson, B. S., Jordan, K. Hormonal and Cognitive Assessment of Spatial Ability and Performance in Engineering Examination Activities. , (2014).
