The present protocol introduces a non-commercial self-developed application for collecting real-time on-site data, including psychological scales, GPS location, heart rate, and blood-oxygen saturation level, as well as the application’s operating procedures. An empirical study conducted in Taiwan in 2020 was used as an application example.
The current protocol aims to showcase the technology integration, providing a detailed description of adopting the HealthCloud app, developed by the Healthy Landscape and Healthy People Lab, National Taiwan University (HLHP-NTU), on smartphones and smartwatches to collect data on users’ real-time psychological and physiological responses and environmental information. A flexible and integrated research method was proposed because it can be difficult to measure multi-dimensional aspects of personal data in on-site studies in landscape and outdoor recreation research. An on-site study conducted in 2020 at the National Taiwan University campus was used as an application example. A dataset of 385 participants was used after excluding invalid samples. During the experiment, participants were asked to walk around campus for 30 min when their heart rate and psychological-scale items were measured, together with several environmental metrics. This work aimed to provide a possible solution to help on-site studies track real-time human responses that match ambient factors. Due to the app’s flexibility, its use on wearable devices shows excellent potential for multidisciplinary research studies.
Real-time data collection
In daily life, people benefit from the physical environment in many ways. For example, positive outcomes, such as psychological1 and heart rate restoration2 have been widely found. In addition, the relationships between ambient factors, such as temperature and humidity, and mental health have been discussed3,4. Studies have also explored the links between physiological and psychological responses, such as heart rate and stress5,6,7,8. A wide range of evidence for psychological and physiological benefits from exposure to nature has been found in well-controlled laboratory studies9,10, which may not have represented the diverse influential factors in the field. Therefore, to measure the relationships among real-time human responses, on-site studies are considered better to reflect the real-life scenario experience and reactions to the environments than laboratory simulations11. Moreover, human reactions to environments may depend on context12. Given the importance of understanding the relationship between people's psychological and physiological health and environmental quality, a real-time self-tracking measurement that can collect various information measures is urgently needed.
Ecological momentary assessments (EMAs) or experience sampling methods (ESMs) may represent solutions for on-site studies13,14. EMAs and ESMs aim to assess humans' momentary responses on-site in real-life scenarios15. By adopting self-tracking techniques, the responses, reactions, and on-site experiences can be measured freshly14. Participants are notified via signals, such as texts or notifications, to implement assessments in so-called signal-contingent sampling schemes15. The term "EMA" is primarily used in health-related studies13, while "ESM" tends to be used in leisure and outdoor recreation studies16. Nonetheless, the terms have occasionally been used interchangeably12.
The possibility of applying EMAs to environmental research studies was discussed by Beute et al.12, who pointed out that they would allow a greater variety of environments to be addressed than merely "natural" or "urban." For example, by adopting ambulatory measurement (such as through GPS location tracking), physiological responses during a walk can be matched with real-time location datasets, providing a richer spatial resolution of environment types and environmental characteristics7. Additionally, the real-time data collection allowed by EMAs ensures a high ecological validity, providing a complementary point of view from laboratory studies.
More and more on-site empirical studies have adopted wearable devices and smartphones to monitor personal health status in daily life and research purposes17,18,19,20. Adopting both of these devices may provide more advantages than using only a smartphone12. First, the access time using smartwatches was shorter than that using phones21, which may cause a reduced interruption burden. Second, watches provide a greater body closeness than smartphones22, and phones can be used as momentary databases to save and upload data. Third, smartwatches nowadays offer multiple sensors to different parameters, such as heart rate variability, electrocardiograms (ECG), and blood pressure23,24,25,26,27. The individual and the overall aspects of human responses can infer certain activities12. Finally, smartphones are usually carried in the pocket for smartphone-based studies, and when it comes to the questionnaires, extra work must be done compared to the case using smartwatches.
However, few studies have explored the relationships between psychological and physiological outcomes and environmental information. Therefore, this study showcases adopting a non-commercial self-developed app, the HealthCloud, on wearable devices, such as smartwatches, and smartphones, to collect real-time psychological, physiological, and environmental information.
The self-developed app and wearable devices
The app for use on wearable devices was developed by the Healthy Landscape and Healthy People Lab, National Taiwan University (HLHP-NTU), to provide more accessible and more flexible ways to track human responses and environmental data, allowing researchers to analyze further the relationships among human health and environmental information (Figure 1).
The app, based on iOS, provides multiple tasks and passive data-collection functions. The app collects self-reported data on the smartwatch, such as psychological-scale items measured through Pop Quiz questions on which users can rate their responses from one to five stars for quick and easy assessment. This type of question intervention may be considered a type of Micro interaction-EMA (µEMA)-an in situ data collection method requiring less attention and having a greater response rate than smartwatch-EMA28. Sensor-monitored physiological response data, including heart rate, heart rate variability, and blood oxygen saturation level, can be measured using the functions of the iOS. Heart rate is measured through the smartwatch's optical heart sensor using a technique called photoplethysmography29. The app detects the amount of blood flow using green LED lights with light-sensitive photodiodes, and the heartbeats per minute are also calculated. The heart rate variability (HRV) and blood oxygen concentration (SpO2) can be detected using apps. For the smartphone, the tasks, such as the Stroop Test (Figure 2B), and Image Capture task (Figure 2C), and the Environment Sound task (Figure 2D), the ambient conditions data, including relative humidity, weather, and altitude, are passively collected from several Application Programming Interfaces.
Figure 1: Overview of the app. The functions of the app on the smartwatch, smartphone, and database. Please click here to view a larger version of this figure.
Figure 2: The app tasks. Examples of the tasks that can be used on the app: from left to right, there is (A) The Pop-up question. (B) The Stroop Test. (C) The Image Capture task. (D) The Environment Sound task. Please click here to view a larger version of this figure.
All data will be uploaded to the backend website (access to cooperative researchers, see Table of Materials). The website provides several primary functions: a map display that shows users' current locations and heart rate (Figure 3), a datasheet for browsing and extracting data (Figure 4), and task configurations for modifying the frequency, priority, and content of the tasks (Figure 5). With such great flexibility and a wide range of measurements, researchers can easily select the previously stated task functions according to the research objectives. In addition, the app can benefit both users and researchers. The app provides their health reports and GPS location trajectories (Figure 6) according to the questions they have answered and their chosen routes. Thus, they can obtain a quick idea of their health status on the day and keep on tracking their health data.
Figure 3: The map displayed on the app database. The map display of the app database provides current information, including locations and heart rate, to the researchers. Please click here to view a larger version of this figure.
Figure 4: Datasheet on the app database. The data report of the display map in the app database, in which data can be exported by filtering the Time, Field, or Tester ID. Please click here to view a larger version of this figure.
Figure 5: The task configuration on the app database. The task priorities, time intervals, language, and content of the questionnaires can be modified. Please click here to view a larger version of this figure.
Figure 6: The health report for the app users. After using the app, the user can receive a set of individual results generated automatically. Please click here to view a larger version of this figure.
Representative study
To showcase the integration of diverse dimensions of data collection using the app on smartphones and smartwatches, an in situ study was conducted in 2020 at the National Taiwan University campus in Taipei City, Taiwan. Participants for the study were recruited on the social media fan page of National Taiwan University through an online form 1 week before the experiment. The form included the research purpose, process, location, participation conditions, a schematic diagram of the research device to be worn, and a space for readers to indicate their willingness to participate and the time at which they could do so. After completion, participants were notified of their experiment's exact time and location by email 2 days before the schedule. Since the research examines psychological changes, physiology, physical activity (walking), and sound and color perception, participants met the following conditions: (1) between 20-36 years old, (2) good physical and mental health, (3) not be in regular use of drugs affecting the central nervous system, (4) not be pregnant or breastfeeding, (5) have no history of cardiovascular disease, (6) can walk for more than 30 min on foot, (7) be able to identify a color.
On the day of the experiment, participants were provided with one set of smartphones and smartwatches, and a route map. Researchers presented a uniform explanation to the participants of the purpose of the research, the research process, the wearable devices, and the matters needing attention in the research process. During the walk, psychological responses were assessed using a Pop Quiz task every 5 min, and physiological responses, such as heart rate, were measured every minute by sensors in the smartwatch. After the experiment, participants were compensated with a 200 NTD equivalent gift card (~7 USD).
For the psychological measurement, this study considered landscape preferences and two aspects of the Perceived Restorative Scale Short Version30, namely, "being away" and "fascination." These aspects were measured by asking participants to rate the statements "This is a place which is away from everyday demands and where I would be able to relax and think about what interests me." and "That place is fascinating; it is large enough for me to discover and be curious about things." on a five-point Likert scale from (1) "strongly disagree" to (5) "strongly agree" to measure individual perceptions of the restorative factors of the environment based on attention restoration theory31. Landscape preference was assessed using a five-point Likert scale with the single question: "How much do you like the setting, for whatever reason?" from (1) "very little" to (5) "very much." The questionnaire was sent using the "Pop Quiz" task with a 5 min time interval, meaning participants received the questionnaire every 5 min.
For physiological measurement, heart rate (HR) while walking was used to represent the participants' physiological outcomes with a 1 min time interval. Environmental information, including GPS data (latitude and longitude), temperature, relative humidity, wind speed, and wind degree, were collected through the smartphone.
The whole protocol follows the National Taiwan University Research Ethics Committee Office's instructions for conducting human-related experiments. During participant recruitment, candidates were informed of their instructions and rights and the experiment's risks in both speech and writing, and the signed consent forms were collected. The app can be installed on smartphones and smartwatches (see Table of Materials).
1. Preparation of the psychological and physiological experiment
2. Participant recruitment
3. Preparation of wearable devices and the app
4. Data collection
5. Data analysis
Question (English) | Question (Chinese) | Indicator | ||
That is a place which is away from everyday demands and where I would be able to relax and think about what interests me. | Being-away | |||
That place is fascinating; it is large enough for me to discover and be curious about things. | Fascination | |||
How much do you like the setting, for whatever reason? | Preference |
Table 1: The format of the Pop Quiz questions. The psychological scale items adopted in this study were used as an example to present the format of the Pop Quiz questions.
Figure 7: "Field name" setting. In the backend website, typing the new field name is needed in black, and then click on Add to mark the target data. Please click here to view a larger version of this figure.
Figure 8: The operating procedures of the app. Users login. (A) Login interface of the app on the smartwatch; app on the smartwatch is started at (B). (C) The main page of the app, where the real-time data were shown. (D) The measurement was set to change the "Field Name" and "Test No." Please click here to view a larger version of this figure.
The original sample consisted of 423 individuals, of which 18 had to be excluded because of poor data quality due to instability of the beta version of the app and another 20 failed to complete all Pop Quiz questions. This led to an effective sample rate of 0.91. A dataset of 385 students (213 females, 172 males) from National Taiwan University were recruited. Participants were between 20-36 years old (M = 23.38, SD = 2.268). Regarding their psychological states, 514 ratings of preference (PREF, M = 3.74, SD = 1.033), 548 ratings of being away (AWAY, M = 3.51, SD = 1.101), and 523 ratings of fascination (FSCN, M = 3.30, SD = 1.135) were collected. For the physiological responses (i.e., heartrate, HR), 14,253 datapoints (Unit = beats per second, M = 107.83, SD = 15.002) were collected. Concerning environmental information, 14,253 datapoints related to GPS latitude (LAT, M = 25.018, SD = 0.002) and longitude (LONG, M = 121.539, SD = 121.533), 14,253 related to temperature (TEMP; Unit = Celsius degree, M = 33.87, SD = 1.517), 14,253 related to relative humidity (RH; Unit = percent, M = 63.25, SD = 6.603), 14,253 related to wind speed (WS; Unit = meters per second, M = 3.58, SD = 1.788), and 12,232 related to wind degree (WD, M = 232.26, SD = 82.952) were collected (Table 2). With these data, variables from different dimensions can be statistically analyzed to verify relationships in turn.
Dimensions | Items | N | Mean | SD | MIN | MAX | |
Psychological responses | PREF | 514 | 3.74 | 1.033 | 1 | 5 | |
AWAY | 548 | 3.51 | 1.101 | 1 | 5 | ||
FASCN | 523 | 3.3 | 1.135 | 1 | 5 | ||
Physiological responses | HR | 14,253 | 107.83 | 15.022 | 65 | 190 | |
Environmental information | LAT | 14,253 | 25.018 | 0.002 | 25.012 | 25.024 | |
LONG | 14,253 | 121.539 | 0.002 | 121.533 | 121.544 | ||
TEMP | 14,253 | 33.87 | 1.517 | 28.73 | 28.79 | ||
RH | 14,253 | 63.25 | 6.603 | 50 | 89 | ||
WS | 14,253 | 3.58 | 1.788 | 0.5 | 7.7 | ||
WD | 12,232 | 232.26 | 82.952 | 40 | 360 |
Table 2: Descriptive statistics for the psychological, physiological, and environmental data. 1. PREF = preference; 2. AWAY = being away; 3. FASCN = fascination; 4. LAT = latitude of GPS location; 5. LONG = Longitude of GPS location; 6. HR = heart rate; 7. TEMP = temperature; 8. RH = relative humidity; 9. WS = wind speed; 10. WD = wind direction.
Purposes of the study and significant findings
Wearable devices, such as smartphones and smartwatches, have been widely used to investigate physiological indicators or syndromes32,33,34, psychological states22,35; environmental information, or behaviors18,36. Most applications of smart devices have focused on one aspect of personal information. To the best of our knowledge, the present work is one of the few that provides an integrated and flexible assessment of psychological, physiological, and environmental data. Furthermore, unlike studies that have used only smartphones as research tools35,37, this protocol took advantage of the smartwatch to provide mobile questionnaires for current psychological state assessment and closely measure physiological data. On the other hand, smartphones were used for data storage, processing, and transfer.
Crucial steps in the protocol
To follow the protocol, it is crucial to consider the activity and schedule of the activities intended to be included. The content of the activity may affect the accuracy of the sensors on the smartwatch. During walking, for example, using smartwatches to collect heart rate showed great validity; however, the validity decreased as the intensity increased38. Second, the detailed instructions for the wearable devices and the app may help obtain standardized results. Improper operation of the devices affected their data collection. In our case, the instructions for the app and devices may need to be expanded, and solutions for common problems should be included. Even though complete instructions for the app and devices were included, a more extended period may be needed for participants to familiarize themselves with these items. Participants who did not own these devices may struggle initially, causing adverse novelty effects; however, the results of previous studies addressing the novelty of smartphones showed that participants using a borrowed phone engaged more than those using their own39. Finally, in this protocol, psychological states were measured using the smartwatch, and the interval between the questionnaires was set according to the study's aims. In previous long-term studies ranging from 2-10 weeks using smartphones as a health intervention, interventions were set to occur a few times a day40,41; in the short-term study, however, the entire experiment took less than 1 h to finish, with an intervention interval of 5 min. The high frequency of tasks may affect individuals' on-site experience.
Advantages and limitations of the protocol and research tools
One advantage of the app is that, by using it, several dimensions of real-time information, including psychological, physiological, and environmental data, can be tracked simultaneously and uploaded automatically. For instance, researchers can design their written questionnaires in any language. The measurement intensity can be adjusted with a high degree of freedom by modifying the time interval for task delivery. Participants can receive tasks on the smartwatch; while answering questions or completing tests, their heart rate, spatial location, and weather data are continuously collected and uploaded to the database. All personal data with helpful information, such as time and location data for tracing their environment or the environment's physical attributes, can also be analyzed.
One limitation of the research tools, the app, and devices, is that it is currently only available on iOS. This may restrict the usability of this protocol. Furthermore, the sizes of smartwatches may make them difficult to read for specific populations, such as older adults. However, wearable devices in human response and environmental studies have been growing noticeably42,43,44, and they are affordable compared to the medical instruments used for physiological data collection. They collect a greater amount and a more comprehensive range of data than smartphones alone45. Therefore, studies that seek to examine daily life scenarios will have significant advantages in carrying convenience and continuous data collection12,45. Another limitation of the app is that the weather data, collected based on the nearest available information, are the same for all the participants if their geographical locations are relatively close. However, relationships and comparisons between subjects from different sites or on different dates can still be feasibly investigated.
Summary and future studies
This study shows the potential of adopting the app on wearable devices as a research tool. According to the protocol (a set of pioneer procedures), three dimensions of information were successfully measured and analyzed. Researchers can subjectively and objectively measure the human-environment relationship by following, expanding, or adapting the protocol. This represents a unique and efficient means of conducting longitudinal or cross-sectional studies. Future studies may emphasize physical environment quality, an essential factor in landscape studies, to address the factors that this protocol did not mention. As a result, the objective measurement of environmental quality, for example, by assessing landscape structures in environmental photos collected using Image Capture tasks in the app, becomes possible. Moreover, the protocol can also be applied to different populations or sites. With this set of standardized procedures, the study's scale can be expanded to regional or national scales instead of merely focusing on one site.
The authors have nothing to disclose.
The Council of Agriculture of Taiwan funded the research project and the HealthCloud app development from 2018 to 2020 [109 agricultural science – 7.5.4-supplementary-#1(1)] ([109 -7.5.4--#1(1)]).
Apple Watch 6 | Apple | For the use of the HealthCloud app, such as Pop-up questions, heart rates measurement. | |
iPhone | Apple | For the use of the HealthCloud app, such as GPS location collection, weather data colledction, data storage, data transfer. | |
HealthCloud | Self-developed | The HealthCloud app, adopting Apple Watch and iPhone, was developed by Healthy Landscape and Healthy People Lab, National Taiwan University (HLHP-NTU) to track human responses. It adopted several APIs such as HealthKit, ResearchKit, Weather API and AppleWatch applications including Breathe app, and Blood Oxygen app to collect physiological status and psychological states, and environmental data in aims of further analyzing the relationships between human health and the environmental information. The link to the app in APP Store is as following: https://apps.apple.com/tw/app/healthcloud/id1446179518?l=en |
|
backend website | The backend website of HealthCloud app for the use of the configuration of the tasks, data exportation, and the display of users. http://healthcloud.hort.ntu.edu.tw/ |
||
HealthKit | Apple | For the use of retrieving the data of physiological responses such as heart rate, heart rate variability, and blood oxygen saturation level. The link to the HealthKit: https://developer.apple.com/documentation/healthkit |
|
ResearchKit | Apple | This kit includes a variety of tasks for the use of research purposes. The functions adopted in HealthCloud app include Image Capture task, environment sound task, Stroop Test, to the Pop Questions of psychological state measurements such as perceived restorativeness scale, landscape preferences. The link to the ResearchKit: https://www.researchandcare.org/ |
|
Weather API | OpenWeather | For the use of collecting the real-time environmental data, including humidity, weather, global positioning system location, altitudes, etc., from the nearest weather station. The link to the Weather API: https://openweathermap.org/api |
|
Breathe app | Apple | For the use of assessing the real-time heart rate variability (HRV). This app was not included in the procedures of this pilot study. However, the HealthlCloud is now capable of retrieving the HRV data collected from Breathe app. The link to the Breathe app: https://apps.apple.com/us/app/breathe/id1459455352 |
|
Blood Oxygen app | Apple | For the use of assessing the real-time Blood Oxygen Concentration level (SpO2). The latest version of HealthlCloud is capable of retrieving the SpO2 data collected from app. This app was not included in the procedures of this pilot study. However, The measurement of Blood Oxygen app: https://support.apple.com/en-us/HT211027 The link to the Blood Oxygen app: https://apps.apple.com/us/app/breathe/id1459455352" |
|
IBM SPSS Statistics 25 | IBM | For the use of statistical analysis. The link to the Blood Oxygen app: https://www.ibm.com/support/pages/downloading-ibm-spss-statistics-25 |