A Method to Study Adaptation to Left-Right Reversed Audition

Atsushi Aoyama

doi:10.3791/56808

JoVE Journal > Behavior

Behavior

A Method to Study Adaptation to Left-Right Reversed Audition

Published: October 29, 2018

doi:

10.3791/56808

Atsushi Aoyama^1,2

¹Faculty of Environment and Information Studies, Shonan Fujisawa Campus (SFC),Keio University, ²School of Information Environment,Tokyo Denki University

Summary

The present study proposes a protocol to investigate the adaptation to left-right reversed audition achieved only by wearable devices, using neuroimaging, which can be an effective tool for uncovering the adaptability of humans to a novel environment in the auditory domain.

Abstract

An unusual sensory space is one of the effective tools to uncover the mechanism of adaptability of humans to a novel environment. Although most of the previous studies have used special spectacles with prisms to achieve unusual spaces in the visual domain, a methodology for studying the adaptation to unusual auditory spaces has yet to be fully established. This study proposes a new protocol to set-up, validate, and use a left-right reversed stereophonic system using only wearable devices, and to study the adaptation to left-right reversed audition with the help of neuroimaging. Although individual acoustic characteristics are not yet implemented, and slight spillover of unreversed sounds is relatively uncontrollable, the constructed apparatus shows high performance in a 360° sound source localization coupled with hearing characteristics with little delay. Moreover, it looks like a mobile music player and enables a participant to focus on daily life without arousing curiosity or drawing attention of other individuals. Since the effects of adaptation were successfully detected at the perceptual, behavioral, and neural levels, it is concluded that this protocol provides a promising methodology for studying adaptation to left-right reversed audition, and is an effective tool for uncovering the adaptability of humans to a novel environments in the auditory domain.

Introduction

Adaptability to a novel environment is one of the fundamental functions for humans to live robustly in any situation. One effective tool for uncovering the mechanism of environmental adaptability in humans is an unusual sensory space that is artificially produced by apparatuses. In the majority of the previous studies dealing with this topic, special spectacles with prisms have been used to achieve left-right reversed vision¹^,²^,³^,⁴^,⁵ or up-down reversed vision⁶^,⁷. Furthermore, exposure to such vision from a few days to more than a month has revealed perceptual and behavioral adaptation¹^,²^,³^,⁴^,⁵^,⁶^,⁷ (e.g., capability to ride a bicycle²^,⁵^,⁷). Moreover, periodic measurements of the brain activity using neuroimaging techniques, such as electroencephalography (EEG)¹, magnetoencephalography (MEG)³, and functional magnetic resonance imaging (fMRI)²^,⁴^,⁵^,⁷, have detected changes in the neural activity underlying the adaptation (e.g., bilateral visual activation for unilateral visual stimulation⁴^,⁵). Although the participant's appearance becomes strange to some extent and great care is needed for the observer to maintain the participant's safety, reversed vision with prisms provides precise three-dimensional (3D) visual information without any delay in a wearable manner. Therefore, the methodology for uncovering the mechanism of environmental adaptability is relatively established in the visual domain.

In 1879, Thompson proposed a concept of pseudophone, "an instrument for investigating the laws of binaural audition by means of the illusions it produces in the acoustic perception of space"⁸. However, in contrast to the visual cases¹^,²^,³^,⁴^,⁵^,⁶^,⁷, few attempts have been made to study the adaptation to unusual auditory spaces, and no noticeable knowledge has been obtained to date. Despite a long history of developing virtual auditory displays⁹^,¹⁰, wearable apparatuses for controlling 3D audition have rarely been developed. Hence, only a few reports examined the adaptation to left-right reversed audition. One traditional apparatus consists of a pair of curved trumpets that are crossed and inserted into a participant's ear canals in a contrariwise manner¹¹^,¹². In 1928, Young first reported the use of these crossed trumpets and wore them continuously for 3 days at most or a total of 85 h to test adaptation to left-right reversed audition. Willey et al.¹² retested the adaptation in three participants wearing the trumpets for 3, 7, and 8 days, respectively. The curved trumpets easily provided left-right reversed audition, but had an issue with the reliability of spatial accuracy, wearability, and strange appearance. A more advanced apparatus for the reversed audition is an electronic system in which left and right lines of head/earphones and microphones are reversely connected¹³^,¹⁴. Ohtsubo et al.¹³ achieved auditory reversal using the first ever binaural headphone-microphones that were connected to a fixed amplifier and evaluated its performance. More recently, Hofman et al.¹⁴ cross-linked complete-in-canal hearing aids and tested adaptation in two participants that wore the aids for 49 h in 3 days and 3 weeks, respectively. Although these studies have reported high performance of sound source localization in the front auditory field, the sound source localization in the backfield and a potential delay of electrical devices have never been evaluated. Especially in Hofman et al.'s study, the spatial performance of the hearing aids was guaranteed for the front 60° in the head-fixed condition and for the front 150° in the head-free condition, suggesting unknown omniazimuth performance. Moreover, the exposure period may be too short to detect phenomena related to the adaptation as compared with the longer cases of reversed vision²^,⁴^,⁵. None of these studies have measured brain activity using neuroimaging techniques. Therefore, the uncertainty in spatiotemporal accuracy, the short exposure periods, and the non-utilization of neuroimaging could be reasons for the small number of reports and the limited amount of knowledge on adaptation to left-right reversed audition.

Thanks to the recent advances in wearable acoustic technology, Aoyama and Kuriki¹⁵ succeeded in constructing a left-right reversed 3D audition using only wearable devices that recently became available and achieved the omniazimuth system with high spatiotemporal accuracy. Moreover, approximately a 1-month exposure to reversed audition using the apparatus exhibited some representative results for MEG measurements. Based on this report, we describe, in this article, a detailed protocol to set-up, validate and use the system, and to test the adaptation to left-right reversed audition with the help of neuroimaging that is performed periodically without the system. This approach is effective for uncovering the adaptability of humans to a novel environment in the auditory domain.

Protocol

All methods described here have been approved by the Ethics Committee of Tokyo Denki University. For every participant, informed consent was obtained after the participant received a detailed explanation of the protocol.

1. Setup of the Left-Right Reversed Audition System

Setup of the Reversed Audition System without a Participant
1. Prepare a linear pulse-code-modulation (LPCM) recorder, binaural microphones, and binaural in-ear earphones.
2. Connect the left and right lines of the microphones crossly to the LPCM recorder so that left-right reversed analogue sound signals are digitalized. Moreover, connect the left and right lines of the earphones straight through to the recorder so that the reversed digitalized signals are immediately played.
  NOTE: In the case of employing the binaural earphone-microphones as binaural earphones, do not use the earphone parts in order to reduce the spillover of the sounds that go through the microphone parts.
3. Put the bodies of the microphones and the earphones together for each ear with slight isolation by sound proofing materials, and cover the microphones with dedicated windscreens for suppressing the wind noise.
4. Insert rechargeable batteries and a large-capacity high-speed memory card into the LPCM recorder and turn it on. Set the recording conditions properly in such a manner that the sound signals are recorded on the memory card as an LPCM format at a sampling rate of 96 kHz with a 24-bit depth.
5. Place the body of the system into a pocket-sized bag.
Setup of the Reversed Audition System with a Participant
1. Instruct a participant to insert the earphones of the reversed audition system tightly into the ear canals.
2. Disconnect the lines for the left and right microphones and connect the dominant-ear side of the microphone straight through to the recorder.
3. Instruct the participant to take off and put on the dominant-ear side of the system repetitively while adjusting the sound volume of the recorder to make the subjective loudness of direct (normal) and indirect (reversed) sounds equal (as close as possible).
4. Check the loudness for the non-dominant ear as well, and connect all the lines of the system back again.
5. Place the system into the participant’s pocket, fix the cords on the participant’s clothes appropriately to prevent them from becoming entangled, and pick up unwanted noises.

2. Validation of the Left-Right Reversed Audition System

NOTE: Perform the following steps to validate the left-right reversed audition system, irrespective of experiments studying adaptation to left-right reversal.

Validation of the Sound Source Localization of the Reversed Audition System
1. Locate a digital angle protractor whose initial direction is defined as 0° at the center of an anechoic room, and assume a virtual circle centered at this point with a radius of 2 m. Along the virtual circle, mark 72 possible sound sources at every 5° from -180° to 175° in a clockwise manner, and set up plane-wave speakers at these points directed towards the center of the circle.
2. Set up a video camera near the center of the room to record the display of the digital protractor.
  NOTE: Since the display of the protractor moves with the protractor’s body, the field of view of the video should be large enough to cover all the possible areas. Moreover, the video camera should be carefully placed in order to not disturb the participant’s sitting position and the sound presentation.
3. Prepare for two sessions of sound source localization: in the first session, the participant does not put on the reversed audition system. In the second session, the participant puts on the equipment, calibrates it, and checks the system (as explained in step 1.2) as quickly as possible.
4. Guide the participants to sit comfortably and blindfolded at the center of the circle facing a 0° sound source and wait for the experiment to start.
5. Conduct two sessions of sound source localization. In both sessions, have the participant use the protractor to indicate the perceived sound direction as precisely as possible without moving the head.
6. For each session, start video-recording the angle display of the protractor, and present 1000-Hz sounds at 65-dB sound pressure level (SPL) from any of the sound sources: the sound at one location is randomly switched to the sound at another location every 10 s in such a way that each location is used once.
  NOTE: Here we use MATLAB with the Psychophysics Toolbox¹⁶^,¹⁷^,¹⁸. Although this toolbox is commonly used to present sounds, any reliable stimulation software can also be used.
7. After each session, stop the video-recording and instruct the participants to take a break for sufficient amount of time.
8. Read the trial-by-trial perceptual angles displayed on the protractor from the recorded video, and evaluate the spatial performance of the reversed audition system by comparing the perceptual angles in the normal and the reversed conditions against the physical angles defined by the direction of sound sources.
Validation of the Delay of the Reversed Audition System
1. Put the reversed audition system on a desk in a calm room with no participants.
2. Disconnect a line to the left microphone, and place a plane-wave speaker and the left earphone as close as possible to the right microphone.
3. Start recording direct (normal) sounds from the speaker and indirect (reversed) sounds from the left earphone simultaneously through the right microphone.
4. Present 1-ms click sounds from the speaker with a moderate inter-stimulus interval at 65-dB SPL.
5. After a sufficient number of trials, stop presenting and recording the sounds.
6. In order to confirm the symmetrical configuration of the system, repeat the same steps above using the right earphone and the left microphone.
7. Read the recorded sound data using software (e.g., MATLAB) and evaluate the difference between the onset timings of the direct (normal) sounds and indirect (reversed) sounds, which corresponds to a potential delay caused by the time spent passing through the electrical path in the system.

3. Studying the Adaptation to Left-Right Reversed Audition

Procedure of the Exposure to Reversed Audition
1. Remind the participants repeatedly of their right to quit the exposure at any time.
  NOTE: Stop the exposure as soon as possible if the participant reports sickness or if an observer notices any sign that the participant wants to quit the exposure for any reason.
2. Prepare a sufficient number of spare rechargeable batteries and large-capacity high-speed memory cards to allow the participant to replace them at anytime.
3. Instruct the participant to wear, calibrate, and check the reversed audition system by themselves during the exposure period, as explained in step 1.2. Perform the same procedure each time the participant wears the system after each interruption.
4. Instruct the participant to perform daily-life activities while wearing the system continuously for approximately a month, except while sleeping, bathing, neuroimaging, and other emergency times. In these cases, ask participants to remove the system and immediately insert earplugs into their ears to prevent recovery of adaptation.
  NOTE: Although it is ideal for the participant to wear the system all day and night, it is strongly recommended that the system not be worn while sleeping and bathing in order to prevent unexpected loud noises and electrical shocks, respectively.
5. Replace the batteries and memory cards routinely before battery exhaustion and memory overcapacity, respectively. Remove the system and replace it with earplugs quickly in a silent place without producing any sound.
6. When a participant needs to move around outside, drive the participant in a car, accompany the participant on the move, or ask them to use safe means of transportation for acts performed alone.
  NOTE: Great care should be taken by the researcher in order to not endanger the participant’s safety during the exposure period, especially when the participant goes outside. Prohibit the participant from performing any dangerous behaviors.
7. In order to facilitate adaptation, instruct the participant to experience situations involving high auditory input, such as walking in a shopping mall or a campus, having a conversation with more than two persons, and playing 3D video games, for as long as possible.
8. Instruct the participant to keep a diary or provide a subjective report to an observer as frequently as possible about perceptual and behavioral changes, experienced events, and anything that the participant notices.
9. After the target exposure period, instruct the participant to take off the reversed audition system.
  NOTE: It is also important to follow up about the perceptual and behavioral changes in order to examine the recovery process from the adaption to left-right reversed audition.
Neuroimaging During the Exposure to Reversed Audition
1. Instruct the participant to train on a task that will be used during the neuroimaging experiments as sufficiently as possible.
  1. For example, train the participant to perform a selective reaction time task in two conditions, compatible and incompatible¹⁵. The compatible condition consists of responding immediately to the right-ear sound with the right index finger and to the left-ear sound with the left index finger. The incompatible condition consists of responding immediately to the right-ear sound with the left index finger and to the left-ear sound with the right index finger.
  2. Use 1000-Hz sounds at 65-dB SPL for 0.1 s with an inter-stimulus interval of 2.5 – 3.5 s, which appears pseudorandomly on either ear side.
2. Before the exposure to reversed audition, conduct a neuroimaging experiment under the trained task.
  1. For example, record either MEG or EEG responses, as well as the left and right finger responses under the selective reaction time task¹⁵. The task consists of two compatible and two incompatible blocks that are alternatively arranged with an inter-block interval of at least 30 s, and with sounds appearing 80 times for each block through the inserted earphones with plastic ear tubes.
    NOTE: Although a 122-channel MEG system was used in Aoyama and Kuriki¹⁵, a multi-channel EEG system is also suitable for this protocol.
  2. For the MEG/EEG recording, set the sampling rate at 1 kHz and the analog recording passband at 0.03 – 200 Hz.
3. During approximately a 1-month exposure to reversed audition, conduct neuroimaging experiments under the trained task every week without the reversed audition system in exactly the same way as in the pre-exposure experiment (step 3.2.2).
  NOTE: The system is removed immediately before and put on immediately after each experiment.
4. One week after the exposure, conduct a neuroimaging experiment under the trained task in exactly the same way as the pre-exposure experiment (step 3.2.2).
5. Analyze the collected data before, during, and after the exposure to left-right reversed audition.
  1. For example, after rejecting the epochs contaminated with eye-related artifacts, removing the offset in the pre-stimulus interval, and setting the low-pass filtering at 40 Hz, average the MEG/EEG data from 100 ms before to 500 ms after the sound onset for the stimulus-response compatible and incompatible conditions¹⁵.
  2. Using an MNE software package¹⁹^,²⁰, estimate the sources of the brain activity with dynamic statistical parametric maps (dSPMs) overlaid on cortical surface images and quantify the intensities of brain activity with minimum-norm estimates (MNEs) for each time point of the averaged data.
  3. Calculate the auditory-motor functional connectivity from single-trial zero-mean MEG/EEG data from 90 to 500 ms after the sound onset for each condition
    NOTE: Here we use MATLAB with the Multivariate Granger Causality Toolbox²¹.
  4. For the behavioral data, calculate the mean reaction times for the stimulus-response compatible and incompatible conditions.

Representative Results

The representative results shown here are based on Aoyama and Kuriki¹⁵. The present protocol achieved left-right reversed audition with high spatiotemporal accuracy. Figure 1 shows the sound source localization in directions over 360° before and immediately after putting on the left-right reversed audition system (Figure 1A), in six participants, as indicated by the cosine similarity. As shown in Figure 1B, the perceptual angles in the normal condition were quite well correlated with the physical angles (positive correlation, adjusted R² = 0.99). The perceptual angles in the reversed condition were also well correlated with the physical angles (negative correlation, adjusted R² = 0.96; see also Figure 4 in Aoyama and Kuriki¹⁵), although there existed a slight perceptual bias toward the counterclockwise rotation, especially for sounds coming from the right-front and the left-back directions. Notably, the perceptual angles in the reversed condition were more correlated with the oppositely arranged perceptual angles in the normal condition (adjusted R² = 0.98) than the physical angles, as shown in Figure 1C. Furthermore, a potential delay of the system was estimated to be a constant 2 ms. The present protocol also achieved a natural wearing appearance, like listening to music with a mobile music player, thereby avoiding any stress of being noticed by other individuals.

Figure 1: Sound source localization in 360° directions, before and immediately after putting on the left-right reversed audition system, in six participants. (A) The constructed left-right reversed audition system. (B) Cosine similarity between perceptual angles and sign-regulated physical angles in the normal (blue) and reversed (red) conditions plotted against (unregulated) physical angles, respectively. While the physical angles are directly used for the cosine similarity in the normal condition, the signs of physical angles are inverted in the reversed condition. (C) Cosine similarity between perceptual angles in the reversed condition and oppositely arranged perceptual angles in the normal condition plotted against physical angles (purple). This figure has been modified from Aoyama and Kuriki¹⁵. Please click here to view a larger version of this figure.

The present protocol revealed perceptual changes to the reversed audition from a relatively early stage during the approximately 1-month exposure. Although a feeling of strangeness was reported just after the exposure, it began to decrease within a week of the exposure and continued to drop further over time. Mirror-image sounds were gradually perceived as normal, which also occurred with visual information and movements. One week after the end of the exposure period, all changes returned to the pre-exposure level. The present protocol detected not only perceptual but also behavioral and neural changes underlying the adaptation. Figure 2 shows changes in behavioral and neural responses during the selective reaction time task over the exposure time in a representative participant. As shown in Figure 2A, the mean reaction times for response-incompatible sounds were overall longer than those for response-compatible sounds from the pre-exposure period to the third week, but became slightly shorter in the fourth week. This relative inversion followed the transient elongation of the mean reaction times irrespective of compatibility in the second week. After the exposure, all mean reaction times returned to the initial level. The MNE intensities of the left and right N1m components exhibited similar trends to the mean reaction times, as shown in Figure 2B, although the compatible-incompatible relationship was inversed. The N1m components are distinct auditory evoked fields observed at about 90 ms after sound onset, and their source was confirmed to be located in the bilateral superior temporal planes using dSPMs. Overall, the intensities in the stimulus-response compatible conditions were higher than those in the incompatible conditions from the pre-exposure period to the third week, but were slightly lower in the fourth week. This relative inversion followed the transient enhancement of the intensities irrespective of compatibility and laterality in the second week. After the exposure, they returned to the initial levels.

Figure 2: Behavioral and neural responses during the selective reaction time task in a representative participant. (A) Mean reaction times for stimulus-response compatible and incompatible conditions. (B) Left and right auditory N1m intensities for stimulus-response compatible and incompatible conditions, as evaluated by minimum-norm estimates. Yellow zones indicate a period exposed to left-right reversed audition. This figure has been modified from Aoyama and Kuriki¹⁵. Please click here to view a larger version of this figure.

Furthermore, the present protocol revealed changes in the functional connectivity across the left and right auditory and motor areas during the selective reaction time task in two participants, as shown in Figure 3. The functional connectivity was tested with the Granger causality test at a threshold of p < 0.05. Initially, these auditory-motor areas communicated with each other irrespective of stimulus and response. However, after exposure to the reversed audition, the auditory-motor connectivity became unstable. Notably, in the second week, the auditory-motor connectivity was disrupted drastically, especially in the right motor-to-auditory feedback and left-to-right motor communication. Immediately after that, the connectivity recovered at the level of the first week, and returned to the initial level after the exposure.

Figure 3: Auditory-motor functional connectivity as tested by Granger causality tests during the selective reaction time task in two participants. Red, yellow, and no arrow(s) indicate the number of participants who showed significance at a threshold of p < 0.05 (N = 2, 1, and 0, respectively). LM and RM denote left and right motor areas, respectively, and LA and RA denote left and right auditory areas, respectively. This figure has been modified from Aoyama and Kuriki¹⁵. Please click here to view a larger version of this figure.

Discussion

The proposed protocol aimed to establish a methodology for studying adaptation to left-right reversed audition as an effective tool for uncovering the adaptability of humans to a novel auditory environment. As evidenced by the representative results, the constructed apparatus achieved left-right reversed audition with high spatiotemporal accuracy. Although the previous apparatuses for reversed audition¹¹^,¹²^,¹³^,¹⁴ were mostly reliable in the front auditory field, this protocol provides high performance in a 360º sound source localization coupled with hearing characteristics. Moreover, a potential delay of 2 ms lost through the electrical path in the system, which has never been evaluated in other electronic apparatuses¹³^,¹⁴, is considered to be negligible due to the human temporal auditory acuity²². Unlike the traditional apparatus of curved trumpets¹¹^,¹² with a strange appearance and uncomfortable fit, the reversed audition system used in the present protocol looks like a mobile music player and enables a participant to focus on daily life without arousing curiosity or drawing attention of other individuals. At this point, it is even superior to the apparatuses for reversed vision using prisms¹^,²^,³^,⁴^,⁵^,⁶^,⁷. Indeed, as evidenced by the representative results, around 1 month of wearing the apparatus achieved adaptation to left-right reversed audition at the perceptual, behavioral, and neural levels. As in previous protocols¹¹^,¹²^,¹³^,¹⁴, it was quite challenging to perform experiments with many participants, due to the long research period and difficulties in participant recruitment. However, individual results provided reliable, rich and valuable information about auditory adaptation (for details, see Aoyama and Kuriki¹⁵). Therefore, the present protocol is much better suited for facilitating the adaptation to reversed audition than any other previous protocols that have failed to noticeably advance knowledge about the adaptation¹¹^,¹²^,¹³^,¹⁴.

As a basic premise, the highest priority in the proposed protocol should be the participant's safety, health, and will during the exposure to the reversed audition. In order to preserve these, an observer must take great care and communicate with the participant as much as possible, especially during and immediately after the exposure period. If any of the conditions are unsatisfactory, an observer must stop the exposure immediately. Apart from that, one of the most critical steps of the protocol is to instruct the participant to experience situations involving high auditory input for as long as possible. Unlike visual cases where the retinal input has fine spatial resolution²³^,²⁴, exposure to reversed audition is less effective due to low auditory spatial resolution²⁵^,²⁶. In addition, non-environmental auditory events rarely occur in daily life, unless a person is subjected to high auditory inputs. Moreover, it is not enough for sounds to be directional and lateralized, but the sounds should also be accompanied by other sensory information or movement to facilitate the adaptation. Without this step, lower, or even no adaptive effect, is expected. Another critical step is to instruct the participant to train on a task as sufficiently as possible before the first neuroimaging experiment so that task performance converges at a certain level. This is necessary for a precise evaluation of the adaptive effect on behavioral and neural responses, because it is quite difficult to dissociate between the adaptive and the task learning effects over time. Preliminary reduction of the task learning effect thus promotes further analysis of the adaptation.

The present protocol can be flexibly modified, depending on the availability of experimental equipment and the purpose of study. For example, to validate the sound source localization of the reversed audition system, it is acceptable to employ another established method for sound source localization, instead of the digital angle protractor, and a sufficiently calm soundproof room, instead of an anechoic room. To study the adaptation to left-right reversed audition, the exposure period can be either shortened or prolonged and the frequency of neuroimaging can be either lower or higher, according to the situation. For further study, it is recommended to perform neuroimaging more frequently after the exposure period to investigate the recovery process after the adaptation. If neuroimaging is unavailable, it is possible to replace neuroimaging experiments by behavioral experiments. In this protocol, there is a possibility that a participant will request temporary suspension of the exposure due to inevitable reasons. Unless the participant agrees to insert earplugs into the ears during the suspended period, the exposure should be terminated due to unknown recovery effects on readaptation; a new experiment should be started with another participant. Another possible issue is that a balance of subjective loudness between left and right sounds becomes uncertain due to physical contact with the system or for other reasons. In that case, it is recommended for the participant to confirm, with the eyes closed, if the sounds emanating from the front are only localized at the front before readjusting the volume.

Even though the present apparatus showed high performance in 360° sound source localization, the results indicated a slight perceptual bias toward the counterclockwise rotation, especially for sounds coming from the right-front and the left-back directions. Assuming that the earphones are properly inserted into the participant's ear canals, two possibilities are considered for the asymmetrical distortion of the localization: individual acoustic characteristics and spillover of unreversed sounds. Acoustic characteristics are typically modeled as head-related transfer functions (HRTFs)²⁷, and common HRTFs are used for any participant in the current version of the apparatus without specific optimization. Thus, there is room to improve the apparatus by implementing individual HRTFs for each ear and participant. In contrast, slight spillover of unreversed sounds is relatively uncontrollable. Although separation of microphone and earphone parts of the system reduces the spillover and usual sounds are unlikely to generate perceptible bone conduction²⁸, it is technically difficult to prevent the spillover completely in a wearable way. Moreover, during the exposure, it is almost impossible to control bone-conducted self-produced voices; thus, there is nothing to do but to assume a symmetric distribution for them. Therefore, it is considered that the implementation of individual HRTFs is the priority to improve the apparatus and achieve more effective adaptation.

To our knowledge, this is the first successful protocol established for studying the long-term adaptation to precise left-right reversed audition with neuroimaging. In addition, this protocol has a great potential for extensive applicability in both auditory and multisensory research. For example, the system incorporating a microcomputer could be set up to induce different alterations in auditory space, such as an overall rightward shift or a compression of auditory space toward the center. Since spatial information is concordantly processed across sensory modalities, altered auditory space could be a strong tool to reveal mechanisms of multisensory spatial recalibration in a way similar to Zwiers et al.²⁹, who reported the effects of wearing prism lenses with spatially compressed vision on sound source localization. Nowadays, it is becoming increasingly popular to use currently available techniques in a multimodal manner, such as the simultaneous use of EEG and fMRI³⁰, and a delayed combined use of transcranial brain stimulation and EEG/MEG³¹. While the simultaneous use of two neuroimaging techniques compensates for their weaknesses reciprocally, the delayed combined use of neurostimulation and neuroimaging techniques reveals brain functions related to after-effects caused by the neurostimulation using the neuroimaging. Notably, an experimental scheme of the present protocol can be regarded as an expanded version of the latter case. Similar to the neurostimulation techniques, continuous wearing of a wearable apparatus with unusual sensory space causes after-effects of adaptation. These effects can be then measured by a neuroimaging technique. Therefore, the delayed combined use of a wearable apparatus and a neuroimaging technique reveals brain functions related to adaptation (as briefly pointed out in Aoyama and Kuriki¹⁵). From a general point of view, this scheme can provide new insights into neuroimaging studies with a variety of adaptive effects. In conclusion, the present protocol, under this scheme, provides a promising methodology for studying left-right reversed audition as a tool to uncover the adaptability of humans to a novel environment in the auditory domain.

Disclosures

The authors have nothing to disclose.

Acknowledgements

This work was partially supported by a grant from JSPS KAKENHI Grant Number JP17K00209. The author thanks Takayuki Hoshino and Kazuhiro Shigeta for technical assistance.

Materials

Linear pulse-code-modulation recorder	Sony	PCM-M10
Binaural microphones	Roland	CS-10EM
Binaural in-ear earphones	Etymotic Research	ER-4B
Digital angle protractor	Wenzhou Sanhe Measuring Instrument	5422-200
Plane-wave speaker	Alphagreen	SS-2101
Video camera	Sony	HDR-CX560
MATLAB	Mathworks	R2012a, R2015a	R2012a for stimulation and R2015a for analysis
Psychophysics Toolbox	Free	Version 3	http://psychtoolbox.org
Insert earphones	Etymotic Research	ER-2
Magnetoencephalography system	Neuromag	Neuromag-122 TM
Electroencephalography system	Brain Products	acti64CHamp
MNE	Free	MNE Software Version 2.7, MNE 0.13	https://martinos.org/mne/stable/index.html
The Multivariate Granger Causality Toolbox	Free	mvgc_v1.0	http://www.sussex.ac.uk/sackler/mvgc/

References

Sugita, Y. Visual evoked potentials of adaptation to left-right reversed vision. Perceptual and Motor Skills. 79 (2), 1047-1054 (1994).
Sekiyama, K., Miyauchi, S., Imaruoka, T., Egusa, H., Tashiro, T. Body image as a visuomotor transformation device revealed in adaptation to reversed vision. Nature. 407 (6802), 374-377 (2000).
Takeda, S., Endo, H., Honda, S., Weinberg, H., Takeda, T. MEG recording for spatial S-R compatibility task under adaptation to right-left reversed vision. Proceedings of the 12th International Conference on Biomagnetism. , 347-350 (2001).
Miyauchi, S., Egusa, H., Amagase, M., Sekiyama, K., Imaruoka, T., Tashiro, T. Adaptation to left-right reversed vision rapidly activates ipsilateral visual cortex in humans. Journal of Physiology Paris. 98 (1-3), 207-219 (2004).
Sekiyama, K., Hashimoto, K., Sugita, Y. Visuo-somatosensory reorganization in perceptual adaptation to reversed vision. Acta psychologica. 141 (2), 231-242 (2012).
Stratton, G. M. Some preliminary experiments on vision without inversion of the retinal image. Psychological Review. 3 (6), 611-617 (1896).
Linden, D. E., Kallenbach, U., Heinecke, A., Singer, W., Goebel, R. The myth of upright vision. A psychophysical and functional imaging study of adaptation to inverting spectacles. Perception. 28 (4), 469-481 (1999).
Thompson, S. P. The pseudophone. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science: Series 5. 5 (50), 385-390 (1879).
Wenzel, E. M. Localization in virtual acoustic displays. Presence: Teleoperators & Virtual Environments. 1 (1), 80-107 (1992).
Carlile, S. . Virtual Auditory Space: Generation and Applications. , (2013).
Young, T. P. Auditory localization with acoustical transposition of the ears. Journal of Experimental Psychology. 11 (6), 399-429 (1928).
Willey, C. F., Inglis, E., Pearce, C. H. Reversal of auditory localization. Journal of Experimental Psychology. 20 (2), 114-130 (1937).
Ohtsubo, H., Teshima, T., Nakamizo, S. Effects of head movements on sound localization with an electronic pseudophone. Japanese Psychological Research. 22 (3), 110-118 (1980).
Hofman, P. M., Vlaming, M. S., Termeer, P. J., van Opstal, A. J. A method to induce swapped binaural hearing. Journal of Neuroscience Methods. 113 (2), 167-179 (2002).
Aoyama, A., Kuriki, S. A wearable system for adaptation to left-right reversed audition tested in combination with magnetoencephalography. Biomedical Engineering Letters. 7 (3), 205-213 (2017).
Brainard, D. H. The Psychophysics Toolbox. Spatial Vision. 10 (4), 433-436 (1997).
Pelli, D. G. The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spatial Vision. 10 (4), 437-442 (1997).
Kleiner, M., Brainard, D., Pelli, D. What’s new in Psychtoolbox-3?. Perception. 36 (14), (2007).
Gramfort, A., et al. MEG and EEG data analysis with MNE-Python. Frontiers in Neuroscience. 7, 267 (2013).
Gramfort, A., et al. MNE software for processing MEG and EEG data. NeuroImage. 86, 446-460 (2014).
Barnett, L., Seth, A. K. The MVGC multivariate Granger causality toolbox: a new approach to Granger-causal inference. Journal of Neuroscience Methods. 223, 50-68 (2014).
Green, D. M. Temporal auditory acuity. Psychological Review. 78 (6), 540-551 (1971).
He, S., Cavanagh, P., Intriligator, J. Attentional resolution and the locus of visual awareness. Nature. 383 (6598), 334-337 (1996).
Anton-Erxleben, K., Carrasco, M. Attentional enhancement of spatial resolution: linking behavioural and neurophysiological evidence. Nature Reviews Neuroscience. 14 (3), 188-200 (2013).
Perrott, D. R., Saberi, K. Minimum audible angle thresholds for sources varying in both elevation and azimuth. Journal of the Acoustical Society of America. 87 (4), 1728-1731 (1990).
Grantham, D. W., Hornsby, B. W., Erpenbeck, E. A. Auditory spatial resolution in horizontal, vertical, and diagonal planes. Journal of the Acoustical Society of America. 114 (2), 1009-1022 (2003).
Xie, B. . Head-Related Transfer Function and Virtual Auditory Display. , (2013).
Stenfelt, S. Acoustic and physiologic aspects of bone conduction hearing. Advances in Oto-Rhino-Laryngology. 71, 10-21 (2011).
Zwiers, M. P., van Opstal, A. J., Paige, G. D. Plasticity in human sound localization induced by compressed spatial vision. Nature Neuroscience. 6 (2), 175-181 (2003).
Huster, R. J., Debener, S., Eichele, T., Herrmann, C. S. Methods for simultaneous EEG-fMRI: an introductory review. Journal of Neuroscience. 32 (18), 6053-6060 (2012).
Veniero, D., Vossen, A., Gross, J., Thut, G. Lasting EEG/MEG aftereffects of rhythmic transcranial brain stimulation: level of control over oscillatory network activity. Frontiers in Cellular Neuroscience. 9, 477 (2015).

Play Video

PDF

DOI

DOWNLOAD MATERIALS LIST

Cite This Article

Aoyama, A. A Method to Study Adaptation to Left-Right Reversed Audition. J. Vis. Exp. (140), e56808, doi:10.3791/56808 (2018).

Automatically Generated