One Dimensional Turing-Like Handshake Test for Motor Intelligence

1. Preparing the System

1. Hardware requirements:
   • Two PHANTOM desktop robots by SensAble Technologies, Inc.
   • 2 Parallel cards.
   • Minimum system requirements: Intel or AMD-based PCs; Windows 2000/XP, 250 MB of disc space.
2. Software requirements:
   • Driver-
     Download drivers from the SensAble technologies website http://www.sensable.com according to the computer operating system.
   • H3DAPI-
     Download the H3DAPI source code and install according to the instructions displayed in the installation walkthrough section in http://www.h3dapi.org/modules/mediawiki/index.php/H3DAPI_Installation
     We updated the code to fit our requirements that were not met in the existing code. The updated files can be downloaded from the handshake tournament website http://www.bgu.ac.il/~akarniel/HANDSHAKE//index.html
     Each file should be placed in the appropriate folder as described in the website above.
     After performing the changes, download CMake and compile the code. The compiling instructions can be found in http://www.h3dapi.org/modules/mediawiki/index.php/H3DAPI_Installation
   • X3D and python
     • Download python 2.5 from http://www.python.org/download/releases/2.5.5/
     • Download the python and x3d codes from the handshake website and place the code files in a dedicated directory, for instance: “C:codeDirectory”
     • Each handshake force model should be written in a separate python file. See Figure 1 for an example of a spring force model.
   • psignifit toolbox version 2.5.6 for Matlab, available at http://www.bootstrap-software.org/psignifit/

Figure 1. Force function in python. An example of a spring force model for a handshake

2. Experimental Protocol

• Open the command window and change to the python and x3d code directory by typing: cd C:codeDirectory.
• Create a folder with the subjects’ names in C:codesDirectory.
• In order to run the experiment type: h3dload code_name.x3d.
• Create a new random file by typing: random_file_name.txt. The random file defines the order in which the different handshakes appear.
• Enter the subjects’ names exactly as in the previously created folder.
• Following the original concept of the classical Turing test, each experiment consists of 3 entities: a human, a computer, and an interrogator. Two subjects (human and interrogator) each hold the stylus of one Phantom haptic device and generate handshake movements. They are asked to follow the instructions that appear on the screen (e.g., :”press Page Up for the first handshake”), for handshake forces to be applied. In all of the following methods, each trial consists of 2 handshakes, and the subjects are required to compare between them. The computer is a simulated handshake model which generates a force signal as a function of time and the one dimensional hand position and its derivatives.
  (1)F_model(t) = Φ[x(t),t] 0 ≤ t ≤ T
  F[x,t] stands for any causal operator, e.g., non-linear time-varying mechanical model of the one dimensional stylus movement, and T is the duration of the handshake. In the current study T=5 seconds.

1. Conducting the “pure” test and calculating the model human likeness grade MHLG_p
   The experiment starts with 12 practice trials in which all the handshakes (n=24) are human, such that the subjects simply shake hands with each other through the telerobotic system. The purpose of these practice trials is to enable the participants to be acquainted with a human handshake in the system.
   In the experiment, we compare four computer models. Each experimental block consists of 4 trials in which we compare the 4 tested models to a human handshake. One of the handshakes in each trial is an interaction with a force generated from one of the four models (a computer), and the other is with a human (the second subject). Therefore, the subjects function as both humans and interrogators. The order of the trials within each block is random and predetermined. Each experiment consists of 10 blocks, such that each computer handshake is repeated 10 times. An initial unanalyzed block is added for general acquaintance with the system and the task.
   For each model, the proportion of handshakes in which the subject chooses the model handshake over the human handshake as more human-like is calculated, to provide a value which is 0.5 when the model is indistinguishable from a human. We multiply this value by two in order to obtain the MHLG_p, such that MHLG=0 is clearly non-human like and MHLG=1 means that the tested handshake is indistinguishable from the human handshake.
2. Conducting the “weighted human-model” test and calculating the model human likeness grade MHLG_w
   In this protocol, there is only one interrogator subject. The other subject functions as the human entity in the handshakes.
   The experiment starts with 30 practice trials in which the interrogator experiences one human handshake and one computer handshake in each trial. In the end of the trial they are asked to choose which of the 2 handshakes was the human handshake. If they succeed, the screen displays “Correct!”, and if they didn’t choose the right handshake, a “Wrong!” message appears.
   After the practice block the experiment is conducted as follows: A trial consists of two handshakes. In one of the handshakes- the stimulus – the interrogator interacts with a combination of forces that comes from the human and a computer handshake model.
   (2) F = α_stimulus• F_human + (1-α_stimulus)•F_{stimulusModel}
   α_stimulus is equally distributed from 0 to 1, e.g:
   α_stimulus= {0, 0.142, 0.284, 0.426, 0.568, 0.710, 0.852, 1}
   The other handshake – the reference – is a fixed combination of forces generated from the human and a reference model:
   (3) F = α_reference• F_human + (1-α_reference)•F_{referenceModel} ; α_reference=0.5
   At the end of each trial the interrogator is requested to choose the handshake that felt more human-like.
   In each experiment we compare two test models and one base model.
   Each experimental block consists of 24 trials comprising each of the linear combinations of the stimulus and the human (eq. 2) for each of the 3 model combinations:
   
   The order of the trials within each block is random and predetermined. Each experiment consists of 10 blocks, such that each combination is repeated 10 times. An initial unanalyzed block is added for general acquaintance with the system and the task.
   We fit a logistic psychometric function¹ to the answers of the interrogator using the psignifit toolbox version 2.5.6 for Matlab, available at http://www.bootstrap-software.org/psignifit/ , with a constrained maximum likelihood method for estimation of the parameters, and find confidence intervals by the bias-corrected and accelerated (BCa) bootstrap method. The curve displays the probability of the interrogator to answer that a stimulus handshake is more human-like, as a function of α_stimulus – α_reference. The point of subjective equality (PSE) is extracted from the 0.5 threshold level of the psychometric curve, indicating the difference between the α_stimulus and α_reference for which the handshakes are perceived to be equally human like. The PSE is used for calculating the MHLG_w according to:
   (4) MHLG_w =0.5-PSE
   A model which is perceived to be as human-like as the reference model yields the MHLG_w value 0.5. The models that are perceived as the least or the most human-like possible, yield MHLG_w values of 0 or 1, respectively.
3. Conducting the “added noise” test and calculating the model human likeness grade MHLG_n
   Similarly to the weighted model-human test, there is only one interrogator subject. The other subject functions as the human entity in the handshakes. The practice block is also the same as in the previous method.
   After the practice, in one of the 2 handshakes – the stimulus – the interrogator interacts with a computer handshake model.
   The other handshake – the reference – is a force generated from a combination of the human and white noise with a frequency range filtered according to the frequencies that appear in the human handshake.
   (5) F = α • F_human + (1-α)•F_noise;
   α is equally distributed from 0 to 1, e.g:
   α = {0, 0.142, 0.284, 0.426, 0.568, 0.710, 0.852, 1}
   At the end of each trial the interrogator is requested to choose the handshake that felt more human-like.
   The experiment is built in the same way the weighted model-human test mentioned above is built. However, while the calibration in the weighted human-model test is performed by comparing different combinations of a base model to a fix combination of itself, in this experiment the base model is substitute by the noise.
   The PSE is extracted from the psychometric curve and defines the MHLG_n
   (6) MHLG_n = 1-PSE
   The models that are perceived as the least or the most human-like possible, yield MHLG_n values of 0 or 1, respectively.

3. Representative Results:

Figure 2 demonstrates the results of one subject for each of the 3 methods. The tested models in all three experiments are 2 viscoelastic models- KB1: spring K=50 N/m, damper B=2 Ns/m; KB2: spring K=20 N/m, damper B=1.3 Ns/m. In the weighted model-human test, the MHLG_w is evaluated by comparing each of the tested models to the elastic base model K=50 N/m.

Figure 2. The MHLG values of two viscoelastic models according to the “pure” test protocol (a), the “weighted model-human protocol” (b) and the “added noise” protocol (c). The error bars in (b) and (c) represent the psychometric curves’ confidence intervals. The black bars represent the MHLG grades for the models, and the gray bars represent those of the base model in (b) and the noise in (c).

The results demonstrate that the viscoelastic model KB2 is perceived as more human like than the other viscoelastic model KB1 using all three evaluation methods.