Fourteen healthy, right-handed subjects (28.6 +/- 5.5 years; 7 men, 7 women) participated in this study. The subjects had normal or corrected to normal vision. Right-handedness was assessed using the Oldfield's questionnaire 32. In addition, the subjects' emotional competence was tested with the German, 20-item version of the Toronto Alexithymia Scale (TAS-20) 333435. None of the participating subjects were classified as alexithymic (mean TAS-20 sum score 34.14 +/- 6.26, range 23). Subjects were also evaluated with the Beck Depression Inventory 36 and the scales of emotional experience (SEE) 37.

From photographs of facial affect 38, we selected for presentation those with happy, sad and neutral facial expressions found to be correctly identified in more than 90 percent of raters; 14 happy, 14 sad, and 14 neutral faces were used. Only faces which had been found to be correctly identified in more than 90 percent of raters 38 were used to ensure that the subjects internally generated the corresponding emotion. This corresponded to a similar approach of a recent study 39. As control stimuli we produced a number of scrambled images from these photographs equal to the number of intact faces. All images were digitized and controlled for luminescence.

Faces were presented for durations ranging between 300 and 500 ms, since this short presentation time is sufficient for conscious visual perception avoiding habituation 40. The presentation time of the faces was jittered to enhance the detection of the stimulation-related BOLD activity changes in this event-related fMRI study. Thereafter, scrambled faces were presented for durations between 11 and 12 s (Figure 1). This long second stimulation period was chosen to provide the subjects sufficient time to engage in appraising the emotional facial expression seen in the first stimulation interval and to allow changs of skin conductance to occur 40. The faces were presented in random order on a laptop connected to a projector (LCD data projector, VPL-S500E, Sony, Toyko, Japan) on a screen which was placed approximately 50 cm from the mirror in the head coil. Immediately before each fMRI scan the subjects were instructed to view the faces in a mind set according to one of the following cognitive instructions: a) identify the emotion expressed in the faces (RECOGNIZE), b) empathize with the emotion expressed in the faces (SHARE EMOTION), c) count the earrings in the faces shown (control condition: DETECT EARRINGS). Each instruction was given twice in random order across the subjects, yielding six separate scans per subject. Thus, the visual stimuli were identical in the different experimental conditions, but the visual information to be processed differed according to the instructions. Since each condition was repeated twice, 28 happy, 28 sad and 28 neutral faces were presented. The presentation was done in random order across the subjects to counteract possible sequence or habituation effects. After each condition, the subjects were debriefed about how well they could perform the task.

The subjects lay supine in the MRI scanner and viewed the faces on a screen via a mirror which was fixed to the head coil. Image presentation was controlled by a TTL-stimulus coming from the MRI-scanner as described in detail elsewhere 41. Scanning was performed on a Siemens Vision 1.5 T scanner (Erlangen, Germany) using an EPI-GE sequence: TR = 5 s, TE = 66 ms, flip angle = 90°. The whole brain was covered by 30 transaxial slices oriented parallel to the bi-commissural plane with in-plane resolution of 3.125 × 3.125 mm, slice thickness of 4 mm, and interslice gap of 0.4 mm. Each acquisition consisted of 255 volumes. The first 3 volumes of each session did not enter the analysis. A high-resolution 3D T1-weighted image (TR = 40 ms, TE = 5 ms, flip angle = 40°) consisting of 180 sagittal slices with in plane resolution of 1.0 × 1.0 mm was also acquired for each subject.

Image data were analysed with SPM2 (Wellcome Department of Cognitive Neurology, London, UK; http://www.fil.ion.ucl.ac.uk/spm). Images were slice-time corrected, realigned, normalized to the template created in the Montreal Neurology Institute (MNI), and spatially smoothed with a 10 × 10 × 10 mm Gaussian filter. The normalization step resampled the images to a voxel size of 2 × 2 × 2 mm. The anatomical T1-weighted image of each subject was co-registered to the mean image of the functional images and also normalized to the MNI-space.

For each of the three instruction sets, the happy, sad and neutral face presentations were modelled as well as the corresponding scrambled faces. The models employed the haemodynamic response function provided by SPM2. Data were temporally filtered using a Gaussian low-pass filter of 4 s and a high-pass filter of 100 s. All data were scaled to the grand mean. Realignment parameters as determined in the realignment step were used as confounding covariates. The duration of all events was modelled with 4 s for face presentations and 8 s for the delay period. The repeated condition images of the 18 experimental conditions were averaged for each of the 14 subjects, yielding a total of 252 averaged condition images. Data were modelled using the canonical haemodynamic response function provided by SPM2. The duration of all events was modelled explicitly with 4 s for face presentations and 8 s for the delay period. To identify the brain areas related to viewing the faces a comparison with viewing the scrambled faces was calculated. Only areas with a p < 0.05 corrected at cluster level with a cluster threshold > 20 voxels were accepted (Table 1).

The PCA employed in house software of which some modules were adapted from SPM2. Extracerebral voxels were excluded from the analysis using a mask derived from the gray matter component yielded by segmentation of the high resolution anatomical 3D image volume into gray matter, white matter and cerebrospinal fluid using the segmentation module of SPM2. Voxel values of the segmented image ranged between 0 and 1; the mask included only those exceeding 0.35, excluding most of white matter. Calculation of the residual matrix is the first step. From a matrix whose rows corresponded to the 252 conditons and columns to the 180 thousands voxels in a single image volume the mean voxel value of each row was subtracted from each element as was the mean voxel value of each column subtracted from each element. Thereafter, the grand mean of all voxel values in the original matrix was added to each element. The result of this normalization procedure is the residual matrix for which the row, column and grand means vanish. Using the singular value decomposition implemented in Matlab, the residual matrix was then decomposed into 252 components, consisting of an image, an expression coefficient, and an eigenvalue for each component. The eigenvalue was proportional to the square root of the fraction of variance described by each component, while the expression coefficients described the amount that each subject and condition contributed to the component. The principal components (PCs) were ranked according to the proportion of variance that each component explains, i.e, PC 1 explains the greatest amount of variance. The expression coefficients and voxel values of a PC were orthonormal and their orthogonality reflected the statistical independence of the PCs. The PC image displays the degree to which the voxels covaried in each PC, their voxel values (loadings) ranged between -1 and 1.

In order to provide a neurophysiological interpretation of the components, statistical tests, e.g. unpaired t-tests and tests of correlation (Pearson) were applied to the expression coefficients. The formal criteria for relevant PCs were: (1) the statistical tests identified the condition differentiating PCs at a significance level of p < 0.001, and (2) the PCs fulfilled the Guttman-Kaiser criterion, the most common retention criterion in PCA 42 in which PCs associated with eigenvalues of the covariance matrix larger in magnitude than the average of all eigenvalues are retained, implying in this analysis that they ranked among the first 61 PCs. The voxels describing the nodes of a neural network associated with a relevant PC image volume fulfilled the conditions that the voxel values lie in the 1^st percentile or the 99^th percentile of the volume's voxel value distribution, and that the voxels belong to clusters of greater than 50 voxels.

The anatomical locations of the peak activations and of the coordinates of the maximal PC loadings of the significant PCs are reported in Talairach space 43. A freely distributed Matlab script 44 effected the transformation from MNI space.

Correlation of the expression coefficients of the significant PCs with the TAS-20, the Beck Depression Inventory, and the SEE scales was conducted using a Pearson two-tailed correlation (SPSS for Windows, Version 12.0.1.). The significance level of the correlations was set to p < 0.01.

Of the four differentiating PCs, PC1 distingished the neural network involved in face indentification, since the involved lingual gyrus, cuneus and precuneus have been known to be involved in face recognition and visual processing 6869. The lingual gyrus has been linked to an early stage of facial processing which occurs before specific identification occurs 70. Conversely, the angular gyrus and the anterior cingulate have been shown previously to be involved in attention 71, and the anterior portion of the superior frontal gyrus has been implicated in theory of mind paradigms 2671.

PC2 represented the functional neural network associated with detecting emotional facial expressions, since the network nodes included bilaterally the fusiform gyrus, corresponding to the so-called fusiform face area. Some studies have suggested that the fusiform face area mediates the lower order processing of simple face recognition 72, while others have implicated it in higher order processing of faces at a specific level 45 including emotional detection 73 and identity discrimination 5864. Also included among the nodes were the posterior portion of the superior frontal and the inferior frontal gyrus, which have been implicated in higher order processing of faces involving the perception of gaze direction 74, attention to faces 75, eye and mouth movements 56, and empathy 2576. Thus, our analysis, relating the concerted action of the fusiform and superior frontal gyrus to the detection of emotional face expressions, substantiates previous work showing that the conjoint activity of the fusiform gyrus and cortex lining the superior temporal sulcus are related to higher order processing of facial features 4553565864727374. We propose that these areas constitute a system involved in facial affect processing.

PC2 also implicated subcortical structures such as the thalamus and the hypothalamus bilaterally, and the cerebellum. These structures belong to the anterior cingulate – ventral striatum – thalamus – hypothalamus loop 767778 and are thought to regulate the emotions, drives, and motivated behavior 27980.

The third statistically relevant PC, PC12, was related to the attentive processing of expressed emotions. Accordingly, the involved cortical areas, e.g. the precuneus, cuneus, middle frontal gyrus, and thalamus have all been linked to tasks related to controlling attention 818283. The inferior frontal gyrus has been shown to be involved in response inhibition during attention 84, while the middle temporal gyrus has been implicated in attention to facial stimuli 47 and to visual attention in general 8182.

PC27 appeared to represent a network mediating a 'feeling' or sense of an observed emotional state. In fact, the superior temporal gyrus has been shown to play an important role in perceiving self/other distinctions, and, importantly, in experiencing a sense of agency 8586. Also, the superior temporal gyrus seems to be important for "theory of mind" capabilities 878889. Similarly, the right inferior parietal lobule has been related to an experienced sense of agency 8586, while the temporal parietal junction has been thought to be crucial to larger networks mediating spatial unity of the self and body 8990, and attention 819192.

Empathic processes allow individuals to quickly asses the emotional states and needs of other individuals while rapidly transmitting our own experiences and needs 93. Thus, emphatic experiences are crucial to rapid and successful social interaction 9495. The recognition of emotional states and cognitive processing of another's emotional expression are critical skillsets utilized by all humans upon perceiving another person's face. In order to properly behave in a social situation, one must understand the context of the situation before an appropriate action can be taken.

The participation of the prefrontal cortex in the four relevant PCs in our study not only support the view that different cognitive functions work together to play a role in distinct neural networks, but, as well, further the notion that the medial pre-frontal cortex is the junction point where different visual, attentive, emotional and higher order cognitive processes come together in order to allow for a subjective reaction towards the exterior world 26. The medial-pre-frontal integration of own cognitive concepts, implicit affective schemata and empathy-based anticipation of the possible reactions of important other individuals allows for an effective selection and adaptive planning of one's own actions. The face as the most significant carrier of emotional information is an important source for these evaluative and appraisal functions.

The four neural networks involved in the recognition of faces and the cognitive control of emotions capture the functions that this study aimed to explore. Our results support numerous studies that hypothesize distinct neural networks for recognizing facial features and emotional expressions 1113141516179697. In particular, although PC 1 and PC 2 differentiated lower-order facial identification and higher-order facial feature processing, the cortical areas within each PC contained brain areas associated with functions effected by the neural networks depicted in the other PCs. Thus, the middle frontal gyrus activated in all four principal components has been shown to be related to inhibition 98, successful recognition of previously studied items 99, and successful error detection, response inhibition, interference resolution, and behavioral conflict resolution associated with completing the Stroop Color-Word task 100. Moreover, the PC representing a basic visual function, PC1 included activation of the right anterior cingulate which has been shown to severely impact the processing of emotional expressions 101102. Apparent also in PC1, the lingual gyrus and cuneus have been implicated in the higher-order processing of emotional expression 103 and the lingual gyrus has been implicated in motional information 104. Interestingly the anterior cingulate has also been thought to cognitively monitor the control of response conflict in information processing 105 and in the regulation of cognitive and emotional processing 106. In a case study, Steeves et al. 61 demonstrated that an intact fusiform face area was sufficient for identifying faces, but a lesion in the occipital face area prevented the patient from higher-order processing of faces such as identity, gender, or emotion.

Out results are also in accordance with the idea that emotion recognition may play a role in the lower order process of face identification, but this role is limited at best and functions mainly as a stepping stone for higher order recognition of emotional expressions. 107. Although the lingual gyrus in PC1 and the fusiform face area in PC2 are located closely anatomically, our analysis suggests that they are involved in distinct neural networks related to different subfunctions, face identification and detection of emotional facial expressions, repectively, of facial processing. Nevertheless, it is possible that the subjects recognized the emotions implicitly also in the DETECT EARRING condition. However, the cognitive instruction in this study was that during the RECOGNIZE and SHARE EMOTION conditions the subjects had to appraise the emotions explicitly. In fact, this difference resulted in additional brain structures involved in the explicit processing conditions.

The involvement of closely adjacent cortical areas in different subfunctions related to the detection of emotional face expressions highlights the difficulty of assigning exclusive functional relevance to a particular area and suggests the multi-functional nature of brain areas which participate in multiple, interconnected neural networks performing lower-order as well as higher-order neural processing.