Earlier this year, researchers in the Visual Recognition Lab at the University of Toronto Scarborough published an interesting study reporting that they were able to reconstruct images of human faces using electroencephalography (EEG) data.

In the study, subjects viewed images of 54 unfamiliar male faces. For each of the faces, one image with a neutral expression was shown, and one with an open-mouthed smile. The images were relatively homogenous: Each showed the face looking forward, viewed from the front, and unobscured (e.g., no accessories or hair obscuring the facial features). The images were also uniformly scaled, aligned, and sized, and their contrast values were normalized. Images were displayed sequentially in a pseudorandom order (no face appearing twice in a row) for 300 milliseconds each. Electrical activity was recorded from scalp electrodes while the subjects viewed the images, using a 64-electrode EEG recording system. The resulting EEG data were then used to reconstruct the images viewed by the subjects, using methods that were broadly similar to ones previously used for facial image reconstruction from functional magnetic resonance imaging (fMRI) data. This video shows examples of reconstructions based on 10-millisecond windows and temporally cumulative data (50-650ms after stimulus onset).

Two methods were used to evaluate the accuracy of the reconstructed images. In the first, each reconstructed image was compared pixel-wise to each original image, and the percentage of instances in which the reconstructed image was more similar to its corresponding original image than any other original image was compared to chance. The accuracy assessed via this method was 69% for neutral faces and 64% for smiling faces using temporally cumulative data. In the second method, naïve subjects were shown one reconstructed image and two original images – one original image corresponding to the reconstructed image and one not – and asked to select the closer original image. The naïve subjects selected the corresponding original image more frequently than would be expected by chance for both neutral and smiling faces. (See Figure 8 of the paper for more details regarding accuracy.)

Image reconstruction from noninvasive brain activity measurements is not new – reconstructions of facial and other images, both static and dynamic, have been demonstrated with fMRI data. (For some examples, look here and here.) And, the authors’ reported levels of accuracy are far from what is likely to be useful in a legal context. But the study is worth taking note of nonetheless, especially in light of the limited information available using EEG and the homogeneity of the original images. If the results are reproducible in other contexts, there could be practical advantages over fMRI-based reconstruction. EEG equipment is portable and relatively inexpensive, and hence, much more accessible. EEG also has better temporal resolution than fMRI, so it could be more suitable for reconstructing dynamic images. The study’s authors say they are working on EEG-based reconstruction of images of objects other than faces, and reconstruction from a subject’s memory – should be interesting to follow.

The full paper is here.