AMITIAE - Tuesday 17 November 2015
Cassandra: Using Reflections in Photographs to Identify Suspects, Acquaintances and Locations
By Graham K. Rogers
Traditional policing has relied on various forms of forensic examination as a way of collecting evidence. Many are familiar with such areas as fingerprint searches, as well as dental identification. A wide range of physical sciences, such as metallurgy, are used to collect evidence from materials used in a crime or found at the scene. Also to be considered in this area are the examination of materials used in bombs which may be chemically analysed to pinpoint their sources. This could also include electrical or mechanical accessories of exploded or unexploded devices and the reassembly of such parts.
More recent developments include forensic computing, as considerable amounts of evidence of culpable activities may be stored on a modern computing device: finding that, or retrieving data when it has been deleted may assist prosecution, or add to the picture of a suspect's activities.
The advent of the digital camera changed presentation of evidence. It is possible to alter any image, although expert analysis should reveal the extent over and above simple enhancement editing. With the relative cheapness of DSLR cameras, along with the widespread nature of smartphones (with cameras incorporated) and other small devices capable of being used to produce photographs, there has been a major expansion in the numbers of images now produced.
As many of these images are in the public domain, for example uploaded to and displayed on social networking sites (e.g. FaceBook) their availability could be a major resource in intelligence gathering, although powerful search and analytical algorithms are needed to make best use of these.
Research by R. Jenkins and C. Kerr in the UK examined the feasibility of being able to pick out subjects from eye reflections in photographs. That study was limited in a number of ways: the use of a digital Hassleblad H2D-39 camera; the use only of corneal reflections; and the concern only with identifying persons. Commenting on this, I noted then that "elements within the image . . . might be identifiable in terms of location were the technology to be improved."
A local photographer opined (in a phone conversation) that when taking photographs of animals or birds, the image was useless unless the eye was in focus. The same would apply to persons, but reflections in a photograph are not limited just to the eye. Subjects may be wearing glasses (sunglasses particularly are good reflectors of light) and it is likely that a photographer may appear in a shot along with other visual details.
While the study by Jenkins and Kerr provides a useful start, the analysis of other reflections in an image may be as valuable in creating a profile of subjects, their whereabouts, and their associates, that there is a case for new search methods to be developed.
The use of images may not produce direct evidence of a person's involvement in such activities, but is more likely to be peripheral in building a case: a sighting of a known associate, for example, at a specific location may be added to a piece of forensic evidence, adding to a profile and thus strengthening the knowledge of a criminal. Subsequent actions by the authorities who will have created a multi-dimensional picture will increase safety for those on the ground, aid in interrogations and help in securing convictions.
Child pornography is different (18 U.S. Code § 2256). Although the US definition gives 18 as the age of a minor, many of the victims are too young to be aware of right and wrong, will do what they are told by an adult, and run the risk of injury or disease. Circulation of images and videos of such acts is secretive, mainly because distribution is against the law in most countries. However, the age of consent may differ depending on the laws in any country.
Nonetheless illegality and secrecy go hand in hand and A. Greenberg, Wired, citing research by G. Owen, of the Computer Chaos Club,as presented to Congress, reported that 83% of traffic to Tor relate to pedophilia. Also reported by Greenberg, the figure was shown to be "only 2 percent of total traffic over Tor's anonymizing network."
Where pornography differs from terrorism in one important aspect is that where terrorists and criminals seek to hide, the most successful pornographic images are of good quality, although this may not follow in all cases, particularly with regard to amateur output. Focus and sharpness are attributes that can be used to collect evidence, build a case, and in some cases secure convictions.
Images from the Nikon cameras had been imported as RAW in Nikon's NEF file format, while those from the iPhones were mainly in JPG (a compressed format). Other output formats might be available depending on specific applications and settings. I used the loupe in Aperture to enlarge those specific parts of the image that had reflections.
Some of the images had been edited but for these purposes that may be immaterial: those that will be used for tracking suspects are likely to be online and may have been edited, enhanced, cropped or sharpened before uploading. In addition, those on sites, such as Facebook or Twitter will not be the best images available: authorities would have to work with what can be found.
Several of the projects were examined to find images that might give up information. Eyes, spectacles, windows, screens and mirrors were all potential sources of information.
Modern high rise buildings are often reflective, although the image may be broken by the number of individual panes; or there may be distortion with a larger expanse of glass. Likewise, a shopping mall or station may have large sections of glass that will reflect details of location or persons that were not in the shot itself.
Other types of surface that reveal some image content include computer screens (including ATMs), especially if the screen is dark.
Using specific reflections was shown to be viable by Jenkins and Kerr, although the equipment used was too specialised. The testing was fairly limited with the controlled images that were used. Nonetheless, it was a pointer to the use of reflections as a way to bring in peripheral data: information that might help fix a location, identify associated or otherwise build a case against a possible suspect. While direct evidence would be preferable, in many cases these types of images would be more valuable in building intelligence on an individual or group.
The use of images of the type that are more likely to be found on a user's computer (or other device) is important as these types of photographs are more likely to be found on social networking sites and thus be more accessible by authorities. An example is the identification of the Facebook page of a suspect in the 13 November 2015 attacks on Paris. Doubtless, security forces in many countries will examine the photographs found on that page in an effort to identify other locations and persons connected to Hadfi.
In these and other images, the reflections might reveal more than the subjects would have wanted.
Graham K. Rogers teaches at the Faculty of Engineering, Mahidol University in Thailand. He wrote in the Bangkok Post, Database supplement on IT subjects. For the last seven years of Database he wrote a column on Apple and Macs. He is now continuing that in the Bangkok Post supplement, Life.
For further information, e-mail to
Back to Home Page