4. A Man One Step Ahead of his Feet
A critical view on the documentary value of digital photographs
- Side: 65-87
- DOI: https://doi.org/10.18261/9788215037394-2020-04
- Publisert på Idunn: 2020-07-20
- Publisert: 2020-07-20
- Creative Commons (CC BY 4.0)
Artikkelen problematiserer digitalfotografiet si rolle som dokument. Sidan fotografiet no både kan endrast og skapast elektronisk utan at det er tydeleg for sjåaren, har bildet i seg sjølv ikkje lenger dokumentarisk verdi. Eg vil støtte denne påstanden ved å analysere eit par manipulerte bilde, kort vise til nokre nyare metodar for bildemanipulering og -skaping, vurdere media sine tiltak for å møte utviklinga og argumentere for eit framlegg til løysing.NØKKELORD: dokumentarfoto, fotomanipulering, fotojuks, photoshop, falske nyhende, deepfake
This article questions the digital photograph’s role as a document. Since photographs today can be both adjusted and created without visible trace, they can no longer as such claim documentary value. I will support this statement by analysing a couple of digitally manipulated images, by briefly presenting a few recent image-adjustment and creation techniques, by assessing the media’s response to this development, and by arguing for a possible solution.
Photographs do not lie. This statement is true. It is also false, and today more false than ever. The McCurry scandal of 2016 was one of the clearest demonstrations of this sordid situation. Other cases, plus recent technical developments within computer software and hardware, clearly support the fact that photographs can indeed lie and do it well. The media do their best to verify the authenticity of their stories, but are, in my opinion, not in a position to guarantee that all their published live or still images are true to fact. I will in this article explain why this is my opinion, and I will be so bold as to suggest an additional form of verification.
Brief technical/historical background
The photographic image, as created by light reflected from a motif through a lens onto a screen inside a camera, documents exactly what is visible through that lens during the time of exposure. This valuable property of photographic image production was praised by Francois Arago even before the invention was publicly announced in 1839 (Larsen, 2008, p. 125), and was elaborated by William H. F. Talbot shortly after (Talbot, 1844–46, p. III and X).
The photograph’s capacity for exact documentation of “what once was” in front of the lens has also been focused in later times (Barthes, 1980/2001, p. 104; Gripsrud/Kjørup, 1993, p. 169; Pagter, 2018, p. 16), even though digital technology during this time has made the photograph as a document much less reliable than before.
The paradox is that the photograph’s capacity for telling the truth – its “ethos”, so to speak – also makes it an effective tool for deception. Photographers may deceive by arranging or influencing the motif (Sherer, 1975, p. 1; Evans, 1978–82, Introduction); editors may deceive by presenting the photograph as coming from a different location (Hareide, 2019, p. 38ff) or situation (Alling-Ode/Tubin, 1993, p. 31; Ritchin, 2008, p. 148); and both may deceive by selecting which motifs to include or exclude (Cole, 2016).
Photographs can also be altered by retouch or montage, techniques dating back to Hippolyte Bayard’s self-portrait as a drowned man in 1839 (Frizot, 1998, p. 30), refined as art in “The Two Ways of Life” by Gustave Rejlander in 1857 (Newhall, 1978, p. 59–60), and, by the end of WW II, sufficiently ill-used to inspire Orwell in the writing of his dystopia “1984”.
The digital revolution has made it possible to perform such altering to the ultimate degree of perfection. It has also made it possible to create perfect photographic-look-a-like images without the need for a physically existent motif.
Traditional photographs consist of grains emulated in a thin, transparent layer upon which attempts at changing image content will leave visible traces of knife, brush or pencil. Digital images consist of dots in a pattern; each given a set of 0’s and 1’s defining a certain number of levels from nothing (dark) to maximum (bright). These levels can be defined or redefined dot by dot or in freely selected groups through manual intervention by an operator, or by a diversity of automatic or autonomous programs. The results are either alterations that, when performed well, are undetectable (Alling-Ode/Tubin, 1993, p. 42), or artificially created images that look exactly like photographs (Ritchin, 2013, p. 6ff).
The McCurry scandal
In May 2016, legendary documentary photographer Steve McCurry was exposed for having planned, arranged and/or digitally altered several of his images. At an exhibition in Turin, Italy, visiting photographer Paolo Viglione noticed a strange detail in a large format street-scene from Havana. He took a picture of the peculiar detail, posted it on the internet and asked if anyone could explain. Few days later, other altered images were discovered, and McCurry found himself as less of a shining star than before. He now defines his work as “visual storytelling” rather than photojournalism (Cade, 2016).
Steve McCurry has been a star of documentary photography for more than four decades. First making a name for himself by illegally entering Afghanistan with a Mujahedeen group in 1979 and telling their story, he has spent the following years producing visual reports for publications such as Time, Paris Match and above all National Geographic. He became a master of composition and colour with a particular fondness for Asia, and he became part of “the great documentary tradition of photographic truth telling” (Bannon, 2011, p. 3).
Trained in the same tradition, I share many of McCurry’s ideals. Judging from his images, and apart from the view on cropping, his ideas of what to record and how to present it fit well with the rules laid down by Cartier-Bresson and other icons of photography in their pursuit of the “perfect picture” (Turner, 1973). McCurry has succeeded more than most in this quest for perfection.
The May 2016 revelations, showing that McCurry’s perfectionism was at least in part due to short-cuts like planning, arranging and altering, came as a shock and a disappointment to many of his fans. This is understandable, without giving me much reason for investigation or debate. What to my mind does provide reason for investigation and debate is what it takes to be exposed for “cheating” as Mr. McCurry has been here, and what to do if and when such “cheating” becomes undetectable.
Method of investigation
My method of investigating the selected material has been to use Adobe Photoshop to study the material in the deepest detail that the material permits. I have looked for clues to and tested diverse theories on what may have been done to the images. Part of this is achieved through image transformation, through adding guidelines and through comparing possibly “doctored” images with images free from suspicion, as well as through proven theories of photographic principles.
In my search for possible motives behind the adjustments, I will also discuss the images in terms of their visual content before and after any proven or presumed alteration.
Material for investigation
For my investigation of the McCurry case, I have had six images at my disposal. The first is a low-resolution copy of the original image from Havana as exhibited in Turin, downloaded from a website belonging to Giancarlo Parisi (figure 4.1). The second is a copy of Mr. Viglione’s original high-resolution image file, kindly provided by him (figure 4.2). In addition, I was lucky enough to be provided with three more images from the same area by my former student José “Pepe” Eduardo Garcia Aldama (figure 4.3, 4.4 and 4.5). Lastly, an aerial view of the location was provided by Google Maps (figure 4.6). These last four served as references due to their status as “truthful”, figure 4.6 being part of an official city map, and Mr. Garcia’s images being RAW files (Kobré, 2008, p. 158–159). Adapted versions of these six images will be referred to as figure 4.1a, 4.1b, etc. Figures will be presented here only when they are necessary as visual texts.
An idealised reconstruction of what I believe has been done, and what could have been done by a more skilled technician, is presented as figure 4.7.
For the presentation of my own case I used my two original JPEG images, included here below the combined result as figure 4.8.
The McCurry case
The best version of figure 4.1 available on the internet had too low a resolution for thorough scrutiny, so the high-resolution detail provided by Mr. Viglione (figure 4.2) was essential to my investigation. My first step was to merge these two into one. Figure 4.1 was enlarged via interpolation to fit the resolution of figure 4.2, which, taken at an angle, was adjusted for barrel-distortion and perspective to match the perspective and distortion of figure 4.1. This new full frame version was rotated 1.8° clockwise to compensate for a slightly tilted camera. The red line indicates vertical centre and the green frame indicates the inserted, high-resolution detail (figure 4.1a).
Zooming in on the central part I could now start searching for tell-tale signs of what had been done to the image, and possibly how (figure 4.1b).
A man one step ahead of his feet
At first sight we notice, as many have done before, that the lower part of a yellow signpost has been cut off and moved to the left, seemingly to serve as a prosthesis for a walking man.
Zooming in closer, we notice that both the man’s feet are still there – he is just one step ahead of them. His right foot has been moved to his left leg and his other shoe, including a torn-off foot, is left behind on the pavement (figure 4.1c).
We can also see that a part of the pavement and stairs has been moved sideways and tilted. A theory aired as the scandal broke was that the clone-tool had been used in order to try to remove the signpost. I can find no support for such a theory in this material. Apart from traces of possible clone-tool use below the man’s right knee, there is no trace of any attempt to, nor any need to, remove the signpost. An alternative explanation is that this part of the picture has been copied and pasted as a second layer on top of the first, moved sideways, tilted, and partly erased to let the lower layer show through. A simple reversal shows that this theory cannot be directly abandoned. Now the shoes fit, and the pavement is straight (figure 4.1d).
Copy – Paste – Move partly delete works like the clone tool, except for the added option of rotating the patch in question. It is normally used to hide something undesirable or extend an image area. Why it has been used here is not obvious. It is, however, a clear example of poor craftsmanship and the direct reason for McCurry being exposed as he was. A man one step ahead of his feet will stumble and fall, which, figuratively speaking, is precisely what McCurry did. Without this sloppy work, the other anomalies might still have gone undetected, McCurry’s reputation might have remained intact, and this article would not have been written.
Poor masonry and other abnormal details
A closer look at the arch framing the man and the signpost reveals several spots of poor masonry. It is well known that Cuba has been under financial difficulties for a long time, but this can hardly explain why an old building has been subject to such poor restoration. Imagining this to be the original form of the building is also out of the question. No mason would build an arch with pillars of such different width and with such uneven fake building blocks, and no owner would accept it. The arch fundament, indicating an uphill turn to the right while the overall view indicates a downhill direction, is equally strange (figure 4.1e, red lines).
Three pavement slabs to the left have identical cracks. This is unusual for slabs, but very common for clone-tool photoshopping (figure 4.1e, green lines).
The back of the blue car seems to be partly penetrated by the building behind it. This is unusual for cars and buildings, but common for images in which someone has tried to add a car where there was none. This little detail, along with the strange masonry and tilted pavement, raises a suspicion that what we have here may be not one image with clumsy photoshopping, but an attempt at creating an image out of more than one original (figure 4.1e, blue line).
Poor ground conditions?
Shifting my focus to the overall picture, I noticed that the buildings do not look right. The laws of perspective state that extended parallel lines converge towards a vanishing point. Accordingly, photographs can be perspective-adjusted so that parallel lines in the motif also appear parallel in the picture. This is normally the case with vertical lines of buildings. Doing this as a part of image post-production is commonly accepted and often compulsory for architecture and cityscapes, in accordance with the perspective element of the Scheimpflug principle as practised with a technical camera or a shift lens (Preus, 1978, p. 59–79).
Having already rotated the image 1.8° clockwise to adjust for a tilted camera, the central building did appear vertical and “normal” in accordance with human vision. In an attempt to make the other buildings also appear vertical in accordance with proven theory (Fjørtoft, 2006, p. 204), I now used the transform tool to make the top of the picture wider and the bottom narrower, keeping the central horizontal part unaltered to avoid changing proportions. This turned out to be impossible. While the façade of the building to the left was vertical, the rest of it was leaning backwards and the building to the right looked wrong indeed (figure 4.1f).
Testing the right-hand side façade provided no better result. Now the buildings on both sides of the street seemed to be leaning forward (figure 4.1g).
Due to a certain tower in Pisa, I had to consider the possibility of poor ground conditions in this part of Havana too, but a similar test performed on an image provided by Mr. Garcia adjusted quite nicely (figure 4.2).
Nothing here indicated poor ground conditions of any kind, poor building construction or poor masonry, and since figure 4.2 is “true to fact”, coming as it does directly from a RAW file, we must assume that the buildings and the ground on which they stand on are in good condition and that the problem must, therefore, rest with the McCurry image.
I have no knowledge of any types of optics able to create anomalies of this kind in a street image, and if any did exist, I find it hard to believe that Mr. McCurry would ever use them. Anomalies like this will, however, easily occur if someone tries to join the left-hand part of one picture with the right-hand part of another, unless both are from a camera in a fixed position. The perspective problem of figure 4.1 is, consequently, yet one more indication of this being a construction from more than one original, photographed from almost, but not quite identical positions.
Looking closer at the central building, I noticed that the first floor and all of the above responded to the laws of horizontal perspective, as they should.
All horizontal lines in this area, extended to the right, did converge towards a vanishing point when tested with a ruler on a large print. The horizontal lines of the ground floor did not (figure 4.1h).
This increased my suspicion that the image in question is not one original, but a merger of more than one and that the anomaly around the man ahead of his feet could be a clumsy attempt at concealing a misfit.
Following this line of thought I started looking for further clues, and found one. The borderline between the stretch of asphalt and the street’s concrete sidewalk shows a small but abrupt sideways shift and turn of direction just behind the right rear wheel of the blue car (figure 4.1i).
A detail of Mr. Garcia’s figure 4.3 gives a clear view of this part of the street with the border between concrete and asphalt appearing straight and unbroken for the entire stretch in question. Mr. McCurry took his pictures in 2014, while Mr. Garcia took his in 2017. By that time a new layer of asphalt had replaced the old one, so figure 4.3 is not conclusive evidence.
Even so, it is natural to expect that both the old and the new layers of asphalt would follow the edge of the concrete sidewalk (figure 4.3, green line).
A closer look at figure 4.3 (red lines) reveals that the concrete sidewalk consists of slabs, probably cast on site, with cracks between them at 90-degree angles to the street.
The pattern this creates makes it difficult to join the left part of one image with the right part of another image along a cut crossing these lines, unless the two originals have the exact same perspective. If what we have here is, as I now had good grounds to believe, a merger between images from slightly different positions and therefore slightly different perspectives, the easiest place to cut would be along a line close to the asphalt, further on through the asphalt and through as little building as possible up through the sky (figure 4.1j).
If my assumption is correct, a thorough investigation of a high-resolution copy of figure 4.1 would most probably show traces of tampering along the path loosely indicated by the red line of figure 4.1j. The part to the left with the couple belongs, as far as I can see, to one original, while the right-hand part with the cars belongs to a second original. It is possible that each of the two cars belong to separate originals, but I have found nothing in my material to support this idea.
The traces of tampering in the central part of the image show a series of actions that are hard to understand, as mentioned (figure 4.1b), unless the part that includes the ground floor of the central building belongs to the image with the couple, while the rest of that building belongs to the image with the cars. What is clear is that the lower part of this area has been separately split, tilted and moved sideways, creating the anomaly that caught Mr. Viglione’s attention and, so to speak, spilled the beans. It is also clear that the ground floor of this building has a horizontal perspective different to that of the upper floors (figure 4.1h).
Furthermore, what seems clear is that the left side of the central block of buildings belongs partly to the left original and partly to the right one. We can see this by zooming in on the anomalies indicated by the yellow lines of figure 4.1e. To my mind, the two almost identical balconies and the two almost identical windows are two versions of one window and one balcony photographed from slightly different positions (figure 4.1k).
An enlarged detail of one of Mr. Garcia’s images proves that this is indeed only one balcony and one window (or ventilation opening) in a building on the other side of a crossing street in the background. Incidentally, in between Mr. McCurry’s and Mr. Garcia’s visits the façade of this building has had a Cuban flag painted on top of a coat of blue (figure 4.4). Due to Mr. McCurry’s positions in the road as opposed to Mr. Garcia’s positions on the pavement, more of this opening is hidden behind the far corner of the block in Mr. McCurry’s images. The appearance of the opening, with different widths, positions and orientations in the two versions of it in Mr. McCurry’s image as it was exhibited in Milan, serves, as far as I can see, as definite proof that the exhibited image is a mix of at least two originals.
The approximate positions of Mr. McCurry and Mr. Garcia while obtaining their respective originals, as estimated from the perspective of their images, are shown in figure 4.5.
Conclusion regarding what has happened to the image from Havana.
This rather tedious investigation into one obvious case of digital tampering was necessary in order for me to reach a well-founded conclusion of what had happened.
If I am right and this is indeed a combination of two separate originals taken a few moments and a few steps apart, as all the signs indicate, the problem of merging them could have been solved by a slight rotation and vertical perspective adjustment, and a slight sideways stretch to conceal the difference in sideways perspective (figure 4.6). If performed with a minimum of skill, the chances are very high that this fakery would never have been spotted at all. This, in my opinion, is the only important fact of the case. It raises the question of how many doctored images go undetected due to professionally performed fakery in the media today.
McCurry has stated that this is “a change that I would have never authorized” and that “the lab technician who made the mistake does not work with me anymore.” I sympathise. Sloppy work should not be tolerated in any trade, and as demonstrated by other doctored images carrying his signature, McCurry has access to professionals capable of far more complicated image adjustments than this (Cade, 2016). He had reason to feel disappointed.
As far as I know, however, McCurry has not admitted to having ordered the change. I have less sympathy for this. I find it much easier to believe that he did so rather than imagining a lab technician taking on such an enterprise without permission.
McCurry has a reputation for perfection in all his work and we know today that some of his perfect images are partly or fully due to planning, influencing or arranging the motif as well as manipulating the resulting images. I hope that the above has made it clear that the image in question here is also a manipulation; a construction from more than one original, and in this case I can easily imagine a reason for wanting this done.
The constructed image tells a beautiful story of lovers walking in step down a street towards a Havana landmark building while another symbol of Cuba today, two vintage American cars, pass by in opposite directions. Except for its poor-quality fakery, the constructed image tells this story well, as most McCurry images do, with good colours, good composition and no disturbing elements. I must presume that the originals, presented separately, did not.
My reason for reconstructing this image was to show that it can be difficult to determine what has been done and, presumably often, that any tampering has been done at all. This leads me to suspect that at least some tampering goes undetected.
Frequency of fraud
My suspicion is strengthened by a report that 20% of the finalists in the 2015 World Press Photo contest were disqualified after examining the original RAW files of the entries (Ming & Laurent, 2015). This option is less available now, as news image agencies today rarely use RAW files (Byford, 2015; personal discussions with AP and Reuters photographers, 2019).
My own case
On a “sundbåt” (city ferry) trip in Kristiansund in 2005, my girlfriend had a pleasant conversation with the captain while buying tickets, and I was surprised not to see any of this contact while looking through the pictures later. All my images showed her looking down when he was looking at her, and vice versa. Human vision perceives and stores information differently than the camera, as we know (Husserl, 2005, p.172, 286), and the direct eye contact between them that I perceived probably never occurred. A manual merger of the left-hand side of one of the images with the right-hand side of a second one, however, recreated my memory of the situation quite nicely.
While working on this article, I repeated the experiment, this time using the “automate” function in Photoshop CS6.
The process took two minutes and produced a result with no trace of tampering.
The only way of exposing this image as fraudulent is by gaining access to the originals, and since these originals were JPG files, I could make the fraud undetectable by replacing the image content of one original with the merged result and deleting the other. Instead, both are included here to illustrate the process.
As my own case shows, parts of images may now be combined into one not only by skilled operators, but can be done so simply, quickly and with better results by using the automated functions of image-editing software. Seamless removal of image content is one option; “stacking” of images creating increased or decreased depth of field is another; group portraits in which all the faces have open eyes and are looking in the same direction is a third – and those faces do not even have to be present at the same time. Some of this technology has already been implemented in modern smart phones.
At the other end of the scale, “deep fake” technology (Sample, 2020) allows individuals to participate in live footage they never participated in, or to make statements they never made, in languages they do not know (Lied, 2019). The options are too numerous to discuss fully in this article. The image does not even have to start as a photograph (West & Bergstrom, 2019).
Automated fraud detection: “Photo Forensics”
There are websites offering programs for a relatively rapid search of anomalies. I here use the word “relative” in the broader sense of the word. All searches take time. One method is to search for differences in JPG compression within the same image. That would be a clear sign of image elements having different sources, but is of little use if the original compression rates and file sizes are identical, as they are in all the images shown here. Clone detection may identify repetition of details (cf. the concrete slabs in figure 4.1e.) and is particularly useful for images including elements of nature. Identical patches of grass, clouds, branches or waves are rarely seen in nature, but are common in doctored photos. However, if the clone tool is used patiently – at close to the pixel level – even this may go undetected.
I have tested a couple of online “photo forensic” services on images created from multiple originals by professional artists such as Ashley Cameron and Erik Johansson, with little and nothing in the line of conclusive evidence. I therefore still believe that people with skill and determination may create deceptions that go undetected, even after thorough investigation.
The bottom line is that even if the same technology that is used to create doctored images is used to detect doctored images, such detection technology does not always succeed even when it is used, which it most often is not due to lack of suspicion, time or resources.
As can be seen, still images or live images that look like photographs can no longer be trusted unquestionably. In most contemporary cases, we would need additional guarantees that they are what we think they are and show what we think they show. News agencies mark their images in order to protect their property, but is that sufficient to verify the authenticity of image content?
The media’s need to be trusted
The media are a vitally important element of any democratic society, providing reliable information as a basis for opinions and decisions. This position is utterly reliant on trust. In order to gain and preserve this trust, the media have to check and double check that all quotes are accurate and that all stated facts are correct.
The media often use photographs as evidence, to the point that legendary Norwegian press photographer Johan Bruun once stated that the purpose of the press photo is to prove that what the writer writes is true (Skau/Aagaaard, 1986, p. 10). Consequently, the media guard the photograph’s role as evidence very jealously, stating that We Do Not Photoshop and Those Who Do Get Fired. The million-dollar question is to what extent the media are able to back up such statements.
As I have attempted to illustrate above, digital photographic images should no longer be taken at face value. They can be changed or created, often very easily, using automated image-adjustment software, and the change may be impossible to detect even after thorough scrutiny. Like analogue photos, they can also deceive in other ways. We have to trust the media to tell us the stories we need them to tell us, but where should we place that trust?
Who to trust, how far and when?
There are people who know, beyond any shadow of a doubt, whether or not a photograph really shows what it claims to show. These people will also, as a rule, know if the motifs they record truly represent the current situation, and they will quite often know if the situation itself was staged or not. You will find them in the field, taking pictures, and they know very well that photographic deception is not just a question of digitally altered images, even if that is receiving the strongest attention right now.
When I read a story written by a reporter, I usually believe what that reporter tells me, and if I do not, I can contact that person and express my doubt, or Google the name and check his or her reputation. When I read a photograph, that option is often closed, since the photographer is often anonymous.
Norwegian law states that all photographers should be credited because it is their right to be identified as authors of their own work. To my mind, all photographers should sign their work because it is their duty, as guarantors, that the story of the photograph is a true record of the event. They are, after all, the primary source of that knowledge, and should take personal responsibility for the stories they tell (unless they face grave personal risk in doing so (Buell, 2002, p. 136)).
Trust is a big word, and no trust should be blind. I do trust that the media, at least the serious media with editors and a reputation to protect, do try their very best to tell the truth, and nothing but the truth, on a daily basis. But I have also hinted at doubts about the media always being capable of discerning between more or less truthful photographs, and I have presented a wish for more responsibility in that area being placed directly where it belongs – with the photographer.
Quite how far I am willing to trust a particular photographer will depend both on the person and the situation, just as is the case with any print journalist, and in order to clarify this I would like to return to where I started.
How far should I trust Steve McCurry?
McCurry was criticised for his work, independent of and just prior to the scandal initiating this article. His perspective of India was attacked by Mr. Teju Cole in a New York Times article in March 2016 (Cole, 2016). To Mr. Cole, India today is laptops, wireless printers and escalators surrounded by a certain amount of chaos, and he finds McCurry’s organised interest for the India of the past utterly boring.
For my part, I am quite happy with McCurry’s lack of fascination for wireless printers in chaotic environments. I have such things at home. They are rather commonplace and, with few exceptions, rather boring to my taste. Taste is individual, as we know, and Mr. Cole is welcome to his own.
I think Mr. McCurry should be extended the same privilege. I do indeed trust him to report on what he finds fascinating in India or Cuba or elsewhere, and trust him to use any means necessary to tell his personal stories in pictures that are well composed, precise and beautiful. No one is able to tell “the whole truth”, but I do trust Mr. McCurry to tell his own to the best of his interest and ability.
Should I trust Mr. McCurry as a photojournalist, judged by the standard of today? Not without caution. His statements about his professional practice have turned out to be less than accurate and I think it wise of him to define himself today as a visual storyteller rather than as a reporter.
As for photojournalists in general, I trust them all as far as I trust any other reporters. To do that, I need to know who they are.