Face Scripting: What Did the Building See?

Film by Shumon Basar, Eyal Weizman, Jane & Louise Wilson

Between 8pm and 9pm on 19th of January 2010, Hamas official Mahmoud al-Mahbouh was killed in room 230 of Al Bustan Rotana Hotel, Dubai. A month later the Dubai Police released a video composed of footage from hundreds of surveillance cameras in Dubai’s airport, shopping malls and hotels that traces the assassination to Mossad agents. Since its broadcast on YouTube, that video has been seen by countless viewers across the globe. It operated as an agent in a murder investigation. Face Scripting is a story, made a year later, that ghosts the Dubai Police’s forensic film. It rehearses the generic architectural syntax of hotel rooms, corridors, and lobbies – those thresholds of blank transition. It also investigates the algorithmic technology of face recognition where unique individuals are identified from the blankness of crowds. This is not a documentary, just one combination culled from an infinite possibility of possible scenarios.

Video: Face Scripting: What Did the Building See?, 2011, single screen projection, surround sound, gauze box: 900x600cm, 2 mirrors: 300x380cm each, 1 HD projector, CCTV monitor, Commissioned by Sharjah Art Foundation and co-produced by the Farook Foundation with support from Luis Augusto Teixeira de Freitas and Mohammed Abdulkader Hafiz

Addendum by Eyal Weizman:

The film, made with Shumon Basar and Jane and Louise Wilson for the Sharjah Biennial of 2011, dealt with the investigation of the assassination in Dubai of Hamas operative Mahmoud Al-Mabhouh in January 2009. The subject of our film was “the story of the story” – it was an examination of the forensic investigation of the Dubai Police’s work.

Here are the basic facts of the story as we unpacked them in the film: The body of al-Mahbouh was discovered on the 19th of July in Al Bustan Rotana Hotel, Dubai. Within 24 hours, the Dubai authorities released a list of suspects, most of them carrying European passports, along with their photos. Three weeks later, Chief of Dubai Police, Dhahi Khalfan Tamin released a video, which was put online, associating Mossad with the killing. The film was co-produced with the Dubai Media Office, the same agency that has promoted Dubai’s brand of real estate development. The film was edited and inter-titled like a murder mystery.

Rather than keep this comprehensive real-time evidence secretly filed in an archive until the conclusion of the investigation, or until legal charges are brought forth, Tamin chose to spectacularized it. The film was indeed fascinating. Since its broadcast on YouTube the video has been seen by millions of viewers across the globe. The power of its evidence was in its becoming infotainment. Crowd sourcing meant that journalists, bloggers and other citizens started to fill in the gaps the film left out, extending the reach of Dubai police. It was once more a demonstration that the principle of diffusion of neo-liberal security is more effective than the intimidating policing and investigative techniques of authoritarian regimes.

The video was composed of footage from hundreds of surveillance cameras in Dubai’s airport, shopping malls and hotels that traced the supposed movement of the assassination squad. The 29 minute long video described the 19 hours leading up to the assassination through the arrival, meeting, movement and exiting of up to 26 different characters – dressed as business men and women, tourists, tennis players. The members of the assassination team are seen arriving separately at the Dubai Airport, casually going through passport control and baggage security, checking into different luxury hotels, gathering in a shopping mall, moving onto another hotel, tracking and following the victim through the lobby, reception, elevators and corridors, some of them changing costumes and wigs, wearing different make up, entering into the hotel room where the dead body of al-Mabhouh was later found, leaving this room, checking out and flying off to different places world wide. It takes the viewer through a series of non-spaces, generic interiors of flow and transit: an interconnected spatial continuum of air-conditioned interior spaces that seems to have no outside. All shots are interior shots. Incidentally, we noticed, the interior of room 230 — the room in which the assassination took place — was the only space not captured on film.

The assassination squad was recorded every time they entered and exited a fold within this space – the cameras being located at the transition between them. So in a sense one could say that these traversals sequenced the video. Doors of different kinds – airport doors, revolving doors at the entrance of hotel lobbies, sliding glass doors at a shopping mall, mirrored doors of an elevator – made the edits. If architecture could be understood an editing machine the door was the cutting blade, we thought.

Face recognition software identified the people involved. The film was thus the result of the intersection of architectural syntax and facial singularities, a narrative flows between spaces and faces, so to say.

Practically, the Dubai police force, extracting data from the face recognition software arrived at the assumed identity of a few suspects. Running backwards, every intersection between two or more agents gave the algorithm scanning the films another line of inquiry. They could thus track the new faces in the crowd until they were meeting further suspects, generating another bifurcation in the investigation. And this bifurcation network might continue ad-infinitum because every time those agents will meet colleagues in Dubai or any other country, more agents will be identified. The entire network had to be dissolved.

Face recognition software was essential because it automated the otherwise manual work of searching for and detecting a single or few faces amongst crowds adding to millions registered on tens of thousands of cameras.

The technology of face recognition was introduced into the princely for dealing with it enormous labor force. It seems that in this city-state, whether you are a tourist or a worker, you clock in and out looking at the camera.

Contemporary facial recognition software, such as those used by the Dubai Police, combine two algorithms: geometric, which looks at the physicality of the head as a three dimensional object – and photometric which studies the image of the face and compares its properties with photographs of faces to eliminate or search for variations. The first concentrate on the underlying skull and the second of the face. Analyzing an image, the geometric algorithm calculates the relative position, size and shape of the eye sockets, nose, nose-bridge, cheekbones, forehead, brain-casing and jaws; it looks for the morphology of the skull under the shadow of the skin.

In detecting and tracing the geometry of the skull only the relevant features from each face/skull were saved, so that the face/skull data could be compressed like a jpg… The useful bits for identification are called “diagnostic fragments” – the rest is deleted. These diagnostic fragments are used to compare with other face/skulls within crowds for matching features.

Because the geometric algorithm extracts the main formal landmarks of the shape of the underlying skull from an image of the subject’s face, it may be seen as the inversion of the ‘skull face superimposition’ that Thomas Keenan and I described in our book on Mengele. Perhaps it is in fact a double inversion, alternately striping and buildings the face from the skull. The advantage is that the skull – unlike the skin – cannot be tempered with, painted, made up stretched or distorted –techniques that are used for camouflaging and are likely to confuse the algorithm.

Camouflage distorts or breaks the form of that which it seeks to hide. Uniform camouflage breaks the unity of the body just like ‘dazzle camouflage’ broke up the visual cohesiveness of warships in World War I. Facial camouflage is directed not (only) at the naked eye but at the recognition capacity of the algorithm. It is usually achieved by special patterns of makeup that deepens or widens the shadow on a person’s face, thus distorting its true depth of its ‘topography’, In this way the shape of the skull could no longer be extracted from the image of the face. Sometimes a strategically-placed sticker, thick eyewear, a band aid, or a beauty mark can do the same job. The person camouflaging should know what is the algorithm doing the identification, so as to attempt to escape it. In the case of the assassins there were analogue camouflaging technique, made to evade digital identification. They sometimes looked like a pixilated equivalence of primitive masks.

In these examinations, it was of course not about the single skull and its morphology, but about a network assembled through it. It is connections and patterns that need to be demonstrated, and this requires a multiplicity of skulls and the connection between them.